TMSID Preservation overkill or worth exploring?

[poppadum]

Plus someone would have to match up the TmsIds to the data in the new EPG.
As unless you fancy doing that by hand, that would just be based on a text search for the programme name.

Does radiotimes.com provide unique IDs for programmes? I haven't noticed any in the output from xmltv (using tv_grab_uk_rt).

If not, won't we have to do some form of text matching anyway to map the programme names to whatever unique ID we decide to allocate a series/episode?

[Tcm2007]

That will be one of the major issues if you want SPs to work reliably.

I'm afraid you'll have to cope with a lot more duplicate copies of recordings.

I believe the Aussies get round it by making EPG maintenance a group communal effort with some kind of wiki-like system.

[gcobb]

I was thinking/hoping that we could pre-seed the database of series IDs in order to make it easy to transfer existing SPs. Either by taking them from existing series info (I need to look at some of the 4000-odd files in my tracker/ directory to see if that is useful) or just by deciding in advance what they will be, even for series which are not currently running (e.g. using a hash of the name or something).

Of course, it wouldn't be perfect because of series which are known by more than one name (F1, Formula 1, Grand Prix, etc.) and because of series with name clashes (e.g. Revealed) but it might allow most SPs to be transferred. Those cases are going to require manual intervention but it would still be possible to pre-assign what ID will be used for them.

Is there any relationship between series IDs and episode IDs or are they completely independent?

[irrelevant]

If there is a way of extracting them from an existing TiVo, even if just reading the SPs people have set up? If a script could be made that would do that, and post the results back to a central website for amalgamation, then I'm sure that would get a good proportion of the ones that people actually want, especially if it's simple to grab and run, like e.g. the install script.

[gcobb]

If you are running tracker then tracker/series has a list of series TmsIDs and the series name - not sure whether it is all series ever seen or series where at least one episode has been recorded (there are plenty I haven't heard of but those could be suggestions).

If you are not running tracker then the tivoweb backup module will backup your SPs which includes the TmsIds and names of all your season passes.

Of course, there is a privacy issue with collecting this info -- we would need a way for people to submit them without recording who submitted what.

Graham

[Tcm2007]

Seriously, the amount of work that would be , compared to setting up SPs afresh is just not worth it. A hour with a copy of the Radio Times and TiVo search and you're done.

[gcobb]

I understand the advice but, seriously, the SPs for the shows which are not currently running are the main point of having the Tivo in my case. We watch little TV but the few things which we do watch we don't want to miss and we rely on the SPs. Preserving that is my main interest in this project.

I am very interested in working on the tools to support crowd-sourcing correct series identification. Using the same series TmsId's as we have today is the easy part of that (although I fully realise that is not all that is necessary in order to be able to move SPs over -- but it would be very useful).

[baggsey]

The integrity of the program data on the TiVo is what gives it its unique search capabilities, and is going to be the core issue in maintaining long-term viability of the service.

As we all know, Tribune Media Services currently allocates a unique seriesid (TmsIds) to every program series or special or film. TiVo relies on the long-term uniqueness of that series id to manage season passes. Without the on-going management and accuracy of the seriesIDs, then we will need to rely on wishlists to identify which programs to record, which will be far less accurate.

However, there ARE ways to get a large proportion of the TmsIds from databases such as the "thetvdb" or zap2it, using APIs available at http://search.cpan.org/~behanw/TVDB-API ... VDB/API.pm . I've used the tool "pytivometathis" over on the pytivo forum use these APIs to batch process the acquisition of appropriate metadata for shows based on programme name.

One drawback is that Tribune has not always been consistent between the allocation of series Ids between the US and the UK. For example, NCIS has a series ID of SH861911 in both the US and UK, wheras Breaking Bad is SH845338 in the UK and SH01009396 in the US.

My recommendation would be to use "thetvdb" as the primary source for getting TMSids from US data, and augment it with what can be extracted from existing TiVos relating to UK programmes. Then as new programmes appear, our community would need to update thetvdb with whatever new code gets generated.

[Tcm2007]

Community update is the key. On day one it will be a crude text match to assign a Tmsid from the database to a show in the EPG, but so long as there's a system for people to correct mistakes and improve the data it will be fine.

It's not as if Tribune are exactly perfect.

[Tcm2007]

I understand the advice but, seriously, the SPs for the shows which are not currently running are the main point of having the Tivo in my case. We watch little TV but the few things which we do watch we don't want to miss and we rely on the SPs. Preserving that is my main interest in this project.

Unfortunately the ones which are not running are the ones which are hard to capture!

Even if we had a perfect database of series and their Tmsids, the first time through matching the EPG data to the database of series will be a little hit and miss.

[gcobb]

Unfortunately the ones which are not running are the ones which are hard to capture!

I have created a website to capture and make available TMSID information. It works by people uploading either SP backup files (created by tivoweb) or tracker series files. It allows queries (by name, with wildcards, or by TMSID) and can export the whole database as a CSV file.

Before I announce the location and encourage people to upload their data, I would like to test the code with a few more SP backup files and tracker series files. If you use tivoweb or tracker and would be willing to send the files to me to test my code, please PM me. I have populated it with my own files which provide 2389 TMSIDs. But the numbers go up to about a million so there are many more to capture! By far the best source is long term tracker users who can provide the series file (although that only gives TmsId and Name -- the SP backups give channel names and genres as well, both of which might be useful in disambiguating entries).

By the way, does anyone have a database for converting genre IDs to text descriptions?

Graham

[Tcm2007]

It's not currently clear that we could make use of any such database - it would depend on how easy it was to build it in to h existing OZTivo system

[gcobb]

It's not currently clear that we could make use of any such database - it would depend on how easy it was to build it in to h existing OZTivo system

Indeed. That will be one of the next things I plan to look at, unless someone else wants to take it on. But the first step is to collect the data.

I believe this can be useful for several different approaches. I hope to create the ideal solution: use it for the initial seeding of data (i.e. for series which are current at the switchover). If that turns out to be too hard, it still may be possible to integrate it later on (so that when series which are not currently showing restart they get an existing ID if one exists). In any case, if new ids are unavoidable, it provides a useful reference for a script to automatically (or maybe semi-automatically) convert existing SPs to use the new ids.

[Tcm2007]

I'm afraid I still think it is sledgehammer/nut - the amount of work needed to recreate your SPs is not that great.

[baggsey]

I have created a website to capture and make available TMSID information. ....... By the way, does anyone have a database for converting genre IDs to text descriptions?

Graham

Graham,

I applaud your efforts to build a TMSid database, but I think that you are probably attempting to duplicate capabilities that already exist. Take a look at http://www.thetvdb.com/ which is an open database for storing all relevant information about tv shows, including TMSids etc. It has a set of defined APIs to access the data through a variety of queries.

If you take a look over at the pytivo forum, and look for Metagenerator you'll be able to download a nice GUI which accesses the database and pulls back all the relevant information, with "TivO Series ID" clearly retrieved. Take a look here http://pytivo.sourceforge.net/forum/met ... -t555.html It's very easy to write a Python or OORexx exec to pull the data.

With regard to Genre mapping, you might want to take a look at this page in the pytivo forum http://pytivo.sourceforge.net/forum/pyt ... ht=baggsey where you can download a genre code file.

My personal view is that whatever process is used to populate the TMSids, it makes sense to use http://www.thetvdb.com as there is already a global community that updates it. However, while the TSMid data for US TV shows is very good, the data for UK-only TV shows is patchy, so your database could be of good use to update thetvdb with TMSids for UK-specific TV shows where the TMSid is blank.