WisconsinEric

EPG from XMLTVListings

Recommended Posts

I am starting a new topic since the current (2/17/19) discussion of TV Guide not updating has gotten overcrowded.

My hope for this topic: just focus on using xmltvlistings.com as an alternative to the non-functional TV Guide EPG.

Has anyone automated the EPG from xmltvlistings similar to what is described for schedulesdirect.org in the "EyeTV Portfolio Software" Forum discussion? Specifically, not being a programmer, I need some guidance in using the API from xmltvlistings (see the link to it in their help menu). I understand what it is designed to do, but the 4MB file it downloads is not identified as an .xmltv file in format, and is not in a format that EyeTV can use. So other steps must be needed.  Thanks anyone for your help.

In comparison, the XMLTListings non-programmers option to simply download a 7-day xmltv EPG file works just fine; after preparing EyeTV*, you just drag the file onto the EyeTV icon. But it would be nice to automate it, get 14-day data, and hopefully the season/episode info (not included in the 7-day file).

* - before using the 7-day EPG direct download, manually change the EPG source to xmltv in EyeTV (use the dropdown menu in Channels) for each active channel in your lineup, and (this is important) save the result as a new lineup. Make sure the lineup is the same as the lineup you select when you sign up for XMLTVListings.com (that site allows you to customize the lineup just as EyeTV does).

Share this post


Link to post
Share on other sites

I did this a little differently. I dragged the xmltv schedule I downloaded onto the EyeTv icon (when quit) in the Finder. EyeTV started and loaded the file. When I changed each Channels listing to wmltv, I was prompted to choose a channel from a list that appeared, to assign to the channel in my list. Although the names on the popup list weren't exactly the same as the names in my Channels list, it was fairly easy to figure out which one corresponded. After I clicked ASSIGN, a schedule for that channel filled that grid row. I did this for all the channels in my Channels list. There were a few that seemed to have no corresponding listing, and these are still on EPG. I never saved the result, but the assignments and the grid persist through quitting and restarting EyeTV. Before starting, I set the Guide in EyeTV prefs to none, then quit EyeTV, although I don't know if this matters.

Edited by Earl Noe

Share this post


Link to post
Share on other sites
1 hour ago, WisconsinEric said:

Has anyone automated the EPG from xmltvlistings similar to what is described for schedulesdirect.org in the "EyeTV Portfolio Software" Forum discussion? Specifically, not being a programmer, I need some guidance in using the API from xmltvlistings (see the link to it in their help menu). I understand what it is designed to do, but the 4MB file it downloads is not identified as an .xmltv file in format

Hmm.  Their documentation says "Get listings in XMLTV Format, for a given lineup."  It looks like there are three options:  get, get_lineups, and get_channels.  Are you using "get"?  What does the 4MB file look like inside?  It should look like the example here: https://www.xmltvlistings.com/help/api/xmltv

Share this post


Link to post
Share on other sites

I gave it a try, and the command to run (or put in a script) is:

curl https://www.xmltvlistings.com/xmltv/get/API Key/LineupID/Days >> xmltv.xml

That produces an xmltv file (named xmltv.xml) that EyeTV happily consumed.

However, "XML TV Listings" data is much more limited than Schedules Direct, and they only allow 5 connections per day, so that limits my ability to test.

 

  • Thanks 1

Share this post


Link to post
Share on other sites
1 hour ago, DC Rackerby said:

Hmm.  Their documentation says "Get listings in XMLTV Format, for a given lineup."  It looks like there are three options:  get, get_lineups, and get_channels.  Are you using "get"?  What does the 4MB file look like inside?  It should look like the example here: https://www.xmltvlistings.com/help/api/xmltv

Hi - I am using 'get' (the first of the 3 API options). The file has the information I want, but the file type is identified by my Mac as text: the file downloads with the file name '14' and no file type, presumably because I chose a 14-day EPG listing option. Whereas the Direct Download option (at https://www.xmltvlistings.com/help/start/listings) downloads a file identified with the correct .xml file type, which can be dragged directly onto the EyeTV icon to load the EPG (as Earl Noe described above, that's the method I've been using manually). Here's the first few lines of what 'get' produces:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE tv SYSTEM "xmltv.dtd">
<tv date="02/18/2019" source-info-url="http://www.tvmedia.ca" source-info-name="TV Media">
  <channel id="3156.stations.xmltv.tvmedia.ca">
    <display-name>ABC (WBAY) Green Bay, WI</display-name>
    <display-name>WBAY-TV</display-name>

etc. (with this detail, there are presumably thousands of lines just for 14 stations in one lineup). That matches the illustration you linked to. I also opened a Direct Download file (the more limited 7-day option); it appears identical in contents to the 'get' API file, but is identified as .xml in the file name. So . . . I wonder if it is a simple matter of just manually reassigning the file type to get EyeTV to recognize it? I'll try that later this evening (I'm recording something right now). What do you think?

 

Share this post


Link to post
Share on other sites
2 hours ago, WisconsinEric said:

Hi - I am using 'get' (the first of the 3 API options). The file has the information I want, but the file type is identified by my Mac as text: the file downloads with the file name '14' and no file type, presumably because I chose a 14-day EPG listing option. Whereas the Direct Download option (at https://www.xmltvlistings.com/help/start/listings) downloads a file identified with the correct .xml file type, which can be dragged directly onto the EyeTV icon to load the EPG (as Earl Noe described above, that's the method I've been using manually). Here's the first few lines of what 'get' produces:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE tv SYSTEM "xmltv.dtd">
<tv date="02/18/2019" source-info-url="http://www.tvmedia.ca" source-info-name="TV Media">
  <channel id="3156.stations.xmltv.tvmedia.ca">
    <display-name>ABC (WBAY) Green Bay, WI</display-name>
    <display-name>WBAY-TV</display-name>

etc. (with this detail, there are presumably thousands of lines just for 14 stations in one lineup). That matches the illustration you linked to. I also opened a Direct Download file (the more limited 7-day option); it appears identical in contents to the 'get' API file, but is identified as .xml in the file name. So . . . I wonder if it is a simple matter of just manually reassigning the file type to get EyeTV to recognize it? I'll try that later this evening (I'm recording something right now). What do you think?

Earlier DC Rackerby had said:

I gave it a try, and the command to run (or put in a script) is:

curl https://www.xmltvlistings.com/xmltv/get/API Key/LineupID/Days >> xmltv.xml

That produces an xmltv file (named xmltv.xml) that EyeTV happily consumed.

However, "XML TV Listings" data is much more limited than Schedules Direct, and they only allow 5 connections per day, so that limits my ability to test.

Confirmed, the problem was just the file type. As soon as I manually assigned a .xml file type to the file downloaded from XMLTVListings, EyeTV recognized/loaded it correctly. You are correct too about the data: despite the longer 2-week listings time period, there are still no data on series/episode, and info about the program is more limited than it was with TV Guide. I guess that leaves Schedules Direct as the only source of series/episode data, but otherwise XMLTVListings produces a very decent EPG with an API to automate downloads and is more user friendly (hey, $5/year cheaper!). It is a fairly new service, based in Ottawa, Ontario, so maybe they will up their game in the future. Thank you, DC Rackerby, for your help and the way of using the 'get' command!

Share this post


Link to post
Share on other sites
19 hours ago, WisconsinEric said:

Hi - I am using 'get' (the first of the 3 API options). The file has the information I want, but the file type is identified by my Mac as text: the file downloads with the file name '14' and no file type, presumably because I chose a 14-day EPG listing option. Whereas the Direct Download option (at https://www.xmltvlistings.com/help/start/listings) downloads a file identified with the correct .xml file type, which can be dragged directly onto the EyeTV icon to load the EPG (as Earl Noe described above, that's the method I've been using manually). Here's the first few lines of what 'get' produces:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE tv SYSTEM "xmltv.dtd">
<tv date="02/18/2019" source-info-url="http://www.tvmedia.ca" source-info-name="TV Media">
  <channel id="3156.stations.xmltv.tvmedia.ca">
    <display-name>ABC (WBAY) Green Bay, WI</display-name>
    <display-name>WBAY-TV</display-name>

etc. (with this detail, there are presumably thousands of lines just for 14 stations in one lineup). That matches the illustration you linked to. I also opened a Direct Download file (the more limited 7-day option); it appears identical in contents to the 'get' API file, but is identified as .xml in the file name. So . . . I wonder if it is a simple matter of just manually reassigning the file type to get EyeTV to recognize it? I'll try that later this evening (I'm recording something right now). What do you think?

 

You could try changing "open with" to EyeTV in GetInfo.

Share this post


Link to post
Share on other sites
8 hours ago, Earl Noe said:

You could try changing "open with" to EyeTV in GetInfo.

Good idea, though simply manually assigning the .xml file type did the trick too (see my post from 2/17). I am moving toward automating getting the EPG, so I'll use the command that DC Rackerby wrote in his post above. That's step 1, downloading the EPG file, which I know works. Step 2 is loading it into EyeTV; in the other discussion thread, there's a command given by Weber and Rackerby should work for this file from xmltvlistings as well (they are using schedulesdirect). Finally, Step 3 is to save this as an executable script, and have it run daily at 5 am or so. For that there have been a few suggestions, so I have to work that out, but since I have a son who does this sort of thing for a living I am going to rely on him. 😀

Share this post


Link to post
Share on other sites
11 hours ago, WisconsinEric said:

I am moving toward automating getting the EPG, so I'll use the command that DC Rackerby wrote in his post above. 

This command:

curl https://www.xmltvlistings.com/xmltv/get/API Key/LineupID/Days >> xmltv.xml

will append the new info to the xmltv.xml file.  So, file will get larger and larger.

This is probably better:

curl https://www.xmltvlistings.com/xmltv/get/API Key/LineupID/Days > xmltv.xml

> redirects output to a file, overwriting the file.

>> redirects output to a file appending the redirected output at the end.

 

Share this post


Link to post
Share on other sites
10 minutes ago, DC Rackerby said:

This command:

curl https://www.xmltvlistings.com/xmltv/get/API Key/LineupID/Days >> xmltv.xml

will append the new info to the xmltv.xml file.  So, file will get larger and larger.

This is probably better:

curl https://www.xmltvlistings.com/xmltv/get/API Key/LineupID/Days > xmltv.xml

> redirects output to a file, overwriting the file.

>> redirects output to a file appending the redirected output at the end.

 

Ah, good point, thank you again (and I learned something). 😀 I assume that (a) the two commands are equivalent if the file does not currently exist, and (b) the file will go to the default location for downloads set up for the web browser. No need to reply unless I'm wrong on either count.

Share this post


Link to post
Share on other sites
1 hour ago, WisconsinEric said:

Ah, good point, thank you again (and I learned something). 😀 I assume that (a) the two commands are equivalent if the file does not currently exist, and (b) the file will go to the default location for downloads set up for the web browser.

As the command is written, the xmltv.xml file will be created in the directory where the command is executed.

If you want it to be created in a particular directory, you would specify that.  For example:

curl https://www.xmltvlistings.com/xmltv/get/API Key/LineupID/Days > ~/Downloads/xmltv.xml

  • Thanks 1

Share this post


Link to post
Share on other sites

This line

type in ."/mc2xml -J sdusername:sdpassword -c us -g zipcode"

Simply results in "-bash command not found" -- and yes I'm inputting the username password and zip code for schedules direct. I've tried just leaving "zipcode" in the command as well as all variations of the opening period and quotes

 

Never mind.. no quotes.. the Word file is written poorly.

 

./mc2xml -J sdusername:sdpassword -c us -g zipcode

 

Edited by NotInUse

Share this post


Link to post
Share on other sites

But... upon opening the xml file with EyeTv.. EyeTV simply hangs and is unresponsive.

 

In fact, opening the XML file with BBEdit hangs that app as well. Perhaps due to the file size.

Edited by NotInUse

Share this post


Link to post
Share on other sites

Okay it seems to work but always hangs EyTV. So, I won't be able to automate this. After letting it sit for some time as "unresponsive" I can force quit, and relaunch --- then I have to do that quirky hardware thing of unplugging and replugging the device (EyeTVHybrid) - it loads and the data is present.

 

I'm uncertain why it gets stuck opening the XML file. It seems to load it, but then has trouble exiting the load.

Edited by NotInUse

Share this post


Link to post
Share on other sites

Okay so it seems the XMLTV file includes ALL channels for my provider in my zip code, not merely those channels "enabled" at schedulesdriect. This is what is bogging down EyeTV.. loading 800 + channels rather than the 50-75 I have enabled.

 

Does anyone know how to restrict the XML to only enabled channels?

Edited by NotInUse

Share this post


Link to post
Share on other sites
15 minutes ago, NotInUse said:

Okay so it seems the XMLTV file includes ALL channels for my provider in my zip code, not merely those channels "enabled" at schedulesdriect. This is what is bogging down EyeTV.. loading 800 + channels rather than the 50-75 I have enabled.

 

Does anyone know how to restrict the XML to only enabled channels?

Did you edit the Channels window in EyeTV to select xmltv from the dropdown EPG menu (has to be done manually for each enabled channel)? You have to do this after you download the .xml EPG file and drag it onto the EyeTV icon. With 50-75 channels enabled that will be a PIA, but you only have to do it once. The channels you have checked as enabled should exactly match the channels you edited schedulesdirect to form your channel lineup.

Share this post


Link to post
Share on other sites

Yeah seems the mc2xml by default pulls every channel for the provider/zip code. The key was to tell mc2XML to ONLY pull specific channels via a mc2xml.chl file in the same directory (just list the channels you want in the file, that's all). It dropped it to actually 114 channels as opposed to 680+,. It still takes EyeTv about 5 minutes to processes the XML, but it DOES process it completely... and all the EPG data is there. (interestingly enough there was a "complete" sound that EyeTV made I had never heard before).

 

And yeah, setting all channels to the XMLTV option took a few minutes, but, as you stated, do it once an then "Save Channels" so it's there. and with the channels saved, I can switch back to an old saved channel lineup to test if the TVGuide data is EVER f*in back online :)

 

(Note I also had to give the bash scripts permission to connect to the web in LittleSnitch)

 

I still want my TVGuide EPG data.. but this is much better than nothing. I do appreciate all the work which went into figuring it all out.

Edited by NotInUse

Share this post


Link to post
Share on other sites

 

8 hours ago, NotInUse said:

Yeah seems the mc2xml by default pulls every channel for the provider/zip code. The key was to tell mc2XML to ONLY pull specific channels via a mc2xml.chl file in the same directory (just list the channels you want in the file, that's all).

You can modify your lineup via the web site (see attached).

Screen Shot 2019-02-20 at 9.44.07 AM.png

Edited by DC Rackerby

Share this post


Link to post
Share on other sites
8 hours ago, NotInUse said:

Yeah seems the mc2xml by default pulls every channel for the provider/zip code.

You can also edit your lineup via Schedules Direct's web site (if you are using that service).

Screen Shot 2019-02-11 at 4.16.11 PM.png

Share this post


Link to post
Share on other sites

DC...

The saved lineup at SchedulesDirect does NOT alter what mc2XML pulls. I have a saved lineup. 

If you read mx2xml.awardspace.info you'll discover that mc2xml by default will always pull all channels. I had a lineup saved at ScedulesDirect, but mc2xml payed no attention to it other than to pull the provider and zip code.

mx2xml.awardspace.info does provide information on how to limit what it pulls. This was an easy enough processes --- simply create a new text file titled "mc2xml.chl" in the same directory as mc2xml and inside that text file list channels you want or don't want. i.e.:

1-5

15-30

-31-45

56

65

A minus sign in front of number is to exclude channel or range of channels. That's all there is to it. 

mc2XML will see the .chl file and pull/exclude the channels listed. More info can eb found at mx2xml.awardspace.info 

Edited by NotInUse

Share this post


Link to post
Share on other sites
1 hour ago, NotInUse said:

DC...

The saved lineup at SchedulesDirect does NOT alter what mc2XML pulls. I have a saved lineup. 

Well, it does if you are using the older service (-T flag in mc2xml), but not with the JSON service (-J flag in mc2xml).

If you click on "Lineup Support" on the Schedules Direct web site, you'll probably see that you have two lineups (one of which you cannot edit).

However, I thought this thread was primarily about xmltvlistings.com

Share this post


Link to post
Share on other sites
On 2/19/2019 at 11:38 AM, DC Rackerby said:

This command:

curl https://www.xmltvlistings.com/xmltv/get/API Key/LineupID/Days >> xmltv.xml

will append the new info to the xmltv.xml file.  So, file will get larger and larger.

This is probably better:

curl https://www.xmltvlistings.com/xmltv/get/API Key/LineupID/Days > xmltv.xml

> redirects output to a file, overwriting the file.

>> redirects output to a file appending the redirected output at the end.

 

Still trying to get automating xmltvlistings to work. Using the curl command above (see below) does not download a file, only creates a 4K file with that filename (and it is not listed as a download in my xmltvlistings account). I am trying to figure out what I'm doing wrong. I know the API works, because I ran it from Firefox; it produces a 4MB .xml file named <14> (for the # of days requested), I do have to manually add the file type to get EyeTV to accept it. In the curl command, does the URL portion have to be a clickable link as you have it? In mine it is not, since I saved the script file as a plain text file (TextEdit wants to save by default as RTF, which I had assumed would not work in Terminal). Here is the script I used (the only one of the 3 commands that works is the 30-second pause, which I put it as a safety margin so the download would be completed before loading into EyeTV). I had previously identified it as executable in Terminal (the file name of the script is xmltvepg):

cd /Users/godfreye/Scripts/
chmod 755 xmltvepg

Anything wrong jump out at you? Thanks for the help.

#!/bin/bash
cd /Users/godfreye/Scripts/
curl https://www.xmltvlistings.com/xmltv/get/[my API key]/[my lineup number]/14 > xmltv.xml
#allow time for download to complete
sleep 30s
#load EPG into EyeTV
open -a EyeTV xmltv.xml

Share this post


Link to post
Share on other sites
31 minutes ago, WisconsinEric said:

Using the curl command above (see below) does not download a file, only creates a 4K file with that filename (and it is not listed as a download in my xmltvlistings account).

I just re-read this.  That sounds like a typo in your URL.

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.