#1 – Broadcast EPG Data

I really thought I would hate the EPG data broadcast by the television networks, but I have to say I’m satisfied with it. I’m not saying I’m impressed by the quality and consistency of TV network data, but at the same time I’m not saying I’m disappointed by it! I can definately see the quality of programme data improving with FreeView providing a unified EPG stream and a move toward MHEG-5.

I normally scan the TV guide for the upcoming week at least once to make sure all my favourites are recorded and to see what’s new. Just keep an eye out for multiple names for some of your favourite shows like “The Simpsons”, “All New Simpsons” and “The Simpsons Marathon” … But once you’ve worked out and recorded all the variations then you’re set!

#2 – Colour Coded Programmes

Annoying at first, but the colour coding of different genres has grown on me. My niece especially likes it because she knows her children’s programming by colour … (she can’t read yet).

At least the TV network are taking the time to categorise the shows they broadcast!

#3 – Intelligent Recording

Don’t you just hate when you schedule your favourite show to record and the show before it runs overtime? You end up missing the last 30 minutes of your show and then get an ear bashing from your better-half because the “stupid Media Center” isn’t doing it’s job! Not so anymore …

It seems that when shows run overtime, the TV networks are performing “last minute” updates to their broadcast EPG, which Media Center is smart enough to detect. This means Media Center automatically adjusts your recording’s end time to ensure all of it will be recorded! Take that live television!

#4 – Seek Preview

One thing I hated about the old Windows 7 and Windows Vista video playback was seeking backwards and forwards. Jumping 10 seconds backwards and 30 seconds forwards gave you a kind of epilepsy as frames jumped about. Now the latest Windows 7 RC version has a “mini” preview window that allows you happily seek without interrupting the main playback. Once you’ve found where you want to be in the preview window, Media Center then jumps to it!

Thumbing through (or should I say clicking through) my diary tonight and realised I’ve only spent about 10 hours in total over the last 14 days working on the compiler. To my surprise, the majority of that time was spent on designing and developing the user interface!

As I’ve mentioned previously, the core of the wmcGuideCompiler7 is complete and runs happily as a background task keeping my test Windows 7 box topped up with EPG goodness from ozTiVo.net. The main issue, as always, is configuration and how to let users sensibly configure their settings without having to hack at XML or flat-file configuration files. So, before I release the alpha, I’ll be finishing off the user interface … So, it might actually turn into a beta by the time it gets released!

Anyway, in the meantime if you have any suggestions/comments on functionality or some useful information then feel free to email me directly at the address below. Just remember time is always tight and I don’t have the luxury of IceTV‘s resources or Niall‘s commercial investment in getting this up and running …

If you hadn’t already worked out from my previous QR Code entry, I proposed to my long-time-suffering girlfriend today so I’ve been busy this week getting a few things finalised for today! I won’t bore you with all the details but, needless to say, I did all the right things … “asked the dad”, “consulted the mum” and “got down on a knee”. Oh, and she said “Yes!”.

So without further delay (tomorrow) …

I think it’s about time to call for a handful of alpha testers next week to start giving me some feedback on all the base product features, and then hopefully if all goes well I’ll add those nice pretty features that users crave for a beta release!

We don’t need that many alpha testers, but I am looking to rank which users this first cut is available for:

1. You’ll need to be a current site sponsor and have an existing epgStream.net license.
2. If you are from Australia, preference will be given to NSW users, preferably Sydney.
3. Only a very limited number of overseas users (New Zealand included) are required!

Just flick me a private email to the address shown below and I’ll get back to you with some basic details as to when and how you’ll get your alpha.

Thanks a bunch!

The reason I’m keen for a wmcGuideCompiler7 beta/pre-release is because our current versions of Windows 7 are, obviously, still in beta. I’m sure that between now and the final Windows 7 release we’ll see some more Media Center features added, existing functionality break, and dozens of bugs fixed!

Cast + Crew Issues

One of the bugs to be fixed in the next few releases of Windows 7 include allowing custom “Cast + Crew” program information. The current Windows 7 beta allows us to load our cast information, but it is not displayed. 

On closer inspection though I think the “Cast + Crew” information being displayed is actually pulled from a Microsoft web service, much like it is currently for DVD information. This might explain why (if you look very closely at the picture shown above) the actor’s name plus their character’s name is displayed (in different fonts even)!

Other Annoying Problems 

There are a few little Media Centre issues that annoy the hell out of me at the moment, so if you are an “official” beta tester, make sure these have been ticked off.

• Faded EPG - Scrolling ahead in the guide to a future date the guide grid performs a “fast forward”, displaying the day you are currently up to. When you stop navigating ahead though the guide fades into the background and the more you fast forward the more transparent it gets!
• Metadata Searches - Trying to perform a guide search for a particular title, actor or keyword but nothing is showing up? Well it’s probably because your Windows indexing Service is sleeping because your machine is in “Power Saver” mode. Could be a few hours before you’re ready to search …
• Duplicating Channels – I’ve noticed that sometimes the scanned channel listings in the EPG replicate multiple times. This means you can sometimes end up with a page full of ABC1 if you leave Media Center running over night … 

* Thanks to “Alexander S.” for burning the word “Mike-rosoft” into my brain …

Aside from my Bruce Schneier books, I really love my well-read “The Art of Computer Programming” (TAOCP) volumes by Donald E. Knuth and was happily side-tracked for a hour as I thumbed through some familiar chapters. Keen to see if there were any new Knuth editions published, I temporarily abandoned the library cleanup and checked on Amazon to find:

The Art of Computer Programming (Vol. 1-3) – US$149.18 

3 updated editions, a boxed set and all hard covered to boot!  How could I not buy these computing classics in the hopes that one day I could pass them onto my computer-literate grand-kids? Needless to say I promptly purchased these tax deductible references and then returned to the purging of my professional library.

Two weeks later (i.e. yesterday) this heavy boxed set turned up on my doorstep, and much to my surprise, in pretty good condition after being shipped from the USA. Ahhhh… nice white pages with no notes in pencil or dead mosquitoes stuck between the pages. Now I can take my tattered old TAOCP paperback copies to work and not worry so much if any “light fingered” work colleages are going to borrow my books for months at a time!

If you haven’t heard of Donald Knuth before then take some time out of your day to find out about his work. This is especially important if you want a better understanding of the “hows and whys” of software development fundamentals, or maybe just need to write lean, mean, industrial strength code.

Oh… errr… I finally sent off the sample Windows 7 EPG MXF data to Mike Hayton this morning for critique. I would have sent it last night but I was too busy flicking through TAOCP, Volume 2. Hopefully Mike will give it the thumbs up and we can start looking at releasing a new wmcGuideCompiler7 version!

The original plan was to release a new version of our Guide Service Proxy application which included a Windows 7 MXF guide compiler. After a little refactoring and a lot of reflection and I decided that we should bundle the compiler as a new application, rather than trying to stuff it into the currently stable Guide Service Proxy.

I spent pretty much late Sunday afternoon transplanting my existing work into a new, clean application base which now has the project name of wmcGuideCompiler7.  This new application base frees me from some of the limitations of the previous Guide Service Proxy base without compromising the great command-line and GUI functionality. You’d be surprised just how many painful lessons we’d learned working with XMLTV and Windows Media Center meta-data, all of which I don’t plan on forgetting with this new project! 

So what next? Well I’ll set aside an hour or two after work this evening and send Mike Hayton a sample MXF file for him to critique. Assuming I get the two thumbs up from Mike, I’ll try and arrange a beta or release candidate when I can scrape together more free time (or if one of my other contract projects are canned).

Vista Media Center‘s EPG really looks like a dog’s breakfast breakfast compared to the new Windows 7 Media Center EPG. It’s great to see all more information loaded onto your screen and I don’t just mean all the extra attributes but the new injection of colours and the clean layouts of the information … It almost makes me want to watch more TV!

You’ll see a lot of the older features like the high-definition icons and episode names, but you’ll also see a lot more information crammed into the programme descriptions! New things like onscreen star ratings, premiere indicators or even the year of the movie help tip the balance when deciding what to record. The only thing I find really disappointing is that you can search for your favourite actor’s shows, but you can’t view the actors in the programme/movie synopsis.

Our current internal Windows 7 build of our Guide Service Proxy application seems to be running fine with no nasty Media Center crashes or loss of guide data. I promised Mike Hayton I’d have a quick word with him before releasing our first public beta, but there isn’t any reason why we wouldn’t release a nicely bundled XMLTV: Guide Pack update with this Windows 7 EPG compatibility over the weekend!

All this is great news for us here in Australia, and well actually for any other country without a native Microsoft guide service (New Zealand, South Africa, etc). Let’s just hope that all this initial work everyone’s been doing for the current Windows 7 Beta lasts through to the final version of Windows 7! Looking through the MXF EPG schema guts the EPG looks pretty solid, I’m just a little worried though that the MXF schema still says 6.x and not 7.x.

Beta testers … Start your engines!

It’s Alive! It’s Alive! 

I managed to dodge all the distractions of a lazy Australian Sunday and build a fairy clean, first cut version of a Windows 7 XMLTV to MXF compiler! Using XMLTV data from ozTiVo.net I was able to load guide data for 15 glorious channels directly into my Windows Media Center guide!

Lots of channel data, not enough channels!

My ASUS Windows 7 test machine is located in study which is good when I have to knuckle down and get some work done in peace. The only problem is that during the daytime, my very sad tuner antenna only picks up SBS channels which means I’m limited in the number of physical channels I can work with. If I happen to tune channels at night I get a much bigger selection … Go figure?!!

From the screenshot sequence above you can see what happens as I add the new XMLTV listing to each scanned channel (SBS, SBS HD, SBS2, etc). You’ll notice the affilate (or network) name after each custom guide listing appears as “epgStream.net”. In the final release of the compiler you should see this replaced with your actual TV network name such as “Nine Network” or “7″ in Australia or I might just keep it as simple as “XMLTV”.

For some extra contrast between channels I’ve overlaid “SBS 3″ and “SBS 4″ channels with “Ten Digital” and “Ten HD” XMLTV data respectively. It’s good to see the little “HD” symbol appear …

Metadata and beyond! 

The new Microsoft EPG schema become a little easier to use with more descriptive schema elements and less complex relationships and references. Overall, this makes the migration from XMLTV to the new MXF structure a much simpler and “flatter” affair.

The biggest improvement I’ve seen is in the handling of the rating, category and keyword referencing. Gone are the days of bit-shifting offset values to set and retrieve reference values which reduces the complexity of the compiler considerably.

How does it work?

Our Guide Service Proxy application has been given a new command-line switch to invoke the compilation and load:

wmcGuideServiceProxy /compile:MXF!6.0.6000.0 /compileSource:XMLTV.xml /load

This means that our application’s next GUI revision will need to be modified to accomodate some extra Windows 7 features, but at least it will be only one application to service all Media Center versions.

Where to next?

Next on the list of things to do is to integrate customised images into the on-screen EPG display. From a few articles I’ve briefly skimmed over it doesn’t look too hard to get this implemented … Though Niall has a few weeks head start over me so I better get cracking!

Any requests?

Sorry guys, I would have posted a little earlier today but had to help a friend of a friend purchase a Toshiba Portege R500 yesterday and have been setting it up for her. Check out these cool videos: http://www.youtube.com/watch?v=4ZboNKnG6oY

Reviewing Mike’s recently released documentation, the internal objects look very similar so hopefully it won’t take long to knock up a working compiler!

Wish me luck!

Loading in your own listings is not a supported scenario. If you try to load in your own listings and things aren’t working, please don’t expect Microsoft support on this. - Mike Hayton

It will be interesting to see what I’ve previously missed when trying to link the MXF channel and service data together. I’ll be looking into this sample more and eventually building a new compiler into our Guide Service Proxy application, but in the meantime you can check out his original post here.

This choice piece of new means that the dreaded hit-n-miss work of disassembling and testing the Windows 7 EPG is made a whole lot easier. Once Mike releases his notes we’ll look at updating our XMLTV: Guide Pack to handle good ‘ole Windows 7.

Mike, whatever Microsoft is paying you … It ain’t enough!

Now that I’m back into “work mode” after my holiday I started investigating the possibility of a custom Windows 7 EPG. After a quick review of Niall’s thread at XP Media Center I decided to roll the sleeves up and get to work! 3 hours later I wished I was back on holidays …

Remember LoadMXF.exe ?

I initially heard good things about LoadMXF which indicated I could load finely crafted EPG XML metadata into Fiji’s Media Center database. Then I started hearing bad things about the lack of Microsoft documentation so decided to have a look for myself at the new internal EPG infrastructure. I must admit I felt better when I glanced at Fiji’s internals as everything seemed normalised and nicely engineered, but it also seemed very complex.

Mike Mayton (Microsoft) hinted at providing some unofficial documentation he was preparing so I decided to wait till it was released before digging any deeper and re-inventing the wheel, so to speak. Unfortunately Mike hasn’t been able to generate any documentation yet, not really his fault as everyone is busy and he was going out of his way to help us Australian users. 

What should my MXF file look like?

The first thing I had to do was work out the basic structure of the MXF format, <MXF> … </MXF> right? Well after a bit of careful reflection (pun intended) and scavaging some local MXF files I determined you need at least the following, non-assembly elements to begin constructing your MXF document:

1. <Providers> – A registered source of EPG data. I presume you can load multiple provider data and hence merge multiple channel data!
2. <Programs> – The list of individual TV programme metadata including titles, descriptions and other bits and pieces you’d expect.
3. <ScheduledEntries> – When and “where” your TV programmes will be scheduled.
4. <Lineups>, <Channels> and <Services> – All the glue you need to bind your EPG to your computer’s region and tuner capabilities.

Looks familiar? Well yes it’s very similar in nature to the previous object classes used by Vista and XP Media Center, but there are dozens of new entities that help make up a really rich Media Center experience. Channel and programme logos, series information, promotions, channel merging, automatic title editing and much much more now seem possible! 

Running LoadMXF.exe for the first time …

“WOW!” I though, “This isn’t too bad after all. I’ve constructed my first MXF file so what was everyone making a fuss about?” – WRONG! I ran LoadMXF with my hand-crafted MXF file and … no errors on screen or in the log! My joy was short lived as searched through all the TV Guide options and I realised my data didn’t seem to be available in Media Center.

My data really is in there!

Perplexed, I double checked all my MXF element references and identifiers and re-ran the LoadMXF process again, but this time stoping and restarting the Media Center Windows Services. No luck.

I though I could sneak a peak at the Media Center SQLite database … Various free SQLite drivers all quickly failed miserably so I decided to have a look at the in-built “MediaCenter.Store.Sqlite” namespace. The only glimmer of hope was the ObjectStoreOpener class but alas it resulted in disappointment once again. When I have a little more time I’ll investigate the whole direct SQLite access arena again … Can anyone recommend one?

Onwards and upwards! Thankfully I found a MXF exporter class that managed to dump all the database data objects. On inspection of the output MXF dump I could see my previous MXF data had been loaded but why wasn’t it appearing in Media Center?

Looking a little harder …

I get the impression it’s quite possible to merge the EIT data with any custom loaded MXF data, but the trick is knowing how all Media Center’s internal data objects all relate to each other (should I be looking into the MergedChannel stored object?). I need to dig a bit deeper into the new Media Center beast to find out what I’m missing, or at least go on a Google rampage and find out what other developers have been doing.

Spoofing those Microsoft Guide servers

After a frustrating 3 hours I came to the conclusion that it might be worth hacking at the Media Center package download protocol to give me a better picture of what is going on between the Media Center client and Microsoft server.

An initial look at the data seems to be ok with the raw MXF data still being compressed before being encrypted. The only thing that seems to have changed with this raw data is that Microsoft are now using AES for encryption and SHA for hashing. For the life of me I can’t see why Microsoft even bother encrypting the data anymore, even more so that it’s typically streaming over a HTTPS connection.

What happens next?

Saturday is going to be a busy day so I might send a few emails and investigate this route more on Sunday. I’m not sure how much time I can spend on this now … Our other developer (Oz) is back overseas and I’ve got another full-time project I’m working on so any help, hints and findings would be greatly appreciated from everyone!