How the iTunes Store could deliver High Definition for the AppleTV

When Apple’s iTunes Music Store introduced video at 320×240 resolution back in October 2005, it raised a number of questions: how would it scale up? How could they manage movies at that paltry resolution? The iPod’s video hardware is so limited, how could it even scale up to 640×480? All of the anxiety about complex solutions were laid to rest after Apple introduced the movies and a firmware upgrade that got a lot more performance out of existing hardware. I argue that similar things will happen in the transition to High-Def video for the Apple TV.

In answer to the above questions, a lot of theories were thrown around in the buildup to the much-rumoured and much-anticipated introduction of movies to the iTunes Store. Theories included having to deliver video at two different sizes and/or bitrates at once, or supporting preferences favouring mobile or high quality. All of them added unnecessary complexity to the simplicity of the iTunes store experience, and felt like a “kludge.”

Apple’s solution was simple: all videos would be delivered at a high-quality 640×480 maximum resolution, encoded with the H.264 video codec. The firmware on the fifth generation iPod was updated, and impressively so (from 900 macroblocks in MPEG-4 to 1200 with H.264 encoding), so no dual delivery formats were needed. Simplicity is maintained.

With the introduction of the Apple TV, and the accompanying hand-wringing about the relatively poor quality of 640×360 pixel movies shown on HDTVs, attention is turning to how Apple plans to deliver High Definition content to the Apple TV. I don’t think the current iPods can be made to understand a high definition stream. While some may imagine a multiple format delivery scenario as predicted before the introduction of movies, I think Apple has another potential trick up its sleeve.

The technology is an extension to H.264, Scalable Video Coding (SVC), and it was scheduled to reach the Final Draft Amendment stage last week. Normally1, that stage means that the standard is all but finalised; nothing but editorial changes are allowed. An aggressive company often can feel confident in releasing a product based on it if, say, they controlled the whole ecosystem around the product….

The idea around SVC is that you have a normal H.264 base layer, and you can add enhancement layers on top of it, whether they be spatial, temporal, or SNR enhancements. That means that you can improve the resolution, frame rate, and/or picture quality by adding additional streams to the base layer. Consider that the 640×360 standard 16:9 videos have half the pixels in each dimension from 720p videos with 1280×720. This is perhaps the most basic application of spatial scalability, but it’s also very relevant. I find it very easy to imagine an additional “track” in QuickTime that fills in three additional pixels out of every four. If a computer (say, my G4 PowerBook) is incapable of handling the full high-def combined stream, it can conveniently fall back to the base layer.

So, in the context of the iTunes ecosystem, these multiple streams and the increased file size need not be delivered to the iPod: with iTunes as the designated hub, the enhancement stream could quite feasibly be stripped when syncing with the iPod. One download, both standard and high definition are there for the appropriate device.

The only thing I’m not sure of is how the audio scenario is going to play out. Encoding and delivering the AC3 (Dolby Digital) stream (as an alternative soundtrack) that most home theatres expect seems very inefficient. AC3 and AAC are kissing cousins, so one could hypothesize a bitstream-to-bitstream transcoding from 5.1 AAC to 5.1 Dolby digital to pump through the optical out, but it also seems unnecessarily computationally expensive.

So the how of High Def seems to have a fair shape. It’s the “when” that isn’t yet clear. In the punditry vacuum where product development is instant and QA is not a factor, it’s feasible to push out updates right away. Strategically, it seems like “as soon as possible” is the answer, as well: a lot of people feel that the HD-DVD/Blu-Ray format wars aren’t close to a resolution, and there’s a window of opportunity for HD video downloads to become the preferred delivery medium.

On the other hand, Apple will want to do it right the first time. It’s a radical enough change to necessitate a new version (perhaps version 8.0) of QuickTime, and a new version of iTunes as well. Would it accompany an announcement of movie downloads in Europe? Would it accompany the “true” widescreen Video iPod? Would it wait for the end-of-year Christmas cycle, when the Apple TV suddenly becomes the must-have accessory that is even more important than the iPod?

I don’t know, but I’m keen to find out. I hope for “soon,” but I suspect it’ll closer to the end of the year.

[disclaimer: This blog entry is merely informed speculation. Although I have some old ties to MPEG, none of this is gleaned from privileged information. The only research I did of this was on public sites. If you follow this blog and past predictions I have made about Apple’s products, you will already know not to lend me too much credence.]

[1] My experience with MPEG standards can attest to the fact that this isn’t always true: I’ll only say that things got complicated in the final stage of standardisation of the MPEG-7 Systems standard, and I personally pissed off a lot of people during one late Friday night’s plenary.