Like a lot of people, I'm pretty excited about HTML5, particularly for including audio and video in web pages. However, while it has great potential, there's still a lot of work to be done before it's easily usable by everyone. I'm not talking so much about bugs in the way browsers handle specific elements or the ongoing codec war, which have been covered in detail in many places, but more about the surrounding ecosystem. There are four issues in particular that I think need addressing.
Problem 1: UI (in)consistency between browsers
While it might be too much to ask for every browser to render an HTML5 video box exactly the same, a bit of consistency would be nice. Case in point: most Flash players, including Youtube, display a 'play' icon over a video before it starts playing. This is presumably so the user can tell that it's a video, not an image, which is particularly relevant when the video controls are hidden. However, most browsers don't do the same with HTML5 video. The key word there is "most". If all browsers didn't, it would simply be a case of providing a poster image that included a play button. However, since some do and some don't, separate posters have to be served to different browsers based on user agent sniffing or the like.
Problem 2: Browser playback performance
My old G4 Mac Mini can play Ogg Theora videos without any problem, even when scaling up to 1280x720. Play the same video in Firefox or Opera from a website and it slows to a crawl. The browser drops frames, the UI buttons don't respond properly, etc. This is when the video is playing at its native resolution inside the web page. Change it to full screen and the problem gets even worse. Now, I'm sure that there's a technical reason why the performance is so bad, but it really needs to be improved. After all, one of the reasons people give for using HTML5 is that Flash's performance is terrible. While that's true, it's still better than Firefox's Ogg Theora performance!
Problem 3: Android performance
The playback on Android (HTC Desire running Android 2.2) isn't too bad, but there is a major problem: it won't reliably stream a video over 3G, even with full signal. During my testing, I discovered that it would refuse to play an H.264 video over 3G, even when I was standing right next to a signal repeater. It just pops up a warning that the video is broken, which makes the user think that there's an error on the website. Switch to a Wi-Fi connection and it works fine. I don't know what the cause of this problem is, but it needs to be fixed. While I greatly prefer Android's openness to the iPhone, the same page works on the iPhone over 3G.
Problem 4: Lack of open source browser streaming support
By "streaming", I mean pseudo-streaming, which provides the ability to jump to any point in a video, even if it hasn't fully loaded yet. For a short video, this isn't a significant problem, but it becomes increasingly important for longer videos. Flash video (flv) streaming has been around for a long time and is supported by Lighttpd, Apache and (I think) nginx. Unfortunately there appears to be no straightforward way of streaming Ogg Theora or WebM on these servers. I've seen various hacks with PHP, but they don't seem well supported or particularly easy to set up. There is an H264 streaming module available for Apache, but it is not fully open source.
Future outlook
I expect that the browser issues will improve, although they will continue to be problematic for some time until users upgrade to fixed versions. Mobile browsers are likely to be the worst for this, as not all users will even be able to upgrade, leaving sites stuck either ignoring users running old browsers, or having to include various hacks for backwards compatibility. I don't see this as being a major problem going forward though.
The future of the server side is much trickier to predict. Given Google's release and promotion of WebM, it seems likely that Ogg Theora will fade away, so I don't expect streaming support for it to added to any web servers. True open source streaming support for H264 is a possibility, but licensing issues may put people off. The fact that H264 has been supported for years in Flash, and yet there is still no streaming module, is particularly telling. That leaves WebM. Google are already promoting WebM heavily to video creators and desktop users, but they could make it a much more attractive proposition to developers and server admins too by creating an open source streaming module for web servers. That certainly seems more likely than a truly open streaming module for H264 in the near future.
As an aside, currently two video formats are essential, and WebM isn't one of them: Ogg Theora and H264. All browsers that support HTML5 video can play at least one of these formats. Browsers that require fallback to Flash can play the H264 video in Goker Cebeci's f4player - this is the only fully open source Flash video player I know of that supports MPEG-4/H264 playback. Other open source players either support flv only (i.e. requiring a third video format), or have licence restrictions (e.g. non-commercial use only). Unfortunately the days when we'll be able to get by with only one video codec still seem to be some way off, but WebM seems the most likely candidate.
I'd like to finish with a little plug: if you need some help adding media to your website, drop us a line. I am certain we will be able to simplify things for you!