Weblogs: Javascript

Full Frontal 2010: JavaScript, HTML5 and Mobile Nirvana

Sunday, November 14, 2010

Remy Sharp's Full Frontal - a JavaScript-focused conference - took place on the 12th November 2010 at the splendid Duke of York down in Brighton. The idea is elegant; Remy invites speakers who he wants to see, and everyone can purchase a ticket to join him. That in itself creates a wonderful and intimate atmosphere that's enhanced by the choice of venue.

This year's theme seems to be what's in store for us in the future both in terms of HTML5 and browsers, so the talks tended to be aspirational, with unstable and crashy demos of functionality that may appear in the browser, and as with all technology-based conferences the product pitch or three.

Alex Russell

Alex Russell (developer of Dojo, now working on ChromeFrame for Google) continued his tirade against Internet Explorer with his talk "Less and More: How HTML5 will change the framework landscape". This talk centered around Microsoft's inability to roll out new features in less than 12 months. And presented ChromeFrame as a mechanism to bring new and useful features into the current range of Internet Explorer browsers.

Alex has spent quite a lot of time thinking about the future capabilities of browsers, and how to create a path to that end. The web is a slow moving environment, and we wish we had a chronometer to figure out how far we have travelled. Moving forward is still hard. To make this effort easier we need to tame the bad things about browsers, stablise new features as DOM implementations, and have a better syntax (one that closer matches what you want the code to do).

In building new features and functions, we need to consider what an idiomatic API would look like, and implement that. Today we see different implementations, and it takes an independent standards body to try to conclude a single common API. So developers banded together to build libraries, abstracting the differences so the developer has to deal with only one API.

Very large companies have to deal with very large problems, and they need to be fully invested in every line of code to be able to succeed. Yahoo's control of YUI means that they can aim it to handle traffic at scale, for example. These are problems that don't affect the majority of developers, but we benefit from them.

Alex takes a look at the CNN homepage and the BBC News homepage and identifies a collection of problems, like containing lots of JavaScript files, mostly not gzipped, 400Kb of JavaScript. That's slow to load and run. And that's just for a news site. Alex asks what can be faster than a news site, It's called news for a reason.

He recognises that the BBC's homepage is high-quality work, the markup is semantic, accessible and crawlable. But they run into problems. The majority of the JavaScript does nothing to the core purpose of the page, it's used to power galleries, and carousels and a ticker animation. Perhaps deferring the loading of JavaScript till the second stage of loading (I interpreted this to mean after the body.load event has fired), and use progressive enhancement to make the page seem faster.

Google hand tunes it's HTML, they have a team of people sitting there watching every single freaking byte. It's expensive. There are compounding factors that make this situation worse. The JavaScript it at the end of the page. How many bytes should you need to render what's just an input box and a button? That's basically all the page does.

Our pages are slow for a number of reasons:

Think of network connections as 10 times slower than disk when requesting assets.

By putting the JavaScript at the end of the DOM we reduce the effect that this has on performance, but the interactive pieces of the page doesn't work until the JavaScript runs. Is that a good user experience?

Alex asks what a blank web page should weigh, a blank page being the starting point for building your website. At the moment a blank page would contain a reference to a JavaScript library (so 25Kb of code), and reset styles. Why does it cost us 30Kb to start a new page or project? We may already be stuck at this point.

JavaScript applications try to subclass HTML and we try specialising behaviour on pages by adding class hooks into the markup. We need to upgrade existing semantics, because our current semantics are already compromised.

Alex mentions Docbook as an aside. It's a much richer syntax, generating much more semantic documents. Statistically no one uses it; because it's XML and there is a tighter finer-grained agreement of the meaning of it's syntax. So it's great for specialised areas like technical documentation, but not good for how the web uses HTML documents today.

The select box in HTML provides a widget for a drop-down menu, bu there's no mapping of UI to it's data. Is that a missing semantic? Is the select box a View or a Model? Alex's point is that HTML doesn't have a large vocabularly, so we use JavaScript to decorate the closest fitting elements. What is the HTML semantics for a bar-chart?

Looking at GMail, the application is indistinguishable from Flash code. It's highly compressed and optimised for display, but not view-sourceable. Markup has stopped evolving, and we're wrapping markup in JavaScript to fill in the missing gaps.

What does it cost to wrap markup with JavaScript? We have to augment the DOM element in a parallel tree, since the core platform isn't good enough to describe what we mean. We're creating towers of Babel, building tall towers based on assumptions that are not a core part of the platform. Tall piles of incompatible junk. How do we alleviate this? We try to force agreements within standards body, good luck with that Alex says.

So what should we be asking for?

We need this: <input type="range">

Alex notes: This is freaking great. Saves loads of code and mess of JavaScript: someone will want it vertical, then make it accessible, then find some way to hook it up to the DOM. The code isn't expressing what you want to say. There's no agreement with tight control over the DOM. This kills us every time. We should be asking the browser to do this..

What we don't want is this:


	navigator.geolocation.getCurrentPosition(
		successCallback,
		errorCallback,
		{ options: ... }
	)

Browsers load plain HTML pages incredibly quick. So we need browsers to implement these features for us at that level first (DOM second, and JavaScript third). A declarative syntax (i.e. HTML) is good, it allows us to crank on, browsers can make it faster without breaking it over time. Once you have DOM your have an API. Declarative begets DOM, DOM begets an API.

The big problem is how to get these features faster. Google Chrome can implement a new feature and roll it out to Chrome users within 6 weeks. The Internet Explorer cycle is years. We continue to use JavaScript as an escape hatch to these problems (as we've already seen on the CNN and BBC News websites), it is really a smart idea?

Alex is preoccupied with how to shrink the gap from introducing a new feature in good browsers and when we can count on it (i.e. working in Internet Explorer). The first approach is to create defensible spaces.

The Mobile web is a fine example of a defensible space. Mobile phone rollover is 12 months in Europe, and much quicker in the US. This beats the pants off the desktop browser. That means, in the mobile web be can start building for the technology edge, and start to shred the load we have to deal with. The Chrome App Store - for Chrome users - benefits developers and users, and benefitting from the 6 week turnaround time for implementing new features.

The baseline loading of HTML is incredibly fast. We must make it a richer and more enjoyable experience for developers. That's why ChromeFrame exists. ChromeFrame is a plugin for Internet Explorer and gives developers a chance to say no to older version of IE. All you need to invoke ChromeFrame is the HTTP header: X-UA-Compatible: Chrome=1 (or as a meta element in an HTML page).

Browsers are exclusive choices, but plugins are not exclusive choices. So it's more likely for Internet Explorer users to use ChromeFrame instead of changing their choice of browsers (assuming there is the ability to change, or there is a choice).

It's the apps that we build that will matter the most. This will help speed up the technology edge. We need to lean on this harder, and speed up adoption of useful rich features.

An interesting talk from Alex Russell, slightly tinged by the product placement of ChromeFrame, but at least we know that bias up-front. This doesn't dilute the idea, and getting more rich functionality as part of the browser is an important step forward for the Web.

Jan Lehnardt (@janl)

In his talk "About the Peer to Peer Web and why you should care", Jan Lehnardt talked about mobile as the prime example of where peer-to-peer fits elegantly. He talked about fragmentation amongst phones, including PPK's point of the heavy fragmentation of webkit browsers to the extent that building for webkit is not a viable approach to tackling these incompatibilities.

Jan covers several pitfalls about the mobile platform, noticeably in Europe. This includes the extortionate fees on mobile data (Data fucking roaming, more expensive than blood, ffs).

Jan argues the case that the mobile phone is an adequate device for peer-to-peer networking, and not having to rely on a server for everything. One smart comment in particular: you cannot add more computers to reduce latency in a network (because the latency is in connecting across a network).

Then Jan switched gears and talked about CouchDB, it's Lotus Note's parentage (typically, I'm the only person in the audience to raise my hand in agreement that Lotus Notes is great). As well as having a native HTTP interface, and talks JSON, CouchDB has one other remarkably killer feature: Replication. CouchDB is basically the Lotus Notes backend implemented in a Web and Cloud friendly manner. The frontend can be CouchDB itself (with CouchApps), or any piece of code that can do HTTP and JSON.

(I'm realising as I write this, replication is a form of peer-to-peer! So perhaps switching gears isn't the right description.)

Consider CouchDB running on your mobile phone. To me, that's an awesome idea. A powerful data store running on a mobile, synchronising and replicating to CouchDB servers in the cloud (or potentially, other mobile devices). That's pretty smart foundations for mobile applications. Easy to communicate and share data with, and built-in rock-solid replication.

That idea alone. wow.

Paul Rouget (@paulrouget)

Paul Rouget is a developer evangelist for Mozilla, and spent his time talking about HTML5 implementations and improvements in Firefox 4, with an emphasis on what will be possible in potentially future Firefox versions. Using a custom hacked Firefox 4 build (running on Ubuntu - no Flash!) he demoed several graphical and audio effects. His talk was titled "Batshit crazy stuff you'll be able to do in browsers"

As with all UK conference, the venue wifi couldn't hold the load of traffic, so most of the connected demonstrations didn't work. They did sound spectactular, including a collaboration space running in browsers, powered by Paul's node.js server. Sharing photos, resizing, altering between multiple browsers at the same time.

Starting with 2D Graphics, Paul demoed a very simple drag and drop interface for photos, with inbuilt resizing and composition with various other images and text, the FileAPI, cross-domain XHT, and seamless storage. The audience was floored when one of the examples included uploading an image from a webcam directly in the browser (another "only on Flash" feature now just in a browser - this was one of the patches Paul did to his Firefox 4 build).

Paul's 3D Graphics demo covered a WebGL powered 3D interface to data. Riffing the Blade-runner title-sequence intro, we flew over a city-scape rendered with 3GL, looking at perpetually updating flickr photos and youtube videos rendered on 3D surfaces, and tweets on rendered 3D LED displays. For a few minutes, we no longer saw a text-based browser, but something else, the typical Hollywood-designed graphical interface.

The audio demonstration was also impressive. Paul showed that the audio data could be exposed to the current document, and used to build graphical effects seen in most software audio players. So Paul demoed a dynamically generated (with firefox-themes) audio room - an intense graphical and audio experience much like the old Demo scene. Impressive that this is all contained within a browser, and required no plugins; and we can look forward to seeing these capabilities in the future.

The next demo looked like a video of a car driving around a city centre. Just a video then. But no. Using mouse drags, we can rotate through the scene as seen from the car as it's driving through. Amazing. Paul then disabled CSS, and we see a series of panarama images. They have literally been styled to look like a seamless video. This resulted in a spontaneous ovation from the audience.

The next demo was even more breathtaking. Paul took live vide action - a pair of hands moving two pieces of paper across a desk, picking it up, letting it float back to the table. Just video, surely. Then Paul dragged a few images onto this surface, and those images were mapped almost perfectly onto the moving paper. I couldn't tell whether this was CSS again, or the merging of video with a Canvas surface - like the audio above. But it was superb.

Paul then showed all the various Browser technologies, SVG, CSS3, plus the plethora of HTML5 technologies that Firefox (and possibly many of the current crop of modern browsers will be supporting), and then hid all the technologies that were not supported in IE9. Just over half of the components disappeared, which is particularly disappointing. Though, to be fair, it was hard to see which components went missing - I was thinking this was a document interface demo, rather than Paul making a particular point.

An awesome Demo, brings great memories of the Demo-scene back in the 286/386 days. But this time, all completely done in Firefox.

Lunch

We had a longer than expected lunch - Las Iguanas was superb, dining with friends. So we missed Dan Webb's talk about "Bringing the same-origin policy to it's knees".

Paul Bakaus (@pbakaus)

Bakaus' talk about "High performance browser games" covered various techniques he used to prototype an isometric game running in a browser. Though his slides were terrible; keywords were in dark blue on a black background, so we endured an entire talk trying to figure out what the hidden keywords were. We mostly failed.

Paul pointed out some interesting gotchas developing games in JavaScript, like Canvas being slow, and techniques to avoid those performance bottlenecks. One key technique is the use of CSS3 Transitions in moulding 2D textures or images onto a 3D surface. (I recall that a retro-gamer implemented Manic Miner in a browser just using JavaScript, Dom Elements and CSS, and lots and lots of timeout loops and events - so these techniques Paul discusses are an evolutionary step above these 2002-ish concepts)

Better yet, with Isometric 3D objects collision detection isn't straight-forward at the best of times. As with old techniques for rendering isometric projections, you sort by z-index, drawing the furtherest away object first, and the nearest object last. Figuring out if the mouse is pointed onto an object isn't just a case of checking whether the mouse is over a certain image, we need to check whether the pixel it's pointing at is an actual visible image, or nothing. Paul talked through his approach of using Canvas as a lookup table, he can check the coordinates of the pixel the mouse is pointing to, and ask the CanvasAPI whether that pixel is transparent. If it is, then he switches to the image underneath the current image and sees if that is transparent. The first occurance of the pixel at that position not being transparent means that there's a collision, and the mouse is pointing at an actual object. An elegant solution, again, a Demo-scene like solution.

Bakaus also noted that 2D graphics don't benefit from hardware acceleration, so his workaround is to map his 2D transformations to 3D (basically setting a z-index of 0), to gain hardware acceleration. That's clever.

Bakaus demoed the isometric projection environment he had build (the Aves Engine, I think) - much like Paul Rouget's demos earlier, mapping flick streams, video, and audio visualiser to 3D surfaces. Bakaus had created a Sims-like environment, with path mapping and animated sprites. Most of the sprites were also non-player characters, so shows that the browser is a sophisticated enough environment to keep a persistent and scripted game world.

Paul talked about using node.js as being advantageous for browser game developers, not just for having one language on both sides of the client-server divide, but also reduces the friction of data-exchange between the two (although, I don't buy the latter argument, JSON is well supported in most, if not all, web-facing components). Still, the usefulness of WebSockets and persistent connections, plus JSON is a perfect networked backend to a browser-based game.

Paul Bakaus currently works for Zynga - hence the Sims like quality of his demo, I suppose. And they are hiring for JavaScript developers who are interested in writing games. I'm glad Paul talked about his own personal projects, since this topic could so easily have gone down a Zynga product/talent pitch route.

Brian LeRoux (@brianleroux)

Brian LeRoux covered his product PhoneGap, which creates an interface between mobile applications and the various data and features on phones - like the Address book, the gyroscope. It was a product pitch, with some guidance about how to get started, and the various platforms PhoneGap runs on, and currently working on supporting.

PhoneGap's mission is to eliminate itself by convincing mobile handset creators to provide API access to their phone's features and capabilities. He did have the Bear Dick Punch on his slides.

Seb Lee-Delisle (@seb_ly)

Seb Lee-Delisle is a rock-star in the Flash community, I've seen his presentations commented on in awe over the last few years (particularly during Flash on the Beach), and he is held in high-regard by many world-class Flash developers. So I was particularly surprised to see him speaking at a JavaScript conference, so interested in hearing his perspectives and takes about both the Flash and Web industries. Thankfully, he produced the best and most engaging talk at this conference, and the most practical demonstration of what we can do now.

Seb acknowledges that he is well-known in the Flash community, but that's not his only forte. With 20 years of development experience, his talents and skills are actually far wider than we pidgeon hole him. He sees himself as a creative developer, building with whatever technologies are available and interesting to him.

Seb took us through step by step (like the old 286/386 game programming tutorials of yore), starting with displaying a particle, and then moving it in various directions. (Live coding, with some issues, but nevertheless made his point very aptly). Then he introduces drag, gravity and momentum in very simple ways, but good enough for a quick visual effect. So within minutes we have some fireworks resulting from a sparkler. Then he switches to using an image instead of a 2x2 pixel square, which increases the visual aspects of animated particles. The same particle animations allows Seb to demo a bouncing ball, using simple collision detection and reversing momentum on each collision.

Seb renders arrays of particles, before each frame he clears down the Canvas and re-renders from scratch (I can just see double buffering with Canvas coming along soon!). This works quite nicely. He then decides not to blank out the canvas between each frame, but paint a slightly opaque layer between each frame. Now each particle leaves a trail that gradually fades out.

Constraining the scope of movement and slowing it down, and using the image of a blurred/softened crescent Seb creates an effective visual interpretation of a column of smoke. Again, the same basic particle animation engine, with some very basic physics rules.

Seb switches to 3D, explaining concepts along the way; the 3D co-ordinate system, mapping that to a 2D surface. Then applying image zoom to resize the image based on it's z-index. With help of Father Ted and some cows, Seb renders the points in a 3D perspective, switches them to cows (small, or far away, I'm not sure). 8 cows positioned at various places in a 3D space rendered on a 2D canvas. The cows are joined by lines, and the cows disappear, leaving a 3D cube. A round of applause by the audience.

So Seb switches back to simple particles and creates an array of particles with 3D coordinates. Looks like a scattergram, until Seb moves his mouse and the whole starfield rotates in 3D. Another round of applause. Seb plays with this effect for a little more, replacing particles with softened star bursts. Now the 3D rotations look marvellous. Seb explains some of the tricks and shortcuts, like alleviating dept-sorting by rendering the starbursts in an additive fashion (superimposed, it makes no difference which ones are rendered first).

Seb then dives into Unity 3D and shows how sophisticated an enviroment it can handle. Sound based on texture, mostly realistic grass weaving, footstep surface mapping, and realistic uneven-terrain mapping. Seb leads us through to demonstrate water effects, and direction specific sound. It all looks very similar to the standard 3D rendering engines driving the first-person type games on the typical gaming console.

Seb demonstrates the collision detection, setting up a landscape populated with barrels (in a triangle formation), various attemps of using 8 barrels up to 100 demonstrate that each barrel reacts to the physics of the game world, even when they have knock-on and cascading effects with each other. We watch various sized triangles of barrels tumble realistically and bounce approrpriately while we throw barrels at this set-up. (As well as grinding Seb's Mac to a halt rendering 100 barrels at real time, just because we can).

To be honest, this was the best presentation of the day (I'm sure the applause received at the end was much louder than any other speaker, only the applause for Julie and Remy at the close was louder). Seb drew us a roadmap of how to get where we are today, to the future everyone else drew a picture of before him. I found the explanation of particle physics useful and enjoyable, particularly taking us through the well-known tricks of transforming co-ordinates in a 3D space into a 2D representation. What particularly impressed me was how Seb could create such wonderful effects very simply and very elegantly. These are not perfect physics engines, but they are good enough for us right now to build some interactive worlds and games.

Seb left us with a warning, the Flash world is mired in effects for the sake of effects, sometimes wallowing in it's own ego. With HTML5 and it's various visual and audio features, lets not go down the same path.

Seb inspires. And I am inspired.

Closing

I like Remy's idea of how to organise a conference. I think it's one that can be replicated in various topics around the World Wide Web with success (given that the organiser has the passion and skills on the topic that Remy has in abundance). The venue compliments Remy's approach; it's an intimate and friendly venue, very little marketing, all passion and knowledge. I'm spoilt because I spent the conference on the first floor, all decked out with sofas, so it was very comfortable for me, and that made the conference so much more enjoyable and relaxing.

How could we replicate this approach?

The choice of speakers was clearly inspired. No big conference-circuit names (famous because they are famous on the conference circuit), just people who work with the technology they talk about, and talking from real experience and knowledge. This is the perfect antidote to the @Media and the misnamed "Future of {tech trend}" series. Brighton seems to have a knack for smaller-scale conferences that actually deliver great amount of value for money (including the trek down from London).

Remy and Julie Sharp have pulled together a remarkable event. Thank you both.

Other coverage

[ Weblog | Categories and feeds | 2011 | 2010 | 2009 | 2008 | 2007 | 2006 | 2005 | 2004 | 2003 | 2002 ]