Full Frontal 2011 - My NotesTuesday, November 29, 2011
Another November, and the third Full Frontal one-day conference took place at the Duke of York cinema in Brighton. Last year's conference was fairly interesting, if a little demo-heavy. This year's seems geared toward development tools and web applications. The headline speaker was ex-Yahoo Nicholas Zakas, but the supporting speakers were more impressive and fresh.
CoffeeScript Design Decisions by Jeremy Ashkenas
I covered Ashkenas on CoffeeScript Design Decisions in a separate blog post. It was that interesting and bloggable.
Respectable Code-Editing in the Browser by Marijn Haverbeke
Marijn Haverbeke talks about code editors in a browser; basically glorified text areas. Text areas themselves are too primitive for building a code editor on; indentation is unworkable, and good luck finding which line of your code you are currently on.
The code editor in a browser received a massive boost from Bespin, a code editor written entirely in canvas, and basically recreated every pixel of a UI. As a demo Bespin was interesting, but using canvas is inappropriate solution. Thankfully this abysmal mess of inaccessibility is now mostly abandoned, and the code editors that have arisen from this work are well on track to being more conducive to the web environment, and more perceptive of the DOM. One significant limitation of canvas is its portability.
So far there are only 3 serious implementations of in-browser code editors:
- ACE, which took the good parts of Bespin, and with new techniques it is much faster and more portable than Bespin.
- Orion, which is an Eclipse project, uses content editable / design mode as a basis.
- CodeMirror, which is Marjin's project. Currently in its second iteration.
All three are open source projects, and to date there isn't a single commercial implementation of a browser-based code editor.
Marijn is the sole developer behind CodeMirror. The first major version of CodeMirror used content-editable (or design mode); this is a browser supported mechanism of inline editing. But Marijn found the feature both underspecified and the browser support buggy or unsuitable for building a code editor. Issues ranged from Internet Explorer inserting paragraphs whenever it could, to the general unavailability of useful events.
After content-editable proved insufficient (code folding was impossible, for example), Marjin started version 2.0 of CodeMirror which was based pure DOM. He limited the size of the DOM by creating a viewport of the document, so only the visible part of the document needed to be rendered on page, thus speeding up the overall rendering of the document. Marjin calls this a fake editor, just lots of changing DOM nodes. The cursor is a lie, the text looks editable, with copy and paste and moving text working. Marjin crafted his own text selection algorithms, and that worked pretty well since CodeMirror has full control over the document model.
CodeMirror with its API approach allows undo/redo, code-folding, and an extensible mechanism for supporting multiple languages with both syntax highlighting and code completion. Building a language extension is about implementing one method, token(), which is in a SAX-like way called for each token within the edited document. There are hooks for managing state through the document. The typical features of a good editor such as search-in-place and text replacement end up being scripts on top of the CodeMirror API.
Internally, the document being edited is stored as a doubly-indexed B-tree as this allows the two way mapping of the actual line of code with the line currently being edited on screen. These two lines can be different because of code-folding, long lines being wrapped, and the region being displayed is offset from the top of the document.
The editor listens for scroll events, and avoids startup freezes as it renders the DOM nodes onces there's sufficient information available to render the current visible region. So scrolling and loading large documents doesn't slow down the interface.
Copy and pasting is a clever hack, CodeMirror detects a right-click and inserts a hidden textarea underneath the cursor. Then the context menu at this point offers copy and paste options, because they are native to text fields.
With demos of the above features and talking through how they were implemented Marjin delivered an interesting tech-heavy talk.
How We Architected Cloud9 IDE by Rik Arends
Rik Arends introduces Cloud9 IDE as the easiest way to work with node.js. Cloud9 IDE is an online IDE and runs on node.js.
We evangelise the Web, but we don't develop on it. Rik accepts the challenge that if many applications can be hosted on the web. why not a developer IDE. Surely web developers should be able to work using a cloud-based IDE. If the web can self-host its entire toolchain isn't this, he asks, the best way of pushing the web forward?
IDEs don't need to be ugly or clunky, so Cloud9 takes a designed approach to IDEs. One year ago it looked a little like Eclipse. Today it will look very familiar to TextMate users (and indeed it takes some useful cues from the Sublime Text Editor). They are even currently importing TextMate themes.
Cloud9 is betting the company on building a cloud-based IDE. It is currently funded by Accell and Atlassian. And, Cloud9 IDE is developed using the Cloud9 IDE. The IDE is open source, and downloadable from github to run on your own sever. (I caught up with Rik later on in the day and he confirmed it can be up and running on a VPS as long as node.js has at least 128Mb of RAM available. So a cleanly configured 256Mb VPS should be a useful starting point).
Rik demos the Cloud9 IDE; projects are brought in via github (after oAuthing with github). The main editing interface uses the ACE code-editing component. This renders a viewport of the current code as DOM Fragments (similar to CodeMirror2, I gather). This means rendering is lightning fast, and syntax highlighting is easy to extend for different languages and markup schemas. One recently added feature to the editor is a mini-map of the code, which is a good visual overview of the code; this is based on the findings that developers recognise code block by how the look, rather than the actual code.
Cloud9 IDE has a Console for running git commands. It's actually an emulation of a server shell, but very limited range of unix commands. One feature coming soon is a link to a virtual machine, and that will allow a real unix console right in the IDE.
The IDE has support for Heroku based deploys (as git commit hooks, I think). The console has code-complete (for example git commands and options).
CloudIDE takes advantage of a number of publicly available libraries and toolkits, including:
- APF - a high performance UI library
- ACE as the code editing component.
- Socket.io which offers realtime socket networking. Rik demoed inline chat and collaborative editing within the CloudIDE interface.
- Connect as the HTTP middleware (which Rik compares to a build-your-own customised Apache server).
Lessons learned using node.js
Rik lists some useful pointers in building applications on node.js:
- structured exceptions do not fit in node.js., it doesn't make sense in an evented system, since there's no clean way of surfacing errors back up to where they can be seen. And because of the asynchronicity of node, you don't want an exception to break the whole call chain.
- Beware of globals. Definitely "use strict" to catch these. Beware of modifying the prototype of objects since these changes persist; prototype changes are global, so this can cause unintended side-effects elsewhere in your code.
- minimise your callback nesting by grouping functions together into a more linear-like order. (I guess this also hints at considering using promises).
- for dense business logic and tests, CPS transforms makes sense, use a code analyser to refactor into a top-down looking code.
From a systems point of view:
- keep an eye on state, preferably have none because it persists across requests
- Monitor nodeJS: use a piece of software called forever which runs node apps in the background and keeps monitoring that it's still alive, restarting it when it fails.
- Log everything so you can reconstruct bugs. Use logging over UDP, so it's a fire-and-forget, for example
- Thick client + stateless server = win
- Put node behind a proxy— makes it very manageable. load balancing, server upgrades, start processes.
- Do not blindly trust a library or module — nodejs is pretty new, so there are many examples of "my first node js module" that then never get updated with fixes and improvements. Don't be afraid to get your hands dirty and building your own libraries and modules, when necessary.
Rik's excellent talk covers useful ground in building and architecting node.js applications. It's also a great example of what is considered an impossible to build application implemented in a highly usable, flexible and powerful way. I guess this is the first serious node.js app I've seen under the covers of, and a great demonstration of what's possible with node.js
This repetition may make sense if this was a last-minute stand-in replacement for another speaker, or one to make up the numbers speaker-wise. But not for a top-billed speaker flown in from the USA, that we as attendees were paying for.
Beyond the Page by Glenn Jones
One page web applications are appearing all over the place, and as Glenn accurately notes they are siloes, but they don't need to be. Glenn shows a couple of demos he has been working on that allow data from one web application to be communicated to another.
His central application is a people store, storing vcards (or the microformat equivalent hcards). He shows how drag and drop between two separate browsers can be used to copy across the vcard information purely in the browser, meaning that this transfer can be done on any web-based application hosted anywhere.
Glenn shows further capabilities, of dragging and dropping vcards from the file system to the browser, and saving vcards from the browser back to the file system.
What impressed me was that the data wasn't just text, or an image, but a set of structured data plus an image. The drag and drop looks very useful, but Glenn notes that the HTML5 specification for dragdrop is a disaster - concurring with PPK's viewpoint. Currently advances in drag-drop seem to be driven by GMail.
Another clever demo was copying in a chunk of HTML (using the Clipboard). Glenn shows that under the covers he is using content-editable / design mode to receive the HTML, and then run a standard parser over that to extract out the structured information. Certainly a sensible use of microformats parser.
As an additional extra Glenn is using the Google Social API which, given an email, returns a list of social network or XFN type information about the person represented by that email address.
The last part of Glenn's talk covered Web Intents. These are used to map common actions with services, such as posting a status, bookmarking a link, editing an image, picking a profile. Glenn shows the two main ways Web Intents improves the flow of information:
Many blogposts have social bookmarking links, instead Glenn shows that replacing these with a "Bookmark this link" and identifying it as such to the browser, that browser can then assign that link to the service that the user uses. So clicking on that link opens the Delicious bookmarking dialogue for one user, and a Pinboard dialogue for another user.
The other direction is for filling in forms - like street addresses. He demos a webintent that maps a "Fill in this address using a profile" link and maps it to his own people store. That way he can select the right profile in his contacts system appropriate for the form such as his personal profile address, or his business profile for his office address.
A very informative talk about the features that are on the way to being ready for use in web sites, features that improve the user experience, and thus help reduce the risk of customers disengaging with sites.
Beyond the Planet of the Geeks by Brendan Dawes
Brendan Dawes is a designer and a geek, and he delights in design of items that escape us - such as Japanese and Brazillian paperclips (and pencils). And he lets his creativity loose. When a high-profile architecture company hires Brendan's company to design a new experience for their portfolio of buildings, they see in Brendan the very same boundless creativity, amazing things happen.
Brendan's narrative draws you in, he shows that sometimes it's good to build things that are not user-friendly because in that way you get experiences that are memorable, enriching and deeply rewarding, as well as expressing a brand identity. One side-effect of having a rich exploratory interface is that it opens the visitor up to serendipitous discovery, one of the most wonderful experiences for curious people.
Brendan's talk is about visualisation of data into an engaging experience. He talks about the ideas that worked, how the idea evolved, how the customer asked to make the logo smaller, about what it's like creating with no limits in place.
Yet he also shows that the idea is still grounded in technology, it's built in HTML5 because it's something Brendan doesn't know (Brendan is a firm believer in moving on to different technologies when you have mastered the current one).
An utterly enjoyable, and inspiring talk, and a nice counter-balance with the other technical-focused talks.
Google Doodles by Marcin Wichary
Marcin works for Google, and his 20% project is helping the Google Doodles team build their next Doodle. He's been involved in several of the new interactive doodles that have appeared on the Google homepage over the last year. From the faithful implementation of PacMan, to the ??? machine, to the magnifying glass doodle, and back to the Jules Verne underwater exploration doodle, and the ??? animation.
After a quick summary of the history of the Google Doodle (from the first Burning Man doodle, right to when Marcin starts to get involved), Marcin dives into the interesting stories behind the interactive ones, covering the usability tests and technical challenges of the doodle. We are shown several versions of the Jules Verne doodle and see the changes made and how they considered IE support.
Marcin talks about how each of the doodles evolved to their end-result, the user-experience factors, the technical limitations and reinventing new animation techniques. Marcin has a healthy interest in retrogame programming techniques, and it's eye-opening some of those techniques have been adapted in these famous doodles. The Martha Graham dancing woman doodle is a magnificent example of his Crushinator technique, which reminded me of the bit-blitting techniques of the Amiga.
Animation tricks like a thick lens holder to hide the coarseness of the rectangular clip-regions, thus saving performance and still producing a jaw-dropping result. Similar to the Dark Sceptre approach to masking large animated characters.
Marcin shows some of the easter eggs present in the Doodles, like the Jules Verne one controllable by the accelerometer built into the Macbook Pros. Many people first learnt about that particular hardware feature because of the Jules Verne doodle. Marcin notes some of the bugs they uncovered, for example the same generation of Macbook pro having different accelerometer chips which resulted in the movements being completely reversed.
A fascinating exploration of animation, interactivity and rich user-experience on the world's most trafficked page. And shows how the old animation techniques are repurposed in the ever-evolving web.
Full Frontal delivers in a way no other conferences do, it's a small audience in a homey venue. Free from marketing and hype, and delivers great value, inspiration, and a focus on the web developer. It is my favourite event.