Paul Irish

Making the www great

A Browser Benchmark That Has Your Back: RoboHornet

The past few years’ browser focus on speed has been great for us and our users. We’ve seen a huge and dramatic performance boost on benchmarks like Sunspider, Kraken, and Octane. But, these benchmarks, often crafted by JavaScript VM engineers, test raw JavaScript performance, which is rarely the bottleneck we have in our apps.

These days, our performance bottlenecks are oftentimes DOM, <canvas> API methods, SVG. Those are our priorities. So then why do we see all browser vendors competing on raw JS when we are hacking our way around to avoid the actual performance pain points? So we wanted to solve that with RoboHornet.

Today, a cross-vendor collection of developers, led by Google, is open-sourcing RoboHornet. As project lead Alex Komoroske put it, “it’s a living, dynamic benchmark that aims to use the collective efforts of the web development community and ultimately get browser vendors to fix real-world performance pain points.” Let’s look at how RoboHornet will hopefully flip the above situation on its head.

  1. Web app developers identify and isolate a specific pain point in their app
  2. That issue is reduced down to a solid benchmark
  3. Propose it to a committee of JavaScript experts from JSPerf, YUI, Google, Facebook, and others
  4. Everyone votes for which issues should make it
  5. Of the top voted items, as long as the committee thinks they are good, they head into the benchmark
  6. The RoboHornet Suite now better represents the web app developer communities biggest priorities for performance
  7. Browser vendors compete on their RoboHornet score, optimizing the slow behaviors that developers prioritize
  8. Everybody wins. Yay ice cream for everyone!

What’s in there now

The current items that RoboHornet tests captures the major pain points of jQuery, Google Apps, Google Maps, Ember, Handlebars and Cappuccino:

  • Adding rows to an existing table
  • Adding columns to an existing table
  • Descendant selectors at different DOM depths
  • Converting a 2D canvas to a data URI
  • Clearing a 2D canvas
  • Table render speed after innerHTML.
  • Scrolling speed using scrollTop
  • Resizing columns in a table
  • Resizing SVGs
  • ES5 getter/setters
  • Referencing the arguments array
  • Scrolling lots of animated GIFs
  • The affect of forcing a reflow by calling offsetHeight
  • Replacing a number of DOM nodes using the Range API
  • localStorage write performance
  • localStorage read performance

You can vote on and review incoming tests in the issue tracker.

RoboHornet represents you

The project is still in alpha and we’d love to get your involvement to better represent everyone. We’re using Benchmark.js internally, as its experience on jsPerf has proven it a trustworthy and reliable benchmark harness.

  1. Please take a look at the site
  2. Watch the RoboHornet Github project
  3. Read how you can be more involved
  4. Submit the performance pain points that your apps run up against.

While you’re at it, go check out the full review of RoboHornet at Tom’s Hardware

Academic Research on Browsers, Frontend Development, and Debugging

University-level education has never really touched too directly on what we do as frontend developers. I know I was self-taught in the ways of the browser, as were all of my friends. Luckily the sort of webapp work we do these days is quite sophisticated so those CS graduates bring much of the great ideas to the JavaScript community. (Alex Sexton has been a personal inspiration here)

Early last year, something clicked in my head when I saw this video: Crossing the Chasm: Pitching Security Research to Mainstream Browser Vendors.

You mean.. there are academics… who are doing research on browsers.. in universities? My my! You mean bright minds think about the same things as us and don’t publish them on Octopress? ;)

Turns out it’s quite an active area of research. So, I’ve put together a list of academic papers, talks, and researchers all centered around frontend development and browsers (Please chime in on the comments if you know of more):

Webapps

Gibraltar: Exposing Hardware Devices to Web Pages Using AJAX UC San Diego, Microsoft Research, Univ. of Singapore, 2012

Modeling and Reasoning about DOM Events Brown University, 2012

Don’t Repeat Yourself: Automatically Synthesizing Client-side Validation Code for Web Applications University of Illinois at Chicago, 2012

Performance

Who Killed My Battery: Analyzing Mobile Browser Energy Consumption Stanford, 2012

How far can client-only solutions go for mobile browser speed? ACM Digital Library, 2012

Race Detection for Web Applications Sofia University & ETH Zurich & IBM T.J. Watson Research Center, 2012

Leo Meyerovich - Berkeley Rsearch UC Berkeley, 2009 (Parallel Browser Execution, Security)

James Mickens - Microsoft Research Microsoft, 2004- (Performance, localStorage, Mashups, Speculative Execution

james mickens is amazing
James Mickens, the Galactic Viceroy of Research Magnificence at Microsoft Research, does awesome work. The above shot is from his MIX11 talk Making Better Web Apps For Today’s Browsers.

Debugging & Tools

Timelapse - Interactive record/reply with Webkit inspector Univ. of Washington, 2012

Cleanroom: Edit-Time Error Detection with the Uniqueness Heuristic Univ. of Washington, 2010

Firebug Research - Academic papers related to Firebug Various, 2012

Realtime Performance Visualizations using Node.js How to NODE, 2011

How Programmers Debug, Revisited: An Information Foraging Theory Perspective Wentworth, OSU, & IBM TJ Watson Research Center, 2010

Security

Crossing the Chasm: Pitching Security Research to Mainstream Browser Vendors Carnegie Mellon, 2011

Benjamin Livshits - Microsoft Research Microsoft, 2004-2012 (Malware, Privacy, JS Benchmarking, JS Compression, JS Static Analysis, Security)

Dynamics of JavaScript Purdue, 2011

Adam Barth - Research Various, 2007- (XSS, CRSF, Security…)

Data Visualization

Highlights of the Infovis Conference, VisWeek 2011 2011

JavaScript

Gregor Richards - JS Research at Purdue Purdue, 2009-2011

Cleanroom University of British Columbia, 2011-2012

Anders Møller - Research Aarhus University, 2002-

Non-JavaScript Tooling

Whyline for Java Natural Programming, 2009

Stacks Plorer Media Computing Group, 2012

Omniscient debugging with TOD Pleiad, 2007

The Elm Programming Language Elm Lang, 2011-2012 (a type-safe, functional reactive language that compiles to HTML, CSS, and JavaScript)

Developers We Admire.

One of the good things about having the eyeballs of many people is that you now have the ability to throw other people into the limelight that completely deserve to be in it. Divya Manian and I have been, for a while now, putting together a list of people in the front-end development community who, in our opinion, could use more broad acknowledgment and attention. They’re also inspiring because…

  1. They are excellently skilled in their area of expertise and go out of their way to teach others how to gain these skills
  2. They participate in open source projects that fuel the community’s knowledge and productivity
  3. They have independent opinions that are derived out of testing, and evidence-based fact finding worthy of learning from

So, here is a list of people we admire:

  • Ryan Seddon: @ryanseddon aka the CSS Ninja develops Modernizr, writes polyfills, and explores the edge of DOM APIs. His recent exploration of source maps lays a lot of groundwork for future web debugging potentials.

  • Hans Christian Renl: @drublic started contributing extensively to HTML5 Boilerplate and has been actively engaged in other projects we have worked on. He also maintains a blog where he shares his learning experiences and explorations of advanced CSS features.

  • Alexis Deveria: @fyrd maintains caniuse.com, and contributes to svg-edit, the open source svg editor. He also created & maintains the canonical timeline of web browsers which has been the basis of other browser history graphs since. He also upstreams all the data from caniuse.com to its API, is used to power sites such as api.html5please.com and html5rocks.com

  • Jason Kiss: @jkiss’s notes on accessibility are like a breath of fresh air into the dense field of accessibility. He tests in as many accessibility technology tools as available and bases his opinions on findings from his research. His articles are extremely valuable to web developers who have long had to base how to do the right thing on superstition and fact-free opinions.

  • Kit Cambridge: @kitcambridge develops some of the best-in-class javascript libraries and assists many open source projects on Github. Take a look around bestiejs to see his fine work and friendly demeanor.

  • Eric Dahlström: @erikdahlstrom is not just the editor of SVG specs, but also helps out often in mailing lists and Stack Overflow with SVG questions. Not only that, he is an unbelievably skilled (obviously!) at hand-coding lightweight SVG, and maintains a repository of SVG images that are worthy of learning from on how to do SVG right.

  • Masataka Yakura: @myakura watches HTML/etc. specification changes, browser bug trackers and mailing lists, and reports the best stuff on his G+

  • Matt Seeley: @innerhtml is a JS hacker at Netflix, who measures constantly and talks about the science of cross-device hardware accelleration

  • Tim Branyen: @tbranyen created Backbone Boilerplate and Backbone Layout Manager to make it easy for everyone to get started with Backbone. He also writes extensively on his blog on dealing with problems of creating web apps.

  • Mathieu Henri: @p01 has an in-depth understanding of JavaScript and shares amazing experiments that are possible within less than 1K of JavaScript such as canvas with audio (warning audio) & Speech Synthesizer. He also frequently contributes to @140bytes on what is possible with 140 bytes of JS.

  • Roman Cortes: @romancortes is one of few who can explore some interesting innovative uses of CSS. His explanation of a 1kb chrismas tree is a whirlwind of frontend knowledge.

  • Peter Beverloo: @beverloo started off a series on his blog documenting what’s new in WebKit/Chromium. He’s since been hired by Google and works on Chrome for Android. His work in distributing edge browser development news has helped thousands of people track the quickly expanding web platform.

  • Erik J Moller: @erikjmoller is to WebGL what @dahlstrom is to SVG. His skill and ability to hand-code webGL is unparalleled. He created a 10-part video on learning webGL (webGL 101) that is archived on youtube not to mention is the main contributor to EmberWind an open source webGL game that is highly performant.

  • Irene Ros: @ireneros dabbles in info-viz tools and has published several of them on GitHub. She is active in contributing back to open-source projects and also maintaining a few of them including Miso Project and underscore.nest.

  • Mike Taylor: @miketaylr is developer relations extraordinare for Opera. He has been a behind-the-scenes force getting a lot of cross-browser compatibility and feature detection complete and distributed.

  • Zoe Mickley Gillenwater: @zomigi wrote an excellent book about CSS3 and publishes informative articles on CSS on her website that are a good resource for anyone who is working with CSS3.

  • Niklas von Hertzen: @Niklasvh created html2canvas, essentially a rendering engine in JavaScript. He’s also had a number of exciting experiements like a Sudoku Solver in CSS.

  • Zoltan Hawryluk: @zoltandulac painstakingly researches making CSS3 features backwards-compatible with IE, in addition to blogging about several discoveries in the process.

Other Unsung Heroes

There are clearly far more people that deserve to have their praises sung. Plenty of people whose names you already know, but also folks like @ZachLeat, @Alex_Gibson, @mbostock, @RobFlaherty, @Raynos2, @bricss and more.

Thank you

We are grateful towards everyone who has helped us along the way to shape the web to be as friendly as possible to web developers, by driving the work on frameworks, libraries, articles and outreach. They do it on their own time, because of their love of and fascination with the web.

All of them are well worth your time to pay attention to and follow and hear their opinions of the web.

Who are yours?

We’ve left out some of the bigger names in this list, so hopefully we can all discover someone new who covers a relevant field extremely well. This is our list, but feel free to leave a comment with anyone you’d put on yours. Or better yet, blog it yourself and link up your list here.

2012.08.27: Heartwarming: @zoltandulac’s writeup of the inspirational frontend developer Walter Zorn 2012.08.29: Gray Ghost Visuals wrote about his heroes