Luke Stevens on Data Driven Design

Luke Stevens is a freelance, Sydney-based web designer with clients in the US, UK and Australia. Luke is speaking at the Web Directions South conference next month on the topic of data driven design. SitePoint’s Matthew Magain cornered him for a chat about this very topic.

First up, could you tell the SitePoint readers a bit about yourself?

Sure. I’m a garden-variety, freelance web designer, no doubt similar to many SitePoint readers. I’ve been freelance for about three-and-a-half years and worked in-house for over five years before that. I have mostly designed and built content-heavy ExpressionEngine sites. All the good things you’ve heard about ExpressionEngine are true – it’s fantastic.

Lately, however, I’ve been working on ideas that I hope will change the way other designers and web professionals think about design on the Web; that’s the data-driven stuff I’ll be talking about at Web Directions South in October. Web analytics, A/B testing – that kind of stuff, from a designer’s perspective.

I’m also working on a book on the same topic, called Performance Based Design. It’s about eighty percent written, and I’m hoping to release it in the next few months.

You’re speaking at Web Directions South this year on this topic, or as it’s also known – data driven design. What do you mean by “performance based design”? Are you talking about navigating Photoshop whilst loaded up on energy drinks and caffeine pills?

Not quite! “Data-driven” sounds extremely boring. You know, maths and statistics and spreadsheets and zzzz …

Really, however, it’s about performance. That’s much sexier. Athletes perform. They also follow data: if they train in a certain way, or work on a different technique, does it improve their times? Can they run faster or throw further?

They don’t just haphazardly do stuff if they want to become better. They experiment, and optimize their performance to the Nth degree so they become the best.

As far as design on the Web goes, performance is based on what users do. Do they buy, click, convert, read, interact, sign up, hang around, or do they just grow frustrated and leave?

I want designers and developers to start thinking about their sites like athletes, and start using data to optimize that kind of user performance: using data to both make life better for users, and fix whatever roadblocks they face. And not in some lame, well-intentioned, feel-good kind of way, but in a measurable, demonstrable, real, data-driven way.

For a long time we’ve just assumed what we think as designers is automatically best for users. However, sometimes those assumptions can be wrong, and hugely costly. Jared Spool of UIE has a great story about how the wrong assumptions of designers were costing a major retailer $300 million a year.

It’s more than about removing roadblocks, though. YouTube recently took a data-driven approach to their home page. They ran a multivariate test with 1,024 combinations of some small changes and improved sign ups 15%, which is incredible.

Designers have been mostly focused on building standards-based sites over the last decade. Where do you see your performance-based approach fitting with the web standards movement?

I think this is the natural evolution of web design beyond the web standards movement. We have standards-based design, and I think – or at least hope – we’ll move to performance-based design as the norm.

Don’t get me wrong – web standards are absolutely vital, as is the work done developing specs, advocating for standards, educating designers, and keeping browser makers accountable. We, as web professionals, are all standing on the shoulders of giants in that regard.

Now, however, we’ve largely figured out the “how” of web standards. The Web is starting to mature, and the next logical step beyond how we can do something is … what performs best?

That’s the question I hope will find its way to the front of designers’ minds, as it forms the basis of performance-based design.

The Web is the most measurable and testable form of design in human history, so I think it’s inevitable that we, as designers, will be focusing more on data and performance in the future.

In the blurb for your presentation, you mention that “data doesn’t mean less creativity and experimentation, it means more.” What do you mean by that?

When you start talking about numbers and metrics and data it sounds like the opposite of creativity. Suddenly you’re a scientist with a spreadsheet, instead of a designer using your imagination and intuition.

But that’s not true. Firstly, web designers already use data to inform their designs; screen resolutions and browser versions are two obvious examples of data points web designers have used for a long time.

Secondly, and most importantly, when you take a data-driven approach and can measure how your designs perform for users, you can try out a bunch of design ideas – no matter how out there – that would’ve otherwise been left as mothballed PSDs on your hard drive.

As well as trying out the ideas you probably already developed, you now know that there are performance gains to be had (which usually translates to real cash money for your client or boss, and ultimately yourself). So it pushes you to be creative and come up with new design variations you may have otherwise not considered.

Great design comes from pushing yourself creatively, and data-driven design lets you do this. You can then push out these ideas to visitors, who use them and vote on which ones they like best (that is, by converting more, or clicking more, or staying longer, and so on).

By letting users test your ideas, you can experiment more and take bigger creative risks, as the data acts as your safety net. It tells you what works and what doesn’t. Without this safety net, we tend to be more conservative and settle for the status quo. We don’t try out the brilliant ideas that may flop or may be a stunning success. But with a data-driven approach, we can.

So, far from stifling creativity, a data-driven approach lets you experiment much more freely.

That sounds great, but how can web designers actually do that?

A/B and multivariate testing is the way to try out these ideas, particularly using Google Website Optimizer. Tools like GWO are pitched as “by marketers, for marketers,” and are usually concerned with optimizing marketing campaigns, specifically landing pages. We, however, as designers really need to make the most of these tools for our own needs, and that’s something I’ll be talking about at Web Directions.

That said, I’d strongly encourage everyone to head over to Google Website Optimizer and become familiar with the tool, and try out a little experiment for themselves. It’s really easy. Google make it easy for non-technical people to get started with GWO, with plenty of help tips and support documentation, so I’m sure SitePoint readers will have no problems at all.

Another tool that may be of interest is the open source A/B and multivariate testing tool, genetify, which takes a different approach to GWO for testing web pages.

Then, once you start testing, please blog the results! As a web design community, we have surprisingly little data on what actually does perform and what doesn’t; so I’d love to see more designers doing tests and publishing the results for us all to learn from.

We figured out standards-based design by blogging about it and discussing it, and I hope we can do the same when it comes to performance-based design.

Doug Bowman famously cited his former employer, Google, testing “41 shades of blue” as one of the reasons why he left them. Where do you draw the line between testing and trusting your designer’s intuition or experience?

I addressed the issues raised by the “41 shades of blue” saga in my essay In Defense of Data-Driven Design. Doug Bowman very kindly linked to the piece on Twitter, saying he agreed with almost all I wrote, which was nice.

To address the question in general though, the line between trusting your intuition as a designer and relying on data is a very interesting one. You can’t test everything, and if you try you end up in the absurd situation Doug found himself in. Yet if you rely on your intuition alone, you may leave performance gains on the table, like the $300 million problem mentioned earlier.

What to do?

Firstly, you might not have to do that much differently at all. Often in the design process, most designers come up with a couple of different options or variations on a theme. Rather than just discarding these ideas as we currently do, they can be developed further and tested. This is great if your client, or boss, has particularly different ideas on what the finished design will be. By working up a couple of options (one you prefer, one they prefer), you can put your money where your mouth is and see what is objectively the best performer.

Beyond that, there are a few rules of thumb to follow:

• Know your web analytics data intimately so you can spot problems and opportunities for performance gains.
• Test real changes to gain real improvements – don’t waste time worrying about inconsequential design details, be bold!
• Some of the best data is qualitative data; that is, feedback from users, as frustrated users don’t leave much of a web analytics trail, so be sure to ask.
• Test things that matter. You don’t necessarily have to test completely different designs. It’s remarkable what a difference testing headlines can have; for example, 37signals boosted sign ups by 30% by testing different headlines.
• If the site has a business goal (that is, to achieve a certain number of page views, conversions, sign ups, or email inquiries), you have to test. Simple as that, so get stuck in and give it a go!

And finally, for more on taking a data-driven approach to design on the Web, come and hear me talk at Web Directions South, and sign up to be notified when my book is out.

Thanks for your time, Luke.

No problem, Matt!

SitePoint readers can take advantage can use the code SITEPOINT when registering for Web Directions South to receive an exclusive discount on their ticket (read more).