Online surveys and usability testing

When you are working professionally with online communications, you typically work with a conceptual separation between:

1. Usability of the communication
2. Relevance of the communication
3. Visual design of the communication

The separation makes sense for the professional because it makes it possible to

1. Optimize usability through improved Information Architecture
2. Optimize relevance through improved content strategy and planning
3. Optimize the design through an improved use of graphical elements

The problem about this separation is that it is only conceptual. It does not exist in real life and it certainly does not exist in the heads of ordinary users visiting websites. For the casual visitor of a website, there is no distinction between usability, relevance and design. If the website is really relevant to the user, she will typically also think that it is really easy to use and maybe even also good looking. Similarly a user visiting two websites with exactly the same content, but with different degrees of usability will also consider the user friendly website much more relevant than the website with low usability.
When we, as professionals, try to determine usability, relevance or design of a solution we therefore need to be very cautious about how we talk to the actual users of a website. If we ask about relevance, we may actually get an answer which is much more related to usability and vice versa.
This all may seem a bit abstract, so let’s provide an example.
In the early days of the Internet I was testing the e-banking solution of Danske Bank. We used qualitative as well as quantitative methods – an online survey and a think-aloud test. Our gut-feelings told us that the usability of this very early version of the online bank would be very low, and this gut-feeling was confirmed in the think-aloud test, where all of the first-time users more or less failed to understand the basics of the application in the 1,5 hours the test lasted. However, and this was the surprising part, in the online surveys we asked a number of usability questions about navigation and general user friendliness of the online bank, and this time the results were in the completely opposite end of the scale – people loved the online bank and found it very easy to use.
The reason, of course, was that the relevance of an online bank was tremendously high. So high that people were willing to spend hours and hours to actually learn how to use the application, and when they had learned how to navigate the not-so-user-friendly application, they actually found it quite easy to use. The users were simply unable to tell the difference between relevance of the online bank and the usability of the online bank.
This is approximately eight years ago, and I told myself that I would never again make the mistake to ask people usability questions in an online survey. If I want to test the usability of a website, I will either use a qualitative think-aloud test or actual tracking data using some kind of metrics system.
Unfortunately it seems like my ancient experience has not become common lore in the web analytics business. More and more often I seem to bump into online surveys on all sorts of websites that are trying to ask people questions about the usability of the website (see example below).
The problem about this kind of surveys is that they simply do not provide the answers that the website owners think they are getting. It may seem like an easy and cheap solution when you buy the survey from the web analytics consultants, and it may seem like you are actually getting answers to your questions, but this is unfortunately not the case!
Online surveys are utterly useless when it comes to determining usability of a website. They are extremely good at determining demographics and content relevance questions, but they are simply unable to give you truthful answers when it comes to the usability of your website.
This may sound very dogmatic and not very pragmatic in the face of real life, where you are often constrained by time and resources. However, being a bit dogmatic when it comes to research methodology may in the long run be more pragmatic, because you will avoid getting the wrong answers to the right questions.

The Long Tail of Social Networking - Google Friend Connect

Google is now running their new program "Google Friend Connect" in a beta (surprise!). Basically the idea is to build the long tail of social networking sites by offering an easy add-on (widget) that can be embedded on your blog, website or whatever you like to publish on the Internet. The widget makes your website a social site where people can join the sites and become members of your own little micro community. In other words - you have become your own micro Zuckerberg :-D
Of course this is too tempting to be ignored, so I have immediately installed the app on this blog (as you can see on the right hand side of my front page of www.petersvarre.dk.
So check it out - join the "Peter Svarre's Blog" community and start the micro social networking revolution right now!!!
And read more about Google Friend Connect here.

Your website is done – time to start working

There is something strange about the way that customers and clients in the web business work together. It is almost common knowledge nowadays that you don’t just launch a website – you launch a web presence! Nevertheless it seems like most relationships between customers and clients are project based, meaning that they focus on launching a website and when the website is launched, the customer-client relationship is winded down to a minimum of support and bug fixing.
There are at least three reasons why this pattern is counterproductive:

1. Experience tells us that most successful websites like Facebook, Wikipedia and Craigslist all started out with one business plan and ended up being successful with a completely different business plan. If you simply launch a website and leave it there, you will never be able to tweak your website into the successful entity which actually serves a customer need.
2. Some of the most successful websites today are community driven, which means that the website constantly changes shape and content based on user input. For community websites the hard work does not end when you launch the website it actually only starts then. When dealing with communities you should be prepared to spend almost more hours and resources on maintaining you website as you did building it.
3. Tracking and KPI measurement is becoming increasingly important – especially in these times of financial tightening. But measuring you website traffic and your KPI’s avail to nothing if you are not also willing to spend resources on optimizing your website based on the statistics. The truly successful website manager in a time of financial crisis is the person who constantly measures and optimizes the website in order to always create optimal ROI for the website. So again, the hard work doesn’t stop when the website is launched. Launch day of the website is simply the beginning of a long and meticulous measure-optimize-measure process, which never ends because a website is a living creature which constantly needs to adopt to changing markets and user needs.

It is puzzling why these simple facts are so often ignored in the offline world – resulting in scores of half-baked monolithic websites which are kind-a-cool, but not really relevant for the target group or the original strategy – simply because the website hasn’t been allowed to develop in a fruitful and ongoing ping-pong relationship between clients and agencies.
I don’t say this often, but maybe we should take a look at how things are done in the offline world. When you launch a store in the real world, you are perfectly aware that you need to hire a bunch of people to maintain the store and sell the products within the store. It is basically not terribly difficult to build a physical store – the hard part is to maintain the store and persuade the customers to keep coming back.