Nothing Like Real Users

Wow, I love the 'labs' concept! 13 days ago we made the FamilySearch Labs site available. Since then the tool has processed about 3,000 pedigrees. We've received lots of feedback from real users on what they love and hate about the experience. Because of the volume and quality of the feedback we were able to quickly isolate and resolve usability issues.

We borrowed the idea from Google Labs to see if we could create an environment where we get user interaction with software substantially earlier in the development process. In the past we've often done paper prototyping to show concepts and task flows to users and get their feedback. Paper prototypes are good at helping you understand if you have the right elements on a page but don't do much to tell you if what your application will be useable. The labs approach has given us a whole new capability in useability testing.

While the labs experience has been a great success, it still doesn't remove the need for first hand observation of users. Here's an example. After we had resolved most of the major issues with the Smart Pedigree Viewer (thanks to Ben Crowther for the name) we decided to go and do some live observation. I spent some time at my favorite family history center watching people use the viewer. In twenty minutes I picked up a handful of usability issues that we never could have discovered without first hand observation. On the other hand, the amount of time we had to spend doing live observation to attain our currently level of useability was substantially reduced.

So what do I like about the labs site:

• Speed: We can change the application, put it on the site, and know if our new features sink or swim within a few hours.
  
• Audience: We've had users from all around the world looking at the application. This is critical for an application that will have an international reach.
  
• Cost: As you can imagine, gathering feedback on an application from several thousand users would be extremely expensive and virtually impossible. The cost to do this through the Labs site is trivial.
  
• Open: The labs culture of letting the users have a peak behind the curtain and sharing often imperfect software early and often creates an openness in communication between the developers and the users that I've never seen before.
  
• Motivation: Working in an environment that allows you to 'publish' your work every couple of days (or more often) is extremely motivating to the development team. It is exciting to get the newest thing into the hands of users.
  
• Community: Bringing the end user into the process helps us all feel like we're on the same team. It creates a sense of community. Everyone is invested in the outcome.
  

I could go on, but the point I'm making is that there is substantial value in the 'labs' concept. I HIGHLY recommend this approach to anyone developing consumer software. I know that some organizations will feel they can't be this open because their competitor will come look at the site too. My response: I believe speed, agility, openness, and community will beat fixed-length secrecy any day. They're going to see your product sooner or later.