Wednesday, August 02, 2006

More on the Problem

Well I’ve been silent for a while. Not that I’ve stopped thinking about the problem of making genealogy accessible to ordinary people. Rather, I’ve been up to my eyeballs in it.

In an effort to describe the problem more fully, here is some information from a test we did last year.

We did some user testing in August of 2005. I won’t disclose the whole scenario but we basically provided a deep set of records for a contiguous set of counties in one state in the US. The records were all digitized and indexed. We took extreme care to find a broad range of users to test making sure that none of them had previously done genealogical research. Then we gave users a shallow pedigree of a family that lived in those counties for several generations and access to the records. The users were extremely successful at deepening the pedigree. One possible interpretation of the test results is that if regular people have easy access to digitized, indexed records, they can succeed. That’s not to say there weren’t some problems along the way.

One of the activities we did to try to understand the results of the test was to build an affinity diagram of all the important things we observed during the test. Here are some of the key learnings from the affinity diagram.

  • People aren’t used to thinking backward through time (death to birth) they are much better at thinking about a person’s life moving forward (birth to death).

  • People start with false assumptions that blind their ability to see the truth. For example, one user assumed that husbands were always older than wives and incorrectly determined that the wife’s age must be wrong because she was older than her husband.

  • Searching on the Internet is not intuitive. Too many or too few search results confuse the user. If there really are no search results, the user should be offered an alternate strategy. For example, if the user searches a census for Bob Johnson in Geneva twp, Walworth Co, Wisconsin and their aren’t any why can’t the system come back with results that say there are no Bob Johnsons in Geneva twp but there are three in Walworth County, would you like to see those?

  • Two strikes and you’re out. Users give search features about two chances before they determine if it is worthwhile or not.

  • Ordinary people seem unwilling to pursue offline sources. They greatly prefer the immediacy of the Internet. Perhaps one of the key measures of the state of the industry should be the mean time for an ordinary person to access a record. Right now I’d guess it is 3 weeks.

  • Users want to save the sources that they find but it is hard. Take a look at clipmarks.com for an interesting way to capture online content.

  • Users have a hard time harvesting appropriate information from a record. One user looking at a census record and talking out loud said, “Years married. Hmm… I don’t need to know that.” The problem of knowing what was meaningful in a record seemed particularly acute when the information was indirect evidence. Forms seemed to help the user get more out of the records.

  • Unfamiliar names and variance in names and dates confused the users.

  • Users need tools to add context and compensate for their inexperience. Things like maps, history, and date calculators would be a great help.

  • Users need to be able to capture clues and get back to them easily in their original context.

  • Users need a place for temporary conclusions – a working draft of their pedigree.

  • Users need help knowing when to draw a conclusion. They struggle with match analysis.

  • Users need substantial help keeping everything they are seeing and doing in context. Experienced genealogists are able to keep it all in their head, novices aren’t.

4 comments:

Dan Lawyer said...

Thanks for the comment Mark. I think there is great value in descendancy research as well. Many cultures outside of North America view their genealogy as a descendancy. The idea of an ancestral approach to the research is very foreign to them. Unfortunately the tools that most of us are used to working with are very week in their ability to support the natural workflow of descendancy research.

Anne McAllister said...

I was really interested to read your analysis of how people approached research. And I was excited about the possibiity that more "deep" documents might be digitized and indexed for general use.

I have been trying to unsnarl a family in Edgefield County, SC for several years -- and it's only been by getting microfilms of equity court records and reading probate inventories that I have been able to even come close to getting more clues which are helping me untangle the identities. I'm sure there are other records that would help, too, but it's a matter of finding out which ones contain the names I need because they are almost never the ones in the record indexes!

Thanks for a thoughtful, interesting and, for me at least, encouraging post.

Anonymous said...

Fascinating experiment and results. I thought you had closed shop. Let the MA know that you are back in action.

Matthew O. Smith said...

It has been great to come across your blog. I have been frustrated by genealogy programs, like PAF, for some now and even started a project, topoged, to address the issues.

However, I have put it on hold in light of what the Church has been doing. After readging your blog I am pleased to see that some of my concerns are shared by others.

Thanks again.