UIDesigner has an interesting post questioning the usefulness of cardsorts in developing information architectures.

I’ve asked before: what are alternatives to card sorting that let us go beyond the superficial? I still haven’t heard any definitive answers though.

Over at Boxes and Arrows, Joe Lamentia gives the rundown on analyzing card sort results in Excel. This is great, because as good as dedicated card sorting tools are, there isn't a great candidate that is dependable in all situations.

Speaking of which, this is a good time to link up the card sorting tools that I know of...if I've missed any drop a line in the comments [list taken from Andy Edmonds @ Uzilla].

• Uzilla's UCardSort is under development and works in the Mozilla browser. Update:I neglected to mention a key difference for UCardSort - it's open source, so you can hack away and add features if it doesn't yet do what you need it to do. (thanks for the reminder Andy).
• IBM's EZ Sort was an early tool that hasn't been further developed. The author has been very helpful in the past when things haven't worked quite right.
• WebSort is a Flash UI web-deployed tool created by Larry Wood of BYU. I'm not sure what commercial licensing arrangements are available.
• WebCAT requires server-side Java, and is free from NIST
• CardZort is a graduate project from Jorge Toro that we've linked up before, and is still under development. He asks that commercial users donate $50

So - did I miss any? If you've used any of these, I'd be curious to hear your experience (I've used EzSort, looked at CardZort, I'm going to install Uzilla's tools and have a look).

Donna Maurer muses on a card sorting weakness - In my last few big card sorts, I have noticed that participants don't really look at the cards and try to form sensible groups based on how the information should be grouped to help them achieve a task. Instead, they try to get rid of most of the cards as quickly as possible....

Good point: almost all participatory research, from card sorting to usability testing to surveys and interviews suffer from the fact that the participants usually don't really want to participate. While our goal is to build a better system, their goal is to finish and get their incentive.

Other than longer education periods and involving users as codesigners (so they are invested in the project itself, not just the incentive), what other ways are there to get more than surface data from a card sort? One immediate thought is to make sure the card set isn't too big (or the usability script too long), since people are more likely to give cursory answers when they've spent a long time on the activity. What else?

Donna Maurer shares her technique for evaluating classification schemes over at Boxes and Arrows. Ten minutes from twenty users means that it's pragmatic, and it addresses classification specifically, instead of being part of a prototype with other issues to evaluate. Here's what you need to do this kind of evaluation:

• A proposed classification system or proposed changes to an existing system. Some uncertainty, mess, and duplication are OK.
• A large set of scenarios that will cover information-seeking tasks using the classification.
• A pile of index cards and a marker.
• Someone to scribe for you.

Looks great - thanks Donna!

Darin Marshall points to Jorge Toro's CardZort card sorting software.

CardZort is a computer application that runs card sorting exercises. Its main purpose is to offer a complete computer-aided system that allows the fast creation and execution of card sorting exercises, and the analysis of the resulting groups via cluster analysis.

Darin says it's less buggy than EZsort - I'm looking forward to trying it out. If it works, then it's well worth the $50 Jorge is asking from people using it for 'professional/lucrative purposes'.

The InformIT article on card sorting Blueprints for the Web: Organization for the Masses (free registration required) is an excerpt from Christina's "Blueprints" book.

Lou pointed to this paper about WebSort, a web-based application developed at Brigham Young University for card-sorting like IBM's EZSort.

We have devloped a web-based interface which allows designers to do electronic "card sort" studies. With it, designers can provide descriptions of features for which they'd like users to provide labels and to "sort" into categories. The results can be used to organize information and services access for "interface" design.