What is card sorting and why is it important to your site’s UX?

What is card sorting?

Card sorting is a type of study for assessing and designing the navigation and structure of a website or app. In card sorting, participants are presented with a list of items (for example, all the products featured in an online supermarket) and they’re asked to group the items in a way that makes the most logical sense.

Depending on the type of card sorting study, participants can also choose names for the groups they’ve put together, forming the potential categories and subcategories of a website.

All of this will hopefully create an easier, more logical and user-centric way of navigating your site.

Card sorting is closely related to tree testing, another type of usability test. In fact, tree testing is often referred to as ‘reverse card sorting’ as it asks users to seek out information, rather than sort it. The two can be used in combination to design a better experience for a website’s users.

You can find out more about tree testing in another of our beginner’s guides, but as for this article, we’ll delve deeper into how card sorting works, the pros and cons, and how you can run your own card sorting study.

If you’d prefer a video version, presented by our hosts Jen and Alex from the UserZoom research team, click below.

What does card sorting have to do with Information Architecture (IA)?

You’ll probably hear the phrase Information Architecture (IA) a lot when you begin testing how people navigate your site, and this simply refers to the way content is presented and accessed from any given page – whether that’s through menus, breadcrumbs, categories, links… whatever takes you from one page to another.

This is how our senior UX researcher Hazel Ho refers to IA:

“Information architecture is the practice of arranging parts of something to make it understandable”

Information Architecture typically focuses on…

• Structure: the way information is laid out i.e. people should be able to predict where to find what they’re looking for
• Organisation – grouping information in a way that makes sense to people
• Labels – ensuring elements are appropriately titled so people can find information

A good IA can help people on your site understand where they are, what’s around, and what to expect.

One of the many ways you can help visitors find their bearings on any page of your website is by offering a clear, comprehensive and logical means of navigation.

This is where card sorting comes in.

What are the benefits of card sorting?

As mentioned earlier, card sorting is a test you can run to validate the effectiveness of your site’s organisation, structuring and labelling.

It will also help you decide how to label your categories and navigation, arrange the subcategories underneath parent categories, decide what needs to go on a homepage and figure out where users are getting lost or confused.

The ultimate benefit is that you’ll be building and improving your navigation by observing how real users will navigate your site and its information architecture, rather than just guessing yourself.

Just because you assume that people will find the ’Fruit and Vegetables’ subcategory under a ‘Fresh’ category, it doesn’t mean people won’t also go looking under ‘Frozen’.

Card sorting is also simple to arrange, the materials needed are cheap to produce and it doesn’t take too long.

What are the disadvantages of card sorting?

There are a few things to be wary of when running card sorting:

• Naturally the results can vary wildly from person to person, or group to group. Much like any user testing, there’s no guarantee of consistency. But it can give you a solid foundation to build your IA.
• Although the test itself can be quick, the analysis can be time consuming.
• All the participants have are the surface details of the content (a title or headline for a webpage) they won’t necessarily know what the content is about or understand any nuance between cards – especially if its a niche subject. Bear this in mind when looking for participants.

How do I set-up a card sorting test?

The beauty of card sorting is that it’s pretty low tech. In fact you only really need some post-it notes, a wall or a large table and some test participants.

Or you can use online software to run the test, saving you the hassle of finding all of the above resources.

Either way, the following steps are involved with running a test…

1) Choose your type of study

Whether online or offline, you’ll have to decide what type of card sort you want to run. There are three possible types of card sorting…

Open card sort

This is the most flexible option. Participants are asked to group cards into categories that makes sense to them, and then they label each category in a way that they feel accurately describes the content.

Closed card sort

Here participants sort cards into category groups that you’ve already labelled and defined. For example, this is handy if you’re launching a new page for ‘Fitbits’ and you’re not sure whether to put it under a parent category ‘Technology’ or ‘Sports and Leisure’.

Hybrid card sort

As you’d expect this is a mixture of open and closed. Participants can sort cards into categories you’ve already defined and then create their own categories if they think your categories are a bit rubbish.

Although UserZoom doesn’t currently offer a hybrid card sort, we can come up with a creative solution based on your needs. Please get in touch for more info.

2) Choose a technique

Here’s some advice on the different means of card sorting…

One-on-one

These are in-person sessions with an observer (also known as moderated). Users are encouraged to express their thoughts out-loud to give a clear picture as to why they are making their decisions. This can be done with physical cards or online software, and you will need a person facilitating/observing.

Group

Another moderated technique, these tests have their participants sort the cards out as a group. The group can be briefed at the beginning and at the end of the session, but they are largely left alone for the duration.

This method can lead to quick decisions, but may also be prone to the pitfalls of group dynamics – i.e. the loudest opinion holds the most sway.

Remote unmoderated

Remote unmoderated tests are online based sessions that require participants to work independently from their own computer. These sessions can be done using open or closed methods.

The sessions are recorded by screen-capture software, and the videos can be analysed at a later date (this is what’s known as ‘unmoderated’)

3) Prepare your cards

You’ll want to create around 50 cards. Fewer cards may not generate enough groupings, more cards will probably lead to fatigue/boredom.

Below is an estimate on the time required to sort a specific number of items.

• 20 minutes for 30 cards
• 30 minutes for 50 cards
• 60 minutes for 100 cards

You then have to decide on the content of your cards. You can do this by brainstorming the types of information you might want to include on your website, or by looking at your existing product inventory, or by looking at what your competitors have on their websites, or even better – ask your users what information they expect to find.

Once you have all your ideas written down on a lovely spreadsheet, you can refine the possibilities until you’re left with only the most relevant cards.

4) Source participants

We recommend testing 50 participants per card sort. 50 participants will allow you the flexibility to exclude participants while maintaining reliable results.

Whether you’re running one-to-one or group sessions, it’s a good idea to recruit participants who will give you real-world data.

Things to bear in mind when working with participants

• If people aren’t receiving an incentive or are not obligated to participate, you’ll need to cast your net wider.
• If you’re using physical cards for a group sort, make sure you have enough room for people to spread out the cards and not crash into one another.
• Let participants know how long the sort should take, so it will help them gauge how much time and effort they should put in.
• For goodness sake, if you’re handwriting the cards, make sure your writing is legible. Better yet, print them out.

Alternatively you can use an online platform (*cough cough*) who can source the participants for you.

Pro tips for card sorting

Here’s a collection of pro tips that you should remember when running card sorting sessions, rounded off with a few ‘pro-pro tips’ from our very own Lee Duddell.

• It will help to number the cards individually on the corner, as you’ll need this for later analysis.
• Remember that you might have to nudge your participants to think out loud.
• If you’re running concurrent group or independent sessions, remember you’ll need to provide enough sets of cards.
• Just running quantitative test will only get you so far, you need qualitative data (the spoken out-loud observations of the participants) to fill in any gaps of understanding.
• Run card sorts in local languages. Qualitative studies in open card sorts that take into account the nuances of local language will be valuable in ensuring your navigation is understood in international versions of your site.
• It’s worth remembering that top level menus communicate to people what they can do on a site. They form a major part of a new visitors’ first impression, therefore it’s worth running tests on these.
• Menus are really hard to A/B test. It’s difficult to drive decent volumes of traffic to split test different configurations of IA, therefore the qualitative approach of card sorting is necessary for menus.
• You will need to run a tree test to validate your analysis. Here’s our beginner’s guide to tree testing.

How to analyse your card sort results

If you’ve used online card-sorting software, this should generate a report for you. UserZoom can generate a visual diagram data in the form of a dendrogram.

A dendrogram is a branching diagram illustrating the strength of relationships between items and between groups of items. They help to visually show the groups of items based on participant’s perceptions of their relationships.

When interpreting dendrograms to identify potential new groupings, look for clusters of items that are both distinct and compact. For instance, the shorter the distance between two items or groups of items, the more similarity they share. However the longer the distance between items or groups of items, the more distinct they are perceived to be from one another.

If you’ve used physical cards – you’ll need to analyse the results yourself.

Here’s some additional advice that’s worth bearing in mind:

• After every sort remember to photograph the layout of the cards.
• Write down the names the participant gave to each grouping (if it’s an open sort) and the corresponding numbers of the cards included under that name.
  • Remember to reshuffle the cards for the next sort.
• Write down qualitative information in the form of user comments.
• Write down quantitative information based on: which cards appeared together most often, how often cards appeared in specific categories.
• It may also be worth noting how much time it took to complete the sort.

You can use this data to find commonalities between the card sorts, or if you have a larger number of cards and sessions you could use an Excel spreadsheet to reveal the relationships and patterns between the cards.

Then you should hopefully have enough information to begin structuring (or restructuring) your site’s information architecture.