What is tree testing and how can it improve your site’s usability?

What is tree testing?

Tree testing is a research method for assessing findability within the hierarchy of a website or app. In tree testing, a text-only version of the site’s hierarchy is laid out in front of test participants, who are asked to highlight a category or page within that structure where they would expect to find a particular item or piece of content.

We’ve touched on tree testing before in our beginner’s guide to card sorting, and although these two methods of user testing share many similarities the method you use to arrive at your results is reversed.

How is tree testing different from card sorting?

Tree testing is also sometimes known as reverse card sorting. In card sorting, test participants are presented with a list of items (for example, all the products in an online supermarket) and asked to group them in a way that makes sense to them.

This puts the onus on the test participants to create their own categories and group items within those categories as they see fit. Tree testing flips this by instead asking testers to work with an existing set of categories and highlight where they believe an item is most likely to be located.

Card sorting would logically take place before a tree test. In fact, tree testing is a good way to validate the results from your card sorting exercises.

Example of a Tree Test

During a tree test, you’ll set a number of tasks for your participant to complete. As an example, here are the results of a test we ran for the ecommerce platform Shopify.

To test the navigation of Shopify, we asked our participants to click where they expect to learn more about ‘enterprise level selling solutions’. Here’s how they did….

The correct option is Enterprise, which can be found underneath Ways to Sell. 93% of participants opened Ways to Sell, with 60% correctly selecting Enterprise afterwards. Another 24% of users clicked on other topics underneath Ways to Sell, with 10% clicking on Pricing.

You can see more from this test by downloading our detailed report: Shopify vs Magneto UX Benchmark Test.

Why ‘tree’?

The structure of your website looks a bit like a tree. There’s a hierarchy of categories and subcategories (or headings and subheadings, or topics and subtopics depending on your favourite UX terminology), all of which branch out from the homepage, which acts as the main tree trunk.

Only there are less squirrels.

Unless you run a squirrel identification website I suppose.

But then your site structure would probably just consist of two paths – grey and red. Amirite?*

What are the benefits of tree testing?

Tree testing helps identify navigation issues with your site or app. You can analyse where a user would expect to find the information they’re looking for, and you can make improvements to your site based on this behaviour.

Tree testing will help you validate the effectiveness of your site’s organisation, structuring and labelling. According to Boxes and Arrows, “while open card sorting is a good ‘detective’ technique, it doesn’t yield the final site structure – it just provides clues and ideas.” This is where tree testing comes in.

Unlike other information architecture tests, tree testing presents participants with realistic task scenarios, so you may learn more about real-world behaviour.

Tree testing is relatively quick, so it’s easier to test variations of a site structure and compare the results. This can also help you provide quantitative results, e.g. Version A had a 47% success rate, compared to Version B’s 22%.

And much like with card sorting, the ultimate benefit is that you’ll be improving your navigation by observing how real users navigate your site rather than just going on assumption.

What are the disadvantages of tree testing?

Donna Spencer, the information architect responsible for the invention of tree testing, points out that the method tests the top-down organisation of a site and the labelling of its topics. Tree testing therefore concentrates solely on your site structure, so you won’t get any feedback on any other areas of navigation or learn how users may use a blend of functionality on your website.

However, if you supplement tree testing with other user research methods, such as basic usability testing or click testing, you can build up a fuller picture of any issues or opportunities with the design and structure of your site.

How do I run tree testing?

Tree testing is typically done remotely, with users on their own computers, in their own environment, completely unmoderated. There are a few different online tools that can help you run unmoderated tree testing, including of course *cough cough* UserZoom. Here’s our own promo video:

If you want to hear more qualitative insight (i.e. a running commentary on why a user is making the decisions they are making) you could organise a moderated session. But these would have to be one-to-one, and you may find your own resources will limit how many you can run.

As opposed to standard user testing, tree testing takes place on a simplified text version of your site that removes everything except the navigable structure of categories and subcategories. This helps keep visual distractions to a minimum and any other methods of navigation (search, breadcrumbs) won’t affect the user’s behaviour or results.

According to NNG, your tree should be a complete list of all your main content categories and all their subcategories. They also point out that even if you’re interested in only testing a specific section of the menu, you shouldn’t exclude the other categories because it assumes that users will know which section to go to.

NNG also recommends providing a tree that is 3, 4 or 5 levels deep and allow your users to be able to go down to the lowest level of subcategories – with each subcategory providing a full list of all options in that area. Just like your actual live website, that your ‘real world’ audience will be exploring.

Tree-testing shouldn’t take longer than 15-20 minutes, and you shouldn’t ask for the user to complete more than 10 tasks. Any more and fatigue/boredom will likely set in, as well as familiarity with the site structure. Results may become skewed as users remember where they’ve seen things before.

We recommend 50 participants for a typical tree test study. This number will “give you the flexibility to exclude participants while maintaining reliable results.” And according to Boxes and Arrows, a good task is clear, specific, and representative of the tasks that actual users will do on the real site.

What should I look for in tree testing results?

Tree testing results tend to be easier to analyse than card sorting results, mainly because of the focus on quantitive data.

The data from a tree test visualises the various paths that users take to get to specific information or content.

This is taken from our own Tree Test Results section, and you can see where both successful and unsuccessful participants clicked during the test.

And here’s an alternative visualization where you can see an overview of all the results for the tasks you’ve set.

According to ExperienceUX, there are other measures you might want to look for in your results:

• Directness: the percentage of users completing the task without hesitation and getting the correct answer first time
• Success: the percentage of users that completed the task vs. failed attempts
• Time: the time it took users to complete a task

You can also look at number of attempts to complete a task, as this can indicate where items are difficult to find.

These results should help you decide whether any problems with your structure relate to the organisation of your content or its labelling.

But again, if some of this data isn’t providing enough helpful insight, you can always carry out some moderated tree tests and ask participants their reasonings in the moment.