Information architecture deliverables: Task testing (with pictures)
It’s good to know what information architecture deliverables you can expect on a project. Whether you’re working on a website, intranet, or another content management system implementation or redesign, there’s a few usual suspects for information architecture deliverables:
Card sorting
Task testing
Content model or metadata model
This article looks at task testing.
What is task testing?
Task testing is a way to run a client’s site structure through some typical scenarios and get quantitative feedback on how easily users can navigate to relevant or appropriate content. It is normally used as a quantitative research method, although it can also be a qualitative method. In this article, I talk about the quantitative method.
Why use task testing?
Task testing gives feedback on the page structure of the site without any content other than the page titles. It can be done on the current structure or new structure.
What does task testing show?
When testing on the current website, I am testing how well each scenario performs in the current website. These quantitative data act as benchmarks against which I can measure the revised site map or site structure. When I test on the new site structure, I am testing how well each scenario performs in the new structure. I might compare the results to the benchmark and I am also looking for a better-than-average success rate.
What do you need to start task testing?
To start task testing, I need a site map for the current site or I need a revised site map.
How do you do task testing?
After interviewing users and stakeholders for the typical tasks or scenarios that users go through on a website, I know the typical scenarios for web content. I create a task testing plan that lists nine to ten typical scenarios and then list pages that would have appropriate information on them. Then I use Treejack (an online tool) to host and run the test.
How do you know if task testing results are “right”?
When doing task testing in Treejack, I’m looking for a better-than-average success rate. I typically use a “grade scale” where anything more than a 70% success rate is passing.
What are some caveats about task testing?
There are several caveats to task testing which I share with all my clients:
Task testing only tests the site structure with the page titles. There is no on page content so users are only able to key in on the page titles to select the right answers.
Scenarios and terminology can be interpreted in different ways. When reviewing the results, I am able to see the different ways something is interpreted. I may not have been able to see this beforehand.
There can be more than one right answer according to users. In Treejack, I can mark more than one page as correct.
Task testing can show what isn’t working, but it can’t necessarily show exactly how to revise the site structure.
Failures need to be analyzed, discussed, the site structure revised and retested.
Failures are ok! They show me where a site map is not working and they are definitive!
Who should review task testing results?
When I plan and run the task testing, I’m the one who reviews the results. My clients review the plan, recruit participants, and review a presentation of the results. We make decisions together on what to change (or not to change) in the site structure.
What do you do once task testing is done?
If I used the current site structure in the testing, I use the results in the content strategy and the revised site map. If I tested on the revised site map, I update the site map.
Do you need to keep updating the task testing?
If my client or I significantly revise the site map, then we test again.
Do you need “real” content in task testing?
You need to use real scenarios, real users, and real page titles in task testing. No on-page content is needed.
Task testing example
Call centre knowledge base
NB: In this example, the page titles don’t really matter. What I want to show here is the overall success, overall directness, and the pie chart. I look at how I made changes between the first round of testing and the second round of testing to make improvements to the site map.
For this client, I created a new site map and tested it. In the first round, the task around overpaying a bill and getting a refund failed miserably! The success of task is 32% and directness is 30%, a failure.
I took a look at the results and considered where users navigated and what they selected as correct answers. The pie tree below shows that there were two to three main areas of the navigation where users looked for answers. The pie tree also shows that users hunted around for the answer and no one answer stood out.
Based on the results, I made revisions to the site map, discussed it with my client, and tested the same task with the same wording but a different site map. With an overall success of 91% and overall directness of 60%, the change we made was definitely appropriate.
From the pie tree, I could tell that there was much less hunting around for the answer. Users had one main branch of the site map where they went to for information.
This example also shows how important it is to:
Test a site map multiple times: A change can have significant impact.
Test major changes to the site map: If we make changes to a site map where quantitative and qualitative data were used to inform the structure, we need to test these changes before implementing them. This way we don’t make detrimental changes to the user experience.
Test the old and new site map: Testing the old (or current) site map and the new site map show us where we’ve made, or haven’t made, improvements.