Introducing the Usability Blueprint.
Designing a usability study involves people from many different fields. Often, we see designers, anthropologists, UX professionals and QA specialists working together to cover all crucial aspects of the usability study. Based on experience from designing and planning multiple user-involving studies, we saw the need for a tool to easily facilitate the collaboration between these parties. Founded on the methodology and toolkit of Design Thinking, we have developed a tool that enables collaboration and thorough development of usability studies. Introducing the Usability Blueprint.
When applied correctly (and that does not require much exercise) the data output found in this tool will provide a cornerstone in your protocol, your moderator guide, your note sheet, and your study report. Here is how it works.
The Usability Blueprint explained
The Usability Blueprint matrix is simple but provides you with a lot of information. Here is how it looks:
The swim lane farthest to the left is called What to test. This is where you put in what it is you want to investigate in your study. These are your test objectives. A simple example could be wanting to learn if users remember to put the cap back onto the pen after having written a message on a post-it. This test objective would go in said swim lane. Let’s use this example to go through the swim lanes:
- Method: Here you state what method you intend to use to gain knowledge about your test objective. In this case we want to observe if the user puts the cap back onto the pen after use.
- How to measure: Here you want to be a bit more specific as to how you make the data collectable. In this case we mark the instances of failure on the note sheet, i.e. we count how many of our users do not put the cap back on. Quantified data like this is much easier and faster to collect and to interpret in the analysis phase.
- Criteria for success: You would have to refer to your QA professional and the risk analysis for this one. For some test objectives, you can tolerate that some users fail in complying with the intended use. In other cases, especially with medical device development, the severity of unintended use could result in serious harm and your criteria for success is zero failure across all respondents. In our case however, the severity of not putting the cap back on is low. In collaboration with our QA professionals we have agreed that the pass-fail ratio must be 4:1. For every four respondents passing this test objective there can be only one failing.
- Definition: For the note-taker to know when you observe a failure during your study, you need to specify exactly what a failure is in this case, and thereby what a failure is not. In the case of the cap, a failure is not having put the cap back onto the pen as the participant leaves the room. The definition depends largely on your specific case and the severity of unintended use according to your risk analysis. For a medical device, the definition of a failure might be that the cap of a drug container should be put back onto the device immediately after use. This is not the case for the pen, as the risk analysis would probably state that the risk of the pen drying out would not occur for hours if the cap is left off, hence the stated definition of a failure.
- Notes: This field is free for you and you colleague to use for communicating and sparring. Also, this is a good place to put in the rational for your definition, a reference to a specific part of your risk analysis, or a note for further development and implementation.
- When: This last field is crucial to the practical planning of your study. To keep efficient both during your study, during reviewing video material, during data analysis and during reporting, everyone involved in the study needs to know not only how, but when which data points are collected. In the case of the pen and the cap, the note-taker will mark the instances during the study. If the note taker would be too busy scribbling down handling steps and the like during a think aloud exercise for instance, it could make sense to do this data collection as part of the video analysis instead.
Now that you have filled in the first row, it is time to fill in other test objectives.
The Usability Blueprint put to use
Having a filled-in matrix enables your team to divide and conquer. One can fill in the protocol, another the moderator guide and a third the note sheet. This is possible because you are now all working in the same direction. As long as you all stick completely to the Usability Blueprint you know what to write in the test protocol, you know what and how to formulate questions in the moderator guide and you know exactly how to make room for collecting the right types of data in the note sheet. Fast paced, easily and effectively.
Try filling in your Usability Blueprint using post-its. This is an even more flexible and collaborative way of designing your study – and it is great for workshops. That is how we do it.
We have used this framework in many studies primarily within medical device development and evaluation. It ensures structured collaboration and a study that is also structured, easy and fast to both develop and execute. It does not waste unnecessary resources and it delivers both usable and insightful output.
Hope you will put the tool to good use too.
Download the Blueprint
Your study in one page
You can use the canvas to describe, design, challenge, and collaborate on your usabiltiy study. Actually, the blueprint work with all types of user studies and in conjunction with other management and execution tools and processes.
We will keep you informed of related products and news. We will not spam you.