System, Intervention, and Implementation Strategy Usability Scales (SUS/IUS/ISUS)

Measures & Guidance > Usability > System, Intervention, and Implementation Strategy Usability Scales (SUS/IUS/ISUS)

The Intervention Usability Scale (IUS), Implementation Strategy Usability Scale (ISUS), and System Usability Scale (SUS) are a set of related scales that support assessment of the usability of interventions, implementation strategies, and systems, where a system may be software program or paper form.

This can support decisions like whether a prototype is ready to advance to more widespread testing or comparisons, such as whether a redesigned intervention is more usable than the original intervention. While they can help answer how usable something is, the scales on their own do not directly guide how to redesign something to be more usable.  

Highlights  

Here are some ways you can use these measure in the design process… 

  • SUS/IUS/ISUS can be administered in early usability tests to provide quick, quantifiable feedback. This allows designers to assess whether users find the intervention interface intuitive and easy to navigate. 
  • Although the SUS/IUS/ISUS provides a general usability score, when paired when paired with qualitative follow-ups, it can help identify specific issues (i.e., confusing navigation or awkward flow). 
  • Repeated use of SUS, or a shorter form measure (see below), throughout the redesign process can track improvements (i.e., benchmarking). If the score increases over time, it signals that changes are positively affecting usability. 

Should you use the System, Intervention, or Implementation Strategy Usability Scale?

Overall, our guidance parallels the name of each scale. If your goal is to assess the usability of the intervention (including any associated supporting tools), then use the Intervention Usability Scale (IUS). If your goal is to assess the usability of a system or artifact, use the System Usability Scale (SUS). SUS was developed primarily for digital systems but may work for other kinds of artifacts such as worksheets. 

There may be times when you need to assess two or all three. However, in these instances, we recommend against using SUS and IUS in the same session with the same participants, as the similarities may lead to fatigue or confusion. Further, because UWAC projects are often investigating the usability of an intervention and usability of a system, it is important for the object or target of the survey to be clear. 

Role of scales in each DDBT phase: When and how to use them 

Even if you use the same scale in each phase, it may have different roles. 

Discover and Test Phases 

From the center perspective, we hope to collect or set up comparisons between the usability of the un-adapted intervention or implementation strategy and the redesigned intervention or implementation strategy, among the intended users and in the intended population. Example comparisons include: 

  • Comparing an un-adapted intervention in discover phase to the redesigned intervention in the test phase using the IUS 
  • In a project with a test phase that has a control condition of the un-adapted intervention/implementation strategy, comparing usability of the un-adapted intervention/implementation strategy (control group) with the adapted intervention/implementation strategy (redesign group) by comparing the IUS/ISUS scores between the control and redesign groups after a period of use in the test phase. 
  • In a project that focuses on redesigning an artifact-based intervention strategy (e.g., software, paper form) to improve usability, comparing the artifact’s usability between the discover phase and test phase using SUS.  

Projects where the un-adapted intervention or implementation strategy cannot be used at all in the destination context without adaptation can prevent collecting meaningful IUS/ISUS/SUS data at this stage. In this situation, please reach out to the Methods Core to discuss what might be useful for both your project and center goals. 

Design Build / Phase 

Teams also may use IUS/ISUS/SUS to assess usability during the design/build phase. Early in this process, a scale might be used with scenarios or storyboards to assess perceived or anticipated usability of a design direction. Interviewers might probe about responses to items that indicate potential usability issues to guide redesign. Later, as prototypes mature, participants can complete the IUS/ISUS/SUS after doing a set of tasks (e.g., using the system or role-playing steps in an intervention or implementation strategy). Usability experts tend to look for a SUS score to reach 70 (a benchmark for satisfactory usability) before moving to a test phase, as doing so earlier will likely mean that significant usability issues remain that would interfere with overall goals for a test phase. It is not known if a similar benchmark carries over to the newer IUS/ISUS. This guide provides further detail on interpreting SUS scores.  

The shorter Usability Metric for User Experience (UMUX) (4-item) or (UMUX-Lite) 2-item might also be used in this iterative process, to avoid measurement fatigue. UMUX-Lite correlates well with SUS, at least for evaluation of technologies – but to convert between UMUX-Lite and SUS, you must use 7-point Likert scales. Currently, there are not published examples of using UMUX or UMUX-Lite for non-digital interventions and implementation strategies. 

As teams wrap up the design/build phase, participants must complete the IUS/ISUS/SUS about their redesigned intervention or implementation strategy. Please report this (and not all iterations) to the center, as we would like to know what usability level is reached by the end of design. 

Adapting the Wording of Scale Items

Before administering the survey, review—and possibly pilot—your selected scale to assess if the wording of items is appropriate for your participants and what you are evaluating. Sometimes, it can make sense to adapt items – you are welcome to meet with the Methods Core to discuss possible adaptations or adding clarifications. Please meet with the Methods Core before dropping items or changing the response scale, as this can inhibit later comparisons with other projects. 

If you need to translate the scale into a different language, the translated scale should be piloted and discussed with community partners. A protocol you can consider using for translating IUS/ISUS/SUS is A systematic approach for accurate translation of instruments: Experience with translating the Connor–Davidson Resilience Scale into Arabic (Toma et al., 2017), which has also been used to translate AIM/IAM/FIM. 

Example of SUS adapted for children and additional examples of SUS adaptations.   

Example use across DDBT phases: PST Aid

Consider the following example: a team is developing a technological tool, PST Aid, to support the delivery of Problem Solving Treatment (PST). The team has already completed the IUS regarding PST in the Discover phase. 

The team chooses to use the IUS and SUS in the design phase to inform their design direction (IUS) and assess readiness of the prototype (SUS), and the IUS in the test phase to compare usability of PST without PST Aid to PST with PST Aid. 

Design phase

In early design workshops, the team uses the IUS to assess participant perceptions about whether PST Aid will make PST, the intervention, more usable. They describe PST (without the Aid) and ask participants to complete the IUS based on it, using “PST” as the object they are evaluating. They then describe PST Aid and how it can be used to support PST delivery, and ask participants to repeat the IUS, this time using “PST, as supported by PST-Aid” as what they are evaluating. 

As the team begins to work on their prototype, they periodically present it to participants and ask them to complete tasks using the prototype. They integrate collection of SUS with their usability testing and interviews. After completing a set of tasks, they ask participants to complete the SUS regarding PST Aid’s usability. In early uses, when they know the prototype has many areas for improvement, they ask participants about items where there is the most opportunity for improvement (e.g., “What problems kept this from being a 10?” for positively worded items). This, along with their observations of teams using the tool, informs their design refinements. As the design matures, they begin paying attention to the total score, as they want to ensure they are above the benchmark of 70 (satisfactory usability) before moving to the test phase. 

Because the team knows that training will be part of how PST Aid is introduced to clinicians, they guide clinicians to anticipate usability post-training for SUS questions, such as “After you have been trained to use [system]…”. Because clients will not receive training, they do not adapt those items in similar ways. 

Test phase

In the test phase, the team uses the IUS to compare usability of PST with PST Aid to PST without PST Aid. Their test phase has a control group (PST without the aid) and a test group PST with PST aid), so they decide it is best to collect IUS during the test phase and compare groups. This is also the comparison in which the center is most interested, as it helps answer the question of whether the introduction of PST Aid makes PST (the intervention) more usable. 

System Usability Scale

You should include qualitative probing about scores in an interview format – consult with Methods Core in best practice in doing this, how long your interview may be, and the order (usability scale then interview; interview then scale, etc.). See also: “Usability Interviews & Task-based Usability Testing” below.  

Goal: To get a baseline usability score and inform if redesign is ready  

Response scale: 1 = Strongly disagree, 2=Disagree, 3=Neutral, 4=Agree, 5=Strongly agree  

  1. I think that I would like to use [system] frequently  
  2. I thought [system] is easy to use  
  3. I think that I would need the support of an expert consultant to be able to use [system] 
  4. I found [system] unnecessarily complex  
  5. I find the various functions in [system] are well integrated  
  6. I thought there was too much inconsistency in [system] 
  7. I would imagine that most people would learn to use [system] very quickly  
  8. I found [system] very cumbersome to use  
  9. I felt very confident using [system] 
  10. I needed to learn a lot of things before I could get going on [system] 

Intervention Usability Scale (IUS)/Implementation Strategy Usability Scale (ISUS)

  1. I like to use [intervention/implementation strategy] frequently  
  2. I find [intervention/implementation strategy] unnecessarily complex  
  3. I think [intervention/implementation strategy] is easy to use  
  4. I need the support of an expert consultant to be able to use [intervention/implementation strategy] 
  5. I find the various components of [intervention/implementation strategy] are well integrated  
  6. I think there is too much inconsistency in [intervention/implementation strategy]  
  7. I would imagine that most people would learn to use [intervention/implementation strategy] very quickly  
  8. I find [intervention/implementation strategy] very cumbersome to use  
  9. I felt very confident using [intervention/implementation strategy] 
  10. I needed to learn a lot of things before I could get going on [intervention/implementation strategy] 

 😠 😞 😐 🙂 😁

Response scale: 1 = Strongly disagree, 2=Disagree, 3=Neutral, 4=Agree, 5=Strongly agree    

Scoring the Usability Scales

Scoring guidelines included here. All three measures are scored the same way. 

Measures & Guidance > Usability > System, Intervention, and Implementation Strategy Usability Scales (SUS/IUS/ISUS)