Appropriateness

Definition 

Perceived fit, relevance, or compatibility of clinical interventions and implementation strategies for a given practice setting, practitioner, or consumer (Proctor et al.)

Rationale

Human-centered design (HCD) is often ultimately concerned with ensuring that products fit users’ contexts in ways that facilitate adoption and integration (Blandford). This is also a strong point of intersection between HCD and implementation science (IS) (Kwan et al.), where contextual appropriateness has been more explicitly defined as the perceived fit, relevance, or compatibility of an innovation for a given setting and the individuals in that setting (Proctor et al., Proctor et al.). Consideration of contextual factors across intervention development, adaptation, and tailoring processes to improve the appropriateness of digital technologies, EBPs, and implementation strategies is often an explicit objective of the application of HCD (Lyon et al., O’Hara et al.). For instance, Haines et al. used contextual inquiry, usability testing, and iterative prototyping to develop implementation strategies intended to enhance the contextual fit and adoption of a care coordination intervention. In addition to designing products or innovations that align with the known constraints of a setting, contextual appropriateness can also be advanced by redesigning aspects of the setting (e.g., workflows) to be more conducive to particular types of products/innovations (i.e., social systems engineering) (Palmer et al.).

Measuring & Understanding Appropriateness

Concrete tools applied in HCD to measure the degree to which health innovations align with their context of use have been drawn largely from the IS literature. Measurement is still limited, as documented in a systematic review that identified only seven instruments addressing appropriateness in physical health care (Khadjesari et al.). The University of Washington (UW) maintains a list of case studies of how UW researchers have studied appropriateness. The Intervention Appropriateness Measure (IAM) (Weiner et al.) has been applied across a range of projects to evaluate the extent to which intervention or implementation strategy development or redesign projects result in improved fit with their target users and setting. For instance, when developing a digital implicit bias intervention for mental health clinicians working in schools, Liu et al. used an IAM benchmark of >80% (i.e., an average score of 4+ on a 1–5 scale) among other indicators to confirm adequate appropriateness following iterative development and testing. Here are some ways that IAM can be used in the design process:

• IAM can be used to test relevance during development to ensure that both educators and adolescents see the intervention as appropriate for their context (e.g., school setting) and needs (e.g., addressing anxiety or depression). This is critical for driving acceptance and uptake. 
• If IAM scores are low, it indicates a mismatch between the intervention and user needs (i.e., a poor contextual adaptation), suggesting that content or strategies need to be adapted to better align with the context. 
• IAM results can guide decisions that can be used to refine the content, for instance on whether to adapt specific modules (e.g., behavioral support) or modify the intervention’s focus (e.g., more emphasis on emotional regulation).   

Appropriateness can be gauged qualitatively through a Goodness-of-Fit interview (Albin et al.). The goodness-of-fit interview is particularly well-suited to probe on clinical intervention and implementation strategy appropriateness issues identified through IAM.

UWAC Reporting Requirements

To support center-wide science, UWAC-funded teams are expected to administer the IAM. The IAM should be administered during the Discover phase (based on the existing intervention or implementation strategy) and at the Test phase (based on the redesigned intervention or implementation strategy and a comparison to the unadapted scores if possible). Instead of administering the IAM during the Discover phase, teams may gauge appropriateness based on existing literature. If possible, IAM should be administered during the Design/Build phase on the redesigned intervention or implementation strategy.

UWAC-funded teams are also expected to administer a Goodness-of-Fit interview, although this is optional for R03s.  Goodness-of-Fit should be administered during the Discover phase (based on the existing intervention or implementation strategy) and at the Test phase (based on the redesigned intervention or implementation strategy and a comparison to the unadapted scores if possible). If possible, Goodness-of-Fit should be administered during the Design/Build phase on the redesigned intervention or implementation strategy.

Intervention Appropriateness Measure (Adult Version) 

1 = Completely disagree, 2 = Disagree, 3 = Neither agree nor disagree, 4 = Agree, 5 = Completely agree   

1. [Intervention or implementation strategy] seems fitting. 
2. [Intervention or implementation strategy] seems suitable.   
3. [Intervention or implementation strategy] seems applicable.   
4. [Intervention or implementation strategy] seems like a good match.

Weiner et al.

Intervention Appropriateness Measure (Child Version)  

😠 😞 😐 🙂 😁

Response scale: 1 = Strongly disagree, 2=Disagree, 3=Neutral, 4=Agree, 5=Strongly agree  

Based on what we just talked about…

1. [The thing] seems fitting. 
2. [The thing] seems like a good match. 

We recommend each team should discuss internally and with UWAC Methods Core if the statements make sense in the context of their intervention and/or implementation strategy, and if there are any contextual or language issues that would affect understanding. There are challenges with translating the IAM to different languages and can sometimes be challenging to understand in English. We recommend you review Toma et al.‘s protocol for translating instruments. Reach out to UWAC Methods Core for inquiries around existing translations. Participants may seek clarification when filling out the survey on specific words or overlap with scale statements. Reach out to UWAC Methods Core to discuss clarification examples or phrasing that anchors statements. Bryan Weiner suggests adding a referent so that respondents have an easier time responding. Below are his suggestions on possible referents: 

• Person: e.g., suitable for nurses (given scope of practice), suitable for my patients, kids like me 
• Place: e.g., suitable for my organization (this clinic, schools) 

Scoring the Intervention Appropriateness Measure

We recommend you calculate the average ratings for each item and an average aggregate score.

• To calculate the average rating for each item, sum all respondents’ scores for that item, and divide by the number of respondents.
• To calculate an average aggregate score, take the average of all respondents’ average scores.

Revised Goodness-of-Fit Interview 

1. What aspects of [CI/IS] are a good fit for your setting? 
2. What aspects of [CI/IS] are a poor fit for your setting? 
3. What could feasibly be changed to improve the fit of [CI/IS] for your setting? (if needed, probe about organizational, personnel, and student/client factors) 
4. Magic Wand Questions 
   1. If you had a magic wand, what would help you learn the [IS]? 
   2. If you had a magic wand, what would you change about this {IS}? 
   3. (potential probe, if you had all the time and money, what would you do to improve fit for your setting? 
5. What would help motivate you to integrate the [CI/IS] into your [WORK/TREATMENT DELIVERY/ETC] philosophy and practice? 

We recommend analyzing Goodness-of-Fit responses with a content analysis approach.