Community focus groups about a COVID-19 individual risk assessment tool: access, understanding and usefulness | BMC Public Health

Thematic analysis of the Fight COVID MKE focus groups revealed three main themes regarding the web-based risk assessment tool: (1) Ease of understanding; (2) Tool utility; and (3) Accessibility. Each theme discussed in the focus groups contained positive and negative perceptions that give insights to how community members perceive and may be influenced by risk assessment tools. Feedback was addressed by risk tool developers after the first four focus groups; thus, later focus groups responded to a revised version of the risk assessment tool with recommended improvements including changing from tables of numbers to charts and figures depicting the information and a video explaining use of the tool and interpretation of the results.

Ease of understanding

Focus group participants commented on the ease of understanding information presented in the risk assessment tool, with some participants finding the information helpful and easy to understand overall. One participant (Mixed-race female, 53 years of age – Focus Group (FG)8) had used the tool prior to their focus group session and noted the tool was “informative and interesting to see”. Other participants that were able to understand the tool spoke to its simplicity and being able to digest the information presented:

“This is very easy to navigate so far that I can see. Comprehensive, and it doesn’t really – it’s not too arcane. So, pretty, pretty easy – pretty easy to navigate, for me, for sure.” (Mixed race male, 29 years of age – FG8)

Specific aspects of the risk assessment tool that participants found to be especially straightforward included components that provided a visual representation of risk, which participants generally found easier to follow and digest than words and numerical explanations. One of these visual representations is a bar graph that compares the risk of death due to COVID-19 in individuals who were (a) not previously vaccinated or naturally infected with COVID-19 to individuals who (b) are maximally protected by way of vaccination and/or previous infection:

“I think the graph itself is good because that shows you the comparison of individuals that are vaccinated vs. unvaccinated. So, I think that could be a powerful tool in encouraging people to consider getting vaccinated. (White female, 73 years of age – FG4)

“The black box with the yellow dots and the blue dots, well, that’s pretty clear to me that if you get vaccinated, look where you’ll be compared to if you don’t. That one I like.” (White female, 77 years of age – FG1)

Other participants felt the tool was not easy to understand. One participant described the tool as being “too complicated” (White female, 68 years of age—FG4). Others felt similarly and spoke of how they had difficulties interpreting information and grasping what the tool was conveying:

“I don’t know that people would look at this and realize it’s an assessment to figure out their COVID risk.” (White male, 74 years of age – FG1)

“I think many of us are saying that it’s a bit unhelpful because there are too many unexplained categories.” (White female, 73 years of age—FG4)

One specific source of confusion for users of the tool in the focus groups pertained to the component that compares life expectancy or loss of life due to COVID-19 to other common causes of death including cancer, flu/pneumonia, and motor vehicle accidents:

“I think this black box, “COVID burden for people like you,” and the life expectancy from other causes, I don’t think people are going to relate to this, or understand how it relates to the information that they put in above.” (White female, 75 years of age – FG1)

“I mean, people don’t know what that means… What does that mean? Maybe other people think, “Wow, that’s really interesting.” But most people would say, “Well, I don’t even know what that means.” It means I live three days longer, or what does that mean? I don’t think people understand.” (White female, 68 years of age – FG4)

In response to these perceptions of specific components of the risk assessment tool, the lead researchers used feedback from initial focus groups to translate these assessments into further improvements of the tool. These changes primarily consisted of changing figures estimating risk by vaccination status to bar graphs instead of numerical data in table format. All subsequent changes made to the tool were updates with COVID mortality data and vaccine effectiveness. Researchers aimed to ease understanding of each component addressed in the focus groups to reach broader populations and facilitate COVID-19 risk perception among those who use the tool. Additionally, the research team uploaded a tutorial video to the webpage explaining the purpose of the risk assessment tool, how to use it, and how to enter and interpret personal data. Five additional Zoom focus groups were held following the addition of the tutorial and the video was played during the subsequent focus groups prior to discussing feedback. Consequently, participants generally found the video aided in their understanding of the risk assessment tool:

“The way [the video] just explained makes perfect sense, how the numbers should be interpreted and stuff. But when I was looking at an actual page, just doing it myself – for example, the category that is meant to say how much time would be taken off of your life if you had COVID, I interpreted that, I think, as that’s how long you’d have left to life if you got COVID or something like that.” (White gender non-conforming, 26 years of age – FG6)

Tool usefulness

The goal of creating a web-based COVID-19 individual risk assessment tool such as this one is not to provide medical advice, but rather to allow individuals the ability to determine individual risk, consider the impact of vaccination, and, as one participant noted, disseminate information that is “data-driven” (Native American male, 36 years of age—FG8). Focus group participants who found the risk assessment tool to be helpful and informative further elaborated on the utility of the tool, specifically discussing the impact that this tool could have on individuals who may not otherwise know how to evaluate their risk:

“I think this is a great tool. I’m playing around with it. I’m seeing the impact that my age, my race, my zip code kind of has on the risk of infection. I think it’s very interesting to see.” (Mixed-race female, 28 years of age – FG8)

“I’m certainly gonna suggest it to my group of non-vaccinated family members and friends. I’m gonna do whatever I can to push them toward that.” (African American female, 55 years of age – FG2)

However, participants noted that although they may have found the tool to be personally useful and would recommend it to others in their communities, this may not hold true or connect with certain populations who harbor negative attitudes with respect to the pandemic and vaccination. While discussing these pervasive negative societal attitudes, many noted that individuals who may not be vaccinated or engaging in protective behaviors “don’t trust the information that’s out there” (White female, 67 years of age—FG1) or remain “skeptical” (White male, 70 years of age- FG4) of the foundational science on which the tool is based. Many had concerns about the political climate surrounding the COVID-19 pandemic:

“I think we’re all thinking the same thing that in this very hyper-politicized climate that we’re all living in right now, there’s a lot of people who feel like it doesn’t matter what you say to them, you know, it’s fake news, it’s misinformation, it’s not real science.” (Mixed-race male, 29 years of age – FG8)

Likewise, some participants discussed how the impact of the risk-assessment tool is ultimately up to individual interpretation and may result in “counterproductive” (White male, 70 years of age—FG4) use. In this thought process, participants examined how a young, vaccinated or unvaccinated individual’s results that show a fairly low risk for severe illness and death from COVID-19 infection may convey a false sense of security and therefore result in lower risk protective behaviors:

“I would be interested to know what the likelihood of someone changing their vaccine plan would be based on seeing this data if it would help anyone to see the value of getting the vaccine or if this would maybe embolden some younger folks to say, well, you know, it’s not that big of a deal, look at the risk of me dying, it’s fine.” (White female, 45 years of age – FG5)

Participants who initially struggled with understanding the purpose of the risk assessment tool and the information it conveyed also displayed doubt and distrust of the tool for use within the public. These participants stated that the information disseminated in the tool seemed largely intended for the academic community and one participant noted that “You’re wasting your time trying to convince the average lay person to get vaccinated with this [tool].” (White male, 74 years of age—FG1).

Others noted that their difficulty in interpreting the results from use of the tool would make it less likely for them to disseminate the information within their own social circles. A few participants questioned how useful this information that may come across as overwhelming will be to the public, especially those who remain unvaccinated due to distrust:

“This is a little bit overwhelming, all of this stuff, for the average lay person, to listen to all of this. There’s got to be a better way to get to people that are not vaccinated than this. Because they’re not going to listen. Frankly, I don’t want to listen. I’m sorry to say that, but that’s just the way I feel.” (White male, 74 years of age – FG1)

Accessibility

When creating a public, web-based tool, it is essential to consider the accessibility and perceived barriers towards using the tool. Some confusion was evident throughout multiple focus groups about the logistics of the tool—how it would be accessible to the public, if its use was limited to healthcare settings, or only to those participating in the Fight COVID MKE study. The research team aimed to communicate in both the focus groups and within the tool web page itself that it is a tool meant to be used by any adult nationwide.

Some participants were unsure whether the tool needed to be used in the presence of their physician who could explain the results. This was noted most often in early focus groups that responded to a tool that was deemed less comprehensible. Participants felt that in order to make a tool like this nationally available and useful to the public, it must be digestible for a layperson with little or no medical background. Additionally, they wondered if all adults would have internet access to the tool. Participants were also concerned about whether some members of the public understood a very important aspect of the tool – chronic health conditions or comorbidities:

“You’re assuming that they know if they have any of these diseases that it increases their risk of COVID.” (White female, 75 years of age – FG1)

Participants pointed out that some people with less adequate or equitable access to healthcare would also be less likely to have been diagnosed with comorbidities. In order to interpret one’s individual risk with regards to COVID-19 infection, there is some assumption with the tool that people understand why their risk may be elevated or not:

“Until I got a decent job, I probably couldn’t answer these questions because I couldn’t afford to go to a doctor. So, how could I say I had a chronic illness?” (African American male, 51 years of age – FG2)