Usability Tests of Medical Devices — Why and How to Perform Them?
The digital transformation over the last few years has brought changes to many sectors, including medicine. That’s why we see a boom in the medical devices segment, a market that is — in Europe alone — expected to reach USD 61.4 billion by 2025. Europe has 27.000 medical technology companies; the US over 6,000 companies that design and manufacture medical devices.
The process of designing and producing medical devices demands companies to fulfill many technical complexities and norms, such as IEC 62366 (Application to Usability Engineering to medical devices), IEC 60601(safety and ergonomic requirements for electronic medical products) for MDR and 21 CFR Part 11 for FDA.
What is more, the FDA and the EU MDR encourage companies to put people in the center of the design and evaluation process to create a safer medical product.
Why is usability testing necessary?
To monitor patients more closely, medical staff uses a vast number of medical devices. Errors during device usage can lead to injuries and even fatal consequences. In fact, more than 250,000 people in the United States die every year due to medical errors, according to a report by Johns Hopkins, which makes it the third leading cause of death after heart disease and cancer. The usage environment is also factor for possible patient harm for example interruption, disturbance or other environmental effects which affect the usage of the medical product.
That’s why safe use should become a no.1 priority for manufacturers. Incomprehensible or complex user interface can be blamed for many patient accidents. Also complex devices like ventilators, infusion pumps, defibrillators, and hybrid drug-device combinations can cause confusion and error during use. This can lead to a higher death toll and hazardous delays or difficulties in drug distribution.
Modern medical devices cover a vast number of patient and medic needs. Also, devices that are used by patients directly need to ensure for a good usability as those people are not trained in using medical devices and therefore might result in incorrect measurements or even put them at risk.
Formative and summative usability tests
In general, usability tests can be split into formative and summative. Although they are both concerned with measuring a user interface’s (UI) efficiency, these two kinds of evaluations do this from different viewpoints.
Formative usability assessments use qualitative techniques and provide observational insights. They are used early in the design process to identify the ease or complexity of design functionality. They serve as a guide to help render a more functional UI. The information they provide can shape and optimize the system’s architecture, pointing out problems and difficulties that can be solved to move forward with the iteration process.
On the other hand, summative usability tests are quantitative, serving as a final check that potential issues and problems have been detected and adequately resolved. They are required for MDR / FDA / CE approval. They also prove that the effect of recurring difficulty points is reduced to an appropriate degree. Manufacturers of medical devices must ensure that their medical devices are based on a user-first approach during this process. They should allow patients and medical staff to perform activities with little to no risk of error that could harm them.
Summative usability testing should determine design and usability problems and possible errors before the device is placed into the market. It can deliver evidence that validates marketing statements. Formative test findings are improved and confirmed by summative tests.
The three steps of summative testing
Preparation
Set up a team of engineers, marketers, regulators, and people with industry-specific skills to prepare a protocol and participate in the testing process. Besides this, you need to define potential use errors (tasks which could be potential harmful), define the acceptance criteria (for example error free rate of x% to pass the test), creating a test plan which will be signed by the client and identify gaps in the documentation. To make sure you get the most precise results, choose a company that specializes in usability testing.
Identify user groups you’re going to test, each consisting of at least 15 participants if you are working for the US market (FDA requirement) or 12 users per group if you are working for the European market (MDR requirement). We like to create Personas for the purpose of selecting user groups which helps to create the right participant screener and is helpful to have in mind for further product or device development. Then you perform a practice run of the test according to the protocol and, if necessary, modify it. Ideally, this will require a person that fits all the screener requirements, but it is not compulsory since you don’t really take any evidence for a report.
Testing
According to IEC 62366–1:2015, a usability test will be conducted. The method for best results is simulated-use testing, which will be used for this usability evaluation. This approach is used to study users interacting with the products and performing actual tasks. This involves the systematic collection of data from test participants using a product and product components in realistic use scenarios but under simulated conditions of use. In contrast to a cognitive walk-through, simulated-use testing allows participants to use the device more independently and intuitively. Simulated use testing can explore user interaction with the device overall, or it can investigate specific human factors considerations identified in the preliminary analyses, such as infrequent or particularly difficult tasks or use scenarios, challenging conditions of use, used by specific user populations.
The length of the test is determined by the number of tasks the individual should perform, and the time it takes to complete them. We mostly have a minimum test session of 60 minutes, depending on complexity of the product up to 90 minutes. Video recording each session can provide additional details you didn’t notice at the time of the test.
Analysis
In this step, the researcher analyzes the notes they’ve taken or watch the recordings they’ve made. Then, they create a report based on their findings and present it to the design team, who should work on further improvements based on the test results. The report is stored within the Usability Engineering File, which is part of the device’s FDA file and part of IEC 62366–1 Usability for engineering file.
Wrap up
By performing usability testing, companies can benefit from several aspects:
● Understand the user’s needs. Before designing a device, you should understand what motivates users to use the medical device in question. Covering another aspect, you can catch some details like difficulties with wearing the device, etc.
● Comprehend the use of medical devices in everyday situations. As medical staff and patients will interact with the device daily, understanding this interaction is crucial for good design. You can achieve this by interviewing them about their experience. For more details, you can take videos of their continued use of the device over an extended period of time to track changes and monitor their
behavior in their own home. Besides the product itself it is also important to include labeling and instruction for use (comprehensibility & readability).
● Create strong relationships with users. Good collaboration with users is crucial to product success. Companies will gain genuine insight into their products by investing in market analysis, strengthening relationships with consumers.
● Design a more effective marketing strategy. Usability testing can help organizations to identify which patients and healthcare providers resonate better with their communication and messages. They can use this feedback to create more personalized campaigns that will offer value to customers.
● Establish baseline user performance and user-satisfaction levels of the product for future product developments and evaluations.
Usability testing is a crucial component of product development, regardless of whether you’re studying products, supporting elements, or interactions. The advantages of usability testing are the same no matter what usability approach is used: to obtain actionable information that producers of medical devices can use to optimize their products, making them focused on users.
To perform precise and effective usability testing, get in touch with us.
