Usability and Practicality of a Novel Mobile Attachment for Aural Endoscopy (endoscope-i): Formative Usability Study

Introduction

Background

Otological diseases make up a large proportion of global disease. In 2015, the Global Burden of Disease Project ranked otitis media as the third-most common short-term disease with an incidence of 471 million worldwide [1]. Furthermore, hearing loss was ranked the fourth-most common chronic disease, affecting over 1 billion people. More recently, in 2017, otitis media had a reported incidence of 318 million cases [2].

The World Health Organization has noted a discrepancy between burden of disease and current resources for many specialties including otology, encouraging advances in technology and telemedicine to help bridge this gap around the world [3]. Recently, with the continual rise of availability of smartphones, mobile apps and attachments have become a way to achieve this, with some apps showing promise for streamlining referrals to ear, nose, and throat (ENT) specialists [4,5]. For example, Biagio showed substantial agreement in otological diagnoses between face-to-face assessments with an otoscope and remote viewing of videos of the findings, termed video-otoscopy [6]. This may help alleviate the aforementioned imbalance of resources by facilitating specialist expertise in remote areas. However, this is dependent on the speed of transfer of the video and the image quality [7]. The key aim of this study was to assess the usability and practicality of a novel otological technology, the endoscope-i, by gaining professional opinions on the effectiveness and comprehensiveness of the system when compared to the current standard, the otoscope (full aims are mentioned in the Test Objectives section). This study explores its potential for improving patient care [8].

The endoscope-i

Overview

During an interview conducted on June 14, 2019, the endoscope-i representative corroborated the following information. The endoscope-i system comprises an adapter designed for a smartphone to be attached to an endoscope, in order to look into a patient’s ear with a wide-angle lens, using a short rigid scope. The system is already commercially available and 3500 have been sold internationally online. The system comes packaged with instructions for use, including an explanatory diagram. Previous trials have been undertaken in Staffordshire, UK, with primary care clinicians using a slightly modified version of the current app. A key group of targeted users of the endoscope-i are audiology staff, nurses, audiologists, general practitioners, health care assistants, and community nurses. Once trained in using the device, it is intended that these users will be able to send the image obtained to an ENT surgeon for assessment of the ear. Therefore, the operator only needs to be skilled in using the device, with appropriate anatomy and physiology knowledge, and not in aural clinical assessment. This would then negate the need for all aural patients to see an ENT surgeon in person and, in turn, could potentially reduce waiting lists.

Differences and Innovations When Compared to the Traditional Otoscope

Firstly, the endoscope-i is attached to a smartphone camera that can be used to view the external auditory canal and tympanic membrane, once it is attached to the scope via the adapter, thereby producing a better image than an otoscope. An app is then used to optimize the image. These high-quality images will also only improve further with the advance in technology and camera quality of mobile phones. An app is then used to optimize the image. Secondly, in current practice, the eye needs to be close to the ear in order to see through the otoscope. With the endoscope-i system, an eyepiece sits on the adapter to view the image, allowing greater distance between the patient and clinician for easier working practice and improved view. This shared image may also be beneficial in the training of medical professionals. In addition, the wide-angle lens of an endoscope produces a broader perspective of the tympanic membrane not afforded by conventional otoscopes.

Another strength of the endoscope-i is its ability to capture high-quality photos and videos. These can be shown to the patient for reassurance and explanation of pathology, or they can be used for a remote referral, such as a junior doctor sending it to a senior colleague or a general practitioner sending it to an ENT surgeon in a distant location. This may enable a specialist management plan to be given for a patient, which could in turn eliminate the need to see the ENT specialist in person, further reducing waiting lists.

Finally, the introduction of the endoscope-i introduces the possibility of simple interventions alongside vision in the future, similar to the mainstream use of endoscopes in major operations, such as in endoscopic tympanoplasty [9].

Cons of the endoscope-i

With a new instrument or device comes a learning curve and a need for learning a new technique. Training is required to utilize the app and adapter but is thought to be minimal, with practice allowing for familiarity with its use. This assumption is tested as one of the aims of this study. The larger part of the training is anticipated to be for the skill of oto-endoscopy itself, which was also assessed. Staff who are not specialized in ENT, audiology, or endoscopy will require initial training in endoscopy, which is more time-consuming than training in only the endoscope-i device.

Further drawbacks are the risks associated with using the device; these include a perforated eardrum at the severe end of the scale and pain on insertion of the endoscope. Significant complications are thought to be rare, with none reported thus far.

Test Objectives

The aims of this formative usability study were to assess the extent to which the endoscope-i could demonstrate effectiveness, efficiency, novel advances, and satisfaction in aural examination on patients attending an outpatient clinic as a part of the clinical assessment process, as per the International Organization for Standardization (ISO) standard, ISO 9241-11 [10]. This study is part of the iterative product development process, and it is different from a summative usability study, which would be conducted for validation purposes [11].

The specific objectives were to gain professional opinions on using the endoscope-i. Specifically, these objectives were as follows:

1. To assess its ability to conduct an aural examination: to be able to assemble the equipment, insert the endoscope into the ear, view the ear drum, and remove the endoscope.
2. To assess its feasibility for integrating its use into clinical practice.
3. To assess its ease and practicality over current practices with otoscopes.
4. To confirm or deny the ability of the device to produce an image of sufficient quality to make a clinical assessment.
5. To assess the usefulness of the current training given for using the device.
6. To gain insights into future recommendations for the product from the targeted specialists.

Methods

Ethics Approval

The project received institutional ethical approval. All participants consented to taking part in the usability study and did so voluntarily.

Participants

The study was advertised via emails and a poster circulated through the audiology and research and development departments within the University of Birmingham and the University Hospitals Birmingham National Health Service (NHS) Foundation Trust as well as through correspondence with professional contacts within the ENT department. All recruits were required to be trained in the skill of endoscopy or otoscopy in order for them to be able to compare it to their current practice. A total of 5 volunteers participated in this study, among which were a lead research nurse, a consultant ENT surgeon, an audiology clinical scientist, an endoscopy nurse, and the head of audiology; these volunteers represented a broad range of experience levels. All participants currently work for the University Hospitals Birmingham NHS Foundation Trust; their roles are listed in Table 1. Information on the endoscope-i was not revealed prior to the study, although 2 of the volunteers had previously seen and used the device through their own professional contact with the developer. The advertising email and flyer mentioned the scope of the study, including the task involved (eg, using an adapter for a smartphone to be attached to an endoscope in order to perform an aural examination).

Tasks and Test Design

A formative usability study took place on June 14, 2019, in the Medical Device Testing and Evaluation Centre (MD-TEC) simulation suite at the Institute of Translational Medicine, Birmingham, on commission of the company. Results were collected, analyzed, and presented independent of the company. The project was fully funded by the European Regional Development Fund. A team of three planned and moderated the study. The study procedure was built around three core elements: users, user environment, and user experience of the device interface [12].

The conducted usability study was structured around a task-based scenario: mannequin heads with accessible ear canals designed for audiology training were used in a mock audiology clinic setting; this can be seen in Figure 1. An endoscope-i representative assisted on the day, giving a training session and a demonstration of the device. The participants were then presented with the device in its case and asked to use it as if they would in a clinical assessment of an ear canal.

The tasks involved in the hands-on session included the following:

1. Take the device out of the box and assemble it according to the instructions.
2. Set the device up to record.
3. Insert the device into the ear.
4. Confirm visual image of the ear canal on the phone.
5. Remove the device from the ear canal.
6. Utilize the app to send the image to a clinician.
7. Dissemble the device and replace it into the box.

For the scenario, the participant acted as the assessing clinician. The company representative was available for issues and queries during the hands-on session, guiding participants if they required, considering it being a formative study [12]. There was also a member of the MD-TEC team present to facilitate the study. The hands-on session was recorded by a high-definition camera on a tripod—with exception of participant 4 who wished not to be recorded—operated by a member of the NHS Foundation Trust’s A/V (audio/video) team. Immediately after the session, a voice recorder was used for the interviews.

‎View this figure

Results

Usability Metrics

Task Completion Success Rate

All participants were able to complete the tasks of assembling the device, carrying out an aural endoscopy, obtaining an image, and dissembling the device; therefore, there was a completion rate of 100% (5/5) for the representative end users. Participants expressed that this was an excellent instrument to obtain a high-quality image that is superior to the current image quality from otoscopes. From the feedback given in this usability study, it is deemed that the relevant clinicians would easily and gratefully adopt this device.

There were no significant errors made during the task-based session of the usability study. On two occasions, it was necessary for the company representative to step in and assist the participant with an aspect of using the device (EI1 and EI4); however, following this, the procedure was then completed completely independently by the participants, suggesting a high error tolerance. Bearing in mind, one of these participants had no prior experience in aural assessment; this indicates that operating the device would be easy to learn for those already assessing ears in practice. Further usability testing would be required to assess the use of the device by lesser-trained health care professionals. All participants (5/5, 100%) were able to complete all seven tasks within 5 minutes, with an average of 3.5 minutes, suggesting it was efficient as well as easy to use.

Usability Overview

All the participants were able to successfully use the endoscope-i and gave positive responses, with comments such as “The instrument itself is excellent” and “Perfect for a busy clinic” (EI1) being made. It was described as “small, compact, clear” (EI1) and was recognized as a “different way of examining a patient,” as there is “no need to get as close [to the patient]” (EI2), but it was also thought of as not difficult to get used to (EI2 and EI4). All participants (5/5, 100%) showed a good awareness of the endoscope-i’s use and purpose, regardless of their previous experience levels of aural examinations, and all seemed keen to integrate it into practice. Their feedback is highly relevant, considering their previous use of alternatives and experience levels in clinical assessments, communication between departments, and transmission systems within the primary and secondary care infrastructure. Figure 1 shows the endoscope-i in use at the point of contact with the patient.

Ease of Use of the Device

Ease of use was specifically named as one of the positive aspects of the device by 2 out of 5 (40%) participants (EI3 and EI5), with Participant EI1 also complimenting the simplicity of putting the parts together. Additionally, all 5 (100%) participants found the endoscope-i easy to use, with phrases such as “easier than anticipated” (EI1) being used to describe it by a participant with no previous experience of the device or of aural examinations and “pretty straightforward” said by a participant who had used a version of it in the past. It is also important to mention that Participant EI3 claimed it was “easy to do even with arthritic fingers.”

Image Quality

One of the advantages of this device over the current practice of using an otoscope was reported as the high-quality image, which was highlighted by all (5/5, 100%) participants. One participant (EI2) described it as “vastly superior” and went on to explain that, in addition to the better quality, this was because the point of vision is next to the ear drum, and the device provides the ability to look around corners to see the full ear drum. They also pointed out that there is also the opportunity to show the image to the patient and the relatives. Storing the actual image was seen as a notable advantage by 4 out of 5 (80%) participants (EI1, EI2, EI4, and EI5) (ie, “the recording element is a good idea”).

The important aspect of image quality was whether it would be of sufficient quality to make a clinical assessment from. The 4 participants out of 5 (80%) (EI2, EI3, EI4, and EI5) with experience in otoscopy or endoscopy positively confirmed that it would be of sufficient quality; the 1 (20%) participant (EI1) without experience said she guessed it would be, taking into account the vast experience she does have in clinical care. It was even described by one participant (EI4) as “far exceed[ing] current image quality.” The image quality can be seen in Figure 1.

Comparison to Current Practice

In comparison to current practice, it was seen as different because, as Participant EI4 explained, there are “clear guidelines on safety and bracing against a patient’s head, which is not able to be done with the endoscope-i.” In addition, Participant EI5 stated that there is “a knack to using it which is different to an otoscope,” which would mean that frequent use would be required to maintain an adequate skill level.

Training Required

The opinions on training needs for the endoscope-i varied between an ongoing competency document for the trainee to complete while being assessed by a competent user in a clinic (EI1), face-to-face group sessions (EI2 and EI4), and video teaching (EI3 and EI5). By mentioning that it would need to be used regularly to gain the specific technique, one participant’s answer was in correlation to Participant EI1 suggesting a competency document. After the initial training, a degree of experience would be required, and the user would need to use the device regularly in order to become fully competent and comfortable with it, according to Participant EI5. Another participant (EI4) also posited that there could be a divide in the acceptance and ease of adaptation of learning between older staff and younger staff due to the difference in familiarity with technology.

For those who suggested group face-to-face training, the emphasis was on having the device in front of the trainees so they could be talked through the process, try it themselves, and have a trainer available for assistance (EI2 and EI4). Those staff who were already trained in endoscopy would require less training than those who were not, since, for those who were inexperienced, “training would be around endoscopy of the ear as a new skill” (EI2) (eg, passing an endoscope through the ear canal without damaging it), in addition to using the adapter and app. A review of anatomy would also be useful, as the user will be getting a totally different view of the ear drum (EI2). Table 2 summarizes the methods of training that were recommended.

iPhone Versus Android

With regard to the app only being compatible with iPhones, it was explored whether this would present an issue for some users. Out of the 5 participants, 2 (40%) (EI1 and EI3) did consider that this may be difficult and would cause some confusion to navigate initially, but Participant EI4 reported that no difficulties were encountered as an Android user.

Design Feedback

The participants were asked to comment on the positive and negative aspects of the device. These responses are expanded upon below and summarized in Table 3.

In terms of assembling the device, it was recorded to be “relatively straightforward” and very easy to put the light source and endoscope together and screw in the battery pack onto the endoscope. However, 2 out of 5 (40%) participants (EI1 and EI2) reported having difficulty with assembly, both assembling the device and the phone in a different order. Participant EI1 found it challenging to clip the adapter, with the endoscope already in place, onto the phone, and Participant EI2 had trouble attaching the endoscope onto the adapter that clips onto the phone. Participant EI2 explained “the weight of the battery makes it very awkward.” Both made suggestions on how to resolve this: Participant EI1 laid the phone flat on the table to then screw it in place, and Participant EI2 attempted changing the order of assembly to attach the heavy battery pack last. It was also stated that the screw-fix to attach the endoscope is fiddly, and Participant EI2 suggested that if the endoscope could clip in to the adapter that would be easier; however, it was recognized that perhaps the screw-fix is the most secure fixation to hold the weight of the phone and battery.

For the software app itself, suggestions were made by Participant EI2 to change the recording and saving buttons so they are both accessible to the thumb, and to switch around the controls such as the focus button, in order for the focus to be easier to access, irrespective of which side the phone is being held on. The app was not found to be intuitive by 1 out of 5 (20%) participants (EI4), because the user is required to flick between the screens and twist it upside down to activate. This participant also found the user position or handhold challenging, as it was “cumbersome to hold” compared to current practice; this view was shared by Participant EI5, positing that it may require practice and experience to find the best way to hold it. Out of the 5 participants, 1 (20%) (EI4) had safety concerns about the distal end entering a patient’s ear when used by junior or inexperienced staff. This could be addressed through the ongoing practical competency element of the training. A final query by a separate participant (EI3) was around storing confidential data in the form of images on the phone. However, the ability to view the images repeatedly after the clinical assessment and share them was also seen to be of benefit.

As a guide for further development with the user in mind, the participants were also encouraged to make usability and design recommendations, resulting in some useful design and handling ideas being proposed. These have been separated into two separate categories of assembly and handling and interface recommendations for ease of assessment; these are all listed in Textbox 1.

The recommendation of adding the rubber bungs to the part entering the patient was offered by 2 out of 5 (40%) participants (EI3 and EI4), due to the consideration that there may be some anxiety attached to the use of an endoscope on a live patient for the first few times by junior staff. Using a speculum on the end of the endoscope was also suggested for the same reason (EI4).

Discussion

Principal Findings

This formative usability study conducted by MD-TEC revealed a high acceptance rate of the endoscope-i, including its concept and method of use. Based on the study, as well as participant feedback, the current design is suitable for the intended purpose and its targeted end users, although some adjustments to the user interface and assembly order of the device may be considered. Furthermore, all participants perceived the device as having superior image quality, inducing a direct positive impact on their usual clinical practice, and having potential to reduce waiting lists and implicitly improve patient safety.

Despite the number of volunteers to the study (N=5), the wide range of experience of the participants was deemed a strength. In particular, one participant was not familiar with aural assessment and had not seen the endoscope-i, two participants were familiar in otoscopy but had not seen the endoscope-i, and two participants were familiar with otoscopy and had previously used the endoscope-i. Those who were familiar with otoscopy could compare their own experience directly against the use of this device; the one that was not trained in otoscopy could be used to see how easy it was for a complete novice to learn to use the device. Regardless of prior experience, a 100% completion rate within 5 minutes (averaged at 3.5 minutes) suggests that the device is easy to learn and use. Another favorable aspect was in the setup of the testing scenario: the use of a mannequin ensured patient safety, the room was designed to mimic clinical practice, and the setting remained uniform across each testing session to reduce variables between participants. The use of different types of feedback (ie, verbal, written, and video) gave a greater level of insight into the device’s usability, the suggestions of which are mentioned in the concluding remarks. The videos, for example, revealed a learning curve where participants initially fumbled to assemble the device, yet soon figured out how to complete the task. In this study, these participants had live training preceding their test; however, other users apart from this study will likely not have this luxury. Therefore, as 80% (4/5) of the participants agreed, a written, annotated, and/or video guide would be extremely helpful for overcoming this learning curve, to use as a reference when using the device; note that this aforementioned reference was not present during the live test for participants in this study, which could have contributed to the fumbling.

Nevertheless, in spite of realistically replicating the use scenario, there are some well-acknowledged limitations of usability studies, such as “testing is always an artificial situation”; personal preferences are only those of the participants and are not representative of the entire user population [13]. In addition, the use of retrospective think-aloud interviews were a good asset to reveal further insights into the usability of the device.

Conclusions

The endoscope-i is a novel device and system that allows the user to take high-quality videos of the eardrum and canal with an improved angle compared to the current otoscope. This study shows that it would enable an aural assessment to be carried out from the image it produces, with the feature to save and export the image allowing for remote examination of patients. This may reduce patient waiting lists to see a specialist in person. On usability testing, the study further found the device to be efficient, effective, error tolerant, and easy to learn. This study did find that further exploration in the following areas might help with the user’s engagement:

1. Explore different options of assembling the device parts and the possibility of a lighter battery pack to assist with this.
2. Consider locations of buttons on the screen in the app interface.
3. Evaluate the possibility of a feature to upload the images directly to the patient notes on an electronic patient record or to print for paper notes.