Measurement of Skin Induration Size Using Smartphone Images and Photogrammetric Reconstruction: Pilot Study

Introduction

Tuberculosis (TB) is one of the leading causes of death among infectious diseases worldwide. The disease progresses through a continuum of infection stages in individuals infected with Mycobacterium tuberculosis bacilli (MTB) from the latent to the active state. In latent tuberculosis infection (LTBI), the bacilli are largely dormant but can produce a detectable immune reaction. The LTBI state indicates previous infection and is strongly associated with an increased risk of progression to active TB, particularly in young children [1,2].

The most widely used method to detect LTBI is the tuberculin skin test (TST), a proxy measure for previous exposure to MTB [3]. TSTs are also used to monitor and estimate prevalence of TB infection in communities [4]. During the administration of a TST, tuberculin purified protein derivative is injected intradermally in the patient’s arm, approximately 3 to 4 inches below the elbow, and the result is assessed after a 48- to 72-hour period. This means that it is necessary for the patient to return to the health facility for assessment of the outcome. During the second visit, a clinician measures, using a ballpoint pen and ruler, and records the size of, the skin induration, if there is any; the result is classified as positive or negative based on consensus thresholds. One of the problems encountered by clinicians is that some patients who are subjected to the TST do not return to their respective health facilities after the specified time to have the result evaluated. Self-assessment of the outcome of TSTs has been explored as an alternative to the required follow-up visit [5-8] and, if accurate, could enhance TB screening efforts by reducing the number of subjects whose readings are not taken because of failure to return for skin test reading [6-8]. However, patients may require training in reading TSTs [5].

As an alternative to direct clinical measurement of TST indurations, we proposed a novel smartphone-based solution, which could capture images of the induration using a camera phone and send these to a central processing center where automated or manual analyses can be performed. Mobile phone usage, particularly smartphones, and global mobile network coverage have risen sharply within the past decade, particularly in developing countries where subscription tripled from 30% to 90% between 2006 and 2014 [9]. At the same time, there was a worldwide decrease in the cost of mobile broadband between 2013 and 2016, with developing and least developed countries experiencing higher reductions than developed countries [10]. These factors increasingly make telemedicine a realistic solution. Telemedicine involves the collection of medical data in one area and transfer to a central processing station for expert analysis; it stands to benefit from the advances in imaging and networking technologies for smartphones. Smartphones are now equipped with powerful cameras and processors, large screens, and various network capabilities [11].

In this study, we addressed the measurement of skin induration resulting from a TST using images captured on a smartphone, with a vision of leveraging the ubiquity of smartphone usage and the capabilities of such phones to enhance the screening for latent TB using TSTs. We assessed the feasibility of measuring induration size for a TST in three-dimensional (3D) scenes reconstructed from spatial images acquired using a smartphone based on agreement with the standard method.

Methods

Image Acquisition

Images were captured using the primary camera of a Samsung Galaxy S7 Edge (Samsung, South Korea) smartphone to produce a set of images per subject. This smartphone has a 12-MP primary camera with a 1.4-µm pixel size, 1/2.5-inch sensor size, and a f/1.7, 26-mm lens. During image capture, particular caution was exercised to ensure sufficient overlap of the regions in adjacent images. This overlap is essential for the 3D scene reconstruction.

Subjects for the study were recruited from postgraduate students and staff members in the Division of Biomedical Engineering at the University of Cape Town. Ethics clearance for the study was obtained from the Human Research Ethics Committee of the University of Cape Town (HREC REF: 250/2016). The purpose and procedure for this study were explained to all subjects who thereafter signed a form consenting to take part in the study. Instead of subjecting each participant to a TST, we used special effects makeup, applied by a professional make-up artist, to mimic a positive TST outcome (induration). We recruited participants with different skin tones, varying from dark to pale. Each mock induration was carefully tailored to mimic the expected appearance of a real induration for that particular skin tone. Examples of the mock indurations are shown in Figure 1. For each mock induration, between 7 and 10 images were captured ensuring coverage of approximately 120° around the arm with the smartphone camera set to autofocus mode. The 120° angle was sufficient to capture the data required for full 3D reconstruction of the mock induration. Before image acquisition, we placed a 10-mm scale bar on the arm, close to the mock induration, for the calibration of the measurements. After image acquisition, the induration size was manually measured by an observer who is experienced in reading TSTs, and these measurements were regarded as the reference standard for assessing the image-based measurements. The manual measurements were recorded by an independent observer to eliminate potential bias (trying to recall the manual measurements) and ensure independence when the experienced TST reader performed the image-based measurements.

‎View this figure

Three-Dimensional Reconstruction and Measurement

Images were transferred from the smartphone to a personal computer for processing using Agisoft PhotoScan (Agisoft LLC, Russia), a commercially available software package that performs photogrammetric processing of digital images. The program can generate 3D spatial data to enable indirect distance, area, and volume measurements of objects of various scales [12]. A mask was manually created for each image to ensure that only the relevant part (background cropped to leave only the arm) was used in the reconstruction. The first stage of 3D reconstruction is the search for and matching of common points on the input images; these points are used to estimate camera positions for each image. Successful completion of this stage requires points to be visible in at least 2 images, and therefore, there is a need for sufficient overlap in adjacent images. The software provides a fully automated workflow and subsequent stages include the refinement of the camera calibration parameters, building of the point cloud model, building of a polygonal mesh, and finally, building of texture. An example of this pipeline is shown in Figure 2. Identification of the scale bar in the reconstructed 3D scene was achieved by the manual placement of markers using mouse clicks. Measurements were subsequently made by placing markers on the arm to indicate the distance to be measured—on the border of the mock induration in the direction transverse to the length of the arm (Figure 3).

Evaluation

Marker placement for both the scale bar identification and the induration measurement was performed by the same observer who made the reference measurements. Additionally, an observer with no prior experience in reading TSTs also placed markers on the 3D arm models; this would allow assessment of the effect of experience on image-based measurements. The 2 observers repeated the image-based measurements so that the reliability of measurement could be assessed. Observers took the image-based measurements separately to avoid bias, and the second measurement for each observer was taken 7 days after the first one. Statistical analysis of the data was performed using the SPSS (IBM Corp, USA) software package.

‎View this figure

‎View this figure

Results

A total of 10 volunteers took part in the study. Mock induration size ranged between 4.0 and 19 mm as measured by the experienced observer according to the standard method used in clinical evaluation of TSTs. First, we studied the agreement between the readings taken by the experienced observer (observer 1) using the standard clinical method and the corresponding image-based measurements by the same observer. In this study, the precision of the clinical standard method was 1 mm (ie, measurements were rounded to the nearest millimeter where necessary), whereas the precision of the image-based method was higher than a thousandth of a millimeter. We also assessed the agreement between the clinical standard measurements and the image-based measurements performed by an inexperienced observer (observer 2), as well as the agreement between the two observers for image-based measurements. Tables 1 and 2 show the intraclass correlation coefficients (ICC) values and their corresponding 95% CI for the various measurements conducted by observers 1 and 2.

Tables 1 and 2 show that ICCs for both observers are well above 0.9, indicating excellent agreement between the observer versus the reference and for intraobserver measurements.

Table 3 shows excellent interobserver agreement with the ICCs higher than 0.9. The ICC shown is computed using the average of the two readings made 7 days apart, by each observer.

Discussion

Principal Findings

Mock skin indurations were used in this study, and their sizes were measured from 3D scenes of the arm reconstructed from planar images captured at several angles using a smartphone. The use of 3D scenery means that real 3D induration size is measured as opposed to planar distance from two-dimensional images. The high ICC values, which indicate excellent agreement between the measurements using the image-based method proposed in this study and those made using the current clinical method, suggest that the proposed method can potentially be used as an alternative to the standard clinical method. Moreover, high values for intra- and interobserver agreement for both experienced and inexperienced users indicate that experience in reading TSTs is not a requirement with the proposed method.

Our method provides the ability to read the results of the skin test at low cost, with high accuracy, in the community. An example where this would be useful is in low-resourced, high burden areas during mass screenings as the dependence on experienced personnel is reduced. In such cases, personnel with little experience or no experience in reading TSTs could be sent out to collect images for later processing. The method also addresses lack of experience in TST reading in low-risk countries. In high-income countries, which currently target a wider pool of patients, the method may also provide an alternative to clinician-based reading of results that reduces the number of TST results that are lost because of patients’ failure to return for the reading.

Results indicate that accurate self-assessment is possible, as patients in possession of an adequately capable smartphone could capture images of the skin induration and send it to a central server for expert processing and recording of induration size. With this approach, the follow-up visit that would normally be required for a TST becomes optional for patients whose induration size indicates a negative result. Self-assessment would increase convenience for the patient as images can be captured in their homes, without the assistance of a health practitioner, and would potentially reduce travel costs. This in turn would also reduce the number of TSTs lost because of patients’ failure to return for assessment. Self-assessment for TSTs has previously been tested with success. For example, subjects have been asked to interpret the outcome of a TST as flat or not with an aim of summoning those with a nonflat reaction for expert evaluation [7]. The sensitivity and specificity for the study were 99.5% and 97%, respectively, with all subjects providing an interpretation. In contrast, the method proposed in this study does not require the patient to make any decision but to merely take images of the site where the TST was administered. Furthermore, our proposed approach goes a step further to produce the measurement for induration size, and the results can be transmitted back to the patient’s phone with the potential to include appropriate messages to encourage attendance at a health care facility for those with a positive test.

A further advantage of the image-based method is that it measures induration size with higher precision than the current clinical standard method. In the standard method, a TST reader has to round off the induration size to the nearest millimeter. This requires the reader to make a judgment during the measurement, and this is a possible source of variation among TST readers that can cause misclassification (positive or negative) of measurements that are close to the threshold [13]. The higher precision of the image-based method is likely to reduce the misclassification rate.

Despite the relative ease with which observers could make measurements from the reconstructed 3D scenes, a few minor problems were encountered. First, the presence of shadows in some of the images made the placement of markers difficult. These shadows were a result of nonuniformity in the lighting in the images, as the camera moves from position to position during image capture. This can easily be addressed using image correction techniques before 3D reconstruction. In addition, despite close inter- and intraobserver agreement, the measurements are still observer-dependent. One way of overcoming this would be automated analysis, wherein an algorithm identifies the scale bar and the points between which the measurement is to be taken. These advances are the subject of an ongoing study.

We identified the calibration strip, which is placed on the arm to provide the scale, as a potential limitation for our method. Although the placement of the calibration strip does not affect results if it is clearly visible in the images, its physical integrity is crucial, and therefore, it requires careful handling by the patient. For example, patients who capture the TST images themselves should avoid bending the calibration strip or even losing it. Training patients on TST reading has previously been identified as a factor for successful implementation of self-assessment [5]; in our case, clinicians would need to educate the patient on the importance of maintaining the integrity of the calibration strip and demonstrate the image acquisition process including optimal placement of the calibration strip. In this ongoing study, we are developing a calibration technique in line with the requirement for automated analysis, which dispenses with the strip. Although mock indurations and a limited number of subjects were sufficient for the current proof-of-concept study, we envisage testing further iterations of our method using real indurations and a larger number of participants. Finally, in this study, we used a high-end smartphone equipped with a camera superior to those fitted on low-end phones that are expected to be more widespread in resource-limited regions. However, at this proof-of-concept stage, our goal was to show that images acquired with a smartphone can be used to measure TST induration size. Smartphone cameras have improved so much in the past few years that primary cameras on modern low-end phones have resolution of 5 MP or better. Furthermore, the 3D reconstruction software that we have used for processing the images works well with low-resolution images [12]. However, in the ongoing study, we plan to explore the effect of camera specifications on the results.

Conclusions

We have shown that a smartphone-based imaging solution has the potential to improve the efficiency of the global TB program by providing the ability to read the results of the skin test at low cost, with high accuracy, in the community. We envisage that the proposed method for induration measurement would enable and enhance latent TB infection screening in high burden, low-resourced regions as well as provide an alternative to clinician-based assessment in high-income regions. By taking advantage of the global ubiquity of smartphones and through further research, the use of smartphone cameras for image acquisition has the potential to bring this method to greater portions of the population through self-assessment and application of telemedicine.