This paper is in the following e-collection/theme issue:

Development of Novel Medical Devices and Innovations for Existing Devices (71) Mobile Health (mhealth) (2777) New Methods (911) Medical Devices for the Elderly (12) Novel Sensors and Data Acquisition Methods in Cardiology (191)

Published on in Vol 3, No 1 (2018): Jan-Dec

Auralife Instant Blood Pressure App in Measuring Resting Heart Rate: Validation Study

Auralife Instant Blood Pressure App in Measuring Resting Heart Rate: Validation Study

Auralife Instant Blood Pressure App in Measuring Resting Heart Rate: Validation Study

Authors of this article:

Timothy B Plante1 Author Orcid Image ;   Anna C O'Kelly2 Author Orcid Image ;   Bruno Urrea3 Author Orcid Image ;   Zane T Macfarlane4 Author Orcid Image ;   Lawrence J Appel5 Author Orcid Image ;   Edgar R Miller III5 Author Orcid Image ;   Roger S Blumenthal6 Author Orcid Image ;   Seth S Martin5, 6, 7 Author Orcid Image
Article Authors Cited by (1) Tweetations (2) Metrics

    Original Paper

    1Department of Medicine, Larner College of Medicine, University of Vermont, Colchester, VT, United States

    2Department of Medicine, Massachusetts General Hospital, Boston, MA, United States

    3Department of Medicine, MedStar Health Union Memorial Hospital, Baltimore, MD, United States

    4Department of Chemistry, Pomona College, Claremont, CA, United States

    5Johns Hopkins University, Welch Center for Prevention, Epidemiology, and Clinical Research, Baltimore, MD, United States

    6Ciccarone Center for the Prevention of Cardiovascular Disease, Johns Hopkins University School of Medicine, Baltimore, MD, United States

    7Malone Center for Engineering in Healthcare, Johns Hopkins University Whiting School of Engineering, Baltimore, MD, United States

    Corresponding Author:

    Timothy B Plante, MD, MHS

    Department of Medicine

    Larner College of Medicine

    University of Vermont

    360 S Park Drive

    Room 206B

    Colchester, VT, 05446

    United States

    Phone: 1 8026563688

    Fax:1 8026568965

    Email: timothy.plante@uvm.edu


    Background: mHealth apps that measure heart rate using pulse photoplethysmography (PPG) are classified as class II (moderate-risk) Food and Drug Administration devices; therefore, these devices need clinical validation prior to public release. The Auralife Instant Blood Pressure app (AuraLife IBP app) is an mHealth app that measures blood pressure inaccurately based on a previous validation study. Its ability to measure heart rate has not been previously reported.

    Objective: The objective of our study was to assess the accuracy and precision of the AuraLife IBP app in measuring heart rate.

    Methods: We enrolled 85 adults from ambulatory clinics. Two measurements were obtained using the AuraLife IBP app, and 2 other measurements were achieved with a oscillometric device. The order of devices was randomized. Accuracy was assessed by calculating the relative and absolute mean differences between heart rate measurements obtained using each AuraLife IBP app and an average of both standard heart rate measurements. Precision was assessed by calculating the relative and absolute mean differences between individual measurements in the pair for each device.

    Results: The relative and absolute mean (SD) differences between the devices were 1.1 (3.5) and 2.8 (2.4) beats per minute (BPM), respectively. Meanwhile, the within-device relative and absolute mean differences, respectively, were <0.1 (2.2) and 1.7 (1.4) BPM for the standard device and −0.1 (3.2) and 2.2 (2.3) BPM for the AuraLife IBP app.

    Conclusions: The AuraLife IBP app had a high degree of accuracy and precision in the measurement of heart rate. This supports the use of PPG technology in smartphones for monitoring resting heart rate.

    JMIR Biomed Eng 2018;3(1):e11057

    doi:10.2196/11057

    Keywords

    mHealth; digital health; heart rate; validation study; photoplethysmography; medical informatics; mobile phones 


    Pulse photoplethysmograms (PPGs) quantify circulation-related color changes in the vascular beds using optical sensors, and it can be used to measure heart rate [1]. Heart rate monitors use PPG are class II (moderate risk) Food and Drug Administration devices, and these devices must undergo clinical validation prior to their release [2]. Several consumer apps leverage the built-in camera and light to obtain PPG-measured heart rate from an illuminated body part (eg, finger). Most of these apps accurately measure heart rate compared with a standard device [3-5].

    The Instant Blood Pressure (IBP) app (AuraLife IBP app, AuraLife, Newport Beach, CA) is an mHealth app that measures blood pressure and heart rate using a smartphone with no additional sensors. Both measurements are presented simultaneously to the user. The app sold >148,000 copies and earned >US $600,000 in revenue within 13 months after its availability [6-8]. It was removed from the app store for unclear reasons in July 2015. We previously reported the inaccuracy of the AuraLife IBP app in measuring blood pressure compared with a validated oscillometric device. Moreover, its measurement process incorporated user-entered demographic and anthropomorphic data in determining blood pressure [6,9,10]. In multiple linear regression modeling of predicted systolic and diastolic blood pressure on user-entered data, sex, age, height, and weight of the participants accounted for only 12% and 12% of the variance in systolic and diastolic blood pressure results for the standard device, respectively, but 66% and 82% of the variability of these measurements for the AuraLife IBP app. The manufacturers of the AuraLife IBP app have raised concerns about our validation protocol and suggested that our findings could be because of hemodynamic changes resulting from the specific protocol [10,11]. These hemodynamic changes would also affect heart rate. However, the precision and accuracy of the app in measuring heart rate have not been reported.

    Determining the accuracy and precision of the AuraLife IBP app in measuring heart rate would provide valuable information for those using the AuraLife IBP app for this purpose and its technology in general. More importantly, this report would provide insight about the quality of the overall validation protocol that we used. The high levels of accuracy and precision of the AuraLife IBP app would support the quality of the overall assessment protocol and confirm the inaccuracy of the blood pressure measurement. Low accuracy or precision in obtaining heart rate could be due to protocol-related hemodynamic changes in patients (too much movement), not enough time between measurements, or app-related performance characteristics, and these would support the manufacturer’s concerns. Finally, it provides the opportunity to determine if the heart rate measurement was also inappropriately dependent on user-entered variables. Herein, we report the accuracy and precision metrics of the AuraLife IBP app in measuring heart rate and the variability of heart rate measurements accounted for by user-entered demographic and anthropomorphic data.


    Validation Protocol

    The methods of this validation protocol have been published elsewhere [6]. We prescreened participants aged ≥18 years for enrollment who were referred from 4 ambulatory clinics at Johns Hopkins University School of Medicine and an ambulatory research site. We excluded participants with contraindications to blood pressure measurement in both arms, internal devices (eg, pacemaker), active arrhythmias, height or weight values outside of the supported range by the AuraLife IBP app (height: <42 or >84 in and weight: <65 or >450 lbs), missing fingers, or inability to follow instructions. Prespecified rules have stated that participants with sequential systolic blood pressure measurements of >12 mmHg or diastolic blood pressure measurements of >8 mmHg (based on international validation guidelines [12]) will be dropped out from the study.

    The participants self-reported date of birth, sex, height, weight, race, ethnicity, highest level of education, history of hypertension, and receipt of antihypertensive medications. The research staff recorded the patient’s date of birth, sex, height, and weight into the app. The participants underwent 5 minutes of quiet sitting. Then, they had 2 pairs of blood pressure and heart rate measurements obtained from each device, of which the sequence was random. The standard devices used were the Omron 907 or Omron 907XL oscillometric noninvasive blood pressure and heart rate monitors, which had an heart rate measurement range of 30 to 199 BPM and an heart rate accuracy within 5% of the reading [13]. These devices were calibrated prior to the enrollment of the first participant. We used the AuraLife IBP app version 1.2.3 installed on an iPhone 5s and iPhone 6 running iOS version 8.3 (Apple Inc, Cupertino, CA), which has a reported heart rate measurement ranging from 39 to 240 BPM [14]. Measurements from the AuraLife IBP app and the standard device were separated by 60 seconds.

    Accuracy

    In accuracy analyses, we compared each individual heart rate measurement obtained using the AuraLife IBP app with the standard heart rate measurement, which was a mean of the heart rate measurements obtained using both standard devices. We calculated the mean relative difference and mean absolute difference between the AuraLife IBP app and standard equipment. Accuracy was visualized with scatterplot and Bland-Altman plot using a short dashed gray line to indicate the mean relative difference of the AuraLife IBP app minus standard and long dashed gray lines to indicate 2 SD.

    Precision

    For precision metrics, we subtracted the second app measurement from the first app measurement, calculating the mean relative difference and mean absolute difference between successive measurements for the same device. This was also performed for the standard device. Precision was visualized with paired coordinate plots. A black dashed line connected the mean of each reading for each device.

    Dependence on User-Entered Variables

    To assess the possibility of dependence of user-entered variables on heart rate obtained using the AuraLife IBP app, we repeated the same regressions that have been previously described [10], that is, we regressed the reported heart rate for the standard device and AuraLife IBP app on the age at the date of study enrollment, gender (male), height in inches, and weight in pounds. We interpreted the correlating R2 as the percentage of the dependent variable (heart rate from each device) explained by the independent variables. We compared R2 values of the AuraLife IBP app and those of the standard device. This study was approved by the institutional review board of the Johns Hopkins University School of Medicine. All analyses were performed with Stata MP 14.2 (StataCorp, College Station, TX).


    Validation Protocol

    In August 2015 and September 2015, we prescreened 105 individuals, of whom 4 did not meet the inclusion criteria of the study (active atrial fibrillation, n=1, or the presence of a pacemaker, n=3). Of the 101 enrolled patients, 3 were not included owing to standard device errors, and 13 were excluded owing to high variation in successive standard device systolic blood pressure measurements (n=7), diastolic blood pressure measurements (n=4), or both (n=2). We were unable to obtain blood pressure and heart rate results for 23 of the attempted 170 AuraLife IBP app measurements because the app encountered an error and was unable to produce a measurement. These were missing from the first pair in 5 individuals (n=5 measurements), the second pair in 4 individuals (n=4 measurements), and both pairs in 7 individuals (n=14 measurements). The complete pairs of the AuraLife IBP app measurements were obtained in 69 individuals.

    The mean (SD) age of the participants was 57 (16) years. Of these participants, 48% (41/85) were men. Moreover, 61% (52/85) were white, 28% (24/85) were black, and 9% (8/85) were Asian (Table 1). The range of the heart rate measurements obtained using the standard device was 46 to 94 BPM for the first measurement, 45 to 94 BPM for the second measurement, and 46 to 94 BPM for the standard measurement (mean of both measurements used in the accuracy analysis). The range of the measurements obtained using the AuraLife IBP app was 46 to 96 BPM for the first measurement and 45 to 99 BPM for the second measurement.

    Accuracy

    The mean relative difference for heart rate obtained using the AuraLife IBP app and standard device was 1.1 (3.5) BPM. The mean absolute difference was 2.8 (2.4) BPM (Table 2). The scatterplot showed a high correlation for heart rate measurements obtained using the AuraLife IBP app and standard devices with Pearson r=0.95 (P<.001), as seen in Figure 1. The Bland-Altman plot showed a nondifferential pattern across the means, as seen in Figure 2, with the dotted line indicating the mean and dashed lines representing 2 SD.

    Table 1. Baseline characteristics.
    DemographicsParticipants (n=85)Range
    Age (years), mean (SD)57 (16)18-81
    Male, n (%)41 (48)
    White, n (%)52 (61)
    Black, n (%)24 (28)
    Asian, n (%)8 (9)
    Hispanic ethnicity, n (%)4 (5)
    Body mass index (kg/m2), mean (SD)28 (6)18-51
    Standard systolic blood pressure (mm Hg), mean (SD)126 (17)92-170
    Standard diastolic blood pressure (mm Hg), mean (SD)70 (11)32-100
    Standard heart rate (beats per minute), mean (SD)68 (11)46-94
    Hypertension, n (%)45 (53)
    On medication, n (%)41 (91)
    Owns a smartphone, n (%)71 (84)
    Table 2. Mean difference between devices.
    Difference (beats per minute)Mean (SD) differenceRange
    Relative1.1 (3.5)−9 to 12
    Absolute2.8 (2.4)0 to 12
    ‎
    Figure 1. Accuracy visualization: Scatterplot of AuraLife Instant Blood Pressure (IBP) app heart rate measurements versus the standard heart rate measurements. BPM: beats per minute.
    View this figure
    ‎
    Figure 2. Accuracy visualization: Bland-Altman plot for the AuraLife Instant Blood Pressure (IBP) app heart rate measurements and the standard measurements. BPM: beats per minute.
    View this figure
    Table 3. Precision metrics for each device.
    DeviceMean (SD) differenceRange
    Standard device (n=85)

    		Relative, beats per minute0.01 (2.2)−6 to 5

    		Absolute, beats per minute1.7 (1.4)0 to 6
    AuraLife IBP App (n=69)

    		Relative, beats per minute−0.1 (3.2)−13 to 7

    		Absolute, beats per minute2.2 (2.3)0 to 13

    Precision

    For the AuraLife IBP app, the mean relative difference between the 69 pairs of measurements was −0.1 (3.2) BPM, whereas the mean absolute difference was 2.2 (2.3) BPM (Table 3). For the standard device, the mean relative difference between the 85 pairs of measurements was 0.01 (2.2) BPM. Meanwhile, the mean absolute difference was 1.7 (1.4) BPM. The paired coordinate plot showed minimal variability between the first and second measurements from each device, as seen in Figure 3, with dashed lines representing mean values.

    Dependence on User-Entered Variables

    The regression of user-entered demographic and anthropomorphic data on the reported heart rate obtained similar R2 values for the standard device and the AuraLife IBP app (Table 4). The independent variables accounted for 19% and 16% of the heart rate variability for the standard device and AuraLife IBP app, respectively.

    ‎
    Figure 3. Precision visualization: Paired coordinate plot for AuraLife Instant Blood Pressure (IBP) app and the standard device. BPM: beats per minute; HR: heart rate.
    View this figure
    Table 4. Regression coefficients.
    DeviceRegression coefficients
    Age (years)MaleHeight (inches)Weight (lbs)ConstantR2
    Standard heart rate−0.24−5.74−0.040.0774.680.19
    AuraLife Instant Blood Pressure app heart rate−0.23−5.730.050.0573.110.16

    Principal Findings

    In this validation study of heart rate measurements obtained using an mHealth app that was previously found to be inaccurate in measuring blood pressure, we revealed that the AuraLife IBP app has a high degree of accuracy and precision for the measurement of heart rate. The heart rate measurement had a similar amount of demographic or anthropomorphic information as the standard device, which is minimal.

    Implications

    Prior assessments of heart rate measuring-devices have generally been positive, and one meta-analysis has reported a pooled correlation coefficient of 0.95 [15], which is identical to that observed in this validation protocol. The use of these apps for the detection of heart rate in the context of arrhythmia is limited. Unfortunately, our protocol intentionally excluded participants with arrhythmias because the standard device is not validated to obtain heart rate or blood pressure measurements in individuals with arrhythmias.

    The accuracy of the heart rate measurements with the AuraLife IBP app provides reassurance that our study protocol did not induce hemodynamic changes, which could have potentially biased the results of our blood pressure validation study. This was a concern raised by the app developers. Hence, results from this validation study of heart rate provide an indirect support to our previous blood pressure validation study about the AuraLife IBP app. Therefore, our protocol may be useful for other researchers interested in comparing the performance characteristics of mHealth apps with those of a validated oscillometric device.

    Limitations

    Because the protocol was primarily designed as a blood pressure validation study, all measurements were obtained at rest and had limited range. Whether the accuracy or precision of the AuraLife IBP app in measuring heart rate will change with exercise or for individuals with resting values at greater extremes is unclear. We did not include individuals with arrhythmias. Thus, whether the performance metrics will be similar among these patients is unclear. Although the standard device is widely used in clinical practice for measuring heart rate and blood pressure, it is not a conventional standard for heart rate or blood pressure measurement in clinical studies. Future studies that use a more conventional method, such as electrocardiography, or those that include trained observers who will use a random-zero sphygmomanometer must be conducted. Finally, we did not test the Android version of the app.

    Conclusions

    The AuraLife IBP app has high accuracy and precision in measuring heart rate in adult ambulatory patients. This further supports the use of PPG technology in smartphones for monitoring resting heart rate.

    Acknowledgments

    We thank Jeanne Charleston, PhD BSN for her assistance in providing training and standard device management; Morgan Grams, MD PhD MHS, Satish Misra, MD, and Haitham Ahmed, MD in their assistance in trial design. We thank the clinical staff at the Johns Hopkins General Internal Medicine, Cardiology, Nephrology, and ProHealth sites for assistance in study execution. We thank the Johns Hopkins legal team for guidance on interactions with AuraLife. This study was supported by the PJ Schafer Cardiovascular Research Grant. TBP was funded by the Institutional National Research Service Awards from the Health Resources and Services Administration (T32HP10025B0) and the National Heart, Lung, and Blood Institute (2T32HL007180-41A1).

    Conflicts of Interest

    SSM reports research support from Google, Apple, Nokia, iHealth, and the Aetna Foundation. There are no other relevant conflicts of interest.

    1. Elgendi M. On the analysis of fingertip photoplethysmogram signals. Curr Cardiol Rev 2012 Feb;8(1):14-25 [FREE Full text] [Medline]
    2. US Food & Drug Administration. Cardiac Monitor Guidance (including Cardiotachometer and Rate Alarm) - Guidance for Industry   URL: https://www.fda.gov/MedicalDevices/ucm073940.htm [accessed 2018-05-15] [WebCite Cache]
    3. Mitchell K, Graff M, Hedt C, Simmons J. Reliability and validity of a smartphone pulse rate application for the assessment of resting and elevated pulse rate. Physiother Theory Pract 2016 Aug;32(6):494-499. [CrossRef] [Medline]
    4. Yan BP, Chan CK, Li CK, To OT, Lai WH, Tse G, et al. Resting and Postexercise Heart Rate Detection From Fingertip and Facial Photoplethysmography Using a Smartphone Camera: A Validation Study. JMIR Mhealth Uhealth 2017 Mar 13;5(3):e33 [FREE Full text] [CrossRef] [Medline]
    5. Poh M, Poh YC. Validation of a Standalone Smartphone Application for Measuring Heart Rate Using Imaging Photoplethysmography. Telemed J E Health 2017 Aug;23(8):678-683. [CrossRef] [Medline]
    6. Plante TB, Urrea B, MacFarlane ZT, Blumenthal RS, Miller III ER, Appel LJ, et al. Validation of the Instant Blood Pressure Smartphone App. JAMA Intern Med 2016 May 01;176(5):700-702. [CrossRef] [Medline]
    7. Federal Trade Commission. Marketers of Blood-Pressure App Settle FTC Charges Regarding Accuracy of App Readings   URL: https:/​/www.​ftc.gov/​news-events/​press-releases/​2016/​12/​marketers-blood-pressure-app-settle-ftc-charges-regarding [accessed 2018-05-15] [WebCite Cache]
    8. Plante TB, O'Kelly AC, Macfarlane ZT, Urrea B, Appel LJ, Miller III ER, et al. Trends in user ratings and reviews of a popular yet inaccurate blood pressure-measuring smartphone app. J Am Med Inform Assoc 2018 Aug 01;25(8):1074-1079. [CrossRef] [Medline]
    9. El Assaad MA, Topouchian J, Darné BM, Asmar RG. Validation of the Omron HEM-907 device for blood pressure measurement. Blood Press Monit 2002 Aug;7(4):237-241. [Medline]
    10. Plante TB, Appel LJ, Martin SS. Critical Flaws in the Validation of the Instant Blood Pressure Smartphone App-A Letter from the App Developers-Reply. JAMA Intern Med 2016 Sep 01;176(9):1410-1411. [CrossRef] [Medline]
    11. Archdeacon R, Schneider R, Jiang Y. Critical Flaws in the Validation of the Instant Blood Pressure Smartphone App-A Letter From the App Developers. JAMA Intern Med 2016 Dec 01;176(9):1410. [CrossRef] [Medline]
    12. Graves J, Quinn D. ISO 81060-2:2013. Non-invasive sphygmomanometers - Part 2: Clinical investigation of automated measurement type. ANSI/AAMI/ISO; 2013.
    13. HEM-907XL Professional Digital Blood Pressure Monitor. Omron Healthcare Wellness & Healthcare Products   URL: https://omronhealthcare.com/wp-content/uploads/hem-907xl_product_guide1.pdf [accessed 2018-07-05] [WebCite Cache]
    14. Instant Blood Pressure Website. 2016. The App   URL: https://web.archive.org/web/20160201161554/http://www.instantbloodpressure.com/the-app/ [accessed 2018-07-06] [WebCite Cache]
    15. De Ridder B, Van Rompaey B, Kampen JK, Haine S, Dilles T. Smartphone Apps Using Photoplethysmography for Heart Rate Monitoring: Meta-Analysis. JMIR Cardio 2018 Feb 27;2(1):e4. [CrossRef]

    ‎
    BPM: beats per minute
    IBP: Instant Blood Pressure
    PPG: pulse photoplethysmogram

    Edited by G Eysenbach; submitted 15.05.18; peer-reviewed by J Goris, S Omboni; comments to author 21.06.18; revised version received 06.07.18; accepted 23.07.18; published 21.11.18

    Copyright

    ©Timothy B Plante, Anna C O'Kelly, Bruno Urrea, Zane T Macfarlane, Lawrence J Appel, Edgar R Miller III, Roger S Blumenthal, Seth S Martin. Originally published in JMIR Biomedical Engineering (http://biomedeng.jmir.org), 21.11.2018.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Biomedical Engineering, is properly cited. The complete bibliographic information, a link to the original publication on http://biomedeng.jmir.org/, as well as this copyright and license information must be included.