<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE article PUBLIC "-//NLM//DTD Journal Publishing DTD v2.0 20040830//EN" "http://dtd.nlm.nih.gov/publishing/2.0/journalpublishing.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink" article-type="review-article" dtd-version="2.0">
  <front>
    <journal-meta>
      <journal-id journal-id-type="publisher-id">JMBE</journal-id>
      <journal-id journal-id-type="nlm-ta">JMIR Biomed Eng</journal-id>
      <journal-title>JMIR Biomedical Engineering</journal-title>
      <issn pub-type="epub">2561-3278</issn>
      <publisher>
        <publisher-name>JMIR Publications</publisher-name>
        <publisher-loc>Toronto, Canada</publisher-loc>
      </publisher>
    </journal-meta>
    <article-meta>
      <article-id pub-id-type="publisher-id">v7i2e42386</article-id>
      <article-id pub-id-type="pmid">38875684</article-id>
      <article-id pub-id-type="doi">10.2196/42386</article-id>
      <article-categories>
        <subj-group subj-group-type="heading">
          <subject>Review</subject>
        </subj-group>
        <subj-group subj-group-type="article-type">
          <subject>Review</subject>
        </subj-group>
      </article-categories>
      <title-group>
        <article-title>Detection of Suicide Risk Using Vocal Characteristics: Systematic Review</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="editor">
          <name>
            <surname>Mavragani</surname>
            <given-names>Amaryllis</given-names>
          </name>
        </contrib>
      </contrib-group>
      <contrib-group>
        <contrib contrib-type="reviewer">
          <name>
            <surname>Upadhyaya</surname>
            <given-names>Pulakesh</given-names>
          </name>
        </contrib>
        <contrib contrib-type="reviewer">
          <name>
            <surname>Teles</surname>
            <given-names>Ariel</given-names>
          </name>
        </contrib>
      </contrib-group>
      <contrib-group>
        <contrib id="contrib1" contrib-type="author" corresp="yes">
          <name name-style="western">
            <surname>Iyer</surname>
            <given-names>Ravi</given-names>
          </name>
          <degrees>BSc, MSc</degrees>
          <xref rid="aff1" ref-type="aff">1</xref>
          <address>
            <institution>Centre for Mental Health</institution>
            <institution>Swinburne University of Technology</institution>
            <addr-line>34 Wakefield Street</addr-line>
            <addr-line>Hawthorn, 3122</addr-line>
            <country>Australia</country>
            <phone>61 456565575</phone>
            <email>raviiyer@swin.edu.au</email>
          </address>
          <ext-link ext-link-type="orcid">https://orcid.org/0000-0001-7699-0846</ext-link>
        </contrib>
        <contrib id="contrib2" contrib-type="author">
          <name name-style="western">
            <surname>Meyer</surname>
            <given-names>Denny</given-names>
          </name>
          <degrees>PhD</degrees>
          <xref rid="aff1" ref-type="aff">1</xref>
          <ext-link ext-link-type="orcid">https://orcid.org/0000-0002-9902-0858</ext-link>
        </contrib>
      </contrib-group>
      <aff id="aff1">
        <label>1</label>
        <institution>Centre for Mental Health</institution>
        <institution>Swinburne University of Technology</institution>
        <addr-line>Hawthorn</addr-line>
        <country>Australia</country>
      </aff>
      <author-notes>
        <corresp>Corresponding Author: Ravi Iyer <email>raviiyer@swin.edu.au</email></corresp>
      </author-notes>
      <pub-date pub-type="collection">
        <season>Jul-Dec</season>
        <year>2022</year>
      </pub-date>
      <pub-date pub-type="epub">
        <day>22</day>
        <month>12</month>
        <year>2022</year>
      </pub-date>
      <volume>7</volume>
      <issue>2</issue>
      <elocation-id>e42386</elocation-id>
      <history>
        <date date-type="received">
          <day>1</day>
          <month>9</month>
          <year>2022</year>
        </date>
        <date date-type="rev-request">
          <day>27</day>
          <month>10</month>
          <year>2022</year>
        </date>
        <date date-type="rev-recd">
          <day>1</day>
          <month>12</month>
          <year>2022</year>
        </date>
        <date date-type="accepted">
          <day>1</day>
          <month>12</month>
          <year>2022</year>
        </date>
      </history>
      <copyright-statement>©Ravi Iyer, Denny Meyer. Originally published in JMIR Biomedical Engineering (http://biomsedeng.jmir.org), 22.12.2022.</copyright-statement>
      <copyright-year>2022</copyright-year>
      <license license-type="open-access" xlink:href="https://creativecommons.org/licenses/by/4.0/">
        <p>This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Biomedical Engineering, is properly cited. The complete bibliographic information, a link to the original publication on https://biomedeng.jmir.org/, as well as this copyright and license information must be included.</p>
      </license>
      <self-uri xlink:href="https://biomedeng.jmir.org/2022/2/e42386" xlink:type="simple"/>
      <abstract>
        <sec sec-type="background">
          <title>Background</title>
          <p>In an age when telehealth services are increasingly being used for forward triage, there is a need for accurate suicide risk detection. Vocal characteristics analyzed using artificial intelligence are now proving capable of detecting suicide risk with accuracies superior to traditional survey-based approaches, suggesting an efficient and economical approach to ensuring ongoing patient safety.</p>
        </sec>
        <sec sec-type="objective">
          <title>Objective</title>
          <p>This systematic review aimed to identify which vocal characteristics perform best at differentiating between patients with an elevated risk of suicide in comparison with other cohorts and identify the methodological specifications of the systems used to derive each feature and the accuracies of classification that result.</p>
        </sec>
        <sec sec-type="methods">
          <title>Methods</title>
          <p>A search of MEDLINE via Ovid, Scopus, Computers and Applied Science Complete, CADTH, Web of Science, ProQuest Dissertations and Theses A&#38;I, Australian Policy Online, and Mednar was conducted between 1995 and 2020 and updated in 2021. The inclusion criteria were human participants with no language, age, or setting restrictions applied; randomized controlled studies, observational cohort studies, and theses; studies that used some measure of vocal quality; and individuals assessed as being at high risk of suicide compared with other individuals at lower risk using a validated measure of suicide risk. Risk of bias was assessed using the Risk of Bias in Non-randomized Studies tool. A random-effects model meta-analysis was used wherever mean measures of vocal quality were reported.</p>
        </sec>
        <sec sec-type="results">
          <title>Results</title>
          <p>The search yielded 1074 unique citations, of which 30 (2.79%) were screened via full text. A total of 21 studies involving 1734 participants met all inclusion criteria. Most studies (15/21, 71%) sourced participants via either the Vanderbilt II database of recordings (8/21, 38%) or the Silverman and Silverman perceptual study recording database (7/21, 33%). Candidate vocal characteristics that performed best at differentiating between high risk of suicide and comparison cohorts included timing patterns of speech (median accuracy 95%), power spectral density sub-bands (median accuracy 90.3%), and mel-frequency cepstral coefficients (median accuracy 80%). A random-effects meta-analysis was used to compare 22 characteristics nested within 14% (3/21) of the studies, which demonstrated significant standardized mean differences for frequencies within the first and second formants (standardized mean difference ranged between −1.07 and −2.56) and jitter values (standardized mean difference=1.47). In 43% (9/21) of the studies, risk of bias was assessed as moderate, whereas in the remaining studies (12/21, 57%), the risk of bias was assessed as high.</p>
        </sec>
        <sec sec-type="conclusions">
          <title>Conclusions</title>
          <p>Although several key methodological issues prevailed among the studies reviewed, there is promise in the use of vocal characteristics to detect elevations in suicide risk, particularly in novel settings such as telehealth or conversational agents.</p>
        </sec>
        <sec sec-type="trial registration">
          <title>Trial Registration</title>
          <p>PROSPERO International Prospective Register of Systematic Reviews CRD420200167413; https://www.crd.york.ac.uk/prospero/display_record.php?ID=CRD42020167413</p>
        </sec>
      </abstract>
      <kwd-group>
        <kwd>voice</kwd>
        <kwd>suicide</kwd>
        <kwd>suicidal</kwd>
        <kwd>biological signal processing</kwd>
        <kwd>machine learning</kwd>
        <kwd>systematic review</kwd>
        <kwd>review methodology</kwd>
        <kwd>risk</kwd>
        <kwd>speech</kwd>
        <kwd>mental health</kwd>
      </kwd-group>
    </article-meta>
  </front>
  <body>
    <sec sec-type="introduction">
      <title>Introduction</title>
      <sec>
        <title>Background</title>
        <p>Telehealth alternatives may soon replace in-person visits to providers of primary health care [<xref ref-type="bibr" rid="ref1">1</xref>]. Telehealth is effective in reducing the severity of mental illness [<xref ref-type="bibr" rid="ref2">2</xref>], leading the Australian government to commit to universal access to telehealth care alternatives [<xref ref-type="bibr" rid="ref3">3</xref>].</p>
        <p>The potential utility of telehealth services to the community is undeniable, being ideally suited to reach sectors of the population that have historically faced barriers to access. In particular, rural and remote communities face unique challenges in terms of both economic disparity and location [<xref ref-type="bibr" rid="ref4">4</xref>]. Telehealth services are also appealing to health care consumers of a younger age (19-44 years) [<xref ref-type="bibr" rid="ref5">5</xref>]. These cohorts substantially overlap with those most at risk of suicide [<xref ref-type="bibr" rid="ref6">6</xref>,<xref ref-type="bibr" rid="ref7">7</xref>].</p>
        <p>Telehealth is also being used in other, more novel ways. Primary health care providers in the United States are increasingly using <italic>forward triage</italic>, where patients are assessed before arrival via telehealth means and often via a conversational agent [<xref ref-type="bibr" rid="ref8">8</xref>]. However, this may prove challenging when mental health is the main presenting issue as suicidality is a feature of most mental health disorders [<xref ref-type="bibr" rid="ref9">9</xref>]. Thus, the transition from in-person provision of health care raises important ethical considerations. For example, how can escalation in suicide risk be accurately and efficiently assessed in the absence of in-person cues?</p>
        <p>In &#62;50 years of research, traditional methods of suicide risk assessment (ie, surveys) have yielded little more than chance accuracy in identifying elevated suicide risk [<xref ref-type="bibr" rid="ref10">10</xref>]. Franklin et al [<xref ref-type="bibr" rid="ref10">10</xref>] suggested that suicide risk assessment would benefit from the use of risk algorithms that can assess multiple predictors simultaneously. However, they did not consider the use of biological markers in their review. Such markers do not rely on patient testimony and may prove more accurate in the assessment of suicide risk [<xref ref-type="bibr" rid="ref11">11</xref>].</p>
        <p>Suicide-related biological marker research has focused mainly on identifying neurobiological changes associated with elevated risk. However, the downstream effects of these neurobiological changes may also be apparent and remain underresearched. In particular, changes in speech production and articulation—the subject of this review—have been associated with elevated suicide risk, as indicated in this section. There is an identified need to leverage these novel technologies in the provision of real-time adaptive personalization of counseling content to match consumer emotions [<xref ref-type="bibr" rid="ref12">12</xref>]. This is consistent with the recent recommendations of Balcombe and De Leo [<xref ref-type="bibr" rid="ref13">13</xref>], who also argue for the real-time tracking of consumer emotions via machine learning–trained predictive models that can assist in delivering more timely and efficient mental health care support at scale.</p>
        <p>In their review of vocal characteristics used to identify suicide risk, Cummins et al [<xref ref-type="bibr" rid="ref14">14</xref>] found that many characteristics could prove viable in the detection and differentiation of suicide risk presentations. They identified 4 types of vocal characteristics used for this purpose: prosodic (long-term changes in rhythm, stress, and intonation), voice production, formant (changes in vocal tract properties), and frequency (pitch). Cummins et al [<xref ref-type="bibr" rid="ref14">14</xref>] noted that the speech of individuals at high risk of suicide is often distinguished by a hollow, toneless, and monotonous quality or by a breathy tone, which corresponds to a marked change in spectral slope (accuracies of 90% when using this variable to predict suicide risk) [<xref ref-type="bibr" rid="ref15">15</xref>]. Cummins et al [<xref ref-type="bibr" rid="ref14">14</xref>] also noted that the second formant bandwidth and power spectral densities between 0 and 1000 Hz are promising candidates for further research (accuracies of 90% were obtained using a combination of these features).</p>
        <p>Homan et al [<xref ref-type="bibr" rid="ref16">16</xref>] recently reviewed the use of both voice signals and text-based data to predict suicide risk. The findings of Cummins et al [<xref ref-type="bibr" rid="ref14">14</xref>] were supported by those of Homan et al [<xref ref-type="bibr" rid="ref16">16</xref>], who also suggested pause length and jitter (the timing of the glottal pulse) as additional candidates. However, the authors did not discuss accuracy of prediction or the methodological specifications informing the systems of classification used. However, both Cummins et al [<xref ref-type="bibr" rid="ref14">14</xref>] and Homan et al [<xref ref-type="bibr" rid="ref16">16</xref>] agree that, despite prevailing methodological issues, namely, small sample sizes, lack of control of covariates, and validity of ground truth, there is evidence that suicide risk does alter the human voice in substantive and important ways and may be predictive of elevated suicide risk. Other authors have also noted the equivocality of current findings and the need for further confirmatory research [<xref ref-type="bibr" rid="ref17">17</xref>].</p>
      </sec>
      <sec>
        <title>Objectives</title>
        <p>Thus, the primary objective of this systematic review was to assess the accuracy of vocal characteristics in differentiating between individuals at risk of suicide and those who are not at risk. The secondary objective was to assess the methodological specifications used in these systems of classification.</p>
      </sec>
    </sec>
    <sec sec-type="methods">
      <title>Methods</title>
      <sec>
        <title>Design</title>
        <p>This systematic review was conducted using the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) standards [<xref ref-type="bibr" rid="ref18">18</xref>] (<xref rid="figure1" ref-type="fig">Figure 1</xref>) and checklist (<xref ref-type="supplementary-material" rid="app1">Multimedia Appendix 1</xref>). The systematic review protocol was registered with PROSPERO on April 28, 2020 (registration CRD420200167413) [<xref ref-type="bibr" rid="ref19">19</xref>]. The Population, Intervention, Comparator, Outcome, and Study Design framework defined the research questions and search terms. The research questions were as follows: Which vocal characteristics can differentiate between high and low risk of suicide among both adult and adolescent populations with a high level of accuracy? and What are the methodological specifications used to derive the vocal characteristics and inform the levels of accuracy obtained?</p>
        <fig id="figure1" position="float">
          <label>Figure 1</label>
          <caption>
            <p>PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) flow diagram.</p>
          </caption>
          <graphic xlink:href="biomedeng_v7i2e42386_fig1.png" alt-version="no" mimetype="image" position="float" xlink:type="simple"/>
        </fig>
      </sec>
      <sec>
        <title>Information Sources</title>
        <p>MEDLINE via Ovid, Scopus, Computers and Applied Sciences Complete, and CADTH, in addition to the gray literature databases Web of Science, ProQuest Dissertations and Theses A&#38;I, Australian Policy Online, and Mednar, were searched initially from January 1, 1990, to December 31, 2020, and updated in January 2022.</p>
      </sec>
      <sec>
        <title>Search Strategy</title>
        <p>Search strategies were developed using Medical Subject Headings and keyword string searches that included synonyms of “suicide,” “vocal,” and “algorithm” as separate blocks. A final block of various vocal characteristics was also added, informed by a preliminary survey of the literature. Gray literature was included to ensure a breadth of sources and that insights from unpublished resources might also be included (ie, theses). As a final step, the reference lists of all the included studies were reviewed to ensure that all possible studies were included. Refer to <xref ref-type="supplementary-material" rid="app2">Multimedia Appendix 2</xref> for all terms and search strategies used.</p>
      </sec>
      <sec>
        <title>Inclusion and Exclusion Criteria</title>
        <p>The participants were human, with no language or age restrictions applied. The focus of inquiry was single or multiple measures of vocal quality, which included measures of volume, pace, pitch, rate, rhythm, fluency, articulation, enunciation, and tone. The presence of suicidal ideation or recent behavior was considered the intervention, whereas the absence of such ideation or behavior was the comparator. The primary outcome was a validated measure of suicide risk, whereas no setting restrictions were applied. The study design included randomized controlled trials, cohort studies only, and other unpublished research (ie, theses).</p>
        <p>We followed the International Classification of Diseases, 11th Revision, which defines <italic>suicidal ideation</italic> as thoughts, ideas, or ruminations about the possibility of ending one’s life; <italic>suicide behavior</italic> as concrete actions that are taken in preparation for fulfilling a wish to end one’s life; and <italic>suicide attempt</italic> as a specific episode of self-harming behavior undertaken with the conscious intention of ending one’s life.</p>
        <p>Studies were excluded when they involved animal populations; were unrelated to either the evaluation of vocal quality or suicide risk; did not involve a comparison group; were single-case studies; or did not provide sufficient detail to establish all the Population, Intervention, Comparator, Outcome, and Study Design criteria.</p>
      </sec>
      <sec>
        <title>Selection and Data Collection Process</title>
        <p>Both authors independently reviewed the title, abstract (step 1), and full text (step 2) of each publication identified in accordance with the inclusion and exclusion criteria. NVivo (version 12; QSR International) [<xref ref-type="bibr" rid="ref20">20</xref>] was used to classify each publication for inclusion (green), in doubt (amber), and exclusion (red), with any in-doubt publications discussed further by the authors before consensus. Each publication was also coded to provide a rationale for exclusion (ie, 1=suicide not the primary focus, 2=non–speech-related, 3=animal study, and 4=no comparison between groups).</p>
      </sec>
      <sec>
        <title>Data Extraction and Quality Assessment</title>
        <p>Information was extracted from the included studies according to the following five categories: (1) participant recruitment and characteristics, (2) preprocessing methodological considerations, (3) vocal characteristics, (4) accuracy, and (5) algorithmic approach to classification.</p>
        <p>The included studies were also assessed for quality of evidence by RI, confirmed by DM using the Oxford Centre for Evidence-Based Medicine Levels of Evidence. Each study was rated with a score of 1 to 5, where randomized controlled trials typically scored higher (score=2) than nonrandomized studies (score=3). Disagreements were resolved through discussion.</p>
      </sec>
      <sec>
        <title>Risk-of-Bias Assessment</title>
        <p>RI assessed the methodological quality of the final studies using the Risk of Bias in Non-randomized Studies tool developed by the Cochrane Collaboration [<xref ref-type="bibr" rid="ref21">21</xref>]. The Risk of Bias in Non-randomized Studies tool involves 3 stages of assessment, including specification of the research question (stage 1) and specification of the effect of interest, result to assess, identification of confounders and cointerventions, risk-of-bias judgment for each domain, and an overall risk-of-bias determination for each study (stage 2). This is then synthesized as an overall risk-of-bias assessment for all studies (stage 3). The risk-of-bias domains include confounding, selection of participants, classification of interventions, deviations from planned interventions, missing data, outcome measurement, reporting of results, and overall bias. The risk of bias was assessed as low, moderate, or high. The risk-of-bias assessments are available in <xref ref-type="supplementary-material" rid="app3">Multimedia Appendix 3</xref> [<xref ref-type="bibr" rid="ref15">15</xref>,<xref ref-type="bibr" rid="ref22">22</xref>-<xref ref-type="bibr" rid="ref41">41</xref>].</p>
      </sec>
      <sec>
        <title>Synthesis Methods</title>
        <p>The included studies were heterogeneous in terms of the vocal characteristics assessed and reporting, with classification accuracies included in some studies and mean outcome measures included in others. A narrative synthesis was used to organize the information from the included studies where mean outcome measures were not reported. The guidelines of Rodgers et al [<xref ref-type="bibr" rid="ref42">42</xref>] were applied, which included a preliminary analysis and exploration of relationships followed by the assessment of the robustness of the synthesis.</p>
        <p>Wherever possible, data were presented in tabular form, with information broadly organized around study and participant characteristics, followed by the 2 study questions: classification accuracy of suicide risk using vocal characteristics in the first section and methodological steps taken in the second section.</p>
        <p>Where mean outcome measures were reported, a random-effects model meta-analysis was conducted to synthesize the available information, although multiple vocal characteristics were typically reported in a small number of studies. Using the R package <italic>metafor</italic> (version 3.8-1; R Foundation for Statistical Computing), standardized mean differences were derived from the mean outcome measures reported. Standardized mean differences for each vocal characteristic were then illustrated using a forest plot. All data used in this systematic review are available in <xref ref-type="supplementary-material" rid="app4">Multimedia Appendix 4</xref>.</p>
      </sec>
    </sec>
    <sec sec-type="results">
      <title>Results</title>
      <p><xref rid="figure1" ref-type="fig">Figure 1</xref> illustrates the number of studies that were initially identified, screened, deemed eligible, and included in the final analysis.</p>
      <sec>
        <title>Summary of the Included Studies</title>
        <p>A total of 1074 studies were initially identified. After careful screening, 21 studies from 4 countries were found to comply with all inclusion and exclusion criteria. These studies are summarized in <xref ref-type="table" rid="table1">Table 1</xref>. The included studies featured 1734 participants overall, with 14% (3/21) of the studies [<xref ref-type="bibr" rid="ref15">15</xref>,<xref ref-type="bibr" rid="ref22">22</xref>,<xref ref-type="bibr" rid="ref23">23</xref>] involving adolescent populations only. The publications by Campbell [<xref ref-type="bibr" rid="ref24">24</xref>], Sanadi [<xref ref-type="bibr" rid="ref25">25</xref>], and Sinha [<xref ref-type="bibr" rid="ref26">26</xref>] were theses, whereas the remaining studies (18/21, 86%) were peer-reviewed journal articles. Most studies (11/21, 52%) were observational in nature, and most studies used participant recordings from either the Vanderbilt II database (8/21, 38%) or the Silverman and Silverman perceptual study (7/21, 33%). These data sources are summarized in <xref ref-type="table" rid="table2">Table 2</xref>. The number of studies published by year is illustrated in <xref rid="figure2" ref-type="fig">Figure 2</xref> and can be seen to increase slightly from 2006 onward.</p>
        <table-wrap position="float" id="table1">
          <label>Table 1</label>
          <caption>
            <p>Sample characteristics of the included studies (N=21).</p>
          </caption>
          <table width="1000" cellpadding="5" cellspacing="0" border="1" rules="groups" frame="hsides">
            <col width="220"/>
            <col width="130"/>
            <col width="130"/>
            <col width="130"/>
            <col width="240"/>
            <col width="150"/>
            <thead>
              <tr valign="top">
                <td>Author, year (country)</td>
                <td>Participants, N</td>
                <td>Design</td>
                <td>Sample</td>
                <td>Assessment measure</td>
                <td>Participant age (years)</td>
              </tr>
            </thead>
            <tbody>
              <tr valign="top">
                <td>Anunvrapong and Yingthawornthuk [<xref ref-type="bibr" rid="ref27">27</xref>], 2014 (Thailand)</td>
                <td>30 female</td>
                <td>Observational</td>
                <td>Psychiatric inpatients</td>
                <td>Psychiatric interview</td>
                <td>—<sup>a</sup></td>
              </tr>
              <tr valign="top">
                <td>Belouali et al [<xref ref-type="bibr" rid="ref28">28</xref>], 2021 (United States)</td>
                <td>124</td>
                <td>Longitudinal</td>
                <td>Veterans</td>
                <td>Patient Health Questionnaire-9</td>
                <td>—</td>
              </tr>
              <tr valign="top">
                <td>Campbell [<xref ref-type="bibr" rid="ref24">24</xref>], 1995 (United States)</td>
                <td>3</td>
                <td>Observational</td>
                <td>Telephone call recordings</td>
                <td>Clinician-rated</td>
                <td>—</td>
              </tr>
              <tr valign="top">
                <td>Figueroa Saavedra et al [<xref ref-type="bibr" rid="ref29">29</xref>], 2020 (Chile)</td>
                <td>100 (60 female and 40 male)</td>
                <td>Cross-sectional</td>
                <td>University students</td>
                <td>Okasha Suicidality Scale</td>
                <td>18-19</td>
              </tr>
              <tr valign="top">
                <td>France et al [<xref ref-type="bibr" rid="ref30">30</xref>], 2000 (United States)</td>
                <td>115 (38 female and 77 male)</td>
                <td>Observational</td>
                <td>Psychiatric inpatients</td>
                <td>Beck Depression Inventory and Hamilton Depression Rating Scale</td>
                <td>25-65</td>
              </tr>
              <tr valign="top">
                <td>Keskinpala et al [<xref ref-type="bibr" rid="ref31">31</xref>], 2007 (United States)</td>
                <td>169 (92 female and 77 male)</td>
                <td>Observational</td>
                <td>Psychiatric inpatients</td>
                <td>Clinician-rated</td>
                <td>—</td>
              </tr>
              <tr valign="top">
                <td>Nik Hashim et al [<xref ref-type="bibr" rid="ref32">32</xref>], 2015 (Malaysia)</td>
                <td>89 (54 female and 35 male)</td>
                <td>Observational</td>
                <td>Psychiatric inpatients</td>
                <td>Beck Depression Inventory-II, Hamilton Depression Rating Scale, Mini International Neuropsychiatric Interview, and Pierce Suicide Intent Scale</td>
                <td>—</td>
              </tr>
              <tr valign="top">
                <td>Nik Hashim et al [<xref ref-type="bibr" rid="ref33">33</xref>], 2015 (Malaysia)</td>
                <td>126</td>
                <td>Controlled study</td>
                <td>Psychiatric inpatients</td>
                <td>Hamilton Depression Rating Scale</td>
                <td>22-62 (mean 42.6, SD 10.2)</td>
              </tr>
              <tr valign="top">
                <td>Ozdas et al [<xref ref-type="bibr" rid="ref34">34</xref>], 2000 (United States)</td>
                <td>20 male</td>
                <td>Controlled study</td>
                <td>Psychiatric inpatients</td>
                <td>Hamilton Depression Rating Scale</td>
                <td>25-65</td>
              </tr>
              <tr valign="top">
                <td>Ozdas et al [<xref ref-type="bibr" rid="ref35">35</xref>], 2004 (United States)</td>
                <td>30 male</td>
                <td>Controlled study</td>
                <td>Psychiatric inpatients</td>
                <td>Clinician-rated</td>
                <td>25-65</td>
              </tr>
              <tr valign="top">
                <td>Ozdas et al [<xref ref-type="bibr" rid="ref36">36</xref>], 2004 (United States)</td>
                <td>30 male</td>
                <td>Controlled study</td>
                <td>Psychiatric inpatients</td>
                <td>Clinician-rated</td>
                <td>25-65</td>
              </tr>
              <tr valign="top">
                <td>Pestian et al [<xref ref-type="bibr" rid="ref37">37</xref>], 2017 (United States)</td>
                <td>379</td>
                <td>Controlled study</td>
                <td>Psychiatric inpatients or outpatients</td>
                <td>Clinician-rated</td>
                <td>Adolescent</td>
              </tr>
              <tr valign="top">
                <td>Sanadi [<xref ref-type="bibr" rid="ref25">25</xref>], 2011 (United States)</td>
                <td>60</td>
                <td>Observational</td>
                <td>Psychiatric inpatients</td>
                <td>Clinician-rated</td>
                <td>—</td>
              </tr>
              <tr valign="top">
                <td>Scherer et al [<xref ref-type="bibr" rid="ref22">22</xref>], 2013 (United States)</td>
                <td>381</td>
                <td>Controlled</td>
                <td>Databases of recordings</td>
                <td>Patient Health Questionnaire-9 and Beck Depression Inventory</td>
                <td>Adult (mean 44.7, SD 12.37) and adolescent (13-17)</td>
              </tr>
              <tr valign="top">
                <td>Scherer et al [<xref ref-type="bibr" rid="ref15">15</xref>], 2015 (United States)</td>
                <td>60</td>
                <td>Controlled</td>
                <td>Psychiatric inpatients</td>
                <td>Columbia Suicide Severity Rating Scale, Suicidal Ideation Questionnaire-Junior, and Ubiquitous Questionnaire</td>
                <td>13-17</td>
              </tr>
              <tr valign="top">
                <td>Sinha [<xref ref-type="bibr" rid="ref26">26</xref>], 2013 (United States)</td>
                <td>17</td>
                <td>Observational</td>
                <td>Psychiatric inpatients</td>
                <td>Clinician-rated</td>
                <td>25-65</td>
              </tr>
              <tr valign="top">
                <td>Subari et al [<xref ref-type="bibr" rid="ref38">38</xref>], 2010 (Malaysia)</td>
                <td>30</td>
                <td>Observational</td>
                <td>Psychiatric inpatients</td>
                <td>Clinician-rated</td>
                <td>25-65</td>
              </tr>
              <tr valign="top">
                <td>Venek et al [<xref ref-type="bibr" rid="ref23">23</xref>], 2017 (United States)</td>
                <td>60</td>
                <td>Controlled</td>
                <td>Psychiatric inpatients</td>
                <td>Columbia Suicide Severity Rating Scale, Suicidal Ideation Questionnaire-Junior, and Ubiquitous Questionnaire</td>
                <td>13-17 (mean 15.47, SD 1.5)</td>
              </tr>
              <tr valign="top">
                <td>Yingthawornsuk et al [<xref ref-type="bibr" rid="ref39">39</xref>], 2006 (United States)</td>
                <td>32 male</td>
                <td>Observational</td>
                <td>Psychiatric inpatients</td>
                <td>Beck Depression Inventory-II</td>
                <td>25-65</td>
              </tr>
              <tr valign="top">
                <td>Yingthawornsuk et al [<xref ref-type="bibr" rid="ref40">40</xref>], 2007 (United States)</td>
                <td>20 female</td>
                <td>Observational</td>
                <td>Psychiatric inpatients</td>
                <td>Beck Depression Inventory-II</td>
                <td>25-65</td>
              </tr>
              <tr valign="top">
                <td>Yingthawornsuk and Shiavi [<xref ref-type="bibr" rid="ref41">41</xref>], 2008 (Thailand)</td>
                <td>25 male</td>
                <td>Observational</td>
                <td>Psychiatric inpatients</td>
                <td>Beck Depression Inventory-II</td>
                <td>25-65</td>
              </tr>
            </tbody>
          </table>
          <table-wrap-foot>
            <fn id="table1fn1">
              <p><sup>a</sup>Not available.</p>
            </fn>
          </table-wrap-foot>
        </table-wrap>
        <table-wrap position="float" id="table2">
          <label>Table 2</label>
          <caption>
            <p>Sources of study participants.</p>
          </caption>
          <table width="1000" cellpadding="5" cellspacing="0" border="1" rules="groups" frame="hsides">
            <col width="240"/>
            <col width="500"/>
            <col width="260"/>
            <thead>
              <tr valign="top">
                <td>Participant source, year</td>
                <td>Details</td>
                <td>Studies</td>
              </tr>
            </thead>
            <tbody>
              <tr valign="top">
                <td>Vanderbilt II database, 1993 [<xref ref-type="bibr" rid="ref43">43</xref>]</td>
                <td>Database of recordings of interviews with individuals responding to an advertisement for low-cost psychotherapy; participants met DSM-IV<sup>a</sup> criteria for major depression</td>
                <td>
                  <list list-type="bullet">
                    <list-item>
                      <p>Anunvrapong and Yingthawornthuk [<xref ref-type="bibr" rid="ref27">27</xref>]</p>
                    </list-item>
                    <list-item>
                      <p>France et al [<xref ref-type="bibr" rid="ref30">30</xref>]</p>
                    </list-item>
                    <list-item>
                      <p>Ozdas et al [<xref ref-type="bibr" rid="ref34">34</xref>-<xref ref-type="bibr" rid="ref36">36</xref>]</p>
                    </list-item>
                    <list-item>
                      <p>Subari et al [<xref ref-type="bibr" rid="ref38">38</xref>]</p>
                    </list-item>
                  </list>
                </td>
              </tr>
              <tr valign="top">
                <td>Cognitive behavioral therapy and psychopharmacology study, 1992 [<xref ref-type="bibr" rid="ref44">44</xref>]</td>
                <td>Database of recordings of psychotherapy sessions comparing the effects of cognitive behavioral therapy with psychopharmacological interventions</td>
                <td>
                  <list list-type="bullet">
                    <list-item>
                      <p>France et al [<xref ref-type="bibr" rid="ref30">30</xref>]</p>
                    </list-item>
                    <list-item>
                      <p>Ozdas et al [<xref ref-type="bibr" rid="ref34">34</xref>,<xref ref-type="bibr" rid="ref36">36</xref>]</p>
                    </list-item>
                  </list>
                </td>
              </tr>
              <tr valign="top">
                <td>Silverman and Silverman perceptual study, nd<sup>b</sup> [<xref ref-type="bibr" rid="ref45">45</xref>]</td>
                <td>Database of recordings of psychotherapy sessions and suicide notes of patients who had attempted or completed suicide within hours to weeks of the recordings</td>
                <td>
                  <list list-type="bullet">
                    <list-item>
                      <p>France et al [<xref ref-type="bibr" rid="ref30">30</xref>]</p>
                    </list-item>
                    <list-item>
                      <p>Ozdas et al [<xref ref-type="bibr" rid="ref34">34</xref>-<xref ref-type="bibr" rid="ref36">36</xref>]</p>
                    </list-item>
                    <list-item>
                      <p>Subari et al [<xref ref-type="bibr" rid="ref38">38</xref>]</p>
                    </list-item>
                  </list>
                </td>
              </tr>
              <tr valign="top">
                <td>Vanderbilt University Hospital emergency department</td>
                <td>Study involving recordings of Vanderbilt University Hospital emergency department inpatient admissions</td>
                <td>
                  <list list-type="bullet">
                    <list-item>
                      <p>Nik Hashim et al [<xref ref-type="bibr" rid="ref32">32</xref>]</p>
                    </list-item>
                    <list-item>
                      <p>Yingthawornsuk et al [<xref ref-type="bibr" rid="ref39">39</xref>,<xref ref-type="bibr" rid="ref40">40</xref>]</p>
                    </list-item>
                    <list-item>
                      <p>Yingthawornsuk and Shiavi [<xref ref-type="bibr" rid="ref41">41</xref>]</p>
                    </list-item>
                  </list>
                </td>
              </tr>
              <tr valign="top">
                <td>Cincinnati Children’s Hospital interview corpus</td>
                <td>60 adolescents enrolled in a prospective study, 30 presenting to the emergency department with suicidal ideation and behaviors versus 30 controls presenting with orthopedic injuries</td>
                <td>
                  <list list-type="bullet">
                    <list-item>
                      <p>Scherer et al [<xref ref-type="bibr" rid="ref15">15</xref>,<xref ref-type="bibr" rid="ref22">22</xref>]</p>
                    </list-item>
                    <list-item>
                      <p>Venek et al [<xref ref-type="bibr" rid="ref23">23</xref>]</p>
                    </list-item>
                    <list-item>
                      <p>Pestian et al [<xref ref-type="bibr" rid="ref37">37</xref>]</p>
                    </list-item>
                  </list>
                </td>
              </tr>
              <tr valign="top">
                <td>DAIC<sup>c</sup>, 2014 [<xref ref-type="bibr" rid="ref46">46</xref>]</td>
                <td>Database of 621 recordings of distressed and nondistressed individuals diagnosed with anxiety, depression, and posttraumatic stress disorder; interviews were conducted in person and via an autonomous agent</td>
                <td>
                  <list list-type="bullet">
                    <list-item>
                      <p>Scherer et al [<xref ref-type="bibr" rid="ref22">22</xref>]</p>
                    </list-item>
                  </list>
                </td>
              </tr>
              <tr valign="top">
                <td>AVEC<sup>d</sup></td>
                <td>Database of 292 audiovisual recordings of interviews with participants with depression conducted via an autonomous agent</td>
                <td>
                  <list list-type="bullet">
                    <list-item>
                      <p>Scherer et al [<xref ref-type="bibr" rid="ref22">22</xref>]</p>
                    </list-item>
                  </list>
                </td>
              </tr>
              <tr valign="top">
                <td>Temuco data set</td>
                <td>Database of recordings of interviews with 60 first-year university students from the Faculty of Health Sciences of the Autonomous University of Chile, Temuco</td>
                <td>
                  <list list-type="bullet">
                    <list-item>
                      <p>Figueroa Saavedra et al [<xref ref-type="bibr" rid="ref29">29</xref>]</p>
                    </list-item>
                  </list>
                </td>
              </tr>
              <tr valign="top">
                <td>Washington DC<sup>e</sup> Veterans Affairs Medical Center trial</td>
                <td>Large ongoing prospective trial of veterans diagnosed with Gulf War syndrome</td>
                <td>
                  <list list-type="bullet">
                    <list-item>
                      <p>Belouali et al [<xref ref-type="bibr" rid="ref28">28</xref>]</p>
                    </list-item>
                  </list>
                </td>
              </tr>
            </tbody>
          </table>
          <table-wrap-foot>
            <fn id="table2fn1">
              <p><sup>a</sup>DSM-IV: Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition.</p>
            </fn>
            <fn id="table2fn2">
              <p><sup>b</sup>nd: no date.</p>
            </fn>
            <fn id="table2fn3">
              <p><sup>c</sup>DAIC: Distress Assessment Interview Corpus.</p>
            </fn>
            <fn id="table2fn4">
              <p><sup>d</sup>AVEC: Audio-Visual Depression Corpus.</p>
            </fn>
            <fn id="table2fn5">
              <p><sup>e</sup>DC: District of Columbia.</p>
            </fn>
          </table-wrap-foot>
        </table-wrap>
        <fig id="figure2" position="float">
          <label>Figure 2</label>
          <caption>
            <p>Cumulative number of publications by year.</p>
          </caption>
          <graphic xlink:href="biomedeng_v7i2e42386_fig2.png" alt-version="no" mimetype="image" position="float" xlink:type="simple"/>
        </fig>
      </sec>
      <sec>
        <title>What Are the Voice Signal Characteristics That Distinguish Elevated Suicide Risk From Other Cohorts?</title>
        <p>Most studies (8/21, 38%) used frequency-based characteristics to differentiate participants at high risk of suicide from depressed and healthy cohorts, whereas 33% (7/21) of the studies used power spectral densities, 29% (6/21) used mel-frequency cepstral coefficients, 24% (5/21) used glottal cycle characteristics, and 14% (3/21) used timing patterns of speech. The highest median level of accuracy was attained using timing patterns of speech (85.5%), followed by power spectral densities (81.5%). Both the minimum and maximum levels of accuracy resulted from the use of power spectral densities (30.1% and 98.1%, respectively). In total, 19% (4/21) of the studies used vocal characteristics from a mixture of categories. For those studies that reported the levels of classification accuracy (15/21, 71%), median accuracies are reported in <xref ref-type="table" rid="table3">Table 3</xref>.</p>
        <table-wrap position="float" id="table3">
          <label>Table 3</label>
          <caption>
            <p>Median classification accuracies of the voice signal characteristics selected in each study.</p>
          </caption>
          <table width="1000" cellpadding="5" cellspacing="0" border="1" rules="groups" frame="hsides">
            <col width="290"/>
            <col width="340"/>
            <col width="180"/>
            <col width="190"/>
            <thead>
              <tr valign="top">
                <td>Primary feature</td>
                <td>Studies</td>
                <td>Accuracy (%), range</td>
                <td>Accuracy (%), median</td>
              </tr>
            </thead>
            <tbody>
              <tr valign="top">
                <td>Frequency-based</td>
                <td>
                  <list list-type="bullet">
                    <list-item>
                      <p>Campbell [<xref ref-type="bibr" rid="ref24">24</xref>]</p>
                    </list-item>
                    <list-item>
                      <p>Francea et al [<xref ref-type="bibr" rid="ref30">30</xref>]</p>
                    </list-item>
                    <list-item>
                      <p>Ozdas et al [<xref ref-type="bibr" rid="ref34">34</xref>]</p>
                    </list-item>
                    <list-item>
                      <p>Beloualia et al [<xref ref-type="bibr" rid="ref28">28</xref>]</p>
                    </list-item>
                    <list-item>
                      <p>Pestiana et al [<xref ref-type="bibr" rid="ref37">37</xref>]</p>
                    </list-item>
                    <list-item>
                      <p>Figueroa Saavedra et al [<xref ref-type="bibr" rid="ref29">29</xref>]</p>
                    </list-item>
                    <list-item>
                      <p>Scherera et al [<xref ref-type="bibr" rid="ref22">22</xref>]</p>
                    </list-item>
                    <list-item>
                      <p>Sinha [<xref ref-type="bibr" rid="ref26">26</xref>]</p>
                    </list-item>
                    <list-item>
                      <p>Venek<sup>a</sup> et al [<xref ref-type="bibr" rid="ref23">23</xref>]</p>
                    </list-item>
                  </list>
                </td>
                <td>61.0-85.0</td>
                <td>77.3</td>
              </tr>
              <tr valign="top">
                <td>Power spectral densities</td>
                <td>
                  <list list-type="bullet">
                    <list-item>
                      <p>France et al [<xref ref-type="bibr" rid="ref30">30</xref>]</p>
                    </list-item>
                    <list-item>
                      <p>Nik Hashima et al [<xref ref-type="bibr" rid="ref32">32</xref>]</p>
                    </list-item>
                    <list-item>
                      <p>Keskinpala et al [<xref ref-type="bibr" rid="ref31">31</xref>]</p>
                    </list-item>
                    <list-item>
                      <p>Sanadi [<xref ref-type="bibr" rid="ref25">25</xref>]</p>
                    </list-item>
                    <list-item>
                      <p>Yingthawornsuk et al [<xref ref-type="bibr" rid="ref39">39</xref>]</p>
                    </list-item>
                    <list-item>
                      <p>Yingthawornsuk et al [<xref ref-type="bibr" rid="ref40">40</xref>]</p>
                    </list-item>
                    <list-item>
                      <p>Yingthawornsuk and Shiavi [<xref ref-type="bibr" rid="ref41">41</xref>]</p>
                    </list-item>
                  </list>
                </td>
                <td>30.1-98.1</td>
                <td>81.5</td>
              </tr>
              <tr valign="top">
                <td>Mel-frequency cepstral coefficients</td>
                <td>
                  <list list-type="bullet">
                    <list-item>
                      <p>Belouali et al [<xref ref-type="bibr" rid="ref28">28</xref>]</p>
                    </list-item>
                    <list-item>
                      <p>Nik Hashim et al [<xref ref-type="bibr" rid="ref32">32</xref>]</p>
                    </list-item>
                    <list-item>
                      <p>Keskinpala et al [<xref ref-type="bibr" rid="ref31">31</xref>]</p>
                    </list-item>
                    <list-item>
                      <p>Ozdas et al [<xref ref-type="bibr" rid="ref35">35</xref>]</p>
                    </list-item>
                    <list-item>
                      <p>Subari et al [<xref ref-type="bibr" rid="ref38">38</xref>]</p>
                    </list-item>
                    <list-item>
                      <p>Yingthawornsuk et al [<xref ref-type="bibr" rid="ref40">40</xref>]</p>
                    </list-item>
                  </list>
                </td>
                <td>60.0-90.0</td>
                <td>78.3</td>
              </tr>
              <tr valign="top">
                <td>Glottal cycle characteristics</td>
                <td>
                  <list list-type="bullet">
                    <list-item>
                      <p>Belouali et al [<xref ref-type="bibr" rid="ref28">28</xref>]</p>
                    </list-item>
                    <list-item>
                      <p>Ozdasa et al [<xref ref-type="bibr" rid="ref35">35</xref>]</p>
                    </list-item>
                    <list-item>
                      <p>Pestian et al [<xref ref-type="bibr" rid="ref37">37</xref>]</p>
                    </list-item>
                    <list-item>
                      <p>Scherera et al [<xref ref-type="bibr" rid="ref15">15</xref>]</p>
                    </list-item>
                    <list-item>
                      <p>Venek et al [<xref ref-type="bibr" rid="ref23">23</xref>]</p>
                    </list-item>
                  </list>
                </td>
                <td>60.0-85.0</td>
                <td>78.9</td>
              </tr>
              <tr valign="top">
                <td>Timing patterns of speech</td>
                <td>
                  <list list-type="bullet">
                    <list-item>
                      <p>Nik Hashim et al [<xref ref-type="bibr" rid="ref32">32</xref>]</p>
                    </list-item>
                    <list-item>
                      <p>Nik Hashim et al [<xref ref-type="bibr" rid="ref33">33</xref>]</p>
                    </list-item>
                    <list-item>
                      <p>Scherer et al [<xref ref-type="bibr" rid="ref22">22</xref>]</p>
                    </list-item>
                  </list>
                </td>
                <td>66.0-100.0</td>
                <td>85.5</td>
              </tr>
            </tbody>
          </table>
          <table-wrap-foot>
            <fn id="table3fn1">
              <p><sup>a</sup>Combined with other voice biometrics.</p>
            </fn>
          </table-wrap-foot>
        </table-wrap>
      </sec>
      <sec>
        <title>A Comparison of 22 Measures for Identifying High Risk of Suicide</title>
        <p>A random-effects model meta-analysis was used to compare 22 different measures nested within 14% (3/21) of the publications. The included studies [<xref ref-type="bibr" rid="ref15">15</xref>,<xref ref-type="bibr" rid="ref22">22</xref>,<xref ref-type="bibr" rid="ref29">29</xref>] involved 80 participants and 22 different vocal characteristics. The standardized mean difference for each vocal characteristic is summarized in the forest plot in <xref rid="figure3" ref-type="fig">Figure 3</xref>.</p>
        <p>Positive standardized mean differences suggest higher levels of the vocal characteristic in the high–suicide-risk cohort, whereas negative standardized mean differences suggest that higher levels of the characteristic are found instead in the low-risk group.</p>
        <p>A subgroup formed by the frequencies of the first and second formants applied to each of the 3 conditions reported by Figueroa Saavedra et al [<xref ref-type="bibr" rid="ref29">29</xref>] was significant in differentiating between participants with and without elevated suicide risk. These significant negative standardized mean differences suggest that those at high risk of suicide feature lower vocal tract resonance frequencies, specifically in the lower formant regions rather than in the higher formant regions (ie, above the second formant of frequencies). Also of note were jitter values in the held “A” vowel condition in the study by Figueroa Saavedra et al [<xref ref-type="bibr" rid="ref29">29</xref>]. Jitter in this condition yielded significant positive differentiation for participants with and without high suicide risk, suggesting that those at higher risk of suicide exhibited higher levels of roughness or hoarseness of articulation.</p>
        <fig id="figure3" position="float">
          <label>Figure 3</label>
          <caption>
            <p>Random-effects meta-analysis forest plot illustrating 22 vocal characteristics in 3 studies.</p>
          </caption>
          <graphic xlink:href="biomedeng_v7i2e42386_fig3.png" alt-version="no" mimetype="image" position="float" xlink:type="simple"/>
        </fig>
      </sec>
      <sec>
        <title>What Are the Methodological Specifications Used?</title>
        <p>Preprocessing is an important stage that occurs before the classification of vocal characteristics. This involves modifications to the voice signal to ensure greater precision in isolating its specific characteristics. <xref ref-type="supplementary-material" rid="app5">Multimedia Appendix 5</xref> illustrates an ideal preprocessing workflow.</p>
        <p>The reviewed studies used a range of software for preprocessing and analysis of the vocal characteristics, including Microsound Editor [<xref ref-type="bibr" rid="ref30">30</xref>,<xref ref-type="bibr" rid="ref34">34</xref>,<xref ref-type="bibr" rid="ref35">35</xref>,<xref ref-type="bibr" rid="ref38">38</xref>,<xref ref-type="bibr" rid="ref47">47</xref>] to identify and remove silence segments, MATLAB [<xref ref-type="bibr" rid="ref22">22</xref>,<xref ref-type="bibr" rid="ref23">23</xref>,<xref ref-type="bibr" rid="ref25">25</xref>,<xref ref-type="bibr" rid="ref26">26</xref>,<xref ref-type="bibr" rid="ref30">30</xref>,<xref ref-type="bibr" rid="ref39">39</xref>,<xref ref-type="bibr" rid="ref48">48</xref>], COVAREP [<xref ref-type="bibr" rid="ref22">22</xref>,<xref ref-type="bibr" rid="ref23">23</xref>,<xref ref-type="bibr" rid="ref37">37</xref>,<xref ref-type="bibr" rid="ref49">49</xref>], and Praat [<xref ref-type="bibr" rid="ref29">29</xref>,<xref ref-type="bibr" rid="ref50">50</xref>] to facilitate subsequent analyses.</p>
        <p>Most studies (11/21, 52%) [<xref ref-type="bibr" rid="ref24">24</xref>,<xref ref-type="bibr" rid="ref27">27</xref>,<xref ref-type="bibr" rid="ref30">30</xref>,<xref ref-type="bibr" rid="ref31">31</xref>,<xref ref-type="bibr" rid="ref34">34</xref>-<xref ref-type="bibr" rid="ref36">36</xref>,<xref ref-type="bibr" rid="ref38">38</xref>-<xref ref-type="bibr" rid="ref41">41</xref>] first converted the signal from analog to digital using a 16-bit recording at a sampling rate of 10 kHz. However, since 2010, recordings were sampled at higher rates. Venek et al [<xref ref-type="bibr" rid="ref23">23</xref>] and Scherer et al [<xref ref-type="bibr" rid="ref15">15</xref>,<xref ref-type="bibr" rid="ref22">22</xref>], for instance, sampled speech at 16 kHz, whereas the remaining studies (19/21, 90%) [<xref ref-type="bibr" rid="ref25">25</xref>,<xref ref-type="bibr" rid="ref26">26</xref>,<xref ref-type="bibr" rid="ref29">29</xref>,<xref ref-type="bibr" rid="ref32">32</xref>,<xref ref-type="bibr" rid="ref51">51</xref>] sampled it at 44.1 kHz.</p>
        <p>All studies (21/21, 100%) then used a band-pass antialiasing filter to restrict the digital signal to a frequency range of 0 to 5000 Hz. Campbell [<xref ref-type="bibr" rid="ref24">24</xref>] was the only author to analyze recordings sourced from telephone calls; thus, a band-pass filter restricting the frequency range to between 300 and 3000 Hz was automatically applied. After filtering, the signal was normalized [<xref ref-type="bibr" rid="ref25">25</xref>,<xref ref-type="bibr" rid="ref26">26</xref>,<xref ref-type="bibr" rid="ref30">30</xref>,<xref ref-type="bibr" rid="ref35">35</xref>,<xref ref-type="bibr" rid="ref36">36</xref>,<xref ref-type="bibr" rid="ref39">39</xref>-<xref ref-type="bibr" rid="ref41">41</xref>] and detrended [<xref ref-type="bibr" rid="ref25">25</xref>,<xref ref-type="bibr" rid="ref30">30</xref>,<xref ref-type="bibr" rid="ref39">39</xref>-<xref ref-type="bibr" rid="ref41">41</xref>] to facilitate comparison between speakers and isolate the variable signal components, respectively.</p>
        <p>Following these steps, the voice signal was differentiated between voiced and unvoiced types in 14% (3/21) of the studies. Subari et al [<xref ref-type="bibr" rid="ref38">38</xref>] categorized voiced segments by the presence of cepstral peaks, Ozdas et al [<xref ref-type="bibr" rid="ref34">34</xref>] differentiated between voiced and unvoiced signals using a discrete wavelet transform, whereas Sinha [<xref ref-type="bibr" rid="ref26">26</xref>] adapted this approach to include 5 band-pass filters instead that selectively identified signal energies corresponding to each subband.</p>
        <p>As noted, power spectral densities were investigated in several studies (6/21, 29%) [<xref ref-type="bibr" rid="ref26">26</xref>,<xref ref-type="bibr" rid="ref30">30</xref>,<xref ref-type="bibr" rid="ref31">31</xref>,<xref ref-type="bibr" rid="ref39">39</xref>-<xref ref-type="bibr" rid="ref41">41</xref>]. Power spectral densities are derived from short-windowed segments of the voice signal. All studies (21/21, 100%) applied nonoverlapping Hamming windows to filter each 40- or 51.2-millisecond signal segment. A total of 10% (2/21) of the studies [<xref ref-type="bibr" rid="ref38">38</xref>] used linear predictive coding applied to 15- and 25.6-millisecond segment durations to derive the first 3 formants and bandwidths.</p>
        <p>Finally, most studies (7/21, 33%) used quadratic discriminant analysis to classify the voice signals of participants at high risk of suicide from other cohorts, and 57% (12/21) of the studies used either maximum likelihood, linear discriminant analysis, or support vector machines. As demonstrated in <xref ref-type="table" rid="table4">Table 4</xref>, the highest median level of accuracy was obtained using quadratic discriminant analysis. Both the minimum and maximum levels of accuracy were also recorded using quadratic discriminant analysis (21.4% and 100%, respectively). The median levels of classification accuracy are summarized in <xref ref-type="table" rid="table4">Table 4</xref>.</p>
        <table-wrap position="float" id="table4">
          <label>Table 4</label>
          <caption>
            <p>Median accuracies achieved by classification algorithm.</p>
          </caption>
          <table width="1000" cellpadding="5" cellspacing="0" border="1" rules="groups" frame="hsides">
            <col width="340"/>
            <col width="390"/>
            <col width="150"/>
            <col width="120"/>
            <thead>
              <tr valign="top">
                <td>Algorithm</td>
                <td>Studies</td>
                <td colspan="2">Classification accuracy (%)</td>
              </tr>
              <tr valign="top">
                <td>
                  <break/>
                </td>
                <td>
                  <break/>
                </td>
                <td>Range</td>
                <td>Median</td>
              </tr>
            </thead>
            <tbody>
              <tr valign="top">
                <td>Maximum likelihood</td>
                <td>
                  <list list-type="bullet">
                    <list-item>
                      <p>Ozdas et al [<xref ref-type="bibr" rid="ref34">34</xref>]</p>
                    </list-item>
                    <list-item>
                      <p>Ozdas et al [<xref ref-type="bibr" rid="ref35">35</xref>]</p>
                    </list-item>
                    <list-item>
                      <p>Ozdas et al [<xref ref-type="bibr" rid="ref36">36</xref>]</p>
                    </list-item>
                    <list-item>
                      <p>Subari et al [<xref ref-type="bibr" rid="ref38">38</xref>]</p>
                    </list-item>
                  </list>
                </td>
                <td>60.0-85.0</td>
                <td>80.0</td>
              </tr>
              <tr valign="top">
                <td>Linear discriminant analysis</td>
                <td>
                  <list list-type="bullet">
                    <list-item>
                      <p>France et al [<xref ref-type="bibr" rid="ref30">30</xref>]</p>
                    </list-item>
                    <list-item>
                      <p>Nik Hashim et al [<xref ref-type="bibr" rid="ref51">51</xref>]</p>
                    </list-item>
                    <list-item>
                      <p>Sanadi [<xref ref-type="bibr" rid="ref25">25</xref>]</p>
                    </list-item>
                    <list-item>
                      <p>Sinha [<xref ref-type="bibr" rid="ref26">26</xref>]</p>
                    </list-item>
                  </list>
                </td>
                <td>30.1-98.1</td>
                <td>79.7</td>
              </tr>
              <tr valign="top">
                <td>Quadratic discriminant analysis</td>
                <td>
                  <list list-type="bullet">
                    <list-item>
                      <p>Nik Hashim et al [<xref ref-type="bibr" rid="ref51">51</xref>]</p>
                    </list-item>
                    <list-item>
                      <p>Keskinpala et al [<xref ref-type="bibr" rid="ref31">31</xref>]</p>
                    </list-item>
                    <list-item>
                      <p>Sanadi [<xref ref-type="bibr" rid="ref25">25</xref>]</p>
                    </list-item>
                    <list-item>
                      <p>Sinha [<xref ref-type="bibr" rid="ref26">26</xref>]</p>
                    </list-item>
                    <list-item>
                      <p>Yingthawornsuk et al [<xref ref-type="bibr" rid="ref39">39</xref>]</p>
                    </list-item>
                    <list-item>
                      <p>Yingthawornsuk et al [<xref ref-type="bibr" rid="ref40">40</xref>]</p>
                    </list-item>
                    <list-item>
                      <p>Yingthawornsuk and Shiavi [<xref ref-type="bibr" rid="ref41">41</xref>]</p>
                    </list-item>
                  </list>
                </td>
                <td>21.4-100.0</td>
                <td>85.4</td>
              </tr>
              <tr valign="top">
                <td>Hierarchical mixed model</td>
                <td>
                  <list list-type="bullet">
                    <list-item>
                      <p>Scherer et al [<xref ref-type="bibr" rid="ref15">15</xref>]</p>
                    </list-item>
                  </list>
                </td>
                <td>69.0-81.0</td>
                <td>75.0</td>
              </tr>
              <tr valign="top">
                <td>Support vector machine</td>
                <td>
                  <list list-type="bullet">
                    <list-item>
                      <p>Pestian et al [<xref ref-type="bibr" rid="ref37">37</xref>]</p>
                    </list-item>
                    <list-item>
                      <p>Scherer et al [<xref ref-type="bibr" rid="ref15">15</xref>]</p>
                    </list-item>
                    <list-item>
                      <p>Venek et al [<xref ref-type="bibr" rid="ref23">23</xref>]</p>
                    </list-item>
                    <list-item>
                      <p>Belouali et al [<xref ref-type="bibr" rid="ref28">28</xref>]</p>
                    </list-item>
                  </list>
                </td>
                <td>61.0-85.0</td>
                <td>75.9</td>
              </tr>
            </tbody>
          </table>
        </table-wrap>
      </sec>
      <sec>
        <title>Risk of Bias and Quality Assessment</title>
        <p>The risk of bias was assessed to explore the variability in the quality of the included publications. None of the included studies were at low risk of bias. In 43% (9/21) of the studies, the risk of bias was moderate [<xref ref-type="bibr" rid="ref15">15</xref>,<xref ref-type="bibr" rid="ref22">22</xref>,<xref ref-type="bibr" rid="ref23">23</xref>,<xref ref-type="bibr" rid="ref26">26</xref>,<xref ref-type="bibr" rid="ref28">28</xref>-<xref ref-type="bibr" rid="ref30">30</xref>,<xref ref-type="bibr" rid="ref32">32</xref>,<xref ref-type="bibr" rid="ref36">36</xref>-<xref ref-type="bibr" rid="ref41">41</xref>], whereas in the remaining 57% (12/21), the risk of bias was assessed as high [<xref ref-type="bibr" rid="ref24">24</xref>,<xref ref-type="bibr" rid="ref25">25</xref>,<xref ref-type="bibr" rid="ref27">27</xref>,<xref ref-type="bibr" rid="ref31">31</xref>,<xref ref-type="bibr" rid="ref33">33</xref>-<xref ref-type="bibr" rid="ref35">35</xref>]. The main sources of bias were confounding factors, selection of participants, and selective reporting of results. The quality of evidence of most (18/21, 86%) of the included studies [<xref ref-type="bibr" rid="ref15">15</xref>,<xref ref-type="bibr" rid="ref22">22</xref>,<xref ref-type="bibr" rid="ref23">23</xref>,<xref ref-type="bibr" rid="ref27">27</xref>-<xref ref-type="bibr" rid="ref37">37</xref>,<xref ref-type="bibr" rid="ref39">39</xref>-<xref ref-type="bibr" rid="ref41">41</xref>] was assessed with a rating of 3, whereas 14% (3/21) of the studies [<xref ref-type="bibr" rid="ref17">17</xref>,<xref ref-type="bibr" rid="ref24">24</xref>,<xref ref-type="bibr" rid="ref52">52</xref>] were assessed at the lowest rating of 5.</p>
      </sec>
    </sec>
    <sec sec-type="discussion">
      <title>Discussion</title>
      <sec>
        <title>Principal Findings</title>
        <p>A systematic review (1995-2021) was undertaken with 2 objectives in mind: to identify which vocal characteristics could accurately differentiate between individuals at high risk of suicide compared with those at lower risk and identify the methodological specifications that inform the derivation of each vocal characteristic and the classification accuracies that result.</p>
        <p>A number of vocal characteristics were found to differentiate between high risk of suicide and comparison cohorts with a high level of accuracy. Of note were the median accuracies obtained using the timing patterns of speech (median accuracy 85.5%) and power spectral densities (median accuracy 81.5%). Furthermore, a random-effects meta-analysis that included 22 vocal characteristics from 14% (3/21) of the studies revealed that frequencies within the lower formants (1 and 2) and jitter provided significant standardized mean differences between high- and low–suicide-risk signals, suggesting that participants at high risk of suicide may have lower vocal tract resonance frequencies while speaking with greater roughness of speech.</p>
        <p>These results are broadly consistent with several recent investigations that have also found significant increases in lower formant frequencies under stressful conditions, suggesting a reduction in articulatory clarity [<xref ref-type="bibr" rid="ref17">17</xref>]; decreases in the quantity of speech among those at high risk of suicide [<xref ref-type="bibr" rid="ref53">53</xref>]; and changes in jitter, a measure of cycle-to-cycle variation in the fundamental frequency that decreases under anxiety-producing conditions [<xref ref-type="bibr" rid="ref17">17</xref>].</p>
        <p>The findings of this study that mel-frequency cepstral coefficients characterize muscle tension and control of the vocal tract, a measure particularly sensitive to changes in stress, are also supported by previous research [<xref ref-type="bibr" rid="ref54">54</xref>]. However, although increases in root mean squared amplitude or loudness have been found among speakers at high risk of suicide in more recent studies [<xref ref-type="bibr" rid="ref55">55</xref>], the use of this variable was limited to only select studies reviewed (2/21, 10%) [<xref ref-type="bibr" rid="ref30">30</xref>,<xref ref-type="bibr" rid="ref37">37</xref>].</p>
        <p>This review also aimed to identify the system specifications used to derive each vocal characteristic and their levels of accuracy. All studies (21/21, 100%) were found to adopt a similar workflow of preprocessing steps that broadly included (1) conversion from analog to digital signals, (2) band-pass filtering, (3) normalization and detrending, (4) differentiation between voiced and unvoiced signals, (5) removal of silent passages, and (6) signal segmentation before classification.</p>
        <p>Although most studies (12/21, 57%) used band-pass filtering to remove frequencies &#62;5000 Hz, only Campbell [<xref ref-type="bibr" rid="ref24">24</xref>] used recordings sourced from telephone calls. As noted by the author, these sources automatically filter signals to between 300 and 3000 Hz. Although noise is reduced, this approach also removes the fundamental frequency from the signal signature but is known to overestimate the first formant frequency values by as much as 13% [<xref ref-type="bibr" rid="ref56">56</xref>]. These may be important considerations for future studies that aim to source data from more novel settings such as telephone helplines or conversational agents.</p>
        <p>Most studies (14/21, 67%) discussed the use of signal normalization to ensure a comparison between speakers. However, only in the study by Subari et al [<xref ref-type="bibr" rid="ref38">38</xref>] was the effect on the overall accuracy of different forms of normalization investigated. Both a maximum likelihood–derived warping factor and normalization based on median third formant values were optimized based on the levels of classification accuracy. The authors noted that the formant-derived approach is preferable given its lower computational load and consideration of the sex of the participant. Normalization is a crucial consideration when the vocal tract of speakers can differ by as much as 7 cm [<xref ref-type="bibr" rid="ref57">57</xref>].</p>
        <p>Detrending was discussed in some studies (5/21, 24%). It is presumed that, by removing the mean signal, the authors sought to reveal the nonstationary signal components that might better differentiate one speaker from another. Although voice signals are known to be stationary only over short time frames (&#60;40 ms) [<xref ref-type="bibr" rid="ref58">58</xref>], several studies (8/21, 38%) used signal segmentation in excess of 50 milliseconds. The strategy of mean removal with potentially nonstationary signals risks dampening low-frequency sounds while attenuating high-frequency sounds and can also introduce secondary artifacts that may cloud ongoing analyses [<xref ref-type="bibr" rid="ref58">58</xref>]. The preference in several studies (5/21, 24%) to detrend via discrete wavelet transform overcomes many of the aforementioned issues and appears well suited to the analysis of nonstationary signals at longer time frames of capture.</p>
        <p>Given the differences in the frequency and amplitude spectra between voiced and unvoiced segments of speech, it is unsurprising that most of the reviewed publications (11/21, 52%) opted to differentiate between these signal types before classification. In total, 3 different approaches to voiced or unvoiced signal differentiation were used: approximation of voiced signals via the presence of cepstral peaks; frequency mapping via discrete wavelet transform using the highest-scale wavelets (2<sup>5</sup>) to categorize voiced signals; and selective band-pass filtering with frequencies &#62;2500 Hz categorized as unvoiced, whereas signals between 320 and 2499 Hz were categorized as voiced. Further investigation is required to determine which approach best optimizes accuracies of classification; however, selective band-pass filtering has the advantage of not altering the signal in any way.</p>
        <p>Regarding those studies that analyzed power spectral densities (9/21, 43%), nonoverlapping Hamming windows were the preferred approach to derive the short–time frame Fourier transform, converting the signal from the time to the frequency domain. This nonstandard approach has the effect of capturing frequencies at regular intervals corresponding to window width while introducing high frequencies as each window tapers into the next. The standard approach that incorporates overlapping windows, smoothing the effects at the tails, seems preferable to the nonstandard approach commonly used [<xref ref-type="bibr" rid="ref59">59</xref>].</p>
        <p>In only a minority of the reviewed studies (3/21, 14%) was supervised machine learning used (support vector machine). However, the highest median levels of accuracy were obtained not when these advanced forms of classification were used but rather when unsupervised quadratic discriminant analysis was used. Contrary to the much touted superiority of supervised machine learning methods, our findings suggest that higher levels of accuracy were instead obtained using less complex classification approaches. However, further investigation using more sophisticated approaches such as neural networks is clearly warranted.</p>
        <p>In only the study by Scherer et al [<xref ref-type="bibr" rid="ref15">15</xref>] was a mixed-effects model used. This approach might better capture the correlated structure of voice signal segments and better account for intraspeaker variance than other approaches.</p>
      </sec>
      <sec>
        <title>Future Directions and Implications for Practice</title>
        <p>Suicide risk has been treated as static, stable, and invariant following initial assessment. Recent studies demonstrate that suicide risk can, in fact, change dramatically over time, suggesting that future studies might use insights from ecological momentary assessment [<xref ref-type="bibr" rid="ref52">52</xref>,<xref ref-type="bibr" rid="ref60">60</xref>]. Alternatively, future studies could adopt the approach of Campbell [<xref ref-type="bibr" rid="ref24">24</xref>] by using trained personnel to assess the changing level of suicide risk across time within each recording as well as between recordings. Such approaches might better reflect the real-time change in suicidality, acknowledging that individuals frequently cycle in and out of risk.</p>
        <p>In the reviewed literature, risk assessments were typically performed by coauthors (eg, Silverman in the study by Campbell [<xref ref-type="bibr" rid="ref24">24</xref>] or Salomon in the study by Sinha [<xref ref-type="bibr" rid="ref26">26</xref>]), an approach that may bias the objective assessment of risk. Future research might use multiple assessors of suicide risk where measures of interrater reliability can be analyzed and possible biases can be isolated and addressed.</p>
        <p>Our analysis of the preprocessing workflows suggests that greater transparency regarding methodological considerations is urgently required. The reviewed publications were clearly aimed at an informed and knowledgeable engineering readership, and it was common to refer to complex methods using technical terms (eg, windowing). It would assist in reproducibility to understand preprocessing decisions in greater detail (ie, window type).</p>
        <p>Certain vocal characteristics have been proven to more accurately differentiate between high and low risk of suicide. In particular, the timing patterns of speech and the vowel space occupied by different speaker articulations hold considerable promise. Also of note are insights derived from power spectral densities and frequency-related categories, underused methods such as the Liljencrants-Fant model of glottal flow, and mel-frequency cepstral coefficients. However, as demonstrated by Pestian et al [<xref ref-type="bibr" rid="ref37">37</xref>] and Venek et al [<xref ref-type="bibr" rid="ref23">23</xref>], the future of research in this field leans toward combining multiple features within high-powered machine learning algorithms such as support vector machine, although it should be noted that lower-powered approaches appeared in this review to yield greater levels of accuracy (ie, quadratic discriminant analysis).</p>
        <p>The power of these advanced machine learning algorithms can only be used with an adequately powered sample. Cummins et al [<xref ref-type="bibr" rid="ref14">14</xref>] called for greater collaboration between research teams to address this ongoing problem. An alternative approach might be to secure greater industry partnerships and look to novel settings with high call volumes such as telemental health. Given that a recent review found poor support for conventional suicide screening methods [<xref ref-type="bibr" rid="ref10">10</xref>], there is a clear case for incorporating voice signal–informed analysis into existing telehealth and other e-services, in particular suicide helplines. These settings typically have large call volumes that increasingly feature elevated risk of suicide, particularly in the COVID-19 era. However, such collaborations also raise other ethical issues such as how best to safeguard callers’ rights to privacy and secure consent.</p>
      </sec>
      <sec>
        <title>Limitations</title>
        <p>There are some limitations to this review. Of note was the lack of specificity in the definitions of high risk of suicide. In only 43% (9/21) of the studies analyzed, the high-risk cohorts were truly reflective of imminent risk of suicide. These studies used recordings of participants sourced from the Silverman database of suicide notes left by patients who had either attempted or completed suicide or, alternatively, from the Cincinnati Children’s Hospital interview corpus, where participants were recruited and interviewed immediately following presentation at emergency departments with acute suicidality. In the remaining studies (12/21, 57%), participants were assigned to the high-risk cohort based on cutoff scores on diverse psychometric tests, including the Beck Depression Inventory and Hamilton Depression Rating Scale. Several large-scale reviews [<xref ref-type="bibr" rid="ref10">10</xref>,<xref ref-type="bibr" rid="ref61">61</xref>-<xref ref-type="bibr" rid="ref63">63</xref>] attest to the low precision and recall of suicide rating scales, suggesting that participants assigned to cohorts with high risk of suicide in these studies may also feature a proportion of false positives and, conversely, a proportion of false negatives in the control groups.</p>
        <p>There was also a diversity of vocal characteristics trialed. Across the reviewed publications, it was challenging to find the same set of features replicated in other samples. It was more common to find a single characteristic combined with others to optimize discrimination between levels of suicide risk. This presented difficulties in determining which features might be reliably accurate across different settings.</p>
        <p>Except for a notable large-scale multicenter trial [<xref ref-type="bibr" rid="ref37">37</xref>], most of the reviewed studies (17/21, 81%) involved small samples, typically &#60;60 participants, sometimes divided between 3 comparison groups [<xref ref-type="bibr" rid="ref38">38</xref>,<xref ref-type="bibr" rid="ref41">41</xref>]. As highlighted by Button et al [<xref ref-type="bibr" rid="ref64">64</xref>], small-sample research is plagued by a number of issues, namely, reduced power, low reproducibility of results, and reductions in the likelihood that the results obtained reflect a true effect. Undoubtedly, these problems are amplified when the sample size of the comparison groups is &#60;10, as was the case in several of the studies reviewed (8/21, 38%) [<xref ref-type="bibr" rid="ref24">24</xref>,<xref ref-type="bibr" rid="ref26">26</xref>,<xref ref-type="bibr" rid="ref34">34</xref>-<xref ref-type="bibr" rid="ref36">36</xref>,<xref ref-type="bibr" rid="ref38">38</xref>,<xref ref-type="bibr" rid="ref40">40</xref>,<xref ref-type="bibr" rid="ref41">41</xref>]. Also of consideration were the high proportion of studies involving psychiatric inpatients (17/21, 81%) and the homogeneity of the databases from which participants were recruited, further limiting generalizability.</p>
        <p>Also of note were the controlled conditions within which the trials took place. It was common for participants to be invited into a room away from extraneous noise and asked to read prescripted text such as the rainbow passage (eg, Yingthawornsuk et al [<xref ref-type="bibr" rid="ref39">39</xref>]) or to articulate prolonged vowel sounds (eg, Scherer et al [<xref ref-type="bibr" rid="ref22">22</xref>]). Although these research protocols increase the likelihood of uncovering candidate vocal characteristics, they also reduce the generalizability of the research findings to other settings, particularly to telephony and other novel eHealth-based applications where these controls are impractical to implement and noise is the rule rather than the exception. Except for the study by Campbell [<xref ref-type="bibr" rid="ref24">24</xref>], no studies sourced participants from these more ecologically valid settings.</p>
        <p>There was a paucity of detail relating to specific preprocessing elements supported by the high risk of bias prevailing among the reviewed studies. One of the cardinal requirements for cumulative science is that methodologies are replicable [<xref ref-type="bibr" rid="ref65">65</xref>]. Publications are increasingly restricting the number of allowable words. Prospective authors might be tempted to limit descriptions of the methodology in favor of the results and discussion. However, most publications also allow for appendixes that can provide supplementary information relating to the methodology.</p>
        <p>However, there were also notable strengths to this review. This review has expanded upon the findings of Cummins et al [<xref ref-type="bibr" rid="ref14">14</xref>] and Homan et al [<xref ref-type="bibr" rid="ref16">16</xref>] in important ways. We have updated the research findings to 2021, and based on the accuracy of discrimination between levels of suicide risk, we were able to identify a number of promising candidate vocal characteristics that warrant further investigation. We were also able to identify and discuss a number of preprocessing steps used before the classification of voice signals.</p>
      </sec>
      <sec>
        <title>Conclusions</title>
        <p>The data indicate that several characteristics successfully differentiate between individuals at high and low risk of suicide. An analysis of power spectral density subbands yielded high accuracies of discrimination between comparison groups (eg, 90.3% accuracy in the study by Yingthawornsuk et al [<xref ref-type="bibr" rid="ref40">40</xref>]); however, the studies that used power spectral densities disagreed on whether the lower- or higher-frequency subbands were of key importance and also disagreed on the need for single or combined feature analyses. Second, several studies (4/21, 19%) found higher formant frequencies and a narrowing of bandwidth among those at elevated risk of suicide [<xref ref-type="bibr" rid="ref22">22</xref>,<xref ref-type="bibr" rid="ref23">23</xref>,<xref ref-type="bibr" rid="ref29">29</xref>,<xref ref-type="bibr" rid="ref30">30</xref>]. Higher levels of predictive accuracy were found when formant features were combined with other features (eg, 80% accuracy in the study by France et al [<xref ref-type="bibr" rid="ref30">30</xref>]). Third, Nik Hashim et al [<xref ref-type="bibr" rid="ref32">32</xref>,<xref ref-type="bibr" rid="ref51">51</xref>] and Scherer et al [<xref ref-type="bibr" rid="ref22">22</xref>] found that the timing patterns of speech in speakers at elevated risk of suicide differed in a number of important ways from those of speakers at low risk of suicide. In particular, pauses were protracted, whereas certain vowel sounds were held for longer periods among those at elevated risk of suicide. Fourth, both the study by Anunvrapong and Yingthawornthuk [<xref ref-type="bibr" rid="ref27">27</xref>] and the study by Ozdas et al [<xref ref-type="bibr" rid="ref35">35</xref>] found that the analysis of mel-frequency cepstral coefficients—which attempts to mimic the energy spectrum of human hearing—successfully differentiated between speakers at high and low risk of suicide. However, a reduced filter bank (first 4 frequencies) yielded greater accuracies. Finally, both Scherer et al [<xref ref-type="bibr" rid="ref22">22</xref>] and Venek et al [<xref ref-type="bibr" rid="ref23">23</xref>] found that certain coefficients of the Liljencrants-Fant model of glottal flow significantly differentiated between high and low risk of suicide, suggesting that those at high risk of suicide often speak in breathier tones. This was particularly apparent among adolescents.</p>
        <p>Although this systematic review revealed a number of limitations in the current literature in this field, the level of accuracy achieved is promising, suggesting that future research, particularly in more novel areas of telemental health, holds considerable promise for the detection and prevention of suicide in the community.</p>
      </sec>
    </sec>
  </body>
  <back>
    <app-group>
      <supplementary-material id="app1">
        <label>Multimedia Appendix 1</label>
        <p>PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) checklist.</p>
        <media xlink:href="biomedeng_v7i2e42386_app1.docx" xlink:title="DOCX File , 31 KB"/>
      </supplementary-material>
      <supplementary-material id="app2">
        <label>Multimedia Appendix 2</label>
        <p>Representative search strategies. STFT: short–time frame Fourier transform.</p>
        <media xlink:href="biomedeng_v7i2e42386_app2.png" xlink:title="PNG File , 104 KB"/>
      </supplementary-material>
      <supplementary-material id="app3">
        <label>Multimedia Appendix 3</label>
        <p>Risk-of-bias assessments.</p>
        <media xlink:href="biomedeng_v7i2e42386_app3.docx" xlink:title="DOCX File , 23 KB"/>
      </supplementary-material>
      <supplementary-material id="app4">
        <label>Multimedia Appendix 4</label>
        <p>Review data.</p>
        <media xlink:href="biomedeng_v7i2e42386_app4.xlsx" xlink:title="XLSX File  (Microsoft Excel File), 1504 KB"/>
      </supplementary-material>
      <supplementary-material id="app5">
        <label>Multimedia Appendix 5</label>
        <p>Ideal preprocessing workflow.</p>
        <media xlink:href="biomedeng_v7i2e42386_app5.docx" xlink:title="DOCX File , 14 KB"/>
      </supplementary-material>
    </app-group>
    <glossary>
      <title>Abbreviations</title>
      <def-list>
        <def-item>
          <term id="abb1">PRISMA</term>
          <def>
            <p>Preferred Reporting Items for Systematic Reviews and Meta-Analyses</p>
          </def>
        </def-item>
      </def-list>
    </glossary>
    <fn-group>
      <fn fn-type="conflict">
        <p>None declared.</p>
      </fn>
    </fn-group>
    <ref-list>
      <ref id="ref1">
        <label>1</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Duffy</surname>
              <given-names>S</given-names>
            </name>
            <name name-style="western">
              <surname>Lee</surname>
              <given-names>TH</given-names>
            </name>
          </person-group>
          <article-title>In-person health care as option B</article-title>
          <source>N Engl J Med</source>
          <year>2018</year>
          <month>01</month>
          <day>11</day>
          <volume>378</volume>
          <issue>2</issue>
          <fpage>104</fpage>
          <lpage>6</lpage>
          <pub-id pub-id-type="doi">10.1056/NEJMp1710735</pub-id>
          <pub-id pub-id-type="medline">29320653</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref2">
        <label>2</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Lawes-Wickwar</surname>
              <given-names>S</given-names>
            </name>
            <name name-style="western">
              <surname>McBain</surname>
              <given-names>H</given-names>
            </name>
            <name name-style="western">
              <surname>Mulligan</surname>
              <given-names>K</given-names>
            </name>
          </person-group>
          <article-title>Application and effectiveness of telehealth to support severe mental illness management: systematic review</article-title>
          <source>JMIR Ment Health</source>
          <year>2018</year>
          <month>11</month>
          <day>21</day>
          <volume>5</volume>
          <issue>4</issue>
          <fpage>e62</fpage>
          <comment>
            <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="https://mental.jmir.org/2018/4/e62/"/>
          </comment>
          <pub-id pub-id-type="doi">10.2196/mental.8816</pub-id>
          <pub-id pub-id-type="medline">30463836</pub-id>
          <pub-id pub-id-type="pii">v5i4e62</pub-id>
          <pub-id pub-id-type="pmcid">PMC6314801</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref3">
        <label>3</label>
        <nlm-citation citation-type="web">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Hunt</surname>
              <given-names>G</given-names>
            </name>
          </person-group>
          <article-title>Doorstop interview on 27 November 2020</article-title>
          <source>Ministers, Department of health and aged care</source>
          <year>2020</year>
          <month>11</month>
          <day>27</day>
          <access-date>2021-02-09</access-date>
          <comment>
            <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="https://www.health.gov.au/ministers/the-hon-greg-hunt-mp/media/doorstop-interview-on-27-november-2020">https://www.health.gov.au/ministers/the-hon-greg-hunt-mp/media/doorstop-interview-on-27-november-2020</ext-link>
          </comment>
        </nlm-citation>
      </ref>
      <ref id="ref4">
        <label>4</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Pfender</surname>
              <given-names>E</given-names>
            </name>
          </person-group>
          <article-title>Mental health and COVID-19: implications for the future of telehealth</article-title>
          <source>J Patient Exp</source>
          <year>2020</year>
          <month>08</month>
          <volume>7</volume>
          <issue>4</issue>
          <fpage>433</fpage>
          <lpage>5</lpage>
          <comment>
            <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="https://journals.sagepub.com/doi/10.1177/2374373520948436?url_ver=Z39.88-2003&#38;rfr_id=ori:rid:crossref.org&#38;rfr_dat=cr_pub  0pubmed"/>
          </comment>
          <pub-id pub-id-type="doi">10.1177/2374373520948436</pub-id>
          <pub-id pub-id-type="medline">33062854</pub-id>
          <pub-id pub-id-type="pii">10.1177_2374373520948436</pub-id>
          <pub-id pub-id-type="pmcid">PMC7415938</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref5">
        <label>5</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Jaffe</surname>
              <given-names>DH</given-names>
            </name>
            <name name-style="western">
              <surname>Lee</surname>
              <given-names>L</given-names>
            </name>
            <name name-style="western">
              <surname>Huynh</surname>
              <given-names>S</given-names>
            </name>
            <name name-style="western">
              <surname>Haskell</surname>
              <given-names>TP</given-names>
            </name>
          </person-group>
          <article-title>Health inequalities in the use of telehealth in the United States in the lens of COVID-19</article-title>
          <source>Popul Health Manag</source>
          <year>2020</year>
          <month>10</month>
          <volume>23</volume>
          <issue>5</issue>
          <fpage>368</fpage>
          <lpage>77</lpage>
          <pub-id pub-id-type="doi">10.1089/pop.2020.0186</pub-id>
          <pub-id pub-id-type="medline">32816644</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref6">
        <label>6</label>
        <nlm-citation citation-type="web">
          <person-group person-group-type="author">
            <collab>World Health Organisation</collab>
          </person-group>
          <article-title>Suicide in the world: Global health estimates</article-title>
          <source>World Health Organisation</source>
          <year>2019</year>
          <access-date>2021-02-12</access-date>
          <comment>
            <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="https://apps.who.int/iris/bitstream/handle/10665/326948/WHO-MSD-MER-19.3-eng.pdf">https://apps.who.int/iris/bitstream/handle/10665/326948/WHO-MSD-MER-19.3-eng.pdf</ext-link>
          </comment>
        </nlm-citation>
      </ref>
      <ref id="ref7">
        <label>7</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Crnek-Georgeson</surname>
              <given-names>KT</given-names>
            </name>
            <name name-style="western">
              <surname>Wilson</surname>
              <given-names>LA</given-names>
            </name>
            <name name-style="western">
              <surname>Page</surname>
              <given-names>A</given-names>
            </name>
          </person-group>
          <article-title>Factors influencing suicide in older rural males: a review of Australian studies</article-title>
          <source>Rural Remote Health</source>
          <year>2017</year>
          <volume>17</volume>
          <issue>4</issue>
          <fpage>4020</fpage>
          <lpage>4</lpage>
          <comment>
            <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="https://www.rrh.org.au/articles/subviewnew.asp?ArticleID=4020"/>
          </comment>
          <pub-id pub-id-type="doi">10.22605/RRH4020</pub-id>
          <pub-id pub-id-type="medline">29031280</pub-id>
          <pub-id pub-id-type="pii">4020</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref8">
        <label>8</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Hollander</surname>
              <given-names>JE</given-names>
            </name>
            <name name-style="western">
              <surname>Carr</surname>
              <given-names>BG</given-names>
            </name>
          </person-group>
          <article-title>Virtually perfect? Telemedicine for Covid-19</article-title>
          <source>N Engl J Med</source>
          <year>2020</year>
          <month>04</month>
          <day>30</day>
          <volume>382</volume>
          <issue>18</issue>
          <fpage>1679</fpage>
          <lpage>81</lpage>
          <pub-id pub-id-type="doi">10.1056/NEJMp2003539</pub-id>
          <pub-id pub-id-type="medline">32160451</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref9">
        <label>9</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Turecki</surname>
              <given-names>G</given-names>
            </name>
            <name name-style="western">
              <surname>Brent</surname>
              <given-names>DA</given-names>
            </name>
          </person-group>
          <article-title>Suicide and suicidal behaviour</article-title>
          <source>Lancet</source>
          <year>2016</year>
          <volume>387</volume>
          <issue>10024</issue>
          <fpage>1227</fpage>
          <lpage>39</lpage>
          <comment>
            <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="http://europepmc.org/abstract/MED/26385066"/>
          </comment>
          <pub-id pub-id-type="doi">10.1016/S0140-6736(15)00234-2</pub-id>
          <pub-id pub-id-type="medline">26385066</pub-id>
          <pub-id pub-id-type="pii">S0140-6736(15)00234-2</pub-id>
          <pub-id pub-id-type="pmcid">PMC5319859</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref10">
        <label>10</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Franklin</surname>
              <given-names>JC</given-names>
            </name>
            <name name-style="western">
              <surname>Ribeiro</surname>
              <given-names>JD</given-names>
            </name>
            <name name-style="western">
              <surname>Fox</surname>
              <given-names>KR</given-names>
            </name>
            <name name-style="western">
              <surname>Bentley</surname>
              <given-names>KH</given-names>
            </name>
            <name name-style="western">
              <surname>Kleiman</surname>
              <given-names>EM</given-names>
            </name>
            <name name-style="western">
              <surname>Huang</surname>
              <given-names>X</given-names>
            </name>
            <name name-style="western">
              <surname>Musacchio</surname>
              <given-names>KM</given-names>
            </name>
            <name name-style="western">
              <surname>Jaroszewski</surname>
              <given-names>AC</given-names>
            </name>
            <name name-style="western">
              <surname>Chang</surname>
              <given-names>BP</given-names>
            </name>
            <name name-style="western">
              <surname>Nock</surname>
              <given-names>MK</given-names>
            </name>
          </person-group>
          <article-title>Risk factors for suicidal thoughts and behaviors: a meta-analysis of 50 years of research</article-title>
          <source>Psychol Bull</source>
          <year>2017</year>
          <volume>143</volume>
          <issue>2</issue>
          <fpage>187</fpage>
          <lpage>232</lpage>
          <pub-id pub-id-type="doi">10.1037/bul0000084</pub-id>
          <pub-id pub-id-type="medline">27841450</pub-id>
          <pub-id pub-id-type="pii">2016-54856-001</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref11">
        <label>11</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Sudol</surname>
              <given-names>K</given-names>
            </name>
            <name name-style="western">
              <surname>Mann</surname>
              <given-names>JJ</given-names>
            </name>
          </person-group>
          <article-title>Biomarkers of suicide attempt behavior: towards a biological model of risk</article-title>
          <source>Curr Psychiatry Rep</source>
          <year>2017</year>
          <volume>19</volume>
          <issue>6</issue>
          <fpage>31</fpage>
          <pub-id pub-id-type="doi">10.1007/s11920-017-0781-y</pub-id>
          <pub-id pub-id-type="medline">28470485</pub-id>
          <pub-id pub-id-type="pii">10.1007/s11920-017-0781-y</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref12">
        <label>12</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Qu</surname>
              <given-names>C</given-names>
            </name>
            <name name-style="western">
              <surname>Sas</surname>
              <given-names>C</given-names>
            </name>
            <name name-style="western">
              <surname>Daudén Roquet</surname>
              <given-names>C</given-names>
            </name>
            <name name-style="western">
              <surname>Doherty</surname>
              <given-names>G</given-names>
            </name>
          </person-group>
          <article-title>Correction: functionality of top-rated mobile apps for depression: systematic search and evaluation</article-title>
          <source>JMIR Ment Health</source>
          <year>2020</year>
          <month>02</month>
          <day>21</day>
          <volume>7</volume>
          <issue>2</issue>
          <fpage>e18042</fpage>
          <comment>
            <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="https://mental.jmir.org/2020/2/e18042/"/>
          </comment>
          <pub-id pub-id-type="doi">10.2196/18042</pub-id>
          <pub-id pub-id-type="medline">32130145</pub-id>
          <pub-id pub-id-type="pii">v7i2e18042</pub-id>
          <pub-id pub-id-type="pmcid">PMC7060497</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref13">
        <label>13</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Balcombe</surname>
              <given-names>L</given-names>
            </name>
            <name name-style="western">
              <surname>De Leo</surname>
              <given-names>D</given-names>
            </name>
          </person-group>
          <article-title>An integrated blueprint for digital mental health services amidst COVID-19</article-title>
          <source>JMIR Ment Health</source>
          <year>2020</year>
          <month>07</month>
          <day>22</day>
          <volume>7</volume>
          <issue>7</issue>
          <fpage>e21718</fpage>
          <comment>
            <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="https://mental.jmir.org/2020/7/e21718/"/>
          </comment>
          <pub-id pub-id-type="doi">10.2196/21718</pub-id>
          <pub-id pub-id-type="medline">32668402</pub-id>
          <pub-id pub-id-type="pii">v7i7e21718</pub-id>
          <pub-id pub-id-type="pmcid">PMC7407255</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref14">
        <label>14</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Cummins</surname>
              <given-names>N</given-names>
            </name>
            <name name-style="western">
              <surname>Scherer</surname>
              <given-names>S</given-names>
            </name>
            <name name-style="western">
              <surname>Krajewski</surname>
              <given-names>J</given-names>
            </name>
            <name name-style="western">
              <surname>Schnieder</surname>
              <given-names>S</given-names>
            </name>
            <name name-style="western">
              <surname>Epps</surname>
              <given-names>J</given-names>
            </name>
            <name name-style="western">
              <surname>Quatieri</surname>
              <given-names>TF</given-names>
            </name>
          </person-group>
          <article-title>A review of depression and suicide risk assessment using speech analysis</article-title>
          <source>Speech Commun</source>
          <year>2015</year>
          <volume>71</volume>
          <fpage>10</fpage>
          <lpage>49</lpage>
          <pub-id pub-id-type="doi">10.1016/j.specom.2015.03.004</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref15">
        <label>15</label>
        <nlm-citation citation-type="confproc">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Scherer</surname>
              <given-names>S</given-names>
            </name>
            <name name-style="western">
              <surname>Pestian</surname>
              <given-names>J</given-names>
            </name>
            <name name-style="western">
              <surname>Morency</surname>
              <given-names>L-P</given-names>
            </name>
          </person-group>
          <article-title>Investigating the speech characteristics of suicidal adolescents</article-title>
          <source>Proceedings of the 2013 IEEE International Conference on Acoustics, Speech and Signal Processing</source>
          <year>2013</year>
          <month>10</month>
          <day>21</day>
          <conf-name>2013 IEEE International Conference on Acoustics, Speech Signal Process</conf-name>
          <conf-date>2013</conf-date>
          <conf-loc>Vancouver</conf-loc>
          <pub-id pub-id-type="doi">10.1109/icassp.2013.6637740</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref16">
        <label>16</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Homan</surname>
              <given-names>S</given-names>
            </name>
            <name name-style="western">
              <surname>Gabi</surname>
              <given-names>M</given-names>
            </name>
            <name name-style="western">
              <surname>Klee</surname>
              <given-names>N</given-names>
            </name>
            <name name-style="western">
              <surname>Bachmann</surname>
              <given-names>S</given-names>
            </name>
            <name name-style="western">
              <surname>Moser</surname>
              <given-names>A</given-names>
            </name>
            <name name-style="western">
              <surname>Duri'</surname>
              <given-names>M</given-names>
            </name>
            <name name-style="western">
              <surname>Michel</surname>
              <given-names>S</given-names>
            </name>
            <name name-style="western">
              <surname>Bertram</surname>
              <given-names>A</given-names>
            </name>
            <name name-style="western">
              <surname>Maatz</surname>
              <given-names>A</given-names>
            </name>
            <name name-style="western">
              <surname>Seiler</surname>
              <given-names>G</given-names>
            </name>
            <name name-style="western">
              <surname>Stark</surname>
              <given-names>E</given-names>
            </name>
            <name name-style="western">
              <surname>Kleim</surname>
              <given-names>B</given-names>
            </name>
          </person-group>
          <article-title>Linguistic features of suicidal thoughts and behaviors: a systematic review</article-title>
          <source>Clin Psychol Rev</source>
          <year>2022</year>
          <month>07</month>
          <volume>95</volume>
          <fpage>102161</fpage>
          <comment>
            <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="https://linkinghub.elsevier.com/retrieve/pii/S0272-7358(22)00046-0"/>
          </comment>
          <pub-id pub-id-type="doi">10.1016/j.cpr.2022.102161</pub-id>
          <pub-id pub-id-type="medline">35636131</pub-id>
          <pub-id pub-id-type="pii">S0272-7358(22)00046-0</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref17">
        <label>17</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Teferra</surname>
              <given-names>BG</given-names>
            </name>
            <name name-style="western">
              <surname>Borwein</surname>
              <given-names>S</given-names>
            </name>
            <name name-style="western">
              <surname>DeSouza</surname>
              <given-names>DD</given-names>
            </name>
            <name name-style="western">
              <surname>Simpson</surname>
              <given-names>W</given-names>
            </name>
            <name name-style="western">
              <surname>Rheault</surname>
              <given-names>L</given-names>
            </name>
            <name name-style="western">
              <surname>Rose</surname>
              <given-names>J</given-names>
            </name>
          </person-group>
          <article-title>Acoustic and linguistic features of impromptu speech and their association with anxiety: validation study</article-title>
          <source>JMIR Ment Health</source>
          <year>2022</year>
          <month>07</month>
          <day>08</day>
          <volume>9</volume>
          <issue>7</issue>
          <fpage>e36828</fpage>
          <comment>
            <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="https://mental.jmir.org/2022/7/e36828/"/>
          </comment>
          <pub-id pub-id-type="doi">10.2196/36828</pub-id>
          <pub-id pub-id-type="medline">35802401</pub-id>
          <pub-id pub-id-type="pii">v9i7e36828</pub-id>
          <pub-id pub-id-type="pmcid">PMC9308078</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref18">
        <label>18</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Liberati</surname>
              <given-names>A</given-names>
            </name>
            <name name-style="western">
              <surname>Altman</surname>
              <given-names>DG</given-names>
            </name>
            <name name-style="western">
              <surname>Tetzlaff</surname>
              <given-names>J</given-names>
            </name>
            <name name-style="western">
              <surname>Mulrow</surname>
              <given-names>C</given-names>
            </name>
            <name name-style="western">
              <surname>Gøtzsche</surname>
              <given-names>PC</given-names>
            </name>
            <name name-style="western">
              <surname>Ioannidis</surname>
              <given-names>JP</given-names>
            </name>
            <name name-style="western">
              <surname>Clarke</surname>
              <given-names>M</given-names>
            </name>
            <name name-style="western">
              <surname>Devereaux</surname>
              <given-names>PJ</given-names>
            </name>
            <name name-style="western">
              <surname>Kleijnen</surname>
              <given-names>J</given-names>
            </name>
            <name name-style="western">
              <surname>Moher</surname>
              <given-names>D</given-names>
            </name>
          </person-group>
          <article-title>The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration</article-title>
          <source>PLoS Med</source>
          <year>2009</year>
          <month>07</month>
          <day>21</day>
          <volume>6</volume>
          <issue>7</issue>
          <fpage>e1000100</fpage>
          <comment>
            <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="http://dx.plos.org/10.1371/journal.pmed.1000100"/>
          </comment>
          <pub-id pub-id-type="doi">10.1371/journal.pmed.1000100</pub-id>
          <pub-id pub-id-type="medline">19621070</pub-id>
          <pub-id pub-id-type="pmcid">PMC2707010</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref19">
        <label>19</label>
        <nlm-citation citation-type="web">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Iyer</surname>
              <given-names>R</given-names>
            </name>
            <name name-style="western">
              <surname>Meyer</surname>
              <given-names>D</given-names>
            </name>
            <name name-style="western">
              <surname>Nedeljkovic</surname>
              <given-names>M</given-names>
            </name>
            <name name-style="western">
              <surname>Seabrook</surname>
              <given-names>L</given-names>
            </name>
          </person-group>
          <article-title>Speech biomarkers of suicide risk: a systematic review</article-title>
          <source>PROSPERO</source>
          <year>2020</year>
          <access-date>2020-01-31</access-date>
          <comment>
            <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="https://www.crd.york.ac.uk/prospero/display_record.php?ID=CRD42020167413">https://www.crd.york.ac.uk/prospero/display_record.php?ID=CRD42020167413</ext-link>
          </comment>
        </nlm-citation>
      </ref>
      <ref id="ref20">
        <label>20</label>
        <nlm-citation citation-type="web">
          <article-title>NVIVO</article-title>
          <source>NVIVO QSR International</source>
          <access-date>2020-02-01</access-date>
          <comment>
            <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="https://www.qsrinternational.com/nvivo-qualitative-data-analysis-software/home">https://www.qsrinternational.com/nvivo-qualitative-data-analysis-software/home</ext-link>
          </comment>
        </nlm-citation>
      </ref>
      <ref id="ref21">
        <label>21</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Sterne</surname>
              <given-names>JA</given-names>
            </name>
            <name name-style="western">
              <surname>Hernán</surname>
              <given-names>MA</given-names>
            </name>
            <name name-style="western">
              <surname>Reeves</surname>
              <given-names>BC</given-names>
            </name>
            <name name-style="western">
              <surname>Savović</surname>
              <given-names>J</given-names>
            </name>
            <name name-style="western">
              <surname>Berkman</surname>
              <given-names>ND</given-names>
            </name>
            <name name-style="western">
              <surname>Viswanathan</surname>
              <given-names>M</given-names>
            </name>
            <name name-style="western">
              <surname>Henry</surname>
              <given-names>D</given-names>
            </name>
            <name name-style="western">
              <surname>Altman</surname>
              <given-names>DG</given-names>
            </name>
            <name name-style="western">
              <surname>Ansari</surname>
              <given-names>MT</given-names>
            </name>
            <name name-style="western">
              <surname>Boutron</surname>
              <given-names>I</given-names>
            </name>
            <name name-style="western">
              <surname>Carpenter</surname>
              <given-names>JR</given-names>
            </name>
            <name name-style="western">
              <surname>Chan</surname>
              <given-names>A</given-names>
            </name>
            <name name-style="western">
              <surname>Churchill</surname>
              <given-names>R</given-names>
            </name>
            <name name-style="western">
              <surname>Deeks</surname>
              <given-names>JJ</given-names>
            </name>
            <name name-style="western">
              <surname>Hróbjartsson</surname>
              <given-names>A</given-names>
            </name>
            <name name-style="western">
              <surname>Kirkham</surname>
              <given-names>J</given-names>
            </name>
            <name name-style="western">
              <surname>Jüni</surname>
              <given-names>P</given-names>
            </name>
            <name name-style="western">
              <surname>Loke</surname>
              <given-names>YK</given-names>
            </name>
            <name name-style="western">
              <surname>Pigott</surname>
              <given-names>TD</given-names>
            </name>
            <name name-style="western">
              <surname>Ramsay</surname>
              <given-names>CR</given-names>
            </name>
            <name name-style="western">
              <surname>Regidor</surname>
              <given-names>D</given-names>
            </name>
            <name name-style="western">
              <surname>Rothstein</surname>
              <given-names>HR</given-names>
            </name>
            <name name-style="western">
              <surname>Sandhu</surname>
              <given-names>L</given-names>
            </name>
            <name name-style="western">
              <surname>Santaguida</surname>
              <given-names>PL</given-names>
            </name>
            <name name-style="western">
              <surname>Schünemann</surname>
              <given-names>HJ</given-names>
            </name>
            <name name-style="western">
              <surname>Shea</surname>
              <given-names>B</given-names>
            </name>
            <name name-style="western">
              <surname>Shrier</surname>
              <given-names>I</given-names>
            </name>
            <name name-style="western">
              <surname>Tugwell</surname>
              <given-names>P</given-names>
            </name>
            <name name-style="western">
              <surname>Turner</surname>
              <given-names>L</given-names>
            </name>
            <name name-style="western">
              <surname>Valentine</surname>
              <given-names>JC</given-names>
            </name>
            <name name-style="western">
              <surname>Waddington</surname>
              <given-names>H</given-names>
            </name>
            <name name-style="western">
              <surname>Waters</surname>
              <given-names>E</given-names>
            </name>
            <name name-style="western">
              <surname>Wells</surname>
              <given-names>GA</given-names>
            </name>
            <name name-style="western">
              <surname>Whiting</surname>
              <given-names>PF</given-names>
            </name>
            <name name-style="western">
              <surname>Higgins</surname>
              <given-names>JP</given-names>
            </name>
          </person-group>
          <article-title>ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions</article-title>
          <source>BMJ</source>
          <year>2016</year>
          <month>10</month>
          <day>12</day>
          <volume>355</volume>
          <fpage>i4919</fpage>
          <comment>
            <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="http://www.bmj.com/lookup/pmidlookup?view=long&#38;pmid=27733354"/>
          </comment>
          <pub-id pub-id-type="doi">10.1136/bmj.i4919</pub-id>
          <pub-id pub-id-type="medline">27733354</pub-id>
          <pub-id pub-id-type="pmcid">PMC5062054</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref22">
        <label>22</label>
        <nlm-citation citation-type="confproc">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Scherer</surname>
              <given-names>S</given-names>
            </name>
            <name name-style="western">
              <surname>Morency</surname>
              <given-names>L</given-names>
            </name>
            <name name-style="western">
              <surname>Gratch</surname>
              <given-names>J</given-names>
            </name>
            <name name-style="western">
              <surname>Pestian</surname>
              <given-names>J</given-names>
            </name>
          </person-group>
          <article-title>Reduced vowel space is a robust indicator of psychological distress: a cross-corpus analysis</article-title>
          <source>Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing</source>
          <year>2015</year>
          <conf-name>IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)</conf-name>
          <conf-date>2015</conf-date>
          <conf-loc>Brisbane</conf-loc>
          <publisher-name>IEEE</publisher-name>
          <pub-id pub-id-type="doi">10.1109/ICASSP.2015.7178880</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref23">
        <label>23</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Venek</surname>
              <given-names>V</given-names>
            </name>
            <name name-style="western">
              <surname>Scherer</surname>
              <given-names>S</given-names>
            </name>
            <name name-style="western">
              <surname>Morency</surname>
              <given-names>L</given-names>
            </name>
            <name name-style="western">
              <surname>Rizzo</surname>
              <given-names>AS</given-names>
            </name>
            <name name-style="western">
              <surname>Pestian</surname>
              <given-names>J</given-names>
            </name>
          </person-group>
          <article-title>Adolescent suicidal risk assessment in clinician-patient interaction</article-title>
          <source>IEEE Trans Affective Comput</source>
          <year>2017</year>
          <month>4</month>
          <day>1</day>
          <volume>8</volume>
          <issue>2</issue>
          <fpage>204</fpage>
          <lpage>15</lpage>
          <pub-id pub-id-type="doi">10.1109/TAFFC.2016.2518665</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref24">
        <label>24</label>
        <nlm-citation citation-type="book">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Campbell</surname>
              <given-names>L</given-names>
            </name>
          </person-group>
          <source>Statistical Characteristics of Fundamental Frequency Distributions in the Speech of Suicidal Patients</source>
          <year>1995</year>
          <publisher-loc>Nashville</publisher-loc>
          <publisher-name>Vanderbilt University</publisher-name>
        </nlm-citation>
      </ref>
      <ref id="ref25">
        <label>25</label>
        <nlm-citation citation-type="web">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Sanadi</surname>
              <given-names>W</given-names>
            </name>
            <name name-style="western">
              <surname>Hasan</surname>
              <given-names>WA</given-names>
            </name>
          </person-group>
          <article-title>Acoustic analysis of speech based on power spectral density features in detecting suicidal risk among female patients dissertation</article-title>
          <source>Vanderbilt University Institutional Repository</source>
          <year>2011</year>
          <access-date>2022-12-09</access-date>
          <comment>
            <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="https://etd.library.vanderbilt.edu/etd-03252011-142343">https://etd.library.vanderbilt.edu/etd-03252011-142343</ext-link>
          </comment>
        </nlm-citation>
      </ref>
      <ref id="ref26">
        <label>26</label>
        <nlm-citation citation-type="web">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Sinha</surname>
              <given-names>A</given-names>
            </name>
          </person-group>
          <article-title>Human and machine recognition of the vocal characteristics of suicide</article-title>
          <source>Vanderbilt University</source>
          <year>2013</year>
          <month>12</month>
          <access-date>2022-12-09</access-date>
          <comment>
            <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="https://ir.vanderbilt.edu/bitstream/handle/1803/15036/SINHA.pdf?sequence=1">https://ir.vanderbilt.edu/bitstream/handle/1803/15036/SINHA.pdf?sequence=1</ext-link>
          </comment>
        </nlm-citation>
      </ref>
      <ref id="ref27">
        <label>27</label>
        <nlm-citation citation-type="confproc">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Anunvrapong</surname>
              <given-names>P</given-names>
            </name>
            <name name-style="western">
              <surname>Yingthawornthuk</surname>
              <given-names>T</given-names>
            </name>
          </person-group>
          <article-title>Characterisation MCFF in depressed speech sample as assessment of suicidal risk</article-title>
          <source>Proceedings of the 2014 International Conference on Advanced Computational Technologies &#38; Creative Media (ICACTCM)</source>
          <year>2014</year>
          <conf-name>International Conference on Advanced Computational Technologies &#38; Creative Media (ICACTCM?) Aug</conf-name>
          <conf-date>2014</conf-date>
          <conf-loc>Pattaya</conf-loc>
          <pub-id pub-id-type="doi">10.15242/IIE.E0814549</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref28">
        <label>28</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Belouali</surname>
              <given-names>A</given-names>
            </name>
            <name name-style="western">
              <surname>Gupta</surname>
              <given-names>S</given-names>
            </name>
            <name name-style="western">
              <surname>Sourirajan</surname>
              <given-names>V</given-names>
            </name>
            <name name-style="western">
              <surname>Yu</surname>
              <given-names>J</given-names>
            </name>
            <name name-style="western">
              <surname>Allen</surname>
              <given-names>N</given-names>
            </name>
            <name name-style="western">
              <surname>Alaoui</surname>
              <given-names>A</given-names>
            </name>
            <name name-style="western">
              <surname>Dutton</surname>
              <given-names>MA</given-names>
            </name>
            <name name-style="western">
              <surname>Reinhard</surname>
              <given-names>MJ</given-names>
            </name>
          </person-group>
          <article-title>Acoustic and language analysis of speech for suicidal ideation among US veterans</article-title>
          <source>BioData Min</source>
          <year>2021</year>
          <month>02</month>
          <day>02</day>
          <volume>14</volume>
          <issue>1</issue>
          <fpage>11</fpage>
          <comment>
            <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="https://biodatamining.biomedcentral.com/articles/10.1186/s13040-021-00245-y"/>
          </comment>
          <pub-id pub-id-type="doi">10.1186/s13040-021-00245-y</pub-id>
          <pub-id pub-id-type="medline">33531048</pub-id>
          <pub-id pub-id-type="pii">10.1186/s13040-021-00245-y</pub-id>
          <pub-id pub-id-type="pmcid">PMC7856815</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref29">
        <label>29</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Figueroa Saavedra</surname>
              <given-names>C</given-names>
            </name>
            <name name-style="western">
              <surname>Otzen Hernández</surname>
              <given-names>T</given-names>
            </name>
            <name name-style="western">
              <surname>Alarcón Godoy</surname>
              <given-names>C</given-names>
            </name>
            <name name-style="western">
              <surname>Ríos Pérez</surname>
              <given-names>A</given-names>
            </name>
            <name name-style="western">
              <surname>Frugone Salinas</surname>
              <given-names>D</given-names>
            </name>
            <name name-style="western">
              <surname>Lagos Hernández</surname>
              <given-names>R</given-names>
            </name>
          </person-group>
          <article-title>Association between suicidal ideation and acoustic parameters of university students' voice and speech: a pilot study</article-title>
          <source>Logoped Phoniatr Vocol</source>
          <year>2021</year>
          <month>07</month>
          <volume>46</volume>
          <issue>2</issue>
          <fpage>55</fpage>
          <lpage>62</lpage>
          <pub-id pub-id-type="doi">10.1080/14015439.2020.1733075</pub-id>
          <pub-id pub-id-type="medline">32138570</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref30">
        <label>30</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>France</surname>
              <given-names>DJ</given-names>
            </name>
            <name name-style="western">
              <surname>Shiavi</surname>
              <given-names>RG</given-names>
            </name>
            <name name-style="western">
              <surname>Silverman</surname>
              <given-names>S</given-names>
            </name>
            <name name-style="western">
              <surname>Silverman</surname>
              <given-names>M</given-names>
            </name>
            <name name-style="western">
              <surname>Wilkes</surname>
              <given-names>DM</given-names>
            </name>
          </person-group>
          <article-title>Acoustical properties of speech as indicators of depression and suicidal risk</article-title>
          <source>IEEE Trans Biomed Eng</source>
          <year>2000</year>
          <volume>47</volume>
          <issue>7</issue>
          <fpage>829</fpage>
          <lpage>37</lpage>
          <pub-id pub-id-type="doi">10.1109/10.846676</pub-id>
          <pub-id pub-id-type="medline">10916253</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref31">
        <label>31</label>
        <nlm-citation citation-type="confproc">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Keskinpala</surname>
              <given-names>H</given-names>
            </name>
            <name name-style="western">
              <surname>Yingthawornsuk</surname>
              <given-names>T</given-names>
            </name>
            <name name-style="western">
              <surname>Wilkes</surname>
              <given-names>D</given-names>
            </name>
            <name name-style="western">
              <surname>Shiavi</surname>
              <given-names>R</given-names>
            </name>
            <name name-style="western">
              <surname>Salomon</surname>
              <given-names>R</given-names>
            </name>
          </person-group>
          <article-title>Screening for high risk suicidal states using mel-cepstral coefficients and energy in frequency bands</article-title>
          <source>Proceedings of the 15th European Signal Processing Conference</source>
          <year>2007</year>
          <conf-name>2007 15th European Signal Processing Conference</conf-name>
          <conf-date>Sep 03-07, 2007</conf-date>
          <conf-loc>Poznan</conf-loc>
          <comment>
            <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&#38;arnumber=7099204&#38;isnumber=7067186"/>
          </comment>
          <pub-id pub-id-type="doi">10.21437/interspeech.2007-144</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref32">
        <label>32</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Nik</surname>
              <given-names>HN</given-names>
            </name>
            <name name-style="western">
              <surname>Wilkes</surname>
              <given-names>M</given-names>
            </name>
            <name name-style="western">
              <surname>Salomon</surname>
              <given-names>R</given-names>
            </name>
          </person-group>
          <article-title>Timing patterns of speech as potential indicators of near-term suicidal risk</article-title>
          <source>Int J Multidisciplinary Curr Res</source>
          <year>2015</year>
          <volume>3</volume>
          <fpage>1102</fpage>
          <lpage>16</lpage>
          <comment>
            <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="http://ipcsit.com/vol58/002-ICSPS2012-P039.pdf"/>
          </comment>
        </nlm-citation>
      </ref>
      <ref id="ref33">
        <label>33</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Hashim</surname>
              <given-names>NW</given-names>
            </name>
            <name name-style="western">
              <surname>Wilkes</surname>
              <given-names>M</given-names>
            </name>
            <name name-style="western">
              <surname>Salomon</surname>
              <given-names>R</given-names>
            </name>
            <name name-style="western">
              <surname>Meggs</surname>
              <given-names>J</given-names>
            </name>
            <name name-style="western">
              <surname>France</surname>
              <given-names>DJ</given-names>
            </name>
          </person-group>
          <article-title>Evaluation of voice acoustics as predictors of clinical depression scores</article-title>
          <source>J Voice</source>
          <year>2017</year>
          <volume>31</volume>
          <issue>2</issue>
          <fpage>256.e1</fpage>
          <lpage>.e6</lpage>
          <pub-id pub-id-type="doi">10.1016/j.jvoice.2016.06.006</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref34">
        <label>34</label>
        <nlm-citation citation-type="confproc">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Ozdas</surname>
              <given-names>A</given-names>
            </name>
            <name name-style="western">
              <surname>Shiavi</surname>
              <given-names>R</given-names>
            </name>
            <name name-style="western">
              <surname>Silverman</surname>
              <given-names>S</given-names>
            </name>
            <name name-style="western">
              <surname>Silverman</surname>
              <given-names>M</given-names>
            </name>
            <name name-style="western">
              <surname>Wilkes</surname>
              <given-names>D</given-names>
            </name>
          </person-group>
          <article-title>Analysis of fundamental frequency for near term suicidal risk assessment</article-title>
          <source>Proceedings of the IEEE International Conference on Systems, Man and Cybernetics</source>
          <year>2000</year>
          <conf-name>Proceedings of the IEEE International Conference on Systems, Man and Cybernetics</conf-name>
          <conf-date>Oct 08-11, 2000</conf-date>
          <conf-loc>Nashville, TN, USA</conf-loc>
          <pub-id pub-id-type="doi">10.1109/icsmc.2000.886379</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref35">
        <label>35</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Ozdas</surname>
              <given-names>A</given-names>
            </name>
            <name name-style="western">
              <surname>Shiavi</surname>
              <given-names>RG</given-names>
            </name>
            <name name-style="western">
              <surname>Wilkes</surname>
              <given-names>DM</given-names>
            </name>
            <name name-style="western">
              <surname>Silverman</surname>
              <given-names>MK</given-names>
            </name>
            <name name-style="western">
              <surname>Silverman</surname>
              <given-names>SE</given-names>
            </name>
          </person-group>
          <article-title>Analysis of vocal tract characteristics for near-term suicidal risk assessment</article-title>
          <source>Methods Inf Med</source>
          <year>2018</year>
          <volume>43</volume>
          <issue>01</issue>
          <fpage>36</fpage>
          <lpage>8</lpage>
          <pub-id pub-id-type="doi">10.1055/s-0038-1633420</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref36">
        <label>36</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Ozdas</surname>
              <given-names>A</given-names>
            </name>
            <name name-style="western">
              <surname>Shiavi</surname>
              <given-names>RG</given-names>
            </name>
            <name name-style="western">
              <surname>Silverman</surname>
              <given-names>SE</given-names>
            </name>
            <name name-style="western">
              <surname>Silverman</surname>
              <given-names>MK</given-names>
            </name>
            <name name-style="western">
              <surname>Wilkes</surname>
              <given-names>DM</given-names>
            </name>
          </person-group>
          <article-title>Investigation of vocal jitter and glottal flow spectrum as possible cues for depression and near-term suicidal risk</article-title>
          <source>IEEE Trans Biomed Eng</source>
          <year>2004</year>
          <volume>51</volume>
          <issue>9</issue>
          <fpage>1530</fpage>
          <lpage>40</lpage>
          <pub-id pub-id-type="doi">10.1109/TBME.2004.827544</pub-id>
          <pub-id pub-id-type="medline">15376501</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref37">
        <label>37</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Pestian</surname>
              <given-names>JP</given-names>
            </name>
            <name name-style="western">
              <surname>Sorter</surname>
              <given-names>M</given-names>
            </name>
            <name name-style="western">
              <surname>Connolly</surname>
              <given-names>B</given-names>
            </name>
            <name name-style="western">
              <surname>Bretonnel Cohen</surname>
              <given-names>K</given-names>
            </name>
            <name name-style="western">
              <surname>McCullumsmith</surname>
              <given-names>C</given-names>
            </name>
            <name name-style="western">
              <surname>Gee</surname>
              <given-names>JT</given-names>
            </name>
            <name name-style="western">
              <surname>Morency</surname>
              <given-names>L</given-names>
            </name>
            <name name-style="western">
              <surname>Scherer</surname>
              <given-names>S</given-names>
            </name>
            <name name-style="western">
              <surname>Rohlfs</surname>
              <given-names>L</given-names>
            </name>
            <name name-style="western">
              <surname>Research Group</surname>
              <given-names>ST</given-names>
            </name>
          </person-group>
          <article-title>A machine learning approach to identifying the thought markers of suicidal subjects: a prospective multicenter trial</article-title>
          <source>Suicide Life Threat Behav</source>
          <year>2017</year>
          <volume>47</volume>
          <issue>1</issue>
          <fpage>112</fpage>
          <lpage>21</lpage>
          <pub-id pub-id-type="doi">10.1111/sltb.12312</pub-id>
          <pub-id pub-id-type="medline">27813129</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref38">
        <label>38</label>
        <nlm-citation citation-type="confproc">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Subari</surname>
              <given-names>K</given-names>
            </name>
            <name name-style="western">
              <surname>Wilkes</surname>
              <given-names>D</given-names>
            </name>
            <name name-style="western">
              <surname>Shiavi</surname>
              <given-names>R</given-names>
            </name>
            <name name-style="western">
              <surname>Silverman</surname>
              <given-names>S</given-names>
            </name>
            <name name-style="western">
              <surname>Silverman</surname>
              <given-names>M</given-names>
            </name>
          </person-group>
          <article-title>Comparison of speaker normalization techniques for classification of emotionally disturbed subjects based on voice</article-title>
          <source>Proceedings of the IEEE EMBS Conference on Biomedical Engineering and Sciences</source>
          <year>2010</year>
          <conf-name>IEEE EMBS Conference on Biomedical Engineering and Sciences</conf-name>
          <conf-date>Nov 30-Dec 2, 2010</conf-date>
          <conf-loc>Kuala Lumpur</conf-loc>
          <pub-id pub-id-type="doi">10.1109/iecbes.2010.5742248</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref39">
        <label>39</label>
        <nlm-citation citation-type="confproc">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Yingthawornsuk</surname>
              <given-names>T</given-names>
            </name>
            <name name-style="western">
              <surname>Keskinpala</surname>
              <given-names>K</given-names>
            </name>
            <name name-style="western">
              <surname>France</surname>
              <given-names>D</given-names>
            </name>
          </person-group>
          <article-title>Objective estimation of suicidal risk using vocal output characteristics</article-title>
          <source>Proceedings of the Interspeech - 9th International Conference on Spoken Language Processing</source>
          <year>2006</year>
          <conf-name>Interspeech - 9th International Conference on Spoken Language Processing</conf-name>
          <conf-date>Sep 17-21, 2006</conf-date>
          <conf-loc>Pittsburgh</conf-loc>
          <comment>
            <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="https://www.isca-speech.org/archive/archive_papers/interspeech_2007/i07_0766.pdf"/>
          </comment>
          <pub-id pub-id-type="doi">10.21437/interspeech.2006-231</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref40">
        <label>40</label>
        <nlm-citation citation-type="confproc">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Yingthawornsuk</surname>
              <given-names>T</given-names>
            </name>
            <name name-style="western">
              <surname>Keskinpala</surname>
              <given-names>K</given-names>
            </name>
            <name name-style="western">
              <surname>Wilkes</surname>
              <given-names>D</given-names>
            </name>
            <name name-style="western">
              <surname>Shiavi</surname>
              <given-names>R</given-names>
            </name>
            <name name-style="western">
              <surname>Salomon</surname>
              <given-names>R</given-names>
            </name>
          </person-group>
          <article-title>Direct Acoustic Feature Using Iterative EM Algorithm and Spectral Energy for Classifying Suicidal Speech</article-title>
          <source>Proceedings of the Interspeech - 8th International Conference on Spoken Language Processing</source>
          <year>2007</year>
          <conf-name>Interspeech - 8th International Conference on Spoken Language Processing</conf-name>
          <conf-date>Aug 27-31, 2007</conf-date>
          <conf-loc>Antwerp, Belgium</conf-loc>
          <pub-id pub-id-type="doi">10.21437/interspeech.2007-144</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref41">
        <label>41</label>
        <nlm-citation citation-type="confproc">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Yingthawornsuk</surname>
              <given-names>T</given-names>
            </name>
            <name name-style="western">
              <surname>Shiavi</surname>
              <given-names>R</given-names>
            </name>
          </person-group>
          <article-title>Distinguishing depression and suicidal risk in men using GMM based frequency contents of affective vocal tract response</article-title>
          <source>Proceedings of the International Conference on Control, Automation and Systems</source>
          <year>2008</year>
          <conf-name>International Conference on Control, Automation and Systems</conf-name>
          <conf-date>Oct 14-17, 2008</conf-date>
          <conf-loc>Seoul</conf-loc>
          <pub-id pub-id-type="doi">10.1109/iccas.2008.4694621</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref42">
        <label>42</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Rodgers</surname>
              <given-names>M</given-names>
            </name>
            <name name-style="western">
              <surname>Sowden</surname>
              <given-names>A</given-names>
            </name>
            <name name-style="western">
              <surname>Petticrew</surname>
              <given-names>M</given-names>
            </name>
            <name name-style="western">
              <surname>Arai</surname>
              <given-names>L</given-names>
            </name>
            <name name-style="western">
              <surname>Roberts</surname>
              <given-names>H</given-names>
            </name>
            <name name-style="western">
              <surname>Britten</surname>
              <given-names>N</given-names>
            </name>
            <name name-style="western">
              <surname>Popay</surname>
              <given-names>J</given-names>
            </name>
          </person-group>
          <article-title>Testing methodological guidance on the conduct of narrative synthesis in systematic reviews</article-title>
          <source>Evaluation</source>
          <year>2009</year>
          <volume>15</volume>
          <issue>1</issue>
          <fpage>49</fpage>
          <lpage>73</lpage>
          <pub-id pub-id-type="doi">10.1177/1356389008097871</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref43">
        <label>43</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Henry</surname>
              <given-names>WP</given-names>
            </name>
            <name name-style="western">
              <surname>Strupp</surname>
              <given-names>HH</given-names>
            </name>
            <name name-style="western">
              <surname>Butler</surname>
              <given-names>SF</given-names>
            </name>
            <name name-style="western">
              <surname>Schacht</surname>
              <given-names>TE</given-names>
            </name>
            <name name-style="western">
              <surname>Binder</surname>
              <given-names>JL</given-names>
            </name>
          </person-group>
          <article-title>Effects of training in time-limited dynamic psychotherapy: changes in therapist behavior</article-title>
          <source>J Consulting Clin Psychol</source>
          <year>1993</year>
          <volume>61</volume>
          <issue>3</issue>
          <fpage>434</fpage>
          <lpage>40</lpage>
          <pub-id pub-id-type="doi">10.1037/0022-006x.61.3.434</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref44">
        <label>44</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Hollon</surname>
              <given-names>SD</given-names>
            </name>
            <name name-style="western">
              <surname>DeRubeis</surname>
              <given-names>RJ</given-names>
            </name>
            <name name-style="western">
              <surname>Evans</surname>
              <given-names>MD</given-names>
            </name>
            <name name-style="western">
              <surname>Wiemer</surname>
              <given-names>MJ</given-names>
            </name>
            <name name-style="western">
              <surname>Garvey</surname>
              <given-names>MJ</given-names>
            </name>
            <name name-style="western">
              <surname>Grove</surname>
              <given-names>WM</given-names>
            </name>
            <name name-style="western">
              <surname>Tuason</surname>
              <given-names>VB</given-names>
            </name>
          </person-group>
          <article-title>Cognitive therapy and pharmacotherapy for depression. Singly and in combination</article-title>
          <source>Arch Gen Psychiatry</source>
          <year>1992</year>
          <volume>49</volume>
          <issue>10</issue>
          <fpage>774</fpage>
          <lpage>81</lpage>
          <pub-id pub-id-type="doi">10.1001/archpsyc.1992.01820100018004</pub-id>
          <pub-id pub-id-type="medline">1417429</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref45">
        <label>45</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Silverman</surname>
              <given-names>M</given-names>
            </name>
            <name name-style="western">
              <surname>Silverman</surname>
              <given-names>S</given-names>
            </name>
          </person-group>
          <article-title>From sound to silence: a preliminary investigation of the use of vocal parameters in the prediction of near-term suicidal risk</article-title>
          <source>J Med Psychotherapy</source>
          <year>2000</year>
          <fpage>1</fpage>
          <lpage>10</lpage>
          <comment>(forthcoming)</comment>
        </nlm-citation>
      </ref>
      <ref id="ref46">
        <label>46</label>
        <nlm-citation citation-type="confproc">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Gratch</surname>
              <given-names>J</given-names>
            </name>
            <name name-style="western">
              <surname>Arstein</surname>
              <given-names>R</given-names>
            </name>
            <name name-style="western">
              <surname>Lucas</surname>
              <given-names>G</given-names>
            </name>
            <name name-style="western">
              <surname>Stratou</surname>
              <given-names>G</given-names>
            </name>
          </person-group>
          <article-title>The distress analysis interview corpus of human and computer interviews</article-title>
          <source>Proceedings of the Ninth International Conference on Language Resources and Evaluation</source>
          <year>2014</year>
          <conf-name>Ninth International Conference on Language Resources and Evaluation</conf-name>
          <conf-date>May 26-31, 2014</conf-date>
          <conf-loc>Reykjavik</conf-loc>
          <pub-id pub-id-type="doi">10.1016/b978-0-12-410393-1.00011-9</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref47">
        <label>47</label>
        <nlm-citation citation-type="web">
          <article-title>Microstudio</article-title>
          <source>MTU Technology Unlimited (MTU)</source>
          <access-date>2020-03-12</access-date>
          <comment>
            <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="http://www.mtu.com/upgrades/microstudio.htm">http://www.mtu.com/upgrades/microstudio.htm</ext-link>
          </comment>
        </nlm-citation>
      </ref>
      <ref id="ref48">
        <label>48</label>
        <nlm-citation citation-type="web">
          <article-title>MATLAB version 7</article-title>
          <source>The Mathworks Inc</source>
          <access-date>2020-03-10</access-date>
          <comment>
            <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="https://au.mathworks.com/products/matlab.html">https://au.mathworks.com/products/matlab.html</ext-link>
          </comment>
        </nlm-citation>
      </ref>
      <ref id="ref49">
        <label>49</label>
        <nlm-citation citation-type="confproc">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Degottex</surname>
              <given-names>G</given-names>
            </name>
            <name name-style="western">
              <surname>Drugman</surname>
              <given-names>T</given-names>
            </name>
            <name name-style="western">
              <surname>Raitio</surname>
              <given-names>T</given-names>
            </name>
            <name name-style="western">
              <surname>Scherer</surname>
              <given-names>S</given-names>
            </name>
          </person-group>
          <article-title>COVAREP A collaborative voice analysis repository for speech technologies</article-title>
          <source>Proceedings of the IEEE International Conference on Acoustics, Speech and Signal processing (ICASSP)</source>
          <year>2014</year>
          <conf-name>2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)</conf-name>
          <conf-date>May 04-09, 2014</conf-date>
          <conf-loc>Florence</conf-loc>
          <pub-id pub-id-type="doi">10.1109/icassp.2014.6853739</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref50">
        <label>50</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Moltu</surname>
              <given-names>C</given-names>
            </name>
            <name name-style="western">
              <surname>Stefansen</surname>
              <given-names>J</given-names>
            </name>
            <name name-style="western">
              <surname>Svisdahl</surname>
              <given-names>M</given-names>
            </name>
            <name name-style="western">
              <surname>Veseth</surname>
              <given-names>M</given-names>
            </name>
          </person-group>
          <article-title>Negotiating the coresearcher mandate - service users' experiences of doing collaborative research on mental health</article-title>
          <source>Disabil Rehabil</source>
          <year>2012</year>
          <volume>34</volume>
          <issue>19</issue>
          <fpage>1608</fpage>
          <lpage>16</lpage>
          <pub-id pub-id-type="doi">10.3109/09638288.2012.656792</pub-id>
          <pub-id pub-id-type="medline">22489612</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref51">
        <label>51</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Nik</surname>
              <given-names>HN</given-names>
            </name>
            <name name-style="western">
              <surname>Wilkes</surname>
              <given-names>M</given-names>
            </name>
            <name name-style="western">
              <surname>Salomon</surname>
              <given-names>R</given-names>
            </name>
            <name name-style="western">
              <surname>Meggs</surname>
              <given-names>J</given-names>
            </name>
          </person-group>
          <article-title>Analysis of timing pattern of speech as possible indicator for near-term suicidal risk and depression in male patients</article-title>
          <source>Int Proceed Comp Sci Inf Tech</source>
          <year>2012</year>
          <volume>58</volume>
          <fpage>6</fpage>
        </nlm-citation>
      </ref>
      <ref id="ref52">
        <label>52</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Hallensleben</surname>
              <given-names>N</given-names>
            </name>
            <name name-style="western">
              <surname>Spangenberg</surname>
              <given-names>L</given-names>
            </name>
            <name name-style="western">
              <surname>Forkmann</surname>
              <given-names>T</given-names>
            </name>
            <name name-style="western">
              <surname>Rath</surname>
              <given-names>D</given-names>
            </name>
            <name name-style="western">
              <surname>Hegerl</surname>
              <given-names>U</given-names>
            </name>
            <name name-style="western">
              <surname>Kersting</surname>
              <given-names>A</given-names>
            </name>
            <name name-style="western">
              <surname>Kallert</surname>
              <given-names>TW</given-names>
            </name>
            <name name-style="western">
              <surname>Glaesmer</surname>
              <given-names>H</given-names>
            </name>
          </person-group>
          <article-title>Investigating the dynamics of suicidal ideation</article-title>
          <source>Crisis</source>
          <year>2018</year>
          <volume>39</volume>
          <issue>1</issue>
          <fpage>65</fpage>
          <lpage>9</lpage>
          <pub-id pub-id-type="doi">10.1027/0227-5910/a000464</pub-id>
          <pub-id pub-id-type="medline">28468557</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref53">
        <label>53</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Galatzer-Levy</surname>
              <given-names>I</given-names>
            </name>
            <name name-style="western">
              <surname>Abbas</surname>
              <given-names>A</given-names>
            </name>
            <name name-style="western">
              <surname>Ries</surname>
              <given-names>A</given-names>
            </name>
            <name name-style="western">
              <surname>Homan</surname>
              <given-names>S</given-names>
            </name>
            <name name-style="western">
              <surname>Sels</surname>
              <given-names>L</given-names>
            </name>
            <name name-style="western">
              <surname>Koesmahargyo</surname>
              <given-names>V</given-names>
            </name>
            <name name-style="western">
              <surname>Yadav</surname>
              <given-names>V</given-names>
            </name>
            <name name-style="western">
              <surname>Colla</surname>
              <given-names>M</given-names>
            </name>
            <name name-style="western">
              <surname>Scheerer</surname>
              <given-names>H</given-names>
            </name>
            <name name-style="western">
              <surname>Vetter</surname>
              <given-names>S</given-names>
            </name>
            <name name-style="western">
              <surname>Seifritz</surname>
              <given-names>E</given-names>
            </name>
            <name name-style="western">
              <surname>Scholz</surname>
              <given-names>U</given-names>
            </name>
            <name name-style="western">
              <surname>Kleim</surname>
              <given-names>B</given-names>
            </name>
          </person-group>
          <article-title>Validation of visual and auditory digital markers of suicidality in acutely suicidal psychiatric inpatients: proof-of-concept study</article-title>
          <source>J Med Internet Res</source>
          <year>2021</year>
          <month>06</month>
          <day>03</day>
          <volume>23</volume>
          <issue>6</issue>
          <fpage>e25199</fpage>
          <comment>
            <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="https://www.jmir.org/2021/6/e25199/"/>
          </comment>
          <pub-id pub-id-type="doi">10.2196/25199</pub-id>
          <pub-id pub-id-type="medline">34081022</pub-id>
          <pub-id pub-id-type="pii">v23i6e25199</pub-id>
          <pub-id pub-id-type="pmcid">PMC8212625</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref54">
        <label>54</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>König</surname>
              <given-names>A</given-names>
            </name>
            <name name-style="western">
              <surname>Riviere</surname>
              <given-names>K</given-names>
            </name>
            <name name-style="western">
              <surname>Linz</surname>
              <given-names>N</given-names>
            </name>
            <name name-style="western">
              <surname>Lindsay</surname>
              <given-names>H</given-names>
            </name>
            <name name-style="western">
              <surname>Elbaum</surname>
              <given-names>J</given-names>
            </name>
            <name name-style="western">
              <surname>Fabre</surname>
              <given-names>R</given-names>
            </name>
            <name name-style="western">
              <surname>Derreumaux</surname>
              <given-names>A</given-names>
            </name>
            <name name-style="western">
              <surname>Robert</surname>
              <given-names>P</given-names>
            </name>
          </person-group>
          <article-title>Measuring stress in health professionals over the phone using automatic speech analysis during the COVID-19 pandemic: observational pilot study</article-title>
          <source>J Med Internet Res</source>
          <year>2021</year>
          <month>04</month>
          <day>19</day>
          <volume>23</volume>
          <issue>4</issue>
          <fpage>e24191</fpage>
          <comment>
            <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="https://www.jmir.org/2021/4/e24191/"/>
          </comment>
          <pub-id pub-id-type="doi">10.2196/24191</pub-id>
          <pub-id pub-id-type="medline">33739930</pub-id>
          <pub-id pub-id-type="pii">v23i4e24191</pub-id>
          <pub-id pub-id-type="pmcid">PMC8057197</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref55">
        <label>55</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Iyer</surname>
              <given-names>R</given-names>
            </name>
            <name name-style="western">
              <surname>Nedeljkovic</surname>
              <given-names>M</given-names>
            </name>
            <name name-style="western">
              <surname>Meyer</surname>
              <given-names>D</given-names>
            </name>
          </person-group>
          <article-title>Using voice biomarkers to classify suicide risk in adult telehealth callers: retrospective observational study</article-title>
          <source>JMIR Ment Health</source>
          <year>2022</year>
          <month>08</month>
          <day>15</day>
          <volume>9</volume>
          <issue>8</issue>
          <fpage>e39807</fpage>
          <comment>
            <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="https://mental.jmir.org/2022/8/e39807/"/>
          </comment>
          <pub-id pub-id-type="doi">10.2196/39807</pub-id>
          <pub-id pub-id-type="medline">35969444</pub-id>
          <pub-id pub-id-type="pii">v9i8e39807</pub-id>
          <pub-id pub-id-type="pmcid">PMC9425169</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref56">
        <label>56</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Künzel</surname>
              <given-names>HJ</given-names>
            </name>
          </person-group>
          <article-title>Beware of the 'telephone effect': the influence of telephone transmission on the measurement of formant frequencies</article-title>
          <source>Forensic Linguistics</source>
          <year>2001</year>
          <volume>8</volume>
          <issue>1</issue>
          <fpage>80</fpage>
          <lpage>99</lpage>
          <pub-id pub-id-type="doi">10.1558/sll.2001.8.1.80</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref57">
        <label>57</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Lammert</surname>
              <given-names>AC</given-names>
            </name>
            <name name-style="western">
              <surname>Narayanan</surname>
              <given-names>SS</given-names>
            </name>
          </person-group>
          <article-title>On short-time estimation of vocal tract length from formant frequencies</article-title>
          <source>PLoS One</source>
          <year>2015</year>
          <volume>10</volume>
          <issue>7</issue>
          <fpage>e0132193</fpage>
          <comment>
            <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="https://dx.plos.org/10.1371/journal.pone.0132193"/>
          </comment>
          <pub-id pub-id-type="doi">10.1371/journal.pone.0132193</pub-id>
          <pub-id pub-id-type="medline">26177102</pub-id>
          <pub-id pub-id-type="pii">PONE-D-14-52345</pub-id>
          <pub-id pub-id-type="pmcid">PMC4503663</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref58">
        <label>58</label>
        <nlm-citation citation-type="book">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Rabiner</surname>
              <given-names>L</given-names>
            </name>
            <name name-style="western">
              <surname>Schafer</surname>
              <given-names>R</given-names>
            </name>
          </person-group>
          <source>Theory and Applications of Digital Speech Processing</source>
          <year>2011</year>
          <publisher-loc>Upper Saddle River</publisher-loc>
          <publisher-name>Pearson</publisher-name>
        </nlm-citation>
      </ref>
      <ref id="ref59">
        <label>59</label>
        <nlm-citation citation-type="book">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Benesty</surname>
              <given-names>J</given-names>
            </name>
            <name name-style="western">
              <surname>Sondhi</surname>
              <given-names>M</given-names>
            </name>
            <name name-style="western">
              <surname>Huang</surname>
              <given-names>Y</given-names>
            </name>
          </person-group>
          <article-title>Introduction to speech processing</article-title>
          <source>Springer Handbook of Speech Processing</source>
          <year>2008</year>
          <publisher-loc>Berlin</publisher-loc>
          <publisher-name>Springer</publisher-name>
          <fpage>1</fpage>
          <lpage>4</lpage>
        </nlm-citation>
      </ref>
      <ref id="ref60">
        <label>60</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Kleiman</surname>
              <given-names>EM</given-names>
            </name>
            <name name-style="western">
              <surname>Turner</surname>
              <given-names>BJ</given-names>
            </name>
            <name name-style="western">
              <surname>Fedor</surname>
              <given-names>S</given-names>
            </name>
            <name name-style="western">
              <surname>Beale</surname>
              <given-names>EE</given-names>
            </name>
            <name name-style="western">
              <surname>Huffman</surname>
              <given-names>JC</given-names>
            </name>
            <name name-style="western">
              <surname>Nock</surname>
              <given-names>MK</given-names>
            </name>
          </person-group>
          <article-title>Examination of real-time fluctuations in suicidal ideation and its risk factors: results from two ecological momentary assessment studies</article-title>
          <source>J Abnorm Psychol</source>
          <year>2017</year>
          <month>08</month>
          <volume>126</volume>
          <issue>6</issue>
          <fpage>726</fpage>
          <lpage>38</lpage>
          <pub-id pub-id-type="doi">10.1037/abn0000273</pub-id>
          <pub-id pub-id-type="medline">28481571</pub-id>
          <pub-id pub-id-type="pii">2017-20211-001</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref61">
        <label>61</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Lotito</surname>
              <given-names>M</given-names>
            </name>
            <name name-style="western">
              <surname>Cook</surname>
              <given-names>E</given-names>
            </name>
          </person-group>
          <article-title>A review of suicide risk assessment instruments and approaches</article-title>
          <source>Mental Health Clinician</source>
          <year>2015</year>
          <volume>5</volume>
          <fpage>216</fpage>
          <lpage>23</lpage>
          <pub-id pub-id-type="doi">10.9740/mhc.2015.09.216</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref62">
        <label>62</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Runeson</surname>
              <given-names>B</given-names>
            </name>
            <name name-style="western">
              <surname>Odeberg</surname>
              <given-names>J</given-names>
            </name>
            <name name-style="western">
              <surname>Pettersson</surname>
              <given-names>A</given-names>
            </name>
            <name name-style="western">
              <surname>Edbom</surname>
              <given-names>T</given-names>
            </name>
            <name name-style="western">
              <surname>Jildevik Adamsson</surname>
              <given-names>I</given-names>
            </name>
            <name name-style="western">
              <surname>Waern</surname>
              <given-names>M</given-names>
            </name>
          </person-group>
          <article-title>Instruments for the assessment of suicide risk: a systematic review evaluating the certainty of the evidence</article-title>
          <source>PLoS One</source>
          <year>2017</year>
          <month>7</month>
          <day>19</day>
          <volume>12</volume>
          <issue>7</issue>
          <fpage>e0180292</fpage>
          <comment>
            <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="https://dx.plos.org/10.1371/journal.pone.0180292"/>
          </comment>
          <pub-id pub-id-type="doi">10.1371/journal.pone.0180292</pub-id>
          <pub-id pub-id-type="medline">28723978</pub-id>
          <pub-id pub-id-type="pii">PONE-D-16-38967</pub-id>
          <pub-id pub-id-type="pmcid">PMC5517300</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref63">
        <label>63</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Carter</surname>
              <given-names>G</given-names>
            </name>
            <name name-style="western">
              <surname>Milner</surname>
              <given-names>A</given-names>
            </name>
            <name name-style="western">
              <surname>McGill</surname>
              <given-names>K</given-names>
            </name>
            <name name-style="western">
              <surname>Pirkis</surname>
              <given-names>J</given-names>
            </name>
            <name name-style="western">
              <surname>Kapur</surname>
              <given-names>N</given-names>
            </name>
            <name name-style="western">
              <surname>Spittal</surname>
              <given-names>MJ</given-names>
            </name>
          </person-group>
          <article-title>Predicting suicidal behaviours using clinical instruments: systematic review and meta-analysis of positive predictive values for risk scales</article-title>
          <source>Br J Psychiatry</source>
          <year>2017</year>
          <month>06</month>
          <day>02</day>
          <volume>210</volume>
          <issue>6</issue>
          <fpage>387</fpage>
          <lpage>95</lpage>
          <pub-id pub-id-type="doi">10.1192/bjp.bp.116.182717</pub-id>
          <pub-id pub-id-type="medline">28302700</pub-id>
          <pub-id pub-id-type="pii">S0007125000281634</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref64">
        <label>64</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Button</surname>
              <given-names>KS</given-names>
            </name>
            <name name-style="western">
              <surname>Ioannidis</surname>
              <given-names>JP</given-names>
            </name>
            <name name-style="western">
              <surname>Mokrysz</surname>
              <given-names>C</given-names>
            </name>
            <name name-style="western">
              <surname>Nosek</surname>
              <given-names>BA</given-names>
            </name>
            <name name-style="western">
              <surname>Flint</surname>
              <given-names>J</given-names>
            </name>
            <name name-style="western">
              <surname>Robinson</surname>
              <given-names>ES</given-names>
            </name>
            <name name-style="western">
              <surname>Munafò</surname>
              <given-names>MR</given-names>
            </name>
          </person-group>
          <article-title>Power failure: why small sample size undermines the reliability of neuroscience</article-title>
          <source>Nat Rev Neurosci</source>
          <year>2013</year>
          <volume>14</volume>
          <issue>5</issue>
          <fpage>365</fpage>
          <lpage>76</lpage>
          <pub-id pub-id-type="doi">10.1038/nrn3475</pub-id>
          <pub-id pub-id-type="medline">23571845</pub-id>
          <pub-id pub-id-type="pii">nrn3475</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref65">
        <label>65</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Asendorpf</surname>
              <given-names>JB</given-names>
            </name>
            <name name-style="western">
              <surname>Conner</surname>
              <given-names>M</given-names>
            </name>
            <name name-style="western">
              <surname>De Fruyt</surname>
              <given-names>F</given-names>
            </name>
            <name name-style="western">
              <surname>De Houwer</surname>
              <given-names>J</given-names>
            </name>
            <name name-style="western">
              <surname>Denissen</surname>
              <given-names>JJ</given-names>
            </name>
            <name name-style="western">
              <surname>Fiedler</surname>
              <given-names>K</given-names>
            </name>
            <name name-style="western">
              <surname>Fiedler</surname>
              <given-names>S</given-names>
            </name>
            <name name-style="western">
              <surname>Funder</surname>
              <given-names>DC</given-names>
            </name>
            <name name-style="western">
              <surname>Kliegl</surname>
              <given-names>R</given-names>
            </name>
            <name name-style="western">
              <surname>Nosek</surname>
              <given-names>BA</given-names>
            </name>
            <name name-style="western">
              <surname>Perugini</surname>
              <given-names>M</given-names>
            </name>
            <name name-style="western">
              <surname>Roberts</surname>
              <given-names>BW</given-names>
            </name>
            <name name-style="western">
              <surname>Schmitt</surname>
              <given-names>M</given-names>
            </name>
            <name name-style="western">
              <surname>Van Aken</surname>
              <given-names>MA</given-names>
            </name>
            <name name-style="western">
              <surname>Weber</surname>
              <given-names>H</given-names>
            </name>
            <name name-style="western">
              <surname>Wicherts</surname>
              <given-names>JM</given-names>
            </name>
          </person-group>
          <article-title>Recommendations for increasing replicability in psychology</article-title>
          <source>Eur J Pers</source>
          <year>2020</year>
          <volume>27</volume>
          <issue>2</issue>
          <fpage>108</fpage>
          <lpage>19</lpage>
          <pub-id pub-id-type="doi">10.1002/per.1919</pub-id>
        </nlm-citation>
      </ref>
    </ref-list>
  </back>
</article>
