Our research approach centered on a comprehensive evaluation of the literature exploring the integration of AI—particularly its subset, ML—within BCI closed-loop systems in health care. The goal was to synthesize current knowledge on the methodologies, algorithms, outcomes, limitations, and emerging directions that define this interdisciplinary field. To achieve this, we developed a targeted search strategy using relevant keywords and Boolean operators, enabling us to identify both theoretical advancements and real-world applications of AI- and ML-enhanced BCIs. This method allowed for a focused analysis of how these technologies are transforming neurological monitoring, cognitive rehabilitation, and personalized patient care.

Our scoping criteria focused on the specific domain of BCI closed-loop systems integrated with ML and AI in health care. We prioritized studies published between 2019‐2024 to ensure the relevance and timeliness of our findings. Our approach included not only technological advancements but also practical challenges and developments in BCI closed-loop systems in health care. Specifically, we reviewed studies that examined the various methods and parameters collected in BCI closed-loop systems (RQ1), ensuring a comprehensive understanding of data acquisition, preprocessing, and real-time feedback mechanisms. We also investigated the ML and AI algorithms used, and the outcomes obtained (RQ2), identifying the overall effectiveness of these algorithms in clinical and experimental settings. In addition, we focused on studies discussing the limitations encountered in current BCI closed-loop systems and proposed future research directions (RQ3), aiming to understand the barriers to implementation, ethical considerations, and technological limitations (RQ4). Exclusion variables from some papers were added, as through our search, we filtered out papers that were not relevant to our goals, any research conducted on animals and not humans, and a lack of focus on Machine Learning.

Our systematic approach involved gathering, critical assessing, integrating, and presenting findings from various research papers on BCI closed-loop systems integrated with ML and AI in health care. We followed a detailed procedure to conduct and report systematic literature reviews, ensuring a rigorous selection process. Initially, we developed a carefully crafted search query to refine our search effectively, using terms such as “BCI OR brain computer interface,” “AND Machine Learning OR AI OR algorithm,” “AND Alzheimer OR Dementia.” Boolean operators like “AND” and “OR” were used strategically to narrow our search. This search spanned 4 major databases: PubMed, IEEE, ACM (Association for Computing Machinery), and Scopus. From these databases, we identified a total of 220 papers: 43 from PubMed, 22 from IEEE, 114 from ACM, and 41 from Scopus. After removing 8 duplicate records, 212 unique records were screened. During the screening phase, 179 records were excluded for reasons such as being out of context (n=84), not relevant to the research questions (n=94), or inaccessible (n=1). Subsequently, the titles and abstracts of the 212 screened records were assessed for eligibility, resulting in 33 full-text articles being reviewed. Of these, 15 reports were excluded due to being theses or books (n=9), report articles (n=4), or of poor quality (n=2). Ultimately, 18 studies met all inclusion criteria and were included in the final review. These papers were selected based on their focus on BCI closed-loop systems in health care, the integration of ML and AI, and their relevance to our study. We prioritized papers displaying rigorous methodologies, including empirical studies, surveys, case studies, experiments, and systematic literature reviews, showcasing innovative approaches, novel insights, or significant findings. In addition to the primary search, we cross-referenced each article’s citations to identify other pertinent papers, ultimately including any that fit our criteria.

Our selection process, guided by PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) [24,25] guidelines as shown in Figure 2, allowed for a comprehensive understanding of the current state and future potential of BCI closed-loop systems in health care. We evaluated sources based on their methodology, innovation, significant findings, and overall relevance.

Our data analysis approach used a systematic data extraction method to rigorously analyze literature focused on BCI closed-loop systems integrated with ML and AI. This approach covered essential aspects such as the methods and parameters used in BCI systems, the ML algorithms used, challenges encountered, proposed solutions, and future research directions. Initially, we conducted an extensive literature review to identify pertinent studies. From this review, we developed a structured extraction matrix aimed at comprehensively capturing thematic elements critical to our study. The matrix included categories such as Title, Methods, Parameters, Machine Learning Algorithms, Challenges/Limitations, Proposed Solutions, Future Research Directions, and Title and Abstract Screening Score (0‐3). These scores would be averaged out among a panel of 3 researchers with a 2 being a “Yes” to our paper list. To validate our methodology, we conducted several validation steps. First, we pilot-tested the matrix with a small sample of 10 papers to ensure it effectively captured relevant information while excluding irrelevant details. Second, we aligned the matrix variable with our research questions to ensure clarity in data extraction. The finalized matrix, formatted in Microsoft Excel, allowed for a smooth, systematic, and comparative analysis across selected papers, including Full Text Screening Score (0‐3). This methodical approach enabled us to extract and synthesize data methodically, allowing anomalies and patterns to naturally emerge. Our synthesis and evaluation of articles were guided by their direct relevance to our study’s focus areas. This systematic approach ensured a robust analysis and provided a solid foundation for our literature review, as reflected in Table 1.

Many studies have explored BCIs with closed-loop systems, but a comprehensive survey focusing on the challenges associated with methods and parameters used in these systems is still lacking. This section addresses this gap by reviewing various preprocessing techniques and the parameters used in BCI closed-loop systems, highlighting their implications for neural activity monitoring and intervention.

The effectiveness of ML and AI algorithms in BCI closed-loop systems is crucial for enhancing patient outcomes, particularly in applications related to neurorehabilitation and cognitive monitoring. These algorithms play a pivotal role in accurately interpreting neural signals, enabling real-time feedback and adaptive responses tailored to individual user needs. Their ability to analyze complex patterns in brain activity allows for improved signal classification and feature extraction, which are essential for ensuring reliable communication between the brain and external devices. Furthermore, the integration of effective ML and AI algorithms facilitates continuous learning and adaptation, ensuring that the BCI system evolves alongside the user’s cognitive state. This adaptability not only enhances the overall user experience but also promotes better engagement and efficacy in therapeutic interventions, making the technology a powerful tool in managing neurological disorders.

Figure 4 illustrates the key machine learning techniques used in BCI closed-loop systems. It categorizes these techniques into preprocessing (eg, Independent Component Analysis [ICA] for noise reduction), data augmentation (eg, generative adversarial networks [GANs] for expanding training data diversity), feature extraction (eg, CNN and transfer learning [TL] for identifying critical signal patterns), and classification (eg, SSVMs and linear discriminant analysis [LDA] for categorizing neural signals). These methods collectively improve the system’s effectiveness by refining the input data, enhancing model training with more varied data, extracting meaningful features, and accurately classifying neural patterns. This multistep approach enables closed-loop BCIs to achieve reliable real-time monitoring and intervention, making them more effective for health care and wellness applications.

In addition, Table 1 outlines the key algorithms and techniques commonly used in BCI systems, while Table 2 offers a detailed comparative evaluation of these machine learning approaches in closed-loop frameworks, emphasizing their applications, adaptability to neurological conditions, performance metrics, and computational complexity, supported by relevant literature.

Machine learning algorithms have significantly enhanced the performance of BCIs. In that respect, one of the most influential techniques would be TL, which borrows knowledge from a source domain to perform better in a target domain. This is of particular importance to BCI, as the collection of data across sessions and subjects is often limited or variable [26]. TL has been successfully applied to data alignment, spatial filtering, feature selection, and classification tasks, dramatically improving the robustness and accuracy of BCIs across different conditions and subjects.

Complementing TL, SVMs have turned out to perform very well in high-dimensional spaces and have performed both linear and nonlinear classification using the kernel trick on data transformation [27]. SVMs have found wide applications in BCI applications, more specifically in classifying EEG signals. The SVM finds the optimal hyperplane that allows separation of different classes, thus always showing good performance in most BCI tasks, such as motor imagery classification and emotion recognition [28].

On the other side, LDA finds out the best separation between several classes by maximizing such a separation by choosing an appropriate linear combination of features. LDA has a nice balance between computational simplicity and performance for BCIs [27].

The role of ICA is paramount in preprocessing methods. ICA is one of the key tools that attempt to separate multivariate signals into additive, independent components [27]. Especially with BCIs, it is very good at isolating artifact components from the neural signals, hence improving the quality of data used for subsequent classification tasks. This step in preprocessing appreciably improves the accuracy of a number of BCI applications.

Moving to more complex models, CNNs have been very triumphant in visual and spatial data, including EEG and functional near-infrared spectroscopy (fNIRS) signals. CNNs are known for their emotional ability to build hybrid brain images that classify the activities taking place in the brain in a very accurate manner for the detection and interpretation of any complex neural pattern [29]. This is critical in applications like mental workload estimation and emotion recognition, where a spatial hierarchy in neural data may be critical for appropriate classification and analysis [33] .

Tree-structured neural networks (TSNN) combine decision trees and neural networks to provide the possibility of hierarchical feature extraction and classification. More importantly, these networks work quite effectively in relation to data: complex and high-dimensional. TSNNs are therefore able to yield promising results on the classification of neural activities and detecting symptoms of neurological disorders with a rich set of neural biomarkers [30]. This fills a gap in the field by providing an optimal balance between accuracy and computational efficiency, needed for real-time BCI applications.

RBMs have made some very great contributions to unsupervised scenes, where instances of the objective are to learn underlying structures of data [31]. In this case, RBMs learn with effective features and reduce dimensionality to improve the performance of classifiers on mental state recognition and motor imagery classification tasks [36].

On the other side, fuzzy models represent the uncertainty and imprecision of EEG data using fuzzy logic. Such models generate rules that are much closer to those resulting from human reasoning and hence are very suitable for processing nonlinear and nonstationary signals. Fuzzy models applied in BCIs include the so-called fuzzy inference systems (FIS) and fuzzy neural networks (FNNs) for classifying EEG patterns, offering both accuracy and interpretability [32].

Another extension to the toolkit of BCIs is GANs. It consists of two neural networks: a generator and a discriminator. These networks counteract in a framework, and each has an opposite goal in a zero-sum game setup. GANs’ application in BCIs is in augmented data generation for improving classifier training, more so when there is not enough data, as GANs increase the robustness, hence the accuracy, of BCI systems by bringing forth more training data. Improvements in these machine learning algorithms have increased the potential of BCIs not only on grounds of performance but also by opening new avenues for possible clinical and practical applications [28].

Table 1 outlines key methods used in BCIs, detailing their roles, applications, and benefits. Techniques like TL, SVM, LDA, and ICA enhance data preprocessing and classification, improving signal quality and performance. CNNs and TSNNs excel in feature extraction and classification of complex neural data, while RBMs and Fuzzy Models handle dimensionality reduction and uncertainty in EEG signals. GANs support data augmentation, boosting robustness and accuracy. These methods collectively optimize the processing of neural signals in closed-loop BCI systems.

TL, SVM, LDA, ICA, CNN, TSNN, RBMs, fuzzy models, and GAN techniques have helped in making BCIs effective and reliable. These algorithms help improve the capability of BCIs to better handle the user’s requirements, reduce calibration time, and realize more accurate and robust control of artificial limbs and other devices.

Some of the limitations facing BCIs’ development and implementation can be summarized as ranging from decoding algorithms through neural and behavioral measurements to computational constraints, as shown in Table 3. The table outlines various challenges associated with BCI technology and proposes corresponding solutions to address these issues. It covers aspects like improving neural signal decoding, enhancing sensor accuracy, and increasing the precision of behavioral measurements. These limitations show requirements for further research in terms of target setting and orientation of work for increasing effectiveness. Further explanations about these are as follows:

AD/ADRD is a progressive neurological disorder, and one of the major causes of death among the older adults [46,47]. Therefore, it is important to acquire new solutions that can enhance the quality of life of patients and their caregivers as the number of people affected increases yearly. The use of BCI technology is considered one of the most promising approaches to this challenge, using state-of-the-art neuroimaging techniques and machine learning algorithms for continuous monitoring and diagnosis.

The proposed framework shown in Figure 5 overcomes the complexity of decoding neural activity in AD/ADRD patients, who often lack the cognitive control normally required by conventional BCIs. It illustrates a closed-loop BCI framework for longitudinal monitoring of AD/ADRD patients, beginning with the acquisition of brain activity through neuroimaging techniques to capture critical neural signals. The collected data undergoes preprocessing, where noise is reduced using artifact detection and removal algorithms, ensuring high-quality signals. Ethical and security concerns are addressed by implementing robust data protection measures to safeguard sensitive patient information. Machine learning algorithms classify the neural signals, accurately distinguishing between various mental states, while feature extraction and domain-specific calibration improve the system’s precision. Electrode calibration ensures reliable signal acquisition, enhancing system performance. Real-time alerts integrate closed-loop monitoring for continuous observation, enabling timely interventions. Ultimately, this framework aims to improve Alzheimer patient care by leveraging Neuralink’s BCI technology [45] for better monitoring and intervention strategies. The different stages are further elaborated below:

After analyzing data, references indicate the most prominent symptoms of AD are severe deficits in communication, cognitive decline, and behavioral changes. Traditional BCIs require active participation; hence, they cannot be used with AD patients. For example, in [48] the authors highlight that traditional BCIs, requiring active control, are not suitable for AD patients due to their cognitive impairments. Instead, the study emphasizes the use of passive BCIs, which leverage preserved affective responses to facilitate basic communication and continuous monitoring [48]. This method makes use of detailed neuroimaging and machine learning components for cognitive and behavioral changes in order to provide such monitoring. Such continual assessment allows for early diagnosis and timely intervention that are crucial in managing AD progression [49]. In that respect, the BCI systems are vulnerable to data breaches and ethical misuses. The security aspects are core parts of the proposed BCI framework, evident in the position of ethical and security considerations at the very center of the framework diagram in Figure 5 [50]. This framework would hardwire strong measures for security against breaches and ethical misuse of information related to a patient. This involves encryption methods, secure storage, and tight access control that ensures only authorized personnel can view sensitive information. Despite technical possibilities, data protection and ethical guidelines are argued to exist, whereas BCI systems are weak in cryptographic, denial-of-service, and sniffing cyber-attacks [50]. Therefore, strong protection measures must be implemented. Implantable BCI devices give real-time data, thereby allowing caregivers to receive instant alerts on a patient’s condition. This approach will improve the quality of care and prevent emergencies.

The envisioned BCI framework addresses the most critical challenges of caring for the patient experiencing Alzheimer disease by efficiently combining advanced neuroimaging techniques with machine learning algorithms as shown in Table 4. In this regard, this approach is relevant for improving patient clinical outcomes while assisting caregivers in handling the complexities of Alzheimer disease management by enhancing neural signal classification, guaranteeing data security, and real-time monitoring [50].

This systematic review synthesized the current evidence from 18 studies on the integration of AI and ML within BCI closed-loop systems for neurorehabilitation, with a specific focus on AD/ADRD. The findings indicate that ML techniques such as TL, CNNs, and SVMs significantly enhance the performance of BCI systems by improving real-time signal classification, feature extraction, and cross-session adaptability. However, the translation of these technological advancements into widespread clinical practice is hampered by significant challenges, including signal variability, computational demands, lengthy calibration, and profound privacy concerns. The proposed framework for longitudinal AD/ADRD monitoring represents a promising, patient-centric application that leverages passive BCI paradigms to circumvent the cognitive demands of traditional systems.

Our findings on the efficacy of ML algorithms like CNNs and TL in BCI systems are strongly supported by the broader literature on AI in digital health. The high accuracy (>90%) of CNNs in classifying complex neural patterns for mental workload and emotion recognition aligns with their proven success in other pattern recognition domains, such as medical imaging. Similarly, the utility of TL in reducing calibration time and improving cross-subject generalization addresses a well-documented bottleneck in BCI research, echoing its successful application in other fields where data scarcity and individual variability are concerns. The review’s identification of passive BCIs as a solution for AD/ADRD patients is particularly insightful. This approach is consistent with a growing trend in digital biomarkers, which seeks to leverage implicit, continuous data from wearables and other sensors for early disease detection and monitoring, moving beyond active user participation. Furthermore, the emphasis on real-time, closed-loop feedback for neurorehabilitation is supported by neuroscientific principles of neuroplasticity. The ability of AI-enhanced BCIs to provide immediate, adaptive intervention is theorized to strengthen neural pathways more effectively than open-loop systems, a hypothesis that is gaining traction in stroke and spinal cord injury rehabilitation. Thus, the results of this review are not isolated but are part of a convergent evolution across AI, neuroscience, and clinical medicine toward more adaptive, data-driven therapeutic interventions.

While the reviewed studies demonstrate significant promise, the evidence base has several important limitations that temper the immediate readiness of these technologies for clinical deployment. The majority of included studies were small-scale, laboratory-based demonstrations. They often involved healthy participants or highly controlled patient groups, lacking the diversity and complexity of real-world clinical environments. This limits the generalizability of the reported high accuracy rates. In addition, as highlighted in the review, there is a pronounced heterogeneity in decoding algorithms, performance metrics, and experimental protocols across studies. The absence of standardized benchmarks makes it difficult to directly compare the performance of different ML models or BCI systems, hindering the identification of optimal approaches. Furthermore, there is a critical gap in long-term longitudinal studies. It remains largely unknown how these systems perform over months or years, how they adapt to disease progression, and whether improvements in signal classification accuracy translate into meaningful clinical outcomes, such as slowed cognitive decline or improved quality of life.

This review itself is subject to certain methodological limitations that should be acknowledged. Limiting the search to studies published between 2019 and 2024, while ensuring timeliness, may have excluded foundational or highly relevant older studies. Furthermore, while major databases were consulted, the exclusion of other potential sources may have led to the omission of pertinent research. Next, the review likely reflects a positive publication bias, as studies with null or negative results are less frequently published. This may create an over-optimistic picture of the current capabilities and reliability of AI-driven BCIs.

Despite following PRISMA guidelines and using a panel of researchers, the processes of screening titles/abstracts and extracting data into a matrix involve a degree of subjective judgment, which could have influenced the final selection and synthesis of the studies.

The findings of this review have several critical implications across different domains:

For clinical practice: in the short term, AI-enhanced BCIs are most likely to find application as sophisticated diagnostic and monitoring tools in specialized neurology centers, aiding in the early and objective detection of cognitive impairment. The proposed framework for AD/ADRD provides a blueprint for developing caregiver alert systems, which could significantly reduce burden and improve patient safety in home-care settings. Clinicians should be aware of these emerging technologies to guide future patient care and manage expectations.

For policy and regulation: the security vulnerabilities and ethical dilemmas identified (eg, data privacy, informed consent for cognitively impaired users) demand urgent attention from policymakers and regulatory bodies like the FDA and EMA. New frameworks are needed to govern the security of neural data, which is arguably the most personal of all health information. Policies must be established to ensure equitable access and prevent misuse, defining clear guidelines for the ethical development and clinical validation of BCI technologies.

For future research: future work must transition from proof-of-concept to robust, clinically focused research. Key priorities should include rigorous, long-term trials with diverse AD/ADRD populations that are essential to validate efficacy and establish clinical utility. In addition, the BCI research community should collaborate to establish common data formats, reporting standards, and performance benchmarks to enable meaningful comparisons. Moreover, research must focus on developing more ergonomic, user-friendly, and low-power hardware that is suitable for prolonged use outside the lab. Creating interpretable ML models will be crucial for building trust among clinicians and patients, allowing them to understand the basis for the system’s classifications and decisions.

This review systematically explored the role of BCI closed-loop systems in health care, with a specific focus on their potential to enhance neurological disorder detection and management through advanced ML and AI techniques. Addressing RQ1, we analyzed various methods and parameters used in BCI closed-loop systems, including signal acquisition, feature extraction, classification, and device output. Key preprocessing techniques such as ICA and TL were identified as crucial for reducing noise and improving signal quality. DBS was also highlighted as a promising intervention for neuropsychological disorders like AD and ADRD.

In evaluating RQ2, we examined the effectiveness of ML and AI algorithms in BCI systems. Techniques like Support SVM, CNN, and RNN demonstrated significant improvements in decoding neural activity, enabling more accurate classification of cognitive states. TL, in particular, showed promise in reducing calibration time, making BCI systems more adaptive to individual users. In addition, BCIs have expanded beyond disease detection, playing a pivotal role in cognitive enhancement, neurofeedback training, and assistive communication.

Despite these advancements, RQ3 highlighted several challenges in the development and implementation of BCI closed-loop systems. Key limitations include high computational costs, long calibration sessions, signal variability across individuals, and security risks such as Poisoning Attacks that could compromise neural signal integrity. Ethical concerns surrounding data privacy and the potential misuse of BCIs also remain pressing issues. Addressing these challenges requires advancements in real-time signal processing, improved sensor technology, and robust cybersecurity frameworks to protect patient data.

To answer RQ4, we proposed a BCI-based framework for longitudinal monitoring of AD/ADRD patients, integrating real-time neural signal acquisition, feature extraction, and ML-based classification for early cognitive decline detection. This framework incorporates real-time alert systems to assist caregivers in proactive intervention, enhancing patient outcomes. In addition, passive BCIs were identified as a viable alternative for patients with severe cognitive impairments, enabling continuous monitoring without requiring active user engagement.

To answer RQ4, we proposed a BCI-based framework for longitudinal monitoring of AD/ADRD patients, integrating real-time neural signal acquisition, feature extraction, and ML-based classification for early cognitive decline detection. This framework incorporates real-time alert systems to assist caregivers in proactive intervention, enhancing patient outcomes. In addition, passive BCIs were identified as a viable alternative for patients with severe cognitive impairments, enabling continuous monitoring without requiring active user engagement.

Building on these advancements, future research should prioritize the refinement of machine learning algorithms to better support real-time signal processing and adaptive learning in dynamic environments. Ethical considerations—such as user consent, data ownership, and secure data handling—must remain central to system design. Continued progress in these areas will be essential for creating scalable, secure, and user-friendly BCI systems that integrate seamlessly into daily life. Ultimately, these innovations will position AI-powered BCIs as transformative tools in improving care, independence, and quality of life for individuals with neurological disorders, particularly those living with AD/ADRD.