Changes for page Methodology

Last modified by manuelmenendez on 2025/03/14 08:31

From version 2.1
edited by manuelmenendez
on 2025/01/27 23:07
Change comment: There is no comment for this version
To version 19.1
edited by manuelmenendez
on 2025/02/14 13:57
Change comment: There is no comment for this version

Summary

Details

Page properties
Content
... ... @@ -1,106 +1,207 @@
1 -=== **Overview** ===
1 +**# Neurodiagnoses AI: Multimodal AI for Neurodiagnostic Predictions**
2 2  
3 -This section describes the step-by-step process used in the **Neurodiagnoses** project to develop a novel diagnostic framework for neurological diseases. The methodology integrates artificial intelligence (AI), biomedical ontologies, and computational neuroscience to create a structured, interpretable, and scalable diagnostic system.
3 +## **Project Overview**
4 +Neurodiagnoses AI implements AI-driven diagnostic and prognostic models for central nervous system (CNS) disorders, adapting the Florey Dementia Index (FDI) methodology to a broader set of neurological conditions. The approach integrates **multimodal data sources** (EEG, neuroimaging, biomarkers, and genetics) and employs **machine learning models** to provide **explainable, real-time diagnostic insights**.##
4 4  
5 -----
6 +## **How to Use External Databases in Neurodiagnoses**
7 +To enhance diagnostic accuracy, Neurodiagnoses integrates data from multiple biomedical and neurological research databases. Researchers can follow these steps to access, prepare, and integrate data into the Neurodiagnoses framework.##
6 6  
7 -=== **1. Data Integration** ===
9 +### **Potential Data Sources**
10 +Neurodiagnoses maintains an updated list of potential biomedical databases relevant to neurodegenerative diseases. ##
8 8  
9 -==== **Data Sources** ====
12 +**Reference: List of Potential Databases**
13 +- **ADNI**: Alzheimer's Disease data ([ADNI](https://adni.loni.usc.edu))
14 +- **PPMI**: Parkinson’s Disease Imaging and biospecimens ([PPMI](https://www.ppmi-info.org))
15 +- **GP2**: Whole-genome sequencing for PD ([GP2](https://gp2.org))
16 +- **Enroll-HD**: Huntington’s Disease Clinical and genetic data ([Enroll-HD](https://www.enroll-hd.org))
17 +- **GAAIN**: Multi-source Alzheimer’s data aggregation ([GAAIN](https://gaain.org))
18 +- **UK Biobank**: Population-wide genetic, imaging, and health records ([UK Biobank](https://www.ukbiobank.ac.uk))
19 +- **DPUK**: Dementia and Aging data ([DPUK](https://www.dementiasplatform.uk))
20 +- **PRION Registry**: Prion Diseases clinical and genetic data ([PRION Registry](https://prionregistry.org))
21 +- **DECIPHER**: Rare genetic disorder genomic variants ([DECIPHER](https://decipher.sanger.ac.uk))
10 10  
11 -* **Biomedical Ontologies**:
12 -** Human Phenotype Ontology (HPO) for phenotypic abnormalities.
13 -** Gene Ontology (GO) for molecular and cellular processes.
14 -* **Neuroimaging Datasets**:
15 -** Example: Alzheimer’s Disease Neuroimaging Initiative (ADNI), OpenNeuro.
16 -* **Clinical and Biomarker Data**:
17 -** Anonymized clinical reports, molecular biomarkers, and test results.
23 +### **1. Register for Access**
24 +- Each external database requires **individual registration** and access approval.
25 +- Ensure compliance with **ethical approvals** and **data usage agreements** before integrating datasets into Neurodiagnoses.
26 +- Some repositories may require a **Data Usage Agreement (DUA)** for sensitive medical data.##
18 18  
19 -==== **Data Preprocessing** ====
28 +### **2. Download & Prepare Data**
29 +- Download datasets while adhering to database usage policies.
30 +- Ensure files meet **Neurodiagnoses format requirements**:
31 + - **Tabular Data**: `.csv`, `.tsv`
32 + - **Neuroimaging Data**: `.nii`, `.dcm`
33 + - **Genomic Data**: `.fasta`, `.vcf`
34 + - **Clinical Metadata**: `.json`, `.xml`##
20 20  
21 -1. **Standardization**: Ensure all data sources are normalized to a common format.
22 -1. **Feature Selection**: Identify relevant features for diagnosis (e.g., biomarkers, imaging scores).
23 -1. **Data Cleaning**: Handle missing values and remove duplicates.
36 +- **Mandatory Fields for Integration**:
37 + - **Subject ID**: Unique patient identifier
38 + - **Diagnosis**: Standardized disease classification
39 + - **Biomarkers**: CSF, plasma, or imaging biomarkers
40 + - **Genetic Data**: Whole-genome or exome sequencing
41 + - **Neuroimaging Metadata**: MRI/PET acquisition parameters
24 24  
25 -----
43 +### **3. Upload Data to Neurodiagnoses**
44 +**Option 1: Upload to EBRAINS Bucket**
45 +- Location: **EBRAINS Neurodiagnoses Bucket**
46 +- Ensure correct **metadata tagging** before submission.##
26 26  
27 -=== **2. AI-Based Analysis** ===
48 + **Option 2: Contribute via GitHub Repository**
49 +- Location: **GitHub Data Repository**
50 +- Create a new folder under `/data/` and include a **dataset description**.
51 +- For large datasets, contact project administrators before uploading.
28 28  
29 -==== **Model Development** ====
53 +### **4. Integrate Data into AI Models**
54 +- Open **Jupyter Notebooks** on EBRAINS to run **preprocessing scripts**.
55 +- Standardize **neuroimaging and biomarker formats** using harmonization tools.
56 +- Use **machine learning models** to handle missing data and feature extraction.
57 +- Train AI models with **newly integrated patient cohorts**.##
30 30  
31 -* **Embedding Models**: Use pre-trained models like BioBERT or BioLORD for text data.
32 -* **Classification Models**:
33 -** Algorithms: Random Forest, Support Vector Machines (SVM), or neural networks.
34 -** Purpose: Predict the likelihood of specific neurological conditions based on input data.
59 +**Reference**: See `docs/data_processing.md` for detailed instructions.
35 35  
36 -==== **Dimensionality Reduction and Interpretability** ====
61 +## **Collaboration & Partnerships**##
62 +# **Partnering with Data Providers**
63 +Neurodiagnoses seeks partnerships with data repositories to:
64 +- Enable **API-based data integration** for real-time processing.
65 +- Co-develop **harmonized AI-ready datasets** with standardized annotations.
66 +- Secure **funding opportunities** through joint grant applications.
37 37  
38 -* Leverage DEIBO (Data-driven Embedding Interpretation Based on Ontologies) to connect model dimensions to ontology concepts.
39 -* Evaluate interpretability using metrics like the Area Under the Interpretability Curve (AUIC).
68 +**Interested in Partnering?**
69 +- If you represent a research consortium or database provider, reach out to explore data-sharing agreements.
70 +- **Contact**: info@neurodiagnoses.com
40 40  
41 -----
72 +## **Final Notes**
73 +Neurodiagnoses continuously expands its data ecosystem to support AI-driven clinical decision-making. Researchers and institutions are encouraged to contribute **new datasets and methodologies**.##
42 42  
43 -=== **3. Diagnostic Framework** ===
75 +For additional technical documentation:
76 +- **GitHub Repository**: [Neurodiagnoses GitHub](https://github.com/neurodiagnoses)
77 +- **EBRAINS Collaboration Page**: [EBRAINS Neurodiagnoses](https://ebrains.eu/collabs/neurodiagnoses)
44 44  
45 -==== **Axes of Diagnosis** ====
79 +If you experience issues integrating data, **open a GitHub Issue** or consult the **EBRAINS Neurodiagnoses Forum**.
46 46  
47 -The framework organizes diagnostic data into three axes:
81 +== **How to Use External Databases in Neurodiagnoses** ==
48 48  
49 -1. **Etiology**: Genetic and environmental risk factors.
50 -1. **Molecular Markers**: Biomarkers such as amyloid-beta, tau, and alpha-synuclein.
51 -1. **Neuroanatomical Correlations**: Results from neuroimaging (e.g., MRI, PET).
83 +To enhance the accuracy of our diagnostic models, Neurodiagnoses integrates data from multiple biomedical and neurological research databases. If you are a researcher, follow these steps to access, prepare, and integrate data into the Neurodiagnoses framework.
52 52  
53 -==== **Recommendation System** ====
85 +=== **Potential Data Sources** ===
54 54  
55 -* Suggests additional tests or biomarkers if gaps are detected in the data.
56 -* Prioritizes tests based on clinical impact and cost-effectiveness.
87 +Neurodiagnoses maintains an updated list of potential biomedical databases relevant to neurodegenerative diseases.
57 57  
58 -----
89 +* Reference: [[List of Potential Databases>>url:https://github.com/Fundacion-de-Neurociencias/neurodiagnoses/blob/main/data/sources/list_of_potential_databases]]
59 59  
60 -=== **4. Computational Workflow** ===
91 +=== **1. Register for Access** ===
61 61  
62 -1. **Data Loading**: Import data from storage (Drive or Bucket).
63 -1. **Feature Engineering**: Generate derived features from the raw data.
64 -1. **Model Training**:
65 -1*. Split data into training, validation, and test sets.
66 -1*. Train models with cross-validation to ensure robustness.
67 -1. **Evaluation**:
68 -1*. Metrics: Accuracy, F1-Score, AUIC for interpretability.
69 -1*. Compare against baseline models and domain benchmarks.
93 +Each external database requires individual registration and access approval. Follow the official guidelines of each database provider.
70 70  
95 +* Ensure that you have completed all ethical approvals and data access agreements before integrating datasets into Neurodiagnoses.
96 +* Some repositories require a Data Usage Agreement (DUA) before downloading sensitive medical data.
97 +
98 +=== **2. Download & Prepare Data** ===
99 +
100 +Once access is granted, download datasets while complying with data usage policies. Ensure that the files meet Neurodiagnoses’ format requirements for smooth integration.
101 +
102 +==== **Supported File Formats** ====
103 +
104 +* Tabular Data: .csv, .tsv
105 +* Neuroimaging Data: .nii, .dcm
106 +* Genomic Data: .fasta, .vcf
107 +* Clinical Metadata: .json, .xml
108 +
109 +==== **Mandatory Fields for Integration** ====
110 +
111 +|=Field Name|=Description
112 +|Subject ID|Unique patient identifier
113 +|Diagnosis|Standardized disease classification
114 +|Biomarkers|CSF, plasma, or imaging biomarkers
115 +|Genetic Data|Whole-genome or exome sequencing
116 +|Neuroimaging Metadata|MRI/PET acquisition parameters
117 +
118 +=== **3. Upload Data to Neurodiagnoses** ===
119 +
120 +Once preprocessed, data can be uploaded to EBRAINS or GitHub.
121 +
122 +* (((
123 +**Option 1: Upload to EBRAINS Bucket**
124 +
125 +* Location: [[EBRAINS Neurodiagnoses Bucket>>url:https://wiki.ebrains.eu/bin/view/Collabs/neurodiagnoses/Bucket]]
126 +* Ensure correct metadata tagging before submission.
127 +)))
128 +* (((
129 +**Option 2: Contribute via GitHub Repository**
130 +
131 +* Location: [[GitHub Data Repository>>url:https://github.com/Fundacion-de-Neurociencias/neurodiagnoses/tree/main/data]]
132 +* Create a new folder under /data/ and include dataset description.
133 +)))
134 +
135 +//Note: For large datasets, please contact the project administrators before uploading.//
136 +
137 +=== **4. Integrate Data into AI Models** ===
138 +
139 +Once uploaded, datasets must be harmonized and formatted before AI model training.
140 +
141 +==== **Steps for Data Integration** ====
142 +
143 +* Open Jupyter Notebooks on EBRAINS to run preprocessing scripts.
144 +* Standardize neuroimaging and biomarker formats using harmonization tools.
145 +* Use machine learning models to handle missing data and feature extraction.
146 +* Train AI models with newly integrated patient cohorts.
147 +* Reference: [[Detailed instructions can be found in docs/data_processing.md>>url:https://github.com/Fundacion-de-Neurociencias/neurodiagnoses/blob/main/docs/data_processing.md]].
148 +
71 71  ----
72 72  
73 -=== **5. Validation** ===
151 +== **Database Sources Table** ==
74 74  
75 -==== **Internal Validation** ====
153 +=== **Where to Insert This** ===
76 76  
77 -* Test the system using simulated datasets and known clinical cases.
78 -* Fine-tune models based on validation results.
155 +* GitHub: [[docs/data_sources.md>>url:https://github.com/Fundacion-de-Neurociencias/neurodiagnoses/blob/main/docs/data_sources.md]]
156 +* EBRAINS Wiki: Collabs/neurodiagnoses/Data Sources
79 79  
80 -==== **External Validation** ====
158 +=== **Key Databases for Neurodiagnoses** ===
81 81  
82 -* Collaborate with research institutions and hospitals to test the system in real-world settings.
83 -* Use anonymized patient data to ensure privacy compliance.
160 +|=Database|=Focus Area|=Data Type|=Access Link
161 +|ADNI|Alzheimer's Disease|MRI, PET, CSF, cognitive tests|ADNI
162 +|PPMI|Parkinson’s Disease|Imaging, biospecimens|[[PPMI>>url:https://www.ppmi-info.org/]]
163 +|GP2|Genetic Data for PD|Whole-genome sequencing|[[GP2>>url:https://gp2.org/]]
164 +|Enroll-HD|Huntington’s Disease|Clinical, genetic, imaging|[[Enroll-HD>>url:https://enroll-hd.org/]]
165 +|GAAIN|Alzheimer's & Cognitive Decline|Multi-source data aggregation|[[GAAIN>>url:https://www.gaain.org/]]
166 +|UK Biobank|Population-wide studies|Genetic, imaging, health records|[[UK Biobank>>url:https://www.ukbiobank.ac.uk/]]
167 +|DPUK|Dementia & Aging|Imaging, genetics, lifestyle factors|[[DPUK>>url:https://www.dementiasplatform.uk/]]
168 +|PRION Registry|Prion Diseases|Clinical and genetic data|[[PRION Registry>>url:https://www.prionalliance.org/]]
169 +|DECIPHER|Rare Genetic Disorders|Genomic variants|DECIPHER
84 84  
171 +If you know a relevant dataset, submit a proposal in [[GitHub Issues>>url:https://github.com/Fundacion-de-Neurociencias/neurodiagnoses/issues]].
172 +
85 85  ----
86 86  
87 -=== **6. Collaborative Development** ===
175 +== **Collaboration & Partnerships** ==
88 88  
89 -The project is open to contributions from researchers, clinicians, and developers. Key tools include:
177 +=== **Where to Insert This** ===
90 90  
91 -* **Jupyter Notebooks**: For data analysis and pipeline development.
92 -* **Wiki Pages**: For documenting methods and results.
93 -* **Drive and Bucket**: For sharing code, data, and outputs.
179 +* GitHub: [[docs/collaboration.md>>url:https://github.com/Fundacion-de-Neurociencias/neurodiagnoses/blob/main/docs/collaboration.md]]
180 +* EBRAINS Wiki: Collabs/neurodiagnoses/Collaborations
94 94  
182 +=== **Partnering with Data Providers** ===
183 +
184 +Beyond using existing datasets, Neurodiagnoses seeks partnerships with data repositories to:
185 +
186 +* Enable direct API-based data integration for real-time processing.
187 +* Co-develop harmonized AI-ready datasets with standardized annotations.
188 +* Secure funding opportunities through joint grant applications.
189 +
190 +=== **Interested in Partnering?** ===
191 +
192 +If you represent a research consortium or database provider, reach out to explore data-sharing agreements.
193 +
194 +* Contact: [[info@neurodiagnoses.com>>mailto:info@neurodiagnoses.com]]
195 +
95 95  ----
96 96  
97 -=== **7. Tools and Technologies** ===
198 +== **Final Notes** ==
98 98  
99 -* **Programming Languages**: Python for AI and data processing.
100 -* **Frameworks**:
101 -** TensorFlow and PyTorch for machine learning.
102 -** Flask or FastAPI for backend services.
103 -* **Visualization**: Plotly and Matplotlib for interactive and static visualizations.
104 -* **EBRAINS Services**:
105 -** Collaboratory Lab for running Notebooks.
106 -** Buckets for storing large datasets.
200 +Neurodiagnoses continuously expands its data ecosystem to support AI-driven clinical decision-making. Researchers and institutions are encouraged to contribute new datasets and methodologies.
201 +
202 +For additional technical documentation:
203 +
204 +* [[GitHub Repository>>url:https://github.com/Fundacion-de-Neurociencias/neurodiagnoses]]
205 +* [[EBRAINS Collaboration Page>>url:https://wiki.ebrains.eu/bin/view/Collabs/neurodiagnoses/]]
206 +
207 +If you experience issues integrating data, open a [[GitHub Issue>>url:https://github.com/Fundacion-de-Neurociencias/neurodiagnoses/issues]] or consult the EBRAINS Neurodiagnoses Forum.