Last modified by robing on 2022/03/25 09:55
From version 81.1
edited by robing
on 2020/05/28 15:12
Change comment: There is no comment for this version
To version 85.1
edited by robing
on 2020/06/22 10:29
Change comment: There is no comment for this version

Summary

Details

Page properties
Content
... ... @@ -7,7 +7,7 @@
7 7  (% class="wikigeneratedid" id="HUseCaseSGA2-SP3-002:IntegratingmultiscaledataA0inareproducibleandadaptablepipeline" %)
8 8  (% style="--darkreader-inline-color:inherit; color:inherit; font-size:24px" %)**Use Case SGA2-SP3-002 KR3.2: Integrating multi-scale data and the output of simulations in a reproducible and adaptable pipeline**
9 9  
10 -Robin Gutzen^^1^^, Giulia De Bonis^^2^^, Elena Pastorelli^^2,3^^, Cristiano Capone^^2^^,
10 +Robin Gutzen^^1,4^^, Giulia De Bonis^^2^^, Elena Pastorelli^^2,3^^, Cristiano Capone^^2^^,
11 11  
12 12  Chiara De Luca^^2,3^^, Michael Denker^^1^^, Sonja Grün^^1,4^^,
13 13  
... ... @@ -43,6 +43,8 @@
43 43  
44 44  This collab illustrates the usage of the [[Neo>>https://neo.readthedocs.io/en/stable/]] and [[Elephant>>https://elephant.readthedocs.io/en/latest/]] tools in performing data analysis with regards to the SGA2-SP3-UC002 KR3.2, investigating sleep, anesthesia, and the transition to wakefulness: see Chapter 1 and Figure 2  of SGA2[[ Deliverable D3.2.1.>>https://drive.ebrains.eu/smart-link/17ac0d6e-e050-4a49-8ca2-e223b70a3121/]], for an overview of the scientific motivations and a description of the UseCase workflow; Chapter 2 (same document) for an introduction to KR3.2; Chapter 3, for a description of the mice ECoG data sets; Chapter 5, about the Slow Wave Analysis Pipeline and Chapter 6 for the mice wide-field GECI data). For details on the datasets used in this collab, please see the References below.
45 45  
46 +[[See the introduction video>>https://www.youtube.com/watch?v=uuAiY6HScM0]]
47 +
46 46  == How the pipeline works ==
47 47  
48 48  The design of the pipeline aims at interfacing a variety of general and specific analysis and processing steps in a flexible modular manner. Hence, it enables the pipeline to adapt to diverse types of data (e.g., electrical ECoG, or optical Calcium Imaging recordings) and to different analysis questions. This makes the analyses a) more reproducible and b) comparable amongst each other since they rely on the same stack of algorithms and any differences in the analysis are fully transparent.
... ... @@ -73,7 +73,7 @@
73 73  Each stage has config files (//pipeline/<stage_name>/configs/config_<profile>.yaml//) to specify which analysis/processing blocks to execute and which parameters to use. General and specific information about the blocks and parameters can be found in the README and config files of each stage. There are preset configuration profiles for the benchmark datasets IDIBAPS ([[ECoG, anesthetized mouse>>https://kg.ebrains.eu/search/?facet_type[0]=Dataset&q=sanchez-vives#Dataset/2ead029b-bba5-4611-b957-bb6feb631396]]) and LENS ([[Calcium Imaging, anesthetized mouse>>https://kg.ebrains.eu/search/instances/Dataset/71285966-8381-48f7-bd4d-f7a66afa9d79]]).
74 74  
75 75  * **Run the notebook**
76 -In the jupyter hub, navigate to //drive/My Libraries/My Library/run_snakemake_in_collab.ipynb//, or where you copied the //pipeline// folder to.
78 +In the jupyter hub, navigate to //drive/My Libraries/My Library/run_snakemake_in_collab.ipynb//, or where you copied the file to.
77 77  Follow the notebook to install the required packages into your Python kernel, set the output path, and execute the pipeline with snakemake.
78 78  
79 79  === ii) Local execution ===
... ... @@ -131,7 +131,7 @@
131 131  * Stage 05 - [[planar velocities>>https://drive.ebrains.eu/smart-link/f4de8073-cb40-47a7-bc82-f97d36dbae25/]]
132 132  * Stage 05 - [[directionality>>https://drive.ebrains.eu/smart-link/5485032d-0121-4cde-9ea2-3e0af3f12178/]]
133 133  
134 -=== Outlook ===
136 +== Outlook ==
135 135  
136 136  * Using the **KnowledgeGraph API **to insert data directly from the Knowledge Graph into the pipeline and also register and store the corresponding results as Analysis Objects. Such Analysis Objects are to incorporate **Provenance Tracking, **using [[fairgraph>>https://github.com/HumanBrainProject/fairgraph]],** **to record the details of the processing and analysis steps.
137 137  * Adding support for the pipeline to make use of **HPC** resources when running on the collab.