Changes for page Provenance of simulation and data analysis workflows
Last modified by adavison on 2022/05/23 22:24
Summary
-
Page properties (1 modified, 0 added, 0 removed)
Details
- Page properties
-
- Content
-
... ... @@ -19,6 +19,17 @@ 19 19 20 20 We present here the current schemas for representing (a) data analysis and (b) simulations in the Knowledge Graph. These schemas will need to be extended to cover neurorobotics simulations, and probably a more explicit representation of pipelines/workflows (the chaining together of multiple analysis / simulation stages) will be needed. 21 21 22 +[[image:Workflow provenance in the EBRAINS KG.svg||alt="KG schema for data analysis"]][[image:Workflow provenance in the EBRAINS KG-2.png||alt="KG schema for data analysis"]] 23 + 24 +[[image:Workflow provenance in the EBRAINS KG.png||alt="KG schema for simulation"]] 25 + 26 +(note that the diagrams do not show Agents; the person who launched each analysis/simulation activity is linked to the activity with a ##wasAssociatedWith## connection). 27 + 28 +(% class="box warningmessage" %) 29 +((( 30 +TODO: insert or link to the detailed schemas for each type 31 +))) 32 + 22 22 == Tools for automated capture of provenance == 23 23 24 24 * on different systems: ... ... @@ -31,8 +31,17 @@ 31 31 32 32 == Communication between computer systems and the KG == 33 33 34 - * localcache and synchronization?45 +Two issues arise: 35 35 47 +(i) fine-grained provenance information may need to be obtained on compute nodes, which may not have network access; 48 + 49 +(ii) failures of provenance upload should not cause the workflows to fail; 50 + 51 +An overall solution for both of these issues would perhaps involve a local cache and later synchronization. 52 + 36 36 == User interfaces for browsing, visualizing, and searching provenance information == 37 37 38 - 55 +(% class="box infomessage" %) 56 +((( 57 +DISCUSSION NEEDED: integrate visualization of prov information into KG Search UI, and/or develop separate app? 58 +)))