Last modified by adavison on 2022/05/23 22:24
From version 19.1
edited by adavison
on 2022/05/23 22:24
Change comment: Update collab owner property to adavison
To version 6.1
edited by adavison
on 2020/08/05 08:46
Change comment: Uploaded new attachment "starting-points.svg", version {1}

Summary

Details

Page properties
Content
... ... @@ -1,6 +1,6 @@
1 1  == Introduction ==
2 2  
3 -Computational provenance is a record of all the steps in a computational scientific workflow, including the code that was run, input data, the computational environment (hardware, OS, compiler versions, library version...), the person who performed each step, and output data.
3 +Computational provenance is a record of all the steps in a computational scientific workflow, including the code that was run, input data, the computational environment (hardware, OS, compiler versions, library version...), and output data.
4 4  
5 5  Capturing computational provenance facilitates:
6 6  
... ... @@ -8,32 +8,18 @@
8 8  * management and tracking of workflows/projects by the scientists/engineers involved
9 9  * evaluation/review by other scientists and engineers
10 10  
11 +
12 +
11 11  == Standards ==
12 12  
13 -The [[W3C PROV standard>>https://www.w3.org/TR/2013/NOTE-prov-overview-20130430/||rel="noopener noreferrer" target="_blank"]] provides a data model and related tools for provenance interchange on the web. The following diagram shows the three base classes of the PROV data model: Entity, Activity, and Agent. These three classes form the basis for the representation of provenance in the EBRAINS Knowledge Graph: every node in the KG has a type which is a subclass of one of these base classes.
15 +Information about the W3C PROV ontology and related tools
14 14  
15 -[[image:starting-points.svg||alt="The three Starting Point classes of the W3C PROV ontology and the properties that relate them."]]
16 16  
17 17  == Storage of provenance in the Knowledge Graph ==
18 18  
19 -We present here the current schemas for representing (a) data analysis and (b) simulations in the Knowledge Graph. These schemas will need to be extended to cover neurorobotics simulations, and probably a more explicit representation of pipelines/workflows (the chaining together of multiple analysis / simulation stages) will be needed.
20 20  
21 -[[image:Workflow provenance in the EBRAINS KG-2.png||alt="KG schema for data analysis"]]
22 -
23 -[[image:Workflow provenance in the EBRAINS KG.png||alt="KG schema for simulation"]]
24 -
25 -(note that the diagrams do not show Agents; the person who launched each analysis/simulation activity is linked to the activity with a ##wasAssociatedWith## connection).
26 -
27 -(% class="box warningmessage" %)
28 -(((
29 -TODO: insert or link to the detailed schemas for each type
30 -)))
31 -
32 32  == Tools for automated capture of provenance ==
33 33  
34 -
35 -Issues to discuss:
36 -
37 37  * on different systems:
38 38  ** HPC systems
39 39  ** neuromorphic systems
... ... @@ -44,17 +44,9 @@
44 44  
45 45  == Communication between computer systems and the KG ==
46 46  
47 -Two issues arise:
33 +* local cache and synchronization?
48 48  
49 -(i) fine-grained provenance information may need to be obtained on compute nodes, which may not have network access;
50 50  
51 -(ii) failures of provenance upload should not cause the workflows to fail;
52 -
53 -An overall solution for both of these issues would perhaps involve a local cache and later synchronization.
54 -
55 55  == User interfaces for browsing, visualizing, and searching provenance information ==
56 56  
57 -(% class="box infomessage" %)
58 -(((
59 -DISCUSSION NEEDED: integrate visualization of prov information into KG Search UI, and/or develop separate app?
60 -)))
38 +
Workflow provenance in the EBRAINS KG-2.png
Author
... ... @@ -1,1 +1,0 @@
1 -XWiki.adavison
Size
... ... @@ -1,1 +1,0 @@
1 -58.6 KB
Content
Workflow provenance in the EBRAINS KG.png
Author
... ... @@ -1,1 +1,0 @@
1 -XWiki.adavison
Size
... ... @@ -1,1 +1,0 @@
1 -49.9 KB
Content
Workflow provenance in the EBRAINS KG.svg
Author
... ... @@ -1,1 +1,0 @@
1 -XWiki.adavison
Size
... ... @@ -1,1 +1,0 @@
1 -159.5 KB
Content
Collaboratory.Apps.Collab.Code.CollabClass[0]
Public
... ... @@ -1,1 +1,1 @@
1 -Yes
1 +No
owner
... ... @@ -1,1 +1,0 @@
1 -adavison
XWiki.XWikiRights[5]
Allow/Deny
... ... @@ -1,1 +1,0 @@
1 -Allow
Levels
... ... @@ -1,1 +1,0 @@
1 -view
Users
... ... @@ -1,1 +1,0 @@
1 -XWiki.XWikiGuest
XWiki.XWikiRights[6]
Allow/Deny
... ... @@ -1,1 +1,0 @@
1 -Allow
Groups
... ... @@ -1,1 +1,0 @@
1 -XWiki.XWikiAllGroup
Levels
... ... @@ -1,1 +1,0 @@
1 -view