Changes for page Technical details
Last modified by lzehl on 2021/07/05 18:57
Summary
-
Page properties (2 modified, 0 added, 0 removed)
Details
- Page properties
-
- Parent
-
... ... @@ -1,1 +1,1 @@ 1 -Collabs.openminds. Documentation.WebHome1 +Collabs.openminds.openMINDS core.WebHome - Content
-
... ... @@ -10,149 +10,83 @@ 10 10 Please find below a documentation of the layout and requirements needed to keep the openMINDS modularity, the syntax of the openMINDS schema template, as well as the openMINDS integration pipeline. 11 11 ))) 12 12 13 -=== The openMINDSumbrella ===13 +=== Overview of the openMINDS layout === 14 14 15 15 (% style="text-align: justify;" %) 16 16 In summary, openMINDS is the overall umbrella for a set of distributed GitHub repositories, each defining a particular metadata model for neuroscience research products. 17 17 18 -(% style="text-align: justify;" %) 19 -The main (or central) [[openMINDS GitHub repository>>https://github.com/HumanBrainProject/openMINDS||rel="noopener noreferrer" target="_blank"]] ingests all these GitHub repositories as [[git-submodules>>https://git-scm.com/docs/git-submodule||rel="noopener noreferrer" target="_blank"]]. Furthermore it stores the openMINDS vocabulary (**##vocab##**), providing general definitions and references for **types** and **properties** used in schemas across all openMINDS repositories (cf. below). And last but not least, it holds the schema representations for all supported metadata formats created by the openMINDS integration pipeline (cf. below). 18 +=== The openMINDS schema template syntax === 20 20 21 21 (% style="text-align: justify;" %) 22 - Forthistowork smoothlyfor theexisting, but alsoforall newopenMINDSmetadatamodels, the corresponding openMINDSsubmodules(GitHub repositories)haveto meetthefollowing requirements:21 +All openMINDS metadata models use a light-weighted schema template syntax for defining the metadata. The correspondingly formatted schema files use the extension: **##.schema.tpl.json##**. 23 23 24 24 (% style="text-align: justify;" %) 25 - **(1)**The openMINDS metadatamodel has tobe located on a**publicGitHubrepository**andpublishedunder an**MITlicense**.24 +Although, as the file extension suggests, this openMINDS schema template syntax is inspired by JSON-Schema, it facilitates or even excludes technical aspects that are generally expected for the openMINDS schemas making it more human-readable, especially for untrained eyes. Behind the scenes, within the openMINDS integration pipeline (cf. below), this schema template syntax is then interpreted and flexibly translated to various formal metadata formats (e.g., JSON-Schema). 26 26 27 27 (% style="text-align: justify;" %) 28 - **(2)**TheGitHubrepositoryshouldhave at leastone**versionbranch** (e.g., "v1").27 +Despite the simplification in comparison to JSON-Schema, the openMINDS schema templates are also, at the core, specially formatted JSON files using a particular syntax, meaning special key-value pairs that define the validation rules of a schema. 29 29 30 30 (% style="text-align: justify;" %) 31 - **(3)** Theversionbranchshould havethe following**maindirectoryfolders**:**##schemas##**(required), **##tests##**(recommended),**##examples##**(recommended),and**##img##** (optional).30 +Please find in the following a full documentation of the openMINDS schema template syntax and how the key-value pairs need to be defined and interpreted. 32 32 33 33 (% style="text-align: justify;" %) 34 - **(4)**The **##schemas##** foldershould containtheschemasof that metadata model implemented inthe**openMINDS schematemplateyntax**(cf. below). The directory of the schemas can be further structured or flat.33 +===== Target & context templates ===== 35 35 36 36 (% style="text-align: justify;" %) 37 - **(5)**The **##tests##**foldershouldcontaintest-instances(JSON-LDs)forthe schemasinaflat directory.Thefile names forthese test-instancesshouldfollowtheconventionof36 +An openMINDS schema //HAS TO HAVE// a **##"_type"##** to be recognized as **target template**. In other words, the **##"_type"##** is used to define the openMINDS namespace of a corresponding schema using the following naming convention: 38 38 39 -(% style="text-align: center;" %) 40 -**##<<XXX>>-<<YYY>>.jsonld##** 41 - 42 -(% style="text-align: justify;" %) 43 -for files that should pass the tests, and 44 - 45 -(% style="text-align: center;" %) 46 -**##<<XXX>>-<<YYY>>-nok.jsonld##** 47 - 48 -(% style="text-align: justify;" %) 49 -for files that should fail the test. In both cases, **##<<XXX>>##** should be replaced with the label of the schema that is tested, and **##<<YYY>>##** with a user defined label for what aspect is tested (e.g., **##person-withoutCI.jsonld##**). 50 - 51 -(% style="text-align: justify;" %) 52 -**(6)** The **##examples##** folder should contain examples for valid instance collections for that metadata model. Each example should receive its own directory (folder) with a **##README.md##** describing the example, and an **##metadataCollection##** subfolder containing the openMINDS instances (JSON-LDs). This subfolder can be further structured or flat. 53 - 54 -(% style="text-align: justify;" %) 55 -**(7)** The **##img##** folder should contain image files used on that GitHub repository (e.g., the logo of the new openMINDS metadata model). The directory of the images can be further structured or flat. 56 - 57 -=== The openMINDS vocabulary === 58 - 59 -(% style="text-align: justify;" %) 60 -Located under the folder **##vocab##** in the main openMINDS GitHub directory, the openMINDS vocabulary is semi-automatically gathered and stored in dedicated JSON files ([[**##types.json##**>>https://raw.githubusercontent.com/HumanBrainProject/openMINDS/v2/vocab/types.json||rel="noopener noreferrer" target="_blank"]] and [[**##properties.json##**>>https://raw.githubusercontent.com/HumanBrainProject/openMINDS/v2/vocab/properties.json||rel="noopener noreferrer" target="_blank"]]). The openMINDS integration pipeline makes sure that both files are updated with each commit to any of the GitHub repositories for the openMINDS metadata models. With that, the openMINDS vocab reflects always an up-to-date status of the general attributes of existing **schemas** and **properties** across all openMINDS metadata models, while providing the opportunity to centrally review and maintain their consistency. In addition, this design allows us to centrally define and maintain multiple references to related schemas and matching schema properties of other metadata initiatives. How this works in detail is explained in the following. 61 - 62 -(% style="text-align: justify;" %) 63 -The **##types.json##** file is an associative array listing all existing openMINDS schemas (via their type). For each openMINDS schema, a small list of general attributes are provided in a nested associative array. Currently, the following attributes are captured: 64 - 65 65 {{code language="json"}} 66 66 { 67 - "OPENMINDS_SCHEMA_TYPE": { 68 - "description": "GENERAL_DESCRIPTION", 69 - "name": "DISPLAY_LABEL", 70 - "translatableTo": [ 71 - "REFERENCE_TO_RELATED_SCHEMA_OF_OTHER_INITIATIVE" 72 - ] 73 - } 40 + "_type": "https:~/~/openminds.ebrains.eu/<<schema-model>>/<<schema-name>>" 74 74 } 75 75 {{/code}} 76 76 77 77 (% style="text-align: justify;" %) 78 - With each newschema committed to one of the openMINDS metadata models, a new entry is appendedto the**##types.json##** file, with the display label automatically derived from therespective schematype and the remaining attributes predefined with a null value. Once an entry for a schema is madein the **##types.json##**file, the values ofall attributes(**##"name"##**, **##"description"##**, and **##"translatableTo"##**)can bemanually edited.All manual editionswill be preserved and notoverwritten whenthefile is updated again with a new commit. In case a schema is deletedfromthe openMINDS metadata models,the correspondingentry in the**##types.json##** file ismarkedasbeingdeprecated(additionalattribute-value pair;**##"deprecated": true##**).It onlycanbepermanentlyremovedfromthe **##types.json##**file,iftheentryis manuallydeleted.45 +where **##<<schema-model>>##** has to be replaced with the label of the openMINDS metadata model the corresponding schema belongs to and **##<<schema-name>>##** exchanged with the name of that schema (written in **##CamelCase##**). 79 79 80 80 (% style="text-align: justify;" %) 81 - Similartothe**##types.json##**file,the **##properties.json##**fileisan associativearray listing allpropertiesacrossallexistingopenMINDS schemas (viatheproperty name). For eachopenMINDSproperty,asmall listof generalattributes are providedinanestedassociative array. Currently,thefollowingattributes are captured:48 +If an openMINDS schema template file //DOES NOT// define a **##"_type"##**, it is interpreted as a **context template** which //HAS TO BE// extended to a target template. 82 82 83 -{{code language="json"}} 84 -{ 85 - "PROPERTY_NAME": { 86 - "description": "GENERAL_DESCRIPTION", 87 - "name": "DISPLAY_LABEL", 88 - "nameForReverseLink": "DISPLAY_LABEL_OF_REVERSED_LINK", 89 - "sameAs": [ 90 - "REFERENCE_TO_MATCHING_SCHEMA-PROPERTY_OF_OTHER_INITIATIVE" 91 - ], 92 - "schemas": [ 93 - "RELATIVE_PATH_TO_OPENMINDS-SCHEMA_USING_THIS_PROPERTY" 94 - ] 95 - } 96 -} 97 -{{/code}} 98 - 99 99 (% style="text-align: justify;" %) 100 - Witheach newpropertycommittedtoaschemaof oneofthe openMINDSmetadatamodels,anew entry is appendedtothe**##properties.json##**file,withthedisplay labeland list ofschemas in which thisproperty occurs automatically derived. The remaining attributesare initially provided with anullvalue. Once an entryforapropertyismade in the**##properties.json##**file,thevalues ofall attributes (**##"name"##**, **##"description"##**,**##"nameForReversedLink"##**, and **##"sameAs"##**)canbe manually edited, exceptfor **##"schemas"##**which will be alwaysautomatically updated. All those manual editionswill be preserved and not overwritten when thefileis updatedagain witha new commit. In case a propertyis notused anymore inany oftheschemasfrom the openMINDS metadatamodels,thecorrespondingentry in the**##properties.json##**file ismarked as being deprecated (additional attribute-value pair; **##"deprecated": true##**). It onlycan bepermanently removedfromthe**##properties.json##** file, if the entry is manually deleted.51 +Context templates are and should be used when multiple openMINDS schemas (target templates) have the same subset of properties. This common subset of properties can then be defined within a single context schema instead of each target template which facilitates the long-term maintenance of these properties. 101 101 102 -=== The openMINDS schema template syntax === 103 - 104 104 (% style="text-align: justify;" %) 105 - AllopenMINDS metadatamodelsaredefinedusingalight-weightedschematemplatesyntax.Althoughthisschematemplatesyntaxis inspiredby JSON-Schema,itoutsourcesmostschematechnicalitiesto behandledbythe openMINDSintegrationpipeline,makingthe openMINDS schemasmorehuman-readable,especiallyforuntrainedeyes.54 +To define that a target template is the extension of a context template, the target template can state under **##"_extends"##** the relative path to the context template. For example, the openMINDS core schema **##Dataset##** (target template) extends the core concept template **##researchProduct##**: 106 106 107 -(% style="text-align: justify;" %) 108 -The few remaining customized technical properties which need additional interpretation or translation to a formal schema languages (e.g. JSON-Schema) have an underscore as prefix (e.g., **##"_type"##**). Within the openMINDS integration pipeline (cf. below), the schema template syntax is interpreted, extended and flexibly translated to various formal schema languages. All further specifications of the openMINDS schema template syntax are described below. 109 - 110 -(% style="text-align: justify;" %) 111 -==== Basic openMINDS schema structure ==== 112 - 113 -(% style="text-align: justify;" %) 114 -All openMINDS schemas need to have the extension **##.schema.tpl.json##** and each schema is defined as a nested associative array (dictionary) with the following conceptual structure: 115 - 116 116 {{code language="json"}} 117 117 { 118 - "_type": "https://openminds.ebrains.eu/LABEL_OF_METADATA_MODEL/SCHEMA_NAME", 119 - "properties": { 120 - "PROPERTY_NAME": { 121 - "type": "DATA_TYPE", 122 - "_instruction": "METADATA_ENTRY_INSTRUCTION" 123 - }, 124 - "required": [ 125 - "PROPERTY_NAME" 126 - ] 58 + "_type": "https:~/~/openminds.ebrains.eu/core/Dataset" 59 + "_extends": "products/researchProduct.schema.tpl.json" 127 127 } 128 128 {{/code}} 129 129 63 +Note that this convention requires the context and corresponding target templates to be located in the same openMINDS metadata model repository. 64 + 130 130 (% style="text-align: justify;" %) 131 - **##"_type"##**definesthe schema type (ornamespace) with the depictednamingconvention, where the label of the respective openMINDS metadata model (e.g., **##"core"##**) and the schema name (format: UpperCamelCase; e.g. **##"ContactInformation"##**) haveto bespecified.Obviously, the schema name should be meaningful and provide some insides into what metadata content the schema covers.66 +===== String properties ===== 132 132 133 133 (% style="text-align: justify;" %) 134 - Under**##"properties"##** a nested associative array is defined, where each key defines the property name (format: lowerCamelCase; e.g. **##"givenName"##**). The correspondingvalue isagain a nested associative array defining the expected data **##"type"##** (cf. below) and the**##"_instructions"##** for entering the correct metadata for the respective property.69 +(//**coming soon**//) 135 135 136 136 (% style="text-align: justify;" %) 137 - Under**##"required"##** a list of property names can be provided that are obligatoryto bepresent in a correctly instantiated metadata instance of the respective schema. If none of the propertiesare required, this key-value pair does not have to be specified.72 +===== Numerical properties ===== 138 138 139 139 (% style="text-align: justify;" %) 140 - ==== Schemas extending schemas ====75 +(//**coming soon**//) 141 141 142 142 (% style="text-align: justify;" %) 143 - Inthe case that several schemas are highly related and contain a common set of properties, it is possibleto define a non-typecontext-schemawith these commonpropertiesthat can be extended and modified by the group of related schemas. All properties and constraints (e.g. required properties, expected data types) defined in the context-schema are passed on to the schemas extending this context-schema. Each of these schemas can define additional properties, or (if necessary) can overwrite the constraints of the context-schema. In order to state that a schema is extending a context-schema, the following additional key-value pair has to be added to the schema template above:78 +===== Object properties ===== 144 144 145 -(% style="text-align: center;" %)146 -** ##"_extends":"RELATIVE_PATH_TO_OPENMINDS-CONTEXT-SCHEMA"##**80 +(% style="text-align: justify;" %) 81 +(//**coming soon**//) 147 147 148 148 (% style="text-align: justify;" %) 149 - This design not only makes it easier to identify highly related schemas, but also facilitates the maintenance of the commonly used properties. A good hands-on example, is the context-schema [[ResearchProduct>>https://raw.githubusercontent.com/HumanBrainProject/openMINDS_core/v3/schemas/products/researchProduct.schema.tpl.json||rel="noopener noreferrer" target="_blank"]] which is extended by the following schema set: [[Dataset>>https://raw.githubusercontent.com/HumanBrainProject/openMINDS_core/v3/schemas/products/dataset.schema.tpl.json||rel="noopener noreferrer" target="_blank"]],[[MetaDataModel>>https://raw.githubusercontent.com/HumanBrainProject/openMINDS_core/v3/schemas/products/metaDataModel.schema.tpl.json]],[[Model>>https://raw.githubusercontent.com/HumanBrainProject/openMINDS_core/v3/schemas/products/model.schema.tpl.json||rel="noopenernoreferrer" target="_blank"]], and [[Software>>https://raw.githubusercontent.com/HumanBrainProject/openMINDS_core/v3/schemas/products/software.schema.tpl.json||rel="noopener noreferrer" target="_blank"]].84 +===== Property arrays ===== 150 150 151 151 (% style="text-align: justify;" %) 152 - Depending on the expected data**##"type"##** additionalconstraints can bemade for the metadata entry of a respective property. Currently, the openMINDS schema template syntax supports the followingdata types: **##"string"##**, ##**"integer"**##, **##"float"##**, **##"boolean"##**, **##"array"##** and **##"object"##**.87 +(//**coming soon**//) 153 153 89 + 154 154 === The openMINDS integration pipeline === 155 155 156 -(//**coming soon**//) If you'd like to learn more about the openMINDS integration pipeline, especially if you'd like to contribute to it, please get in touch with us (the openMINDS development team) via the issues on the openMINDS or openMINDS_generator GitHub or the support email: openminds@ebrains.eu 157 - 158 -{{putFootnotes/}} 92 +(//**coming soon**//)