Changes for page 3 Try it yourself!

Last modified by maaike on 2022/07/06 10:08
From version 10.1
edited by maaike
on 2022/07/06 08:27
Change comment: There is no comment for this version
To version 2.3
edited by maaike
on 2022/06/28 10:10
Change comment: There is no comment for this version

Summary

Details

Page properties
Title
... ... @@ -1,1 +1,1 @@
1 -3 Try it yourself!
1 +Try it yourself!
Content
... ... @@ -6,29 +6,29 @@
6 6  1. Find datasets that contain NIfTI files
7 7  1. Find the software that can be used to open .smr file formats?
8 8  
9 -[[Log in to your EBRAINS account and try running the examples yourself!>>https://lab.ch.ebrains.eu/hub/user-redirect/lab/tree/shared/Practical%20Guide%20to%20Using%20the%20EBRAINS%20Knowledge%20Graph%20in%20(your)%20Research%20-%20User%20Examples/query.ipynb]]
9 +=== ===
10 10  
11 -=== Example 1 - How many datasets used human subjects? ===
11 +==== Example 1 - How many datasets used human subjects? ====
12 12  
13 -==== **Search UI** ====
13 +**Search UI**
14 14  
15 15  In the filter function, select "Homo Sapiens" under species. This filters the available datasets in the Knowledge Graph for human subjects only.
16 16  
17 -[[[[image:HumanData.png]]>>https://search.kg.ebrains.eu/?facet_type[0]=Dataset&facet_Dataset_speciesFilter[0]=Homo%20sapien&category=Dataset&species[0]=Homo%20sapiens]]
17 +[[https:~~/~~/search.kg.ebrains.eu/?facet_type[0]=Dataset&facet_Dataset_speciesFilter[0]=Homo%20sapiens>>https://search.kg.ebrains.eu/?facet_type[0]=Dataset&facet_Dataset_speciesFilter[0]=Homo%20sapiens]]
18 18  
19 -[[https:~~/~~/search.kg.ebrains.eu/?facet_type[0]=Dataset&facet_Dataset_speciesFilter[0]=Homo%20sapien&category=Dataset&species[0]=Homo%20sapiens>>https://search.kg.ebrains.eu/?facet_type[0]=Dataset&facet_Dataset_speciesFilter[0]=Homo%20sapien&category=Dataset&species[0]=Homo%20sapiens]]
19 +**Query Builder**
20 20  
21 -==== **Query Builder** ====
22 22  
23 -To search for datasets containing human subjects only, we will therefore execute the query against the "dataset version" data structure. We want to know the persistent identifier and name of the dataset version, so we declare the "id" and "lookup label" first. Since our objective is to filter dataset based on the species of the subjects, we need to specify "sudied specimen" in our query too. We have four specimen categories, "subjects", "subject group", "tissue sample", and "tissue sample collection". We add a "type filter" to restrict our results to "Subject" and "Subject group" since we are not currently not interested in "Tissue samples" and "Tissue sample collections". To ensure that we only get datasets with human subjects, we can define the "Species" under "studied specimen" (again, you can add a type filter to exclude "Strain" since this is irrelevant for human subjects). For the "Species", we want the "label" to contain "homo sapiens" which is why we add a filter "CONTAINS" with the value "Homo sapiens".
22 +==== Example 2 - Find datasets with male adult subjects ====
24 24  
25 -For graph databases, like the EBRAINS Knowledge Graph, it is very easy to create very long and complex queries. We can simplify deeply nested structures by "flattening" the query. This is only possible when a property only has 1 nested property ("child"). In our query, this is the case for the "Studied specimen" and the "Species".
24 +**Search UI**
26 26  
27 -Once we have build your query, we can go to the "execute query" section (the play button on the left) and run the query, we can see the total number of dataset versions.
28 -\\Please note that this number can differ from the one you figured out in the search UI. The reason for this is, that the search UI does only count the newest dataset version whilst the query also returns older dataset versions.
26 +All the metadata in the knowledge graph is represented by nodes and their relationships by the edges. Most of the "basic" metadata is visualised in the Search UI to make it easy for the user to find datasets that fit certain criteria without needing to know how to navigate and traverse a graph structure.
29 29  
30 -{{code language="json" layout="LINENUMBERS"}}
31 -{
28 +**Query Builder**
29 +
30 +{{code language="python"}}
31 +query = {
32 32   "@context": {
33 33   "@vocab": "https://core.kg.ebrains.eu/vocab/query/",
34 34   "query": "https://schema.hbp.eu/myQuery/",
... ... @@ -47,135 +47,54 @@
47 47   },
48 48   "structure": [
49 49   {
50 - "propertyName": "query:shortName",
51 - "path": "https://openminds.ebrains.eu/vocab/shortName"
52 - },
53 - {
54 54   "propertyName": "query:id",
55 55   "path": "@id"
56 56   },
57 57   {
58 - "propertyName": "query:studiedSpecimen",
59 - "required": true,
60 - "filter": {
61 - "op": "CONTAINS",
62 - "value": "Homo sapiens"
63 - },
64 - "path": [
65 - {
66 - "@id": "https://openminds.ebrains.eu/vocab/studiedSpecimen",
67 - "typeFilter": [
68 - {
69 - "@id": "https://openminds.ebrains.eu/core/Subject"
70 - },
71 - {
72 - "@id": "https://openminds.ebrains.eu/core/SubjectGroup"
73 - }
74 - ]
75 - },
76 - {
77 - "@id": "https://openminds.ebrains.eu/vocab/species",
78 - "typeFilter": [
79 - {
80 - "@id": "https://openminds.ebrains.eu/controlledTerms/Species"
81 - }
82 - ]
83 - },
84 - "https://openminds.ebrains.eu/vocab/name"
85 - ]
86 - }
87 - ]
88 -}
89 -{{/code}}
90 -
91 -=== Example 2 - Find datasets with male adult subjects ===
92 -
93 -==== **Search UI** ====
94 -
95 -All the metadata in the knowledge graph is represented by nodes and their relationships by the edges. Most of the "basic" metadata is visualised in the Search UI to make it easy for the user to find datasets that fit certain criteria without needing to know how to navigate and traverse a graph structure. When searching for "male adult" subjects in the search UI, we find datasets that have these keywords in any of the text summarised on the dataset card (it is a 'fuzzy search' : [[https:~~/~~/search.kg.ebrains.eu/?category=Dataset&q=male%20and%20adult>>https://search.kg.ebrains.eu/?category=Dataset&q=male%20and%20adult]]). To ensure we only look for the any specimen (subjects or samples) originating from male adult mice, we need to write a query and extract the metadata programmatically.
96 -
97 -==== **Query Builder** ====
98 -
99 -For dataset versions that use male adult subjects, we can filter datasets using these 2 properties. The easiest way is to add a required filter to biological sex that is "EQUAL" to "male" and the age category "EQUAL" to "adult". By selecting the filter "EQUAL" instead of "CONTAINS", we ensure that only datasets with adult animals are found. If we want to be more general and include all subjects from the onset of sexual maturity, we can use "CONTAINS" instead as this will include subjects with the age category "prime adult", "young adult" and "late adult" as well.
100 -
101 -We have again taken advantage of the type filter (set it to subjects and subject groups), and we flattened the query where possible. This means that you now find multiple elements in the "path" for "biological sex" and for the "age category".
102 -
103 -Try it yourself and check out the differences between the results of the flattened and unflattened queries!
104 -
105 -{{code language="json" layout="LINENUMBERS"}}
106 -{
107 - "@context": {
108 - "@vocab": "https://core.kg.ebrains.eu/vocab/query/",
109 - "query": "https://schema.hbp.eu/myQuery/",
110 - "propertyName": {
111 - "@id": "propertyName",
112 - "@type": "@id"
113 - },
114 - "path": {
115 - "@id": "path",
116 - "@type": "@id"
117 - }
118 - },
119 - "meta": {
120 - "type": "https://openminds.ebrains.eu/core/DatasetVersion",
121 - "responseVocab": "https://schema.hbp.eu/myQuery/"
122 - },
123 - "structure": [
124 - {
125 125   "propertyName": "query:shortName",
126 126   "path": "https://openminds.ebrains.eu/vocab/shortName"
127 127   },
128 128   {
129 - "propertyName": "query:id",
130 - "path": "@id"
131 - },
132 - {
133 133   "propertyName": "query:studiedSpecimen",
134 - "path": {
135 - "@id": "https://openminds.ebrains.eu/vocab/studiedSpecimen",
136 - "typeFilter": [
137 - {
138 - "@id": "https://openminds.ebrains.eu/core/Subject"
139 - },
140 - {
141 - "@id": "https://openminds.ebrains.eu/core/SubjectGroup"
142 - }
143 - ]
144 - },
59 + "path": "https://openminds.ebrains.eu/vocab/studiedSpecimen",
145 145   "required": true,
146 146   "structure": [
147 147   {
148 - "propertyName": "query:lookupLabel",
149 - "path": "https://openminds.ebrains.eu/vocab/lookupLabel"
150 - },
151 - {
152 152   "propertyName": "query:id",
153 153   "path": "@id"
154 154   },
155 155   {
156 156   "propertyName": "query:biologicalSex",
68 + "path": "https://openminds.ebrains.eu/vocab/biologicalSex",
157 157   "required": true,
158 - "filter": {
159 - "op": "EQUALS",
160 - "value": "male"
161 - },
162 - "path": [
163 - "https://openminds.ebrains.eu/vocab/biologicalSex",
164 - "https://openminds.ebrains.eu/vocab/name"
165 - ]
70 + "structure": {
71 + "propertyName": "query:name",
72 + "path": "https://openminds.ebrains.eu/vocab/name",
73 + "required": true,
74 + "filter": {
75 + "op": "EQUALS",
76 + "value": "male"
77 + }
78 + }
166 166   },
167 167   {
168 168   "propertyName": "query:studiedState",
82 + "path": "https://openminds.ebrains.eu/vocab/studiedState",
169 169   "required": true,
170 - "filter": {
171 - "op": "EQUALS",
172 - "value": "adult"
173 - },
174 - "path": [
175 - "https://openminds.ebrains.eu/vocab/studiedState",
176 - "https://openminds.ebrains.eu/vocab/ageCategory",
177 - "https://openminds.ebrains.eu/vocab/name"
178 - ]
84 + "structure": {
85 + "propertyName": "query:ageCategory",
86 + "path": "https://openminds.ebrains.eu/vocab/ageCategory",
87 + "required": true,
88 + "structure": {
89 + "propertyName": "query:name",
90 + "path": "https://openminds.ebrains.eu/vocab/name",
91 + "required": true,
92 + "filter": {
93 + "op": "EQUALS",
94 + "value": "adult"
95 + }
96 + }
97 + }
179 179   }
180 180   ]
181 181   }
... ... @@ -183,9 +183,9 @@
183 183  }
184 184  {{/code}}
185 185  
186 -=== Example 3 - Find datasets that contain NIfTI files ===
105 +==== Example 3 - Find datasets that contain NIfTI files ====
187 187  
188 -==== **Search UI** ====
107 +**Search UI**
189 189  
190 190  Files are organised based on their file format (i.e. file extension) and the software that could be used to open these files. In openMINDS this is captured with the content type. In the filter function, select "application/vnd.nifti.1" under content types to select all the dataset with a NIfTI 1 file format.
191 191  
... ... @@ -193,18 +193,10 @@
193 193  
194 194  [[https:~~/~~/search.kg.ebrains.eu/?facet_type[0]=Dataset&facet_Dataset_contentTypes[0]=application%2Fvnd.nifti.1>>https://search.kg.ebrains.eu/?facet_type[0]=Dataset&facet_Dataset_contentTypes[0]=application%2Fvnd.nifti.1]]
195 195  
196 -==== **Query Builder** ====
115 +**Query Builder**
197 197  
198 -To find datasets with a particular file format in it, we can either write a query for 1) the file extension or 2) based on the content type. The difference between the two approaches is that the first approach just looks at the file extension without considering the what type of file format it is and what software can be used to open it. For example, both nifti 1 and nifti 2 files have the same extension. The nifti 2 format is an update of nifti 1 and will not be recognised as a valid nifti 1 format. This is important when considering what program to use when opening the files. To be able to differentiate, we describe the files with content types that tell the user what type of file format it is and we have linked a number of software applications to that content type to facilitate reuse of the data.
199 -
200 -For these examples, we are showing the unflattened version of the query. Try it yourself to create a flattened version!
201 -
202 -**Query datasets based on file extension**
203 -
204 -We can restrict the search results with a filter using a required field. In this particular case a filter that "ENDS_WITH" a value (e.g. .nii.gz) could be used. We can use .nii for normal nifti files or .nii.gz for compressed nifti files.
205 -
206 -{{code language="json" layout="LINENUMBERS"}}
207 -{
117 +{{code language="python"}}
118 +query = {
208 208   "@context": {
209 209   "@vocab": "https://core.kg.ebrains.eu/vocab/query/",
210 210   "query": "https://schema.hbp.eu/myQuery/",
... ... @@ -255,139 +255,3 @@
255 255   ]
256 256  }
257 257  {{/code}}
258 -
259 - **Query datasets based on content type**
260 -
261 -To find datasets using the content type structure, we can use a filter that "CONTAINS" a value (e.g. nifti). We are now not distinguishing between nifti 1 or nifti 2 files, but select all files that are classified as "nifti". If you want to narrow down the results further, you could replace "nifti" by "nifti.1" or "nifti.2".
262 -
263 -{{code language="json" layout="LINENUMBERS"}}
264 -{
265 - "@context": {
266 - "@vocab": "https://core.kg.ebrains.eu/vocab/query/",
267 - "query": "https://schema.hbp.eu/myQuery/",
268 - "propertyName": {
269 - "@id": "propertyName",
270 - "@type": "@id"
271 - },
272 - "path": {
273 - "@id": "path",
274 - "@type": "@id"
275 - }
276 - },
277 - "meta": {
278 - "type": "https://openminds.ebrains.eu/core/DatasetVersion",
279 - "responseVocab": "https://schema.hbp.eu/myQuery/"
280 - },
281 - "structure": [
282 - {
283 - "propertyName": "query:shortName",
284 - "path": "https://openminds.ebrains.eu/vocab/shortName"
285 - },
286 - {
287 - "propertyName": "query:id",
288 - "path": "@id"
289 - },
290 - {
291 - "propertyName": "query:repository",
292 - "path": "https://openminds.ebrains.eu/vocab/repository",
293 - "required": true,
294 - "structure": {
295 - "propertyName": "query:contentTypePattern",
296 - "path": "https://openminds.ebrains.eu/vocab/contentTypePattern",
297 - "required": true,
298 - "structure": {
299 - "propertyName": "query:contentType",
300 - "path": "https://openminds.ebrains.eu/vocab/contentType",
301 - "required": true,
302 - "structure": {
303 - "propertyName": "query:name",
304 - "path": "https://openminds.ebrains.eu/vocab/name",
305 - "required": true,
306 - "filter": {
307 - "op": "CONTAINS",
308 - "value": "nifti"
309 - }
310 - }
311 - }
312 - }
313 - }
314 - ]
315 -}
316 -{{/code}}
317 -
318 -=== (% id="cke_bm_3314S" style="display:none" %) (%%)Example 4 - Find the software that can be used to open .smr file formats? ===
319 -
320 -==== **Search UI** ====
321 -
322 -To find software that can open a particular file format like the Spike2 file format (.smr), we can select the category "software" and then filter based on "input format". We select "application/vnd.spike2.sonpy.son" to ensure we only get software for this file format.
323 -
324 -[[[[image:SoftwareSearch.png||alt="Software Search"]]>>https://search.kg.ebrains.eu/?category=Software&inputFormats[0]=application%2Fvnd.spike2.sonpy.so]]
325 -
326 -[[https:~~/~~/search.kg.ebrains.eu/?category=Software&inputFormats[0]=application%2Fvnd.spike2.sonpy.son>>https://search.kg.ebrains.eu/?category=Software&inputFormats[0]=application%2Fvnd.spike2.sonpy.son]]
327 -
328 -
329 -==== **Query Builder** ====
330 -
331 -For this question, we will execute the query against the "softwareVersion data structure. We ask for the name, version and input type of the software. We further refine our query by restricting the result to software that can open files with the file extension ".smr". We get the same 3 software types as in the search, and we immediately see that one of the software types has multiple versions that can open this kind of files.
332 -
333 -(This query is not flattened).
334 -
335 -You may have noticed that even though you can get the same results in the Search UI and the Query Builder, the details you need to define to get to that result are not always the same. For example, in the query we can rely on the file extension (i.e. ".smr"), whereas in the Search UI we sometimes need to have specific details about the file you are trying to open, such as the software/data aquisition/analysis package (i.e. Spike2) that was used. You may keep this is mind when tackling a particular question. Queries can be made as detailed and broad as you want, whereas the Search UI is a snapshot of the most common and basic information that is available.
336 -
337 -{{code language="json" layout="LINENUMBERS"}}
338 -{
339 - "@context": {
340 - "@vocab": "https://core.kg.ebrains.eu/vocab/query/",
341 - "query": "https://schema.hbp.eu/myQuery/",
342 - "propertyName": {
343 - "@id": "propertyName",
344 - "@type": "@id"
345 - },
346 - "path": {
347 - "@id": "path",
348 - "@type": "@id"
349 - }
350 - },
351 - "meta": {
352 - "type": "https://openminds.ebrains.eu/core/SoftwareVersion",
353 - "responseVocab": "https://schema.hbp.eu/myQuery/"
354 - },
355 - "structure": [
356 - {
357 - "propertyName": "query:id",
358 - "path": "@id"
359 - },
360 - {
361 - "propertyName": "query:shortName",
362 - "path": "https://openminds.ebrains.eu/vocab/shortName"
363 - },
364 - {
365 - "propertyName": "query:versionIdentifier",
366 - "path": "https://openminds.ebrains.eu/vocab/versionIdentifier"
367 - },
368 - {
369 - "propertyName": "query:inputFormat",
370 - "path": "https://openminds.ebrains.eu/vocab/inputFormat",
371 - "required": true,
372 - "structure": [
373 - {
374 - "propertyName": "query:name",
375 - "path": "https://openminds.ebrains.eu/vocab/name"
376 - },
377 - {
378 - "propertyName": "query:fileExtension",
379 - "path": "https://openminds.ebrains.eu/vocab/fileExtension",
380 - "required": true,
381 - "filter": {
382 - "op": "CONTAINS",
383 - "value": ".smr"
384 - }
385 - }
386 - ]
387 - }
388 - ]
389 -}
390 -{{/code}}
391 -
392 -
393 -
HumanData.png
Author
... ... @@ -1,1 +1,0 @@
1 -XWiki.maaike
Size
... ... @@ -1,1 +1,0 @@
1 -400.8 KB
Content
SoftwareSearch.png
Author
... ... @@ -1,1 +1,0 @@
1 -XWiki.maaike
Size
... ... @@ -1,1 +1,0 @@
1 -203.7 KB
Content