kg-spatial-search

Last modified by oschmid on 2023/08/22 11:23

KG Spatial Search

A how-to guide to use the spatial search extension of the EBRAINS KG

What is KG Spatial Search?

The Spatial Search extension is an additional index which allows to search for registered datasets in the KG by spatial search - meaning that it is possible to find datasets based on their locality in the 3-dimensional space by geometries.

Why is there an additional API to the KG Query API?

Although in a first version, it was attempted to directly integrate the spatial search query capabilities into the KG Query API, this approach has turned out to not be ideal. The reasons for this is the increased complexity introduced into the Query API for a very specific use-case covering the needs for a small subset of the standard users who query the KG. This is why it was decided to provide the functionality as an extension instead allowing to query the ids of the datasets through the spatial search and subsequently ask the KG for the semantic meta information.

What data is registered in the spatial search?

Theoretically, all data which is anchored in their respective coordinate space can be registered in the KG Spatial Search. We currently provide ingestion pipelines for data annotated with the locareJSON format. Please note that complex, planar and volumetric geometries such as polyhedrons, spheres, etc. are translated into point clouds of a configurable density during the ingestion process to ensure better performance at query time for the cost for potential false-negatives in highly fine-granular queries. Please also note that currently only datasets can be registered for which the spatial location information is regarded to be non-sensitive from a GDPR perspective.

How do I query the KG Spatial Search?

Simply use the API endpoint at https://spatial.kg.ebrains.eu/cores/ebrains/spatial_objects by running queries according to the following examples:

Get started: Query by a "bounding box" (hyperrectangle)

1
2
3
4
5
6
7
curl -X 'POST' \
 'https://spatial.kg.ebrains.eu/cores/ebrains/spatial_objects' \
 -H 'accept: application/json' \
 -H 'Content-Type: application/json' \
 -d '{
  "filters": "inside(hyperrectangle{[50.0,146.0,227.0], [51.0, 149.0, 228.0], \"AMB-CCF_v3-RAS\"})"
}'

As you can see, you're sending a POST request to the endpoint at https://spatial.kg.ebrains.eu/spatial-search/cores/ebrains/spatial_objects  with a payload defining a geometry of interest. In this case, we're looking for all objects that are inside a hyperrectangle defined by its lowest and highest point in the coordinate system of the coordinate space AMB-CCF_v3-RAS. This query will return you an array of ids of the objects located within the geometry which you then can use to conveniently query the KG either through the Instance API or the Query API to access detailed meta information.

Query by hypersphere

Alongside the possibility to use hyperrectangles for querying the spatial search, you can also use hyperspheres:

1
2
3
{
 "filters": "inside(hypersphere{[50.0,146.0,227.0], 2, \"AMB-CCF_v3-RAS\"})"
}

It is defined by the center of the sphere with its coordinates, the radius in coordinate units and the coordinate space of the given coordinates (in this case again AMB-CCF_v3-RAS)

Advanced use: Union geometries for complex queries

To build arbitrarily complex geometries, it is possible to combine hyperrectangles and hyperspheres via (nested) unions:

1
2
3
{
 "filters": "union(union(inside(hyperrectangle{[50.0,146.0,227.0], [51.0, 149.0, 228.0], \"AMB-CCF_v3-RAS\"}),    inside(hyperrectangle{[120.0,146.0,227.0], [150.0, 180.0, 300.0], \"AMB-CCF_v3-RAS\"})),inside(hypersphere{[50.0,146.0,227.0], 2, \"AMB-CCF_v3-RAS\"}))"
}

How to get the available spaces?

To get a list of available spaces (to know which coordinate systems are available), please run

1
2
3
4
5
curl -X 'POST' \
 'https://spatial.kg.ebrains.eu/spaces' \
 -H 'accept: application/json' \
 -H 'Content-Type: application/json' \
 -d '{}'

to retrieve a list of spaces. To get more information about the space, you can then ask for a specific one with

1
2
3
4
curl -X 'GET' \
 'https://spatial.kg.ebrains.eu/spaces/AMB-CCF_v3-RAS' \
 -H 'accept: application/json' \
 -H 'Content-Type: application/json'

Technical details of the ingestion pipeline

The current pipeline is regularly parsing locareJSON files registered in the KG.

  • Points are translated to the data structure of spatial search without manipulation
  • Polyhedrons are translated via the vtk library into a point cloud of (currently) a density of 1 coordinate unit (by ray-casting on an obb-tree)
  • Other geometries are work-in-progress

The resulting files (all representing point-clouds) as well as the space definitions are uploaded to the bucket of this collab. A cron job running on the spatial search server pulls the files from the repo and rebuilds the databases index in regular intervals.