Attention: The Keycloak upgrade has been completed. As this was a major upgrade, there may be some unexpected issues occurring. Please report any issues you find to support by using the contact form found at https://www.ebrains.eu/contact/. Thank you for your patience and understanding. 


Version 2.1 by chaney08 on 2021/07/20 20:14
Show last authors
1
2
3 This page describes a workflow that you can follow to use the Collaboratory.drive as a back-end for your service to store and read data inside a private user space.
4
5 == Overview ==
6
7 Your OIDC client can be setup to have a service account linked to it. This service account being seen as a user by Keycloak, it can log in the Collaboratory.drive to have its user account synchronised there.
8
9 From this point, everything is set up to let your service account create and share files and folders with existing users. This can be achieved by using the existing Seafile API (the tool behind the Collaboratory.drive).
10
11 == Creating a service account in IAM ==
12
13 If you do not yet have an OIDC client for your service, see [[Registering an OIDC client>>doc:Collabs.the-collaboratory.Documentation IAM.FAQ.OIDC Clients explained.1\. Registering an OIDC client.WebHome]].
14
15 Once you have an OIDC account, you need to [[modify your client>>doc:Collabs.collaboratory-community-apps.Community App Developer Guide.WebHome||anchor="HModifyingyourclient"]] to allow service accounts:
16
17 {{code language="bash"}}
18 # Set your registration token and client id
19 clb_reg_token="..."
20 clb_client_id="my-awesome-client"
21
22 # Update the client. Note that the client ID appears both in the endpoint URL and in the body of the request.
23 curl -X PUT https://iam.ebrains.eu/auth/realms/hbp/clients-registrations/default/${clb_client_id} \
24 -H "Authorization: Bearer ${clb_reg_token}" \
25 -H 'Content-Type: application/json' \
26 -d '{
27 "clientId": "'${clb_client_id}'",
28 "serviceAccountsEnabled": true
29 }' |
30
31 # Pretty print the JSON response
32 json_pp
33
34 {{/code}}
35
36 == Creating the service account in the Drive ==
37
38 Once you have created the service account in IAM, the service account needs to be created in the Drive. This step requires IAM admin privileges so you cannot do it yourself. Send a request to [[support@ebrains.eu>>mailto:support@ebrains.eu]] instead and make sure to include the following information:
39
40 1. The URL of this page: [[Create an IAM service account>>https://wiki.ebrains.eu/bin/view/Collabs/collab-devs/How%20To/Create%20an%20IAM%20service%20account/]]
41 1. Your client ID: e.g. "my-awesome-client"
42
43 Never share tokens or secrets.
44
45 == Creating the user space ==
46
47 Once support confirms that the service account has been activated in the Drive, you can proceed by creating the user space in the Drive. The user space will be created in the Drive, in the default Library of the service account. Each user of your service will have its own user space.
48
49 The general process is the following:
50
51 1. Fetch a token for your service account to be able to discuss with the Drive API.
52 1. Your service needs to get its default library id. It is where it will create the users' spaces.
53 1. For a given user, your service should create a folder, using a unique identifier (either the sub or username).
54
55 From this point, your service can now store and read data linked to users' accounts.
56
57 If your service needs the data to be available in Jupyter Notebooks, it will need to share it with the user:
58
59 1. Inside the user folder, your service should create a folder with a name that would be common across users.
60 1. Your service should now share the inside folder with the common name with the individual user.
61
62 This way, notebooks will be able to refer to your service data with a common path for all users.
63
64 === Getting an API token ===
65
66 The first step is to fetch an access token for your service account. This can be done with the following request:
67
68 {{code language="bash"}}
69 # Set the client id and secret
70 clb_client_id=...
71 clb_client_secret=...
72
73 # Call the token endpoint
74 curl -X POST https://iam.ebrains.eu/auth/realms/hbp/protocol/openid-connect/token \
75 -d 'grant_type=client_credentials' \
76 -d "client_id=${clb_client_id}" \
77 -d "client_secret=${clb_client_secret}" \
78 -d "scope=openid email collab.drive" |
79
80 # Pretty print the JSON response
81 json_pp
82 {{/code}}
83
84 The response will be similar to:
85
86 {{code language="javascript"}}
87 {
88 "expires_in" : 108000,
89 "not-before-policy" : 1563261088,
90 "access_token" : "eyJhbGciOiJSU...",
91 "session_state" : "4882dbae-56dc-4a91-b8ae-8ad07117d4af",
92 "refresh_expires_in" : 14400,
93 "refresh_token" : "eyJhbGciOiJIUz...",
94 "token_type" : "bearer",
95 "scope" : "openid email collab.drive"
96 }
97 {{/code}}
98
99 Fetch the access token from this response.
100
101 The next step is to get a Drive API token with the access token:
102
103 {{code language="bash"}}
104 # Set the access token value
105 clb_access_token=...
106
107 # Call the token endpoint
108 curl -X GET https://drive.ebrains.eu/api2/account/token/ \
109 -H "Authorization: Bearer ${clb_access_token}"
110 {{/code}}
111
112 The response will look like ##1c1345da8a99b36168afef92ef7f83af8b4ca6f0##. This is the API token that you will need to use in your ##Authorization## header to communicate with the Collaboratory.drive API.
113
114 {{warning}}
115 Note that, unlike access tokens , the Drive API token needs an authorisation header in the form of "##Authorization: **Token** your-api-token"## (and not Bearer!).
116 {{/warning}}
117
118 === Fetching your service account's default library ===
119
120 Note: from this point, you can refer to the [[documentation of Seafile>>https://download.seafile.com/published/web-api/v2.1-admin/]] to make API calls to the Collaboratory.drive.
121
122 {{code language="bash"}}
123 # Set the API token
124 clb_api_token=...
125
126 # Call the default-repo endpoint
127 curl -X GET https://drive.ebrains.eu/api2/default-repo/ \
128 -H "Authorization: Token ${clb_api_token}" |
129
130 # Pretty print the JSON response
131 json_pp
132 {{/code}}
133
134 You will get a response in the form of:
135
136 {{code language="javascript"}}
137 {
138 "repo_id": "175a7b2e-f9f5-4f3c-a9e9-84c4e995f1ea",
139 "exists": true
140 }
141 {{/code}}
142
143 The `repo_id` is the identifier of the default library of your service account.
144
145 === Creating a folder for a given user ===
146
147 In order to isolate the data of individual users, you should create a folder for each user, based on a unique identifier of the user. The most secure identifier you can use is the `sub` as it is an internal unique identifier of IAM.
148
149 The `username` is also a valid unique identifier but, in some rare cases, they might be claimed by a different user in the future.
150
151 Creating a folder is done with the following call:
152
153 {{code language="bash"}}
154 # Set the API token, your library id and the folder path
155 clb_api_token=...
156 clb_repo_id=...
157 clb_folder_path=/my/user/folder/path
158
159 # Call the default-repo endpoint
160 curl -X POST \
161 "https://drive.ebrains.eu/api2/repos/${clb_repo_id}/dir/" \
162 -H "Authorization: Token ${clb_api_token}" \
163 -d "p=${clb_folder_path}"
164 -F 'operation=mkdir' |
165
166 # Pretty print the JSON response
167 json_pp
168 {{/code}}
169
170 Please note that you cannot create a path of folders all at once. You will need to make one call at each level of your path.
171
172 === Writing data to the user space ===
173
174 TODO
175
176 === Fetching data from the user space ===
177
178 TODO
179
180 == Sharing data with notebooks ==
181
182 The Collaboratory.drive is mounted in the Jupyter container of the user. This means that notebooks can access data stored in the Collaboratory.drive of the user.
183
184 Let's imagine your service generates data that should be ingested by a notebook. In order for the notebook to be usable by any user, the path to the data must be the same for each user.
185
186 When sharing a folder ##/path/to/the/shared_folder## with a user, the folder becomes reachable at the path ##drive/share with me/shared_folder##. As you can see, only the name of the folder is shared with the user.
187
188 Imagine you want to generate data per user and you want notebooks to refer to this data in a folder named ##//my-awesome-client//-data##. You will need to create a folder in your service account's default library in the following form:
189
190 ##/path/to/user/spaces/in/your/library/**${user-id}**///my-awesome-client//-data##.
191
192 Make sure the folder starts with your OIDC client name or some other unique name. Once the folder is created, you will need to share it with the user.
193
194 === Sharing a folder with a user ===
195
196 {{code language="bash"}}
197 # Set the parameters
198 clb_api_token=...
199 clb_repo_id=...
200 clb_username=...
201 clb_folder_path="/path/to/user/spaces/in/your/library/${clb_username}/my-awesome-client-data"
202 clb_permission=rw # r=read, w=write
203
204 # Call the default-repo endpoint
205 curl -X PUT "https://drive.ebrains.eu/api2/repos/${clb_repo_id}/dir/shared_items/" \
206 -H "Authorization: Token ${clb_api_token}" \
207 -d "p=${clb_folder_path}"
208 -F 'share_type=user' \
209 -F "permission=${clb_permission}" \
210 -F "username=${clb_username}@humanbrainproject.eu"
211 {{/code}}
212
213 You will get a response in the form of:
214
215 {{code language="javascript"}}
216 // TODO
217 {{/code}}
218
219 (% class="wikigeneratedid" id="HH4Won27tAppearinToC" %)
220