Last modified by luehrs on 2023/04/12 09:26
From version 8.1
edited by luehrs
on 2023/04/12 07:26
Change comment: There is no comment for this version
To version 7.1
edited by luehrs
on 2023/04/12 07:19
Change comment: There is no comment for this version

Summary

Details

Page properties
Content
... ... @@ -144,78 +144,3 @@
144 144  ##Job id: 532086a8-30a9-11eb-b0da-fa163e1833c1##
145 145  
146 146  Users can use multiple ##~-~-os-token## to specify OS tokens for different projects.
147 -
148 -==== Bulk transfer ====
149 -
150 -Users can use bulk transfer by using the ##~-~-file## option. Then they have to submit a JSON file.
151 -
152 -##$fts-rest-transfer-submit \
153 --s https:~/~/fts.bsc.es:8446/ \##
154 -##~-~-access-token $tok \##
155 -##~-~-file <your_file>.json##
156 -
157 -The format of such files is described here: [[https:~~/~~/fts3-docs.web.cern.ch/fts3-docs/fts-rest/docs/bulk.html>>url:https://fts3-docs.web.cern.ch/fts3-docs/fts-rest/docs/bulk.html]]. Note that the files the user transfers this way still need to be from and to the same object store, i.e., she/he can only transfer a list of files from project A at site 1 to project B at site 2 in one job.
158 -
159 -As it is the same for any Swift-to-Swift transfer, users have to supply OS project ids, and in some cases, OS tokens as well. Users can use the command line options to do so or put it in the ##params## section when using a JSON file. An example of the JSON file for a bulk transfer is shown below:
160 -
161 -##{
162 -"files": [
163 -{
164 - "sources": [
165 - "swifts:~/~/swift.bsc.es/testcontainer/cp1"
166 - ],
167 - "destinations": [
168 - "swifts:~/~/object.cscs.ch/testcontainer/bulk/cp1"
169 - ]
170 -},
171 -{
172 - "sources": [
173 - "swifts:~/~/swift.bsc.es/testcontainer/cp2"
174 - ],
175 - "destinations": [
176 - "swifts:~/~/object.cscs.ch/testcontainer/bulk/cp2"
177 - ]
178 -}
179 -],
180 -"params":{             "os_project_id":"c7ccbe370791496fa44632e6ccb10405:7cc304cc62d44ab180d4a28427e2b64b",
181 - "os_token":[ 
182 - "7cc304cc62d44ab180d4a28427e2b64b:gAAAAA…"
183 - ]
184 -}##
185 -
186 -=== Query a transfer ===
187 -
188 -This command checks the status of the transfer job; a detailed FTS job state machine can be found here: [[https:~~/~~/fts3-docs.web.cern.ch/fts3-docs/docs/state_machine.html>>url:https://fts3-docs.web.cern.ch/fts3-docs/docs/state_machine.html]].
189 -
190 -##$fts-rest-transfer-status \
191 --s https:~/~/fts.bsc.es:8446/ \
192 -~-~-access-token $tok \
193 -<your_job_id>##
194 -
195 -The job ID is given when the user submits the transfer.
196 -
197 -=== List transfers ===
198 -
199 -This command lists the ongoing transfers at the FTS endpoint.
200 -
201 -##$fts-rest-transfer-list \
202 --s https:~/~/fts.bsc.es:8446/ \
203 -~-~-access-token $tok##
204 -
205 -=== Cancel a transfer ===
206 -
207 -This command cancels the transfer job.
208 -
209 -##$fts-rest-transfer-cancel \
210 --s https:~/~/fts.bsc.es:8446/ \
211 -~-~-access-token $tok \
212 -<your_job_id>##
213 -
214 -== Notes ==
215 -
216 -The configuration of the service allows to:
217 -
218 -* Transfer object files with a maximum size of 5GB.
219 -Note that files larger than 512MB will be stored as large objects in OpenStack, namely, files are split when larger than 512 MB; each chunk is 256 MB. Note that users may still view/download the files as a whole like normal objects after the upload is complete. Information on large object support can be found here: [[https:~~/~~/docs.openstack.org/swift/latest/overview_large_objects.html>>url:https://docs.openstack.org/swift/latest/overview_large_objects.html]].
220 -* Manage up to 1000 concurrent requests.
221 -This means that up to 1000 object files, or chunks of an object file, can be sent to the storage system concurrently;