Changes for page User Guide: Swift Transfer with FTS
Last modified by luehrs on 2023/04/12 09:26
Summary
-
Page properties (1 modified, 0 added, 0 removed)
Details
- Page properties
-
- Content
-
... ... @@ -64,9 +64,9 @@ 64 64 65 65 To install the RPM package one needs to run the following command: 66 66 67 - {{{[promt]$ yum install fts-rest-client-1.0.0-%{_release}.el7.noarch}}}67 +##[promt]$ yum install fts-rest-client-1.0.0-%{_release}.el7.noarch## 68 68 69 -Should the user prefer to build the fts-rest-client him/herself, the scripts can be found in the fts-rest-flask repository here: [[https:~~/~~/gitlab.cern.ch/fts/fts-rest-flask/-/tree/swift>>url:https://gitlab.cern.ch/fts/fts-rest-flask/-/tree/swift]]. The user can clone the repository, switch to the swift branch, and cd into it. We advise creating a venv for this project. Then run pip install . or python setup.py install to install the fts package. After that, the user may run the scripts in src/cli the same way he/she uses the FTS commands. 69 +Should the user prefer to build the fts-rest-client him/herself, the scripts can be found in the fts-rest-flask repository here: [[https:~~/~~/gitlab.cern.ch/fts/fts-rest-flask/-/tree/swift>>url:https://gitlab.cern.ch/fts/fts-rest-flask/-/tree/swift]]. The user can clone the repository, switch to the swift branch, and ##cd## into it. We advise creating a ##venv## for this project. Then run ##pip install .## or ##python setup.py## install to install the fts package. After that, the user may run the scripts in src/cli the same way he/she uses the FTS commands. 70 70 71 71 === Valid OIDC token === 72 72 ... ... @@ -75,21 +75,34 @@ 75 75 For CLI users, Device Code Flow ([[RFC 8628>>url:https://www.rfc-editor.org/rfc/rfc8628.html]]) is the recommended authorisation grant to obtain OIDC tokens. The user can install [[oidc-agent>>url:https://indigo-dc.gitbook.io/oidc-agent/]] to perform such workflows. To use oidc-agent with Fenix AAI, a development version 4.4.1 is needed; such packages can be found here: [[https:~~/~~/repo.data.kit.edu/devel/>>url:https://repo.data.kit.edu/devel/]]. 76 76 77 77 1. Add the following line to /etc/oidc-agent/pubclients.config (for the first time): 78 +##APP-EF25A717-2059-461D-8275-A71CC098B82B@https:~/~/central-proxy.fenix-ri.eu## 79 +1. Initiate oidc-agent: 80 +##[promt]$ eval $(oidc-agent)## 81 +1. The user needs to run the oidc-gen command to generate account info for Fenix: 82 +##[promt]$ oidc-gen ~-~-pub ~-~-issuer https:~/~/central-proxy.fenix-ri.eu \ 83 +~-~-configuration-endpoint https:~/~/central-webapp.fenix-ri.eu/oidc/.well-known/openid-configuration \ 84 +~-~-flow device ~-~-no-save <define_an_account_name>## 85 +Follow the instructions of the output, login to Fenix AAI. 86 +1. Obtain an OIDC token: 87 +##[promt]$ oidc-token <defined_account_name>## 78 78 79 - {{{APP-EF25A717-2059-461D-8275-A71CC098B82B@https://central-proxy.fenix-ri.eu}}}89 +== How to perform a data transfer with FTS == 80 80 81 - 1.Initiate oidc-agent:91 +In this section, the user can find useful commands and examples. More detailed commands are available at [[FTS3 Documentation - CERN>>url:https://fts3-docs.web.cern.ch/fts3-docs/]]. 82 82 83 - [promt]$eval$(oidc-agent)93 +=== FTS commands === 84 84 85 - 1.Theuser needstorunthe oidc-gen commandto generate account info for Fenix:95 +The FTS transfers are managed by the following commands: 86 86 87 -[promt]$ oidc-gen ~-~-pub ~-~-issuer https:~/~/central-proxy.fenix-ri.eu \ ~-~-configuration-endpoint https:~/~/central-webapp.fenix-ri.eu/oidc/.well-known/openid-configuration \ 97 +* fts-rest-transfer-submit //to submit transfer job// 98 +* fts-rest-transfer-status //to query a transfer job// 99 +* fts-rest-transfer-list //to list transfer jobs// 100 +* fts-rest-transfer-cancel //to cancel a transfer job// 88 88 89 - ~-~-flowdevice~-~-no-save <define_an_account_name>102 +=== Source/destination URL format === 90 90 91 -Fo llowthe instructions of the output,login toFenixAAI.104 +FTS is a protocol-specific transfer service. If the user wants to transfer from/to a Swift endpoint, she/he should specify ##swifts:~/~/ + storage name## instead of ##https:~/~/ + storage name## in the source/destination URL. The ##swifts:~/~/## is to specify she/he is using Swift protocol with FTS. 92 92 93 - 1.Obtain anOIDCtoken:106 +The format of a Swift url to submit to FTS is ##swifts:~/~/ + storage name + container name + (folder name(s)) + object name##. 94 94 95 - [promt]$oidc-token<defined_account_name>108 +For example, ##swifts:~/~/swift.bsc.es/testcontainer/data.txt## locates the data.txt file in container testcontainer in the BSC object store.