Changes for page User Guide: Swift Transfer with FTS

Last modified by luehrs on 2023/04/12 09:26

From 3.4 to 3.3 From 6.1 to 5.1

From version 5.1

edited by luehrs
on 2023/04/12 07:16

Change comment: Uploaded new attachment "image-20230412091641-1.png", version {1}

To version 3.4

edited by luehrs
on 2023/04/12 07:11

Change comment: There is no comment for this version

Raw
Rendered

Summary

Page properties (1 modified, 0 added, 0 removed)
Attachments (0 modified, 0 added, 1 removed)
- image-20230412091641-1.png

Details

Page properties

Content

@@ -64,9 +64,9 @@
  To install the RPM package one needs to run the following command:
--##[promt]$ yum install fts-rest-client-1.0.0-%{_release}.el7.noarch##
++{{{[promt]$ yum install fts-rest-client-1.0.0-%{_release}.el7.noarch}}}
--Should the user prefer to build the fts-rest-client him/herself, the scripts can be found in the fts-rest-flask repository here: [[https:~~/~~/gitlab.cern.ch/fts/fts-rest-flask/-/tree/swift>>url:https://gitlab.cern.ch/fts/fts-rest-flask/-/tree/swift]]. The user can clone the repository, switch to the swift branch, and ##cd## into it. We advise creating a ##venv## for this project. Then run ##pip install .## or ##python setup.py## install to install the fts package. After that, the user may run the scripts in src/cli the same way he/she uses the FTS commands.
++Should the user prefer to build the fts-rest-client him/herself, the scripts can be found in the fts-rest-flask repository here: [[https:~~/~~/gitlab.cern.ch/fts/fts-rest-flask/-/tree/swift>>url:https://gitlab.cern.ch/fts/fts-rest-flask/-/tree/swift]]. The user can clone the repository, switch to the swift branch, and cd into it. We advise creating a venv for this project. Then run pip install . or python setup.py install to install the fts package. After that, the user may run the scripts in src/cli the same way he/she uses the FTS commands.
  === Valid OIDC token ===
@@ -75,34 +75,21 @@
  For CLI users, Device Code Flow ([[RFC 8628>>url:https://www.rfc-editor.org/rfc/rfc8628.html]]) is the recommended authorisation grant to obtain OIDC tokens. The user can install [[oidc-agent>>url:https://indigo-dc.gitbook.io/oidc-agent/]] to perform such workflows. To use oidc-agent with Fenix AAI, a development version 4.4.1 is needed; such packages can be found here: [[https:~~/~~/repo.data.kit.edu/devel/>>url:https://repo.data.kit.edu/devel/]].
 . Add the following line to /etc/oidc-agent/pubclients.config (for the first time):
--##APP-EF25A717-2059-461D-8275-A71CC098B82B@https:~/~/central-proxy.fenix-ri.eu##
--1. Initiate oidc-agent:
--##[promt]$ eval $(oidc-agent)##
--1. The user needs to run the oidc-gen command to generate account info for Fenix:
--##[promt]$ oidc-gen ~-~-pub ~-~-issuer https:~/~/central-proxy.fenix-ri.eu \
--~-~-configuration-endpoint https:~/~/central-webapp.fenix-ri.eu/oidc/.well-known/openid-configuration \
--~-~-flow device ~-~-no-save <define_an_account_name>##
--Follow the instructions of the output, login to Fenix AAI.
--1. Obtain an OIDC token:
--##[promt]$ oidc-token <defined_account_name>##
--== How to perform a data transfer with FTS ==
++{{{APP-EF25A717-2059-461D-8275-A71CC098B82B@https://central-proxy.fenix-ri.eu}}}
--In this section, the user can find useful commands and examples. More detailed commands are available at [[FTS3 Documentation - CERN>>url:https://fts3-docs.web.cern.ch/fts3-docs/]].
++1. Initiate oidc-agent:
--=== FTS commands ===
++[promt]$ eval $(oidc-agent)
--The FTS transfers are managed by the following commands:
++1. The user needs to run the oidc-gen command to generate account info for Fenix:
--* fts-rest-transfer-submit             //to submit transfer job//
--* fts-rest-transfer-status              //to query a transfer job//
--* fts-rest-transfer-list                   //to list transfer jobs//
--* fts-rest-transfer-cancel             //to cancel a transfer job//
++[promt]$ oidc-gen ~-~-pub ~-~-issuer https:~/~/central-proxy.fenix-ri.eu \ ~-~-configuration-endpoint https:~/~/central-webapp.fenix-ri.eu/oidc/.well-known/openid-configuration \
--=== Source/destination URL format ===
++~-~-flow device ~-~-no-save <define_an_account_name>
--FTS is a protocol-specific transfer service. If the user wants to transfer from/to a Swift endpoint, she/he should specify ##swifts:~/~/ + storage name## instead of ##https:~/~/ + storage name## in the source/destination URL. The ##swifts:~/~/## is to specify she/he is using Swift protocol with FTS.
++Follow the instructions of the output, login to Fenix AAI.
--The format of a Swift url to submit to FTS is ##swifts:~/~/ + storage name + container name + (folder name(s)) + object name##.
++1. Obtain an OIDC token:
--For example, ##swifts:~/~/swift.bsc.es/testcontainer/data.txt## locates the data.txt file in container testcontainer in the BSC object store.
++[promt]$ oidc-token <defined_account_name>