Data Transfer with Globus

All RCC storage systems are configured as Globus endpoints to facilitate the transfer of data. Globus can be used in a number of different ways:

  • Transfer large amounts of data between RCC storage systems (e.g. Archival to HPC/GPFS)
  • Transfer large amounts of data from an RCC storage system to a personal computer with Globus Connect Personal
  • Transfer large amounts of data between RCC storage systems and storage systems at other institutions
  • Publish and curate data sets with Globus Data Publication
  • Set up automated transfers with the Globus CLI utility or your own application with the Globus APIs As an alternative to SFTP, Globus is a more fully-featured and reliable tool for transferring large datasets.

Setting up a Globus Account and Endpoint

Before you can transfer data to or from an RCC storage system, you must sign up for a Globus account. A Globus Connect Personal endpoint is necessary to transfer data to and from your personal computer.

  1. Visit http://globus.org and sign up for an account. Then sign in.
  2. Browse to Endpoints on the left menu bar.
  3. Select Create new endpoint from the upper right of the screen.
  4. Select Globus Connect Personal
  5. Follow the steps on screen
  6. Check this will NOT be a high assurance endpoint

Transferring Data

Once you have setup an account and endpoint, you can transfer data anytime.

  1. With the Globus Client running, log in to http://globus.org and browse to the File Manager tab
  2. Click Start here under Collection and enter the name of your endpoint or find it in the list
  3. Navigate to the path in your file tree that you wish to transfer files to or from
  4. Select the file or folder you wish to transfer
  5. In the hamburger menu (three horizontal lines) select Transfer or Sync to...
  6. Select the second endpoint and choose the path you want to transfer files to or from
  7. Press the Start button to initiate the transfer
  8. Your files will now be queued for transfer. You can monitor the progress of your transfer by viewing the View Transfer Activity Page on the Globus website.

RCC Endpoints

All RCC storage systems are available as Globus endpoints. You may find these endpoints by searching for endpoints beginning with fsurcc# or by entering their full names:

Globus CLI

Globus provides a cross-platform CLI utility for initiating transfers and managing endpoints from the command line or via automated scripts. See the Globus CLI documentation for installation and usage instructions.

Globus APIs

Globus provides both HTTP and Python APIs for integrating Globus tasks into your own applications. See the Globus APIs documentation for more information.

Getting Help

If you still need help with Globus, please refer to the Globus documentation or submit a support ticket to the RCC staff.