CyberInfrastructure Training to Advance Climate Scienc (CI-TRACS) - Data Movement Workshop

Presenters

• Luke Nelson - Hawaiʻi Data Science Institute Fellow
• Sean Cleveland - University of Hawaiʻi System - Cyberinfrastructure Research Scientist

Past Presenter and Contributor

• Alan Whinery - University of Hawaiʻi System - Chief Internet Engineer

Synopsis

This workshop will be an introduction to understanding the challenges and options in moving scientific data over the network. In particular, attendees will learn about some of the different network infrastructure and tools available and the use cases to apply them towards and highlight any disadvantages or drawbacks to a particular technology. Lessons will also address what to do if a transfer is experiencing less than expected performance, potential common contributors to a transfer bottleneck and when and who to ask for assistance will also be covered. Different data transfer tools like SFTP, Globus and Rclone for moving data will be covered in hands-on exercises.

Prerequisites

• Basic SSH and Command line experience. We recommend the participants to go through shell-novice if you are new to the command-line
• Have an account on Mana
• Have UH Duo/MFA enabled
• Have a modern web browser

Learning Outcomes:

By the end of this workshop attendees will know how to:

• Demonstrate understanding of advantages and disadvantages of various bulk data transfer tools.
• Apply understanding to decide, based on their current computing, storage and network infrastructure and that of their end-point sites, appropriate solutions for a data transfer workflow.
• Demonstrate the ability to identify, communicate and mitigate potential bottlenecks in collaboration with campus cyberinfrastructure and network operators.
• Transfer data using Globus
• Transfer data using Rclone