Move data to/from Jetstream's object storage using Globus

Jetstream offers object storage that can be accessed via swift and S3 APIs. Object storage on Jetstream is more flexible than the virtual disks used by Jetstream VMs and attached volumes.

This guide explains how you can transfer research data into (or out of) a bucket in Jetstream’s object storage using Globus. While the data is in a bucket, you can access it using the Swift or S3 APIs or any application that’s been adapted to use object storage.

 

Prerequisites

  1. You must have permission to access Jetstream’s object storage. The usual way to gain this permission is to request and receive a storage allocation on Jetstream.

  2. You must have permission to use Jetstream’s OpenStack APIs. Assuming you already have a Jetstream allocation, you can request API access.

  3. You must create one or more buckets in Jetstream’s object storage before you can access them via Globus in Step 3. You can use the OpenStack Horizon portal or the OpenStack CLI to do this. ( Remember, Horizon calls buckets “containers.”) Your buckets can be private: you’ll provide credentials for accessing your buckets during Step 2.

 

Step 1 - Create a Jetstream VM with a floating IP address

Step 2 - Create an S3 storage gateway

Step 3 - Access your Globus s3 bucket

Step 4 - Shelve your VM until you need it again.