Atmosphere Command Line Interface

"atmo", a python-based command line interface for ATMOSPHERE based on CLIFF (the same package underlying the "openstack" commands) is available.

Documentation and Installation:

Documentation for the package, including installation instructions, are at: https://atmosphere-cli.readthedocs.io

We encourage users to install atmo locally for each user:

1 pip install --user atmosphere-cli

 Make sure you install the package named atmosphere-cli

JETSTREAM SPECIFIC SETUP:

add the following to your .bashrc or a file called .env in your $HOME directory :

1 2 export ATMO_BASE_URL="https://use.jetstream-cloud.org" export ATMO_AUTH_TOKEN="fff28djdFAKETOKENfff28djdFAKETOKENfff28djd"

TOKEN:

To get a Personal Access Token:

  • Launch a browser and login: use.jetstream-cloud.org

  • In the top right, you'll see your username with a drop-down menu: 

  • Click on "Settings" 

  • Click on "Show More" under Advanced.

  • Scroll down to "Persona Access Tokens" and Click on the "+" button

  • Give your token a name and click on "Create Token" 

  • Save this Token by clicking on the "Copy" button and pasting it somewhere you can recover it. 


    THIS IS YOUR ONLY CHANCE TO SAVE THIS TOKEN.
    YOU WILL HAVE TO DELETE THE TOKEN AND START OVER IF YOU DO NOT SAVE IT.

  • You will likely wish to save this token into your .env in the ATMO_AUTH_TOKEN line as above.

ATMO USAGE:

USAGE
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 (atmosphere-cli-<STRING>) prompt> atmo --help usage: atmo [--version] [-v | -q] [--log-file LOG_FILE] [-h] [--debug] [--atmo-base-url <atmosphere-base-url>] [--atmo-auth-token <atmosphere-auth-token>] [--atmo-api-server-timeout <atmosphere-api-server-timeout>] Atmosphere CLI optional arguments: --version show program's version number and exit -v, --verbose Increase verbosity of output. Can be repeated. -q, --quiet Suppress output except warnings and errors. --log-file LOG_FILE Specify a file to log output. Disabled by default. -h, --help Show help message and exit. --debug Show tracebacks on errors. --atmo-base-url <atmosphere-base-url> Base URL for the Atmosphere API (Env: ATMO_BASE_URL) --atmo-auth-token <atmosphere-auth-token> Token used to authenticate with the Atmosphere API (Env: ATMO_AUTH_TOKEN) --atmo-api-server-timeout <atmosphere-api-server-timeout> Server timeout (in seconds) when accessing Atmosphere API (Env: ATMO_API_SERVER_TIMEOUT) Commands: allocation source list List allocation sources for a user. allocation source show Show details for an allocation source. complete print bash completion command (cliff) group list List groups for a user. group show Show details for a group. help print detailed help for another command (cliff) identity list List user identities managed by Atmosphere. identity show Show details for a user identity. image list List images for user. image search Search images for user. image show Show details for an image. image version list List image versions for an image. image version show Show details for an image version. instance actions Show available actions for an instance. instance attach Attach a volume to an instance. instance create Create an instance. instance delete Delete an instance. instance detach Detach a volume from an instance. instance history List history for instance. instance list List instances for user. instance reboot Reboot an instance. instance redeploy Redeploy to an instance. instance resume Resume an instance. instance shelve Shelve an instance. instance show Show details for an instance. instance start Start an instance. instance stop Stop an instance. instance suspend Suspend an instance. instance unshelve Unshelve an instance. maintenance record list List maintenance records for Atmosphere. maintenance record show Show details for a maintenance record. project create Create a project. project list List projects for a user. project show Show details for a project. provider list List cloud providers managed by Atmosphere. provider show Show details for a cloud provider. size list List sizes (instance configurations) for cloud provider. size show Show details for a size (instance configuration). version Show Atmosphere API version. volume create Create a volume. volume delete Delete a volume. volume list List volumes for a user. volume show Show details for a volume. (atmosphere-cli-<STRING>) prompt> atmo provider list +----+--------------------------------------+--------------------------------+-----------+----------------+--------+--------+------------+ | id | uuid | name | type | virtualization | public | active | start_date | +----+--------------------------------------+--------------------------------+-----------+----------------+--------+--------+------------+ | 4 | f906a5ee-34a8-499a-9185-a35feb3d6f01 | Jetstream - Indiana University | OpenStack | KVM | True | True | 2016-01-28 | | 5 | 3ff65aa8-505b-48c3-aef1-aa0ada14c756 | Jetstream - TACC | OpenStack | KVM | True | True | 2016-02-16 | +----+--------------------------------------+--------------------------------+-----------+----------------+--------+--------+------------+



Launching an instance with ATMO

To launch an instance you'll need to know the UUIDs for:

  1. cloud provider (identity)

  2. flavor-size

  3. image source on that cloud

  4. project

  5. XSEDE allocation.

If we wanted to launch an "m1.small" of "Ubuntu 18.04 Devel and Docker" on the Indiana cloud with "my-U18-Indiana" we will need to do:

1: Get UUID of cloud provider (aka IDENTITY UUID)

IDENTITY UUID
1 2 3 4 5 6 7 8 9 10 (atmosphere-cli-<STRING>) prompt> atmo identity list +--------------------------------------+--------+--------------------------------+-----------+--------------+---------------+ | uuid | name | provider | quota_cpu | quota_memory | quota_storage | +--------------------------------------+--------+--------------------------------+-----------+--------------+---------------+ | 88dea875-a8d6-483a-bdb3-4446651e4a71 | beckbw | Jetstream - Indiana University | 132 | 360 | 500 | | 08653592-af99-4f47-8c6d-3709a6deaf02 | beckbw | Jetstream - TACC | 132 | 360 | 500 | +--------------------------------------+--------+--------------------------------+-----------+--------------+---------------+ >>>>> REMEMBER IDENTITY-UUID (e.g. 88dea875-a8d6-483a-bdb3-4446651e4a71 for Indiana )

2: Get UUID and SIZE-ALIAS of flavor on chosen cloud provider

FLAVOR UUID
1 2 3 4 5 6 7 8 9 10 11 12 13 14 (atmosphere-cli-<STRING>) prompt> atmo size list +--------------------------------------+------------+-------+--------------------------------+-----+--------+------+--------+------------+ | uuid | name | alias | provider | cpu | memory | disk | active | start_date | +--------------------------------------+------------+-------+--------------------------------+-----+--------+------+--------+------------+ ... | 4912d26c-db76-4095-a007-25bdb8e2def8 | m1.small | 2 | Jetstream - TACC | 2 | 4096 | 20 | True | 2016-03-08 | | 4612070a-791b-4874-88bb-e15636805ec7 | m1.small | 2 | Jetstream - Indiana University | 2 | 4096 | 20 | True | 2016-01-29 | | f663b727-786f-47f5-9122-9b1b1c4f3529 | m1.medium | 3 | Jetstream - TACC | 6 | 16384 | 60 | True | 2016-03-08 | | d3aab647-d81a-4842-9cdc-3a9fcbb5c588 | m1.medium | 3 | Jetstream - Indiana University | 6 | 16384 | 60 | True | 2016-01-29 | ... +--------------------------------------+------------+-------+--------------------------------+-----+--------+------+--------+------------+ >>>>> REMEMBER SIZE-UUID (e.g. 4612070a-791b-4874-88bb-e15636805ec7 for m1.small on Indiana ) >>>>> REMEMBER SIZE-ALIAS (e.g. 2 for m1.small on Indiana )

3a: Get UUID of image

IMAGE UUID
1 2 3 4 5 6 7 8 9 10 11 12 13 (atmosphere-cli-<STRING>) prompt> atmo image list --tag-name "Featured" +--------------------------------------+----------------------------------------+------------+-----------+------------+ | uuid | name | created_by | is_public | start_date | +--------------------------------------+----------------------------------------+------------+-----------+------------+ ... | c45c94f9-64d1-5e49-8942-0440718ef1f2 | Matlab Minimal | jfischer | True | 2018-10-26 | | 4afad299-3b41-5133-8326-10900b7aa6e7 | Ubuntu 18.04 Devel and Docker | jfischer | True | 2018-10-25 | | 977e1233-54a4-51f5-97a9-560efd53b975 | Centos 7 (7.5) Development GUI | jfischer | True | 2018-10-25 | | c97c4d5e-fe15-5156-b519-0cdb4021492b | Ubuntu 16.04 Devel and Docker | jfischer | True | 2018-10-25 | ... +--------------------------------------+----------------------------------------+------------+-----------+------------+ >>>>> REMEMBER IMAGE-UUID (e.g. 4afad299-3b41-5133-8326-10900b7aa6e7 for Ubuntu 18.04 Devel and Docker )

3b: Get UUID of the desired VERSION of the image

VERSION UUID
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 (atmosphere-cli-<STRING>) prompt> atmo image show 4afad299-3b41-5133-8326-10900b7aa6e7 (IMAGE-UUID from above) +-------------+------------------------------------------------------------------------------------------------+ | Field | Value | +-------------+------------------------------------------------------------------------------------------------+ | id | 717 | | uuid | 4afad299-3b41-5133-8326-10900b7aa6e7 | | name | Ubuntu 18.04 Devel and Docker | | description | Ubuntu 18.04 LTS Development + GUI support + Docker | | | | | | Based on Ubuntu cloud image for 18.04 LTS with basic dev tools, GUI/Xfce added | | | | | | Installation size ~ 4.4 GB | | created_by | jfischer | | versions | 1.0 (fe8deab2-e72f-40a5-aef4-5b908e3d5a6f) | | | 1.8 (dfe3c474-0711-4ca6-bef9-5b61c5a7267b) | | | 1.10 (3edba696-b13b-4661-ab80-3d1a74a2d34e) | | | 1.11 (309300be-2dc8-45b8-890e-7623b272efec) | | | 1.12 (9c9e9963-84e7-4340-812c-e91304c2fe41) | | | 1.13 (1df21c86-c940-41d8-88d6-e231fb0d401f) | | tags | desktop, base, vnc, development, Ubuntu, Featured, docker, docker-compose | | url | https://use.jetstream-cloud.org/api/v2/images/4afad299-3b41-5133-8326-10900b7aa6e7?format=json | | is_public | True | | start_date | 2018-10-25 | | end_date | | +-------------+------------------------------------------------------------------------------------------------+ >>>>> REMEMBER VERSION-UUID (e.g. 1df21c86-c940-41d8-88d6-e231fb0d401f for Ubuntu 18.04 Devel and Docker Version 1.13 )

3c: Get UUID of that VERSION specific to the chosen cloud (aka. MACHINE UUID)

LAUNCHING INSTANCE
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 (atmosphere-cli-<STRING>) prompt> atmo image version show 1df21c86-c940-41d8-88d6-e231fb0d401f (VERSION-UUID from above) +-------------------+--------------------------------------------------------------------------------+ | Field | Value | +-------------------+--------------------------------------------------------------------------------+ | id | 1df21c86-c940-41d8-88d6-e231fb0d401f | | name | 1.13 | | image_name | Ubuntu 18.04 Devel and Docker | | image_description | Ubuntu 18.04 LTS Development + GUI support + Docker | | | | | | Based on Ubuntu cloud image for 18.04 LTS with basic dev tools, GUI/Xfce added | | | | | | Installation size ~ 4.4 GB | | created_by | jfischer | | change_log | v1.13 - patched up to 10-25-18 | | machines | Jetstream - Indiana University (8567d635-ab74-4013-9a21-30821b9a7301) | | | Jetstream - TACC (8567d635-ab74-4013-9a21-30821b9a7301) | | allow_imaging | True | | min_mem | None | | min_cpu | None | | start_date | 2018-10-25T14:55:21.136101Z | +-------------------+--------------------------------------------------------------------------------+ >>>>> REMEMBER MACHINE-UUID (e.g. 8567d635-ab74-4013-9a21-30821b9a7301 for Ubuntu 18.04 version 1.13 on Indiana )

4: Get UUID of the PROJECT to which you will assign the instance

PROJECT UUID
1 2 3 4 5 6 7 8 9 10 (atmosphere-cli-<STRING>) prompt> atmo project list +--------------------------------------+-----------+--------+------------+------------+--------+-----------+---------+-------+ | uuid | name | owner | created_by | start_date | images | instances | volumes | links | +--------------------------------------+-----------+--------+------------+------------+--------+-----------+---------+-------+ ... | a9d28bb7-44d6-4914-8d1a-64673d12ebf2 | testing | beckbw | beckbw | 2016-02-10 | 0 | 7 | 3 | 1 | ... +--------------------------------------+-----------+--------+------------+------------+--------+-----------+---------+-------+ >>>>> REMEMBER PROJECT-UUID (e.g. a9d28bb7-44d6-4914-8d1a-64673d12ebf2 for "testing" project )

5: Get UUID of the ALLOCATION against which you the instance will charge service units

ALLOCATION UUID
1 2 3 4 5 6 7 8 9 (atmosphere-cli-<STRING>) prompt> atmo allocation source list +--------------------------------------+--------------+-----------------+--------------+------------------+------------+ | uuid | name | compute_allowed | compute_used | global_burn_rate | start_date | +--------------------------------------+--------------+-----------------+--------------+------------------+------------+ ... | fd7de822-6398-4bfb-b6dd-47958b21178d | TG-TRA160003 | 5000000 | 3981007.652 | 47.75 | 2017-06-13 | +--------------------------------------+--------------+-----------------+--------------+------------------+------------+ >>>>> REMEMBER ALLOCATION-UUID (e.g. fd7de822-6398-4bfb-b6dd-47958b21178d for "TG-TRA160003" allocation )

6a: CREATE INSTANCE USING MACHINE UUID

CREATE INSTANCE from MACHINE UUID
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 Now we can combine all those to launch an instance using the following syntax: (atmosphere-cli-<STRING>) prompt> atmo instance create --identity <IDENTITY-UUID> --size-alias <SIZE-ALIAS> --source-alias <MACHINE-UUID> --project <PROJECT-UUID> --allocation-source-id <ALLOCATION-UUID> <NAME-OF-INSTANCE> (atmosphere-cli-<STRING>) prompt> atmo instance create --identity 88dea875-a8d6-483a-bdb3-4446651e4a71 --size-alias 2 --source-alias 8567d635-ab74-4013-9a21-30821b9a7301 --project a9d28bb7-44d6-4914-8d1a-64673d12ebf2 --allocation-source-id fd7de822-6398-4bfb-b6dd-47958b21178d "my-U18-Indiana" +-------------------+--------------------------------------+ | Field | Value | +-------------------+--------------------------------------+ | id | 28982 | | uuid | 1611a4c9-b32a-42d9-a575-cadedcbd9813 | | name | my-U18-Indiana | | username | USER | | allocation_source | TG-TRA160003 | | image_id | 717 | | image_version | 1.13 | | launched | 2018-11-29 | | image_size | m1.small | | provider | Jetstream - Indiana University | +-------------------+--------------------------------------+

OR

6b: ALTERNATIVE: CREATE INSTANCE USING ONLY IMAGE UUID (merge steps 3a-c)

CREATE INSTANCE from IMAGE UUID
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 You can also use the "--image" option to combine the "image list", "image show", "image version show", and "--source-alias" steps into one step that automatically chooses the latest version and machine appropriate to the cloud identity you have specified: (atmosphere-cli-<STRING>) prompt> atmo instance create --identity <IDENTITY-UUID> --size-alias <SIZE-ALIAS> --image <IMAGE-UUID> --project <PROJECT-UUID> --allocation-source-id <ALLOCATION-UUID> <NAME-OF-INSTANCE> (atmosphere-cli-<STRING>) prompt> atmo instance create --identity 88dea875-a8d6-483a-bdb3-4446651e4a71 --size-alias 2 --project a9d28bb7-44d6-4914-8d1a-64673d12ebf2 --allocation-source-id fd7de822-6398-4bfb-b6dd-47958b21178d --image 4afad299-3b41-5133-8326-10900b7aa6e7 "my-newVM" +-------------------+--------------------------------------+ | Field | Value | +-------------------+--------------------------------------+ | id | 29132 | | uuid | 9165de94-0a3f-4245-9193-733f530ef0b2 | | name | my-newVM | | username | USER | | allocation_source | TG-TRA160003 | | image_id | 717 | | image_version | 1.14 | | launched | 2018-12-05 | | image_size | m1.small | | provider | Jetstream - Indiana University | +-------------------+--------------------------------------+



OPTIONAL COMMAND LINE AUTO-COMPLETION:

auto-completion functions as in bash and allows you to use <tab> to fill in missing information



OPTIONAL COMMAND LINE AUTO-COMPLETION
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 # <OPTIONAL> Install command line completion. Allows for tab-based auto-completion (atmosphere-cli-<STRING>) prompt> atmo complete >> ~/.bash_aliases (atmosphere-cli-<STRING>) prompt> . ~/.bash_aliases (atmosphere-cli-<STRING>) prompt> atmo <tab> allocation complete help identity image instance project provider size version volume (atmosphere-cli-yMCrgFr7) prompt> atmo instance <tab> actions attach create delete detach history list reboot redeploy resume shelve show start stop suspend unshelve (atmosphere-cli-yMCrgFr7) prompt> atmo provider <tab> list show (atmosphere-cli-yMCrgFr7) prompt> atmo provider list +----+--------------------------------------+--------------------------------+-----------+----------------+--------+--------+------------+ | id | uuid | name | type | virtualization | public | active | start_date | +----+--------------------------------------+--------------------------------+-----------+----------------+--------+--------+------------+ | 4 | f906a5ee-34a8-499a-9185-a35feb3d6f01 | Jetstream - Indiana University | OpenStack | KVM | True | True | 2016-01-28 | | 5 | 3ff65aa8-505b-48c3-aef1-aa0ada14c756 | Jetstream - TACC | OpenStack | KVM | True | True | 2016-02-16 | +----+--------------------------------------+--------------------------------+-----------+----------------+--------+--------+------------+





Troubleshooting:

Error message or problem

Solution

Error message or problem

Solution



1 2 (atmosphere-cli-<string>) prompt:~$ atmo identity list ERROR: {'detail': 'Authentication credentials were not provided.'}



You likely have the WRONG/INVALID  ATMO_AUTH_TOKEN string.

Obtain a new Personal Access Token from the Atmosphere GUI as described above and reset your ATMO_AUTH_TOKEN environment variable.

I've changed/started/deleted an instance but
atmo instance list
doesn't reflect the changes.

Go to the GUI and do a force-refresh (⌘-R). Refreshing the GUI refreshes the API. Currently there is no force-refresh for the CLI.





1 2 ERROR: {'detail': '<DATE> - v32 deployment: Atmosphere is down for a Scheduled Maintenance, Today between 9am - 4pm MST.'}



Probably self-explanatory, right? 



Useful atmo-cli based scripts:

These are provided as-is and are not formally supported.

script

Solution

script

Solution



atmo_launch_instance.sh

uses atmo commands to find all needed UUID to launch an instance.



1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 (atmosphere-cli-yMCrgFr7) prompt> ./atmo_launch_instance.sh -h Usage: atmo_launch [+/-d] [ - :DEFAULT, NODEBUG | +:DEBUG] -c <provider cloud; TACC|Indiana; DEFAULT=Indiana> -s <flavor; DEFAULT="m1.small"> -i <image search string; DEFAULT="Ubuntu 16.04"> -v <image version string; DEFAULT=<latest> > -p <project name; DEFAULT="testing"> -a <allocation; DEFAULT="TG-TRA160003"> -n <provider cloud; TACC|Indiana; DEFAULT=Indiana> [-h] [this usage message] (atmosphere-cli-yMCrgFr7) prompt> ./atmo_launch_instance.sh +d -v 1.13 -c Indiana -n "bootscoot-IU" Searching... idval 88dea875-a8d6-483a-bdb3-4446651e4a71 szval 2 imgval c97c4d5e-fe15-5156-b519-0cdb4021492b verval 04f6c686-6b8d-4be9-b277-16bf2a4a16e6 machstr 7505ea37-2fbb-499f-b150-c18fade5ce26 machval 7505ea37-2fbb-499f-b150-c18fade5ce26 srcval 7505ea37-2fbb-499f-b150-c18fade5ce26 prjval a9d28bb7-44d6-4914-8d1a-64673d12ebf2 allocval fd7de822-6398-4bfb-b6dd-47958b21178d nameval bootscoot-IU atmo instance create --identity 88dea875-a8d6-483a-bdb3-4446651e4a71 --size-alias 2 --source-alias 7505ea37-2fbb-499f-b150-c18fade5ce26 --project a9d28bb7-44d6-4914-8d1a-64673d12ebf2 --allocation-source-id fd7de822-6398-4bfb-b6dd-47958b21178d bootscoot-IU +-------------------+--------------------------------------+ | Field | Value | +-------------------+--------------------------------------+ | id | 22134 | | uuid | 9c1b79ce-9fcc-443d-978d-2536f8833d94 | | name | bootscoot-IU | | username | beckbw | | allocation_source | TG-TRA160003 | | image_id | 107 | | image_version | 1.13 | | launched | 2018-04-18 | | image_size | m1.small | | provider | Jetstream - Indiana University | +-------------------+--------------------------------------+



instance_select.sh

select from menu of your current instances

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 prompt> atmo instance show $(./instance_select_dev.sh |grep ^[0-9]| tail -n 1 | awk '{print $3}') 1) BECK-C74-IU 4) BECK-C74-TACC 7) beck-C74-IU 2) BECK-U16-TACC 5) beckbri-U16-TACC 8) Quit 3) BECK-U16-IU 6) bootscoot-IU Select:3 Select:5 Select:7 Select:8 +-------------------+--------------------------------------+ | Field | Value | +-------------------+--------------------------------------+ | id | 22135 | | uuid | db4a937b-9ef2-47fc-9cce-34f6476087dd | | name | beck-C74-IU | | username | beckbw | | identity | Username: beckbw, Project:beckbw | | project | testing | | allocation_source | TG-TRA160003 | | compute_allowed | 5000000 | | compute_used | 1096774.11 | | global_burn_rate | 24.0 | | user_compute_used | 2887.29 | | user_burn_rate | 2.0 | | image_id | 241 | | image_version | 1.22 | | image_usage | 1096774.11 | | launched | 2018-04-18 | | image_size | m1.small | | image_cpu | 2 | | image_mem | 4096 | | image_disk | 20 | | status | active | | activity | | | ip_address | 149.165.157.11 | | provider | Jetstream - Indiana University | | web_desktop | True | | shell | False | | vnc | True | +-------------------+--------------------------------------+ prompt>