Get Started on Google Cloud with CLI
After working with Amazon Web Services for a few years, I decided to take Google Cloud Platform for a spin. In this post we will get set up with Google Cloud Platform (GCP), and use the CLI to interact with it for a very basic use case (launch, delete an instance). I will also refer to AWS counterpart(s) as and when it makes sense. The idea is to conceptualize automated creation and teardown of entire environments using the idea of projects in GCP.
Before we proceed, it is important to understand the concept of Projects in Google Cloud Platform. A project does not really have a direct counterpart in AWS. A project is a collection of resources and services organized to work together. One project is associated with one billing account. Any communication outside of the project boundaries needs to occur via an external network connection.
Setup
-
Sign up for Google Cloud Platform Free Tier.
-
Download and unzip the Google Cloud SDK. While AWS Command Line Interface is called AWS CLI, GCP’s is called Cloud SDK.
-
cd google-cloud-sdk -
./install.shto add the CLI SDK to our path. -
gcloud initto initialize the SDK. This will trigger an OAuth flow, authorizing the SDK to make API calls on our behalf. -
Once authorized, we will se a list of projects to pick from (If there are any existing ones), or to create a new project. Let us create a new project.
You are logged in as: [your_email_at_gmail.com].
Pick cloud project to use:
[1] cp100-166810
[2] manish-test-162406
[3] Create a new project
Please enter numeric choice or text value (must exactly match list
item): 3
Enter a Project ID. Note that a Project ID CANNOT be changed later.
Project IDs must be 6-30 characters (lowercase ASCII, digits, or
hyphens) in length and start with a lowercase letter. project-for-my-blog
Your current project has been set to: [project-for-my-blog].
...
Your Google Cloud SDK is configured and ready to use!
There are several files created in ~/.config/gcloud which contain the config, logs, credentials, etc. Similar to AWS, which uses ~/.aws folder.
Exploring the CLI
The typical syntax of gcloud CLI is gcloud [flags ] <group | command>. This is similar to AWS CLI, where it is aws <command> <subcommand> [<subcommand> ...] [params].
Here is a compare and contrast to show configuration
bash-3.2$ gcloud config configurations list
NAME IS_ACTIVE ACCOUNT PROJECT DEFAULT_ZONE DEFAULT_REGION
default True your_email_at_gmail@gmail.com project-for-my-blog
vs.
bash-3.2$ aws configure list
Name Value Type Location
---- ----- ---- --------
profile <not set> None None
access_key <not set> None None
secret_key <not set> None None
region us-east-1 config-file ~/.aws/config
To list out components installed with the CLI, we can use gcloud components list command. This is very different than AWS CLI.
bash-3.2$ gcloud components list
Your current Cloud SDK version is: 155.0.0
The latest available version is: 155.0.0
┌─────────────────────────────────────────────────────────────────────────────────────────────────────────────┐
│ Components │
├───────────────┬──────────────────────────────────────────────────────┬──────────────────────────┬───────────┤
│ Status │ Name │ ID │ Size │
├───────────────┼──────────────────────────────────────────────────────┼──────────────────────────┼───────────┤
│ Not Installed │ App Engine Go Extensions │ app-engine-go │ 96.6 MiB │
│ Not Installed │ Bigtable Command Line Tool │ cbt │ 3.9 MiB │
│ Not Installed │ Cloud Datalab Command Line Tool │ datalab │ < 1 MiB │
│ Not Installed │ Cloud Datastore Emulator │ cloud-datastore-emulator │ 15.4 MiB │
│ Not Installed │ Cloud Datastore Emulator (Legacy) │ gcd-emulator │ 38.1 MiB │
│ Not Installed │ Cloud Pub/Sub Emulator │ pubsub-emulator │ 21.0 MiB │
│ Not Installed │ Emulator Reverse Proxy │ emulator-reverse-proxy │ 14.5 MiB │
│ Not Installed │ Google Container Registry's Docker credential helper │ docker-credential-gcr │ 2.3 MiB │
│ Not Installed │ gcloud Alpha Commands │ alpha │ < 1 MiB │
│ Not Installed │ gcloud Beta Commands │ beta │ < 1 MiB │
│ Not Installed │ gcloud app Java Extensions │ app-engine-java │ 128.6 MiB │
│ Not Installed │ gcloud app PHP Extensions (Mac OS X) │ app-engine-php-darwin │ 21.9 MiB │
│ Not Installed │ gcloud app Python Extensions │ app-engine-python │ 6.1 MiB │
│ Not Installed │ kubectl │ kubectl │ 14.8 MiB │
│ Installed │ BigQuery Command Line Tool │ bq │ < 1 MiB │
│ Installed │ Cloud SDK Core Libraries │ core │ 6.0 MiB │
│ Installed │ Cloud Storage Command Line Tool │ gsutil │ 2.9 MiB │
│ Installed │ Default set of gcloud commands │ gcloud │ │
└───────────────┴──────────────────────────────────────────────────────┴──────────────────────────┴───────────┘
To install or remove components at your current SDK version [155.0.0], run:
$ gcloud components install COMPONENT_ID
$ gcloud components remove COMPONENT_ID
To update your SDK installation to the latest version [155.0.0], run:
$ gcloud components update
As we can see, we have the basic components pre-installed, and others can be installed/removed at any time.
gcloud Interactive Shell
Just like in AWS we can use the super awesome AWS Shell, for gcloud, the interactive shell is a part of the Alpha component. It can be installed with gcloud components install alpha, and can be run with gcloud alpha shell.
bash-3.2$ gcloud alpha shell
gcloud>
-------------------------------------------------------------------------------
your_email_at_gmail@gmail.com | project-for-my-blog | ctrl-q: Quit | ctrl-t: Help ON
This has the contextual help as well as smart auto-complete/auto-suggest features similar to the aws-shell.
Creating a Project via gcloud CLI
This is where it gets really interesting, specially when compared to AWS. Before the project can be put to any use, it needs to have Cloud APIs enabled, and to do that, it needs to have billing enabled.
Verify the billing account, which has been initialized after authentication we did during gcloud init.
gcloud> alpha billing accounts list
ID NAME OPEN
XXXXXX-XXXXXX-XXXXXX My Billing Account True
Next, we associate, or link our project, project-for-my-blog with this account-id, like so -
gcloud> alpha billing accounts projects link project-for-my-blog --account-id=XXXXXX-XXXXXX-XXXXXX
billingAccountName: billingAccounts/XXXXXX-XXXXXX-XXXXXX
billingEnabled: true
name: projects/project-for-my-blog/billingInfo
projectId: project-for-my-blog
gcloud>
We’re not done yet. We need to associate services to this project. We can list the available services by using list --available command.
gcloud> service-management list --available
NAME TITLE
picker.googleapis.com Google Picker API
bigquery-json.googleapis.com BigQuery API
chromewebstore.googleapis.com Chrome Web Store API
tracing.googleapis.com Google Tracing API
youtube.googleapis.com YouTube Data API v3
youtubeanalytics.googleapis.com YouTube Analytics API
clouderrorreporting.googleapis.com Stackdriver Error Reporting API
...
...
Since we need the compute services enabled, we will enable it by using service-management --enable <service-name>.
gcloud> service-management enable compute-component.googleapis.com
Waiting for async operation operations/projectSettings.544ddc35-4780-414d-a814-XXXXXXXXX to complete...
Operation finished successfully. The following command can describe the Operation details:
gcloud service-management operations describe operations/projectSettings.544ddc35-4780-414d-a814-XXXXXXXX
Now that we got the billing and the service-management sorted out, we can issue compute commands.
gcloud> compute instances list
Listed 0 items.
gcloud>
Creating an Instance
Next, we will create an instance. This is much easier compared to AWS, as gcloud defaults pretty much everything except for the name and zone of the instance.
gcloud> compute instances create first-gcloud-instance --zone us-west1-a
Created [https://www.googleapis.com/compute/v1/projects/project-for-my-blog/zones/us-west1-a/instances/first-gcloud-instance].
NAME ZONE MACHINE_TYPE PREEMPTIBLE INTERNAL_IP EXTERNAL_IP STATUS
first-gcloud-instance us-west1-a n1-standard-1 10.138.0.3 35.xxx.yyy.zz RUNNING
We can verify the default values by issuing a describe command, just like AWS.
gcloud> compute instances describe first-gcloud-instance --zone us-west1-a
canIpForward: false
cpuPlatform: Intel Broadwell
creationTimestamp: '2017-05-16T01:24:55.090-07:00'
disks:
- autoDelete: true
boot: true
deviceName: persistent-disk-0
index: 0
interface: SCSI
kind: compute#attachedDisk
licenses:
- https://www.googleapis.com/compute/v1/projects/debian-cloud/global/licenses/debian-8-jessie
mode: READ_WRITE
source: https://www.googleapis.com/compute/v1/projects/project-for-my-blog/zones/us-west1-a/disks/first-gcloud-instance
type: PERSISTENT
id: '4643304423181191609'
kind: compute#instance
machineType: https://www.googleapis.com/compute/v1/projects/project-for-my-blog/zones/us-west1-a/machineTypes/n1-standard-1
metadata:
fingerprint: CFAAAAAAA=
kind: compute#metadata
name: first-gcloud-instance
networkInterfaces:
- accessConfigs:
- kind: compute#accessConfig
name: external-nat
natIP: 35.xxx.yyy.zz
type: ONE_TO_ONE_NAT
kind: compute#networkInterface
name: nic0
network: https://www.googleapis.com/compute/v1/projects/project-for-my-blog/global/networks/default
networkIP: 10.138.0.3
subnetwork: https://www.googleapis.com/compute/v1/projects/project-for-my-blog/regions/us-west1/subnetworks/default
scheduling:
automaticRestart: true
onHostMaintenance: MIGRATE
preemptible: false
selfLink: https://www.googleapis.com/compute/v1/projects/project-for-my-blog/zones/us-west1-a/instances/first-gcloud-instance
serviceAccounts:
- email: 999999999999-compute@developer.gserviceaccount.com
scopes:
- https://www.googleapis.com/auth/cloud.useraccounts.readonly
- https://www.googleapis.com/auth/devstorage.read_only
- https://www.googleapis.com/auth/logging.write
- https://www.googleapis.com/auth/monitoring.write
- https://www.googleapis.com/auth/pubsub
- https://www.googleapis.com/auth/service.management.readonly
- https://www.googleapis.com/auth/servicecontrol
- https://www.googleapis.com/auth/trace.append
status: RUNNING
tags:
fingerprint: 42AAAAAAAAA=
zone: https://www.googleapis.com/compute/v1/projects/project-for-my-blog/zones/us-west1-a
gcloud>
To summarize, the instance is a Debian8 (Jessie) VM, type n1-standard-1 (1 vCPU, 3.75G RAM), has a Public IP, a 10GB Standard Persistent Disk (HDD).
In AWS terms, the AMI is Debian GNU/Linux 8 (Jessie), instance type is m3.medium, with a 10G HDD Root Volume, no EBS volume, in the Default VPC’s Public Subnet.
We can ssh into this instance. Note that we did not deal with any keypair like we do with AWS. However, as we try to ssh into this instance, gcloud will create a keypair for us.
gcloud> compute ssh first-gcloud-instance --zone us-west1-a
WARNING: The public SSH key file for gcloud does not exist.
WARNING: The private SSH key file for gcloud does not exist.
WARNING: You do not have an SSH key for gcloud.
WARNING: SSH keygen will be executed to generate a key.
Generating public/private rsa key pair.
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /Users/mpandit/.ssh/google_compute_engine.
Your public key has been saved in /Users/mpandit/.ssh/google_compute_engine.pub.
The key fingerprint is:
SHA256:************************************ mpandit@C02STG51GTFM
The key's randomart image is:
+---[RSA 2048]----+
..
..
..
..
+----[SHA256]-----+
Updating project ssh metadata...\Updated [https://www.googleapis.com/compute/v1/projects/project-for-my-blog].
Updating project ssh metadata...done.
Waiting for SSH key to propagate.
Warning: Permanently added 'compute.4643304423181191609' (ECDSA) to the list of known hosts.
The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.
Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
mpandit@first-gcloud-instance:~$
mpandit@first-gcloud-instance:~$ uname -a
Linux first-gcloud-instance 3.16.0-4-amd64 #1 SMP Debian 3.16.39-1+deb8u2 (2017-03-07) x86_64 GNU/Linux
Feel free to play around with the instance. Here is how to install Apache2.
mpandit@first-gcloud-instance:~$ sudo apt-get install apache2
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following extra packages will be installed:
...
Processing triggers for libc-bin (2.19-18+deb8u7) ...
Processing triggers for systemd (215-17+deb8u6) ...
Processing triggers for sgml-base (1.26+nmu4) ...
mpandit@first-gcloud-instance:~$ telnet localhost 80
Trying ::1...
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
However, if we try to use the public IP from our browser, the connection will fail. This is because only ssh (port 22) access is allowed by default.
We can compare this to AWS where the Default VPC Security Group allows all traffic to all ports from 0.0.0.0/0.
Deleting the Instance
Deleting the instance is also straightforward, needing only the name and the zone.
gcloud> compute instances list
NAME ZONE MACHINE_TYPE PREEMPTIBLE INTERNAL_IP EXTERNAL_IP STATUS
first-gcloud-instance us-west1-a n1-standard-1 10.138.0.3 35.xxx.yyy.zz RUNNING
gcloud> compute instances delete my-gcloud-instance --zone us-west1-a
The following instances will be deleted. Attached disks configured to
be auto-deleted will be deleted unless they are attached to any other
instances. Deleting a disk is irreversible and any data on the disk
will be lost.
- [my-gcloud-instance] in [us-west1-a]
Do you want to continue (Y/n)? Y
Deleted [https://www.googleapis.com/compute/v1/projects/project-for-my-blog/zones/us-west1-a/instances/first-gcloud-instance].
gcloud>
Deleting the Project
Finally, we can delete the project. This is not an instantaneous action though - as GCP lets us undelete a project for up to a certain time (30 days?). Once the project is deleted, an email is sent informing the same. Deleting a project will delete all the resources associated with it.
gcloud> projects list
PROJECT_ID NAME PROJECT_NUMBER
project-for-my-blog project-for-my-blog 28XXXXXXXXXXXX
gcloud> projects delete project-for-my-blog
Your project will be deleted.
Do you want to continue (Y/n)? Y
Deleted [https://cloudresourcemanager.googleapis.com/v1/projects/project-for-my-blog].
You can undo this operation for a limited period by running:
$ gcloud projects undelete project-for-my-blog
gcloud>
Conclusion
While this is barely scrarching the surface of GCP, in this post we were able to get a brief introduction to using GCP via command line, and drive a very basic use case in an automated manner with no UX involvement. Please tweet/DM me with your feedback, and what would you like to see next.