After working with Amazon Web Services for a few years, I decided to take Google Cloud Platform for a spin. In this post we will get set up with Google Cloud Platform (GCP), and use the CLI to interact with it for a very basic use case (launch, delete an instance). I will also refer to AWS counterpart(s) as and when it makes sense. The idea is to conceptualize automated creation and teardown of entire environments using the idea of projects in GCP.
Before we proceed, it is important to understand the concept of Projects in Google Cloud Platform. A project does not really have a direct counterpart in AWS. A project is a collection of resources and services organized to work together. One project is associated with one billing account. Any communication outside of the project boundaries needs to occur via an external network connection.
Sign up for Google Cloud Platform Free Tier.
Download and unzip the Google Cloud SDK. While AWS Command Line Interface is called AWS CLI, GCP’s is called Cloud SDK.
./install.shto add the CLI SDK to our path.
gcloud initto initialize the SDK. This will trigger an OAuth flow, authorizing the SDK to make API calls on our behalf.
Once authorized, we will se a list of projects to pick from (If there are any existing ones), or to create a new project. Let us create a new project.
You are logged in as: [your_email_at_gmail.com]. Pick cloud project to use:  cp100-166810  manish-test-162406  Create a new project Please enter numeric choice or text value (must exactly match list item): 3 Enter a Project ID. Note that a Project ID CANNOT be changed later. Project IDs must be 6-30 characters (lowercase ASCII, digits, or hyphens) in length and start with a lowercase letter. project-for-my-blog Your current project has been set to: [project-for-my-blog]. ... Your Google Cloud SDK is configured and ready to use!
There are several files created in
~/.config/gcloud which contain the config, logs, credentials, etc. Similar to AWS, which uses
Exploring the CLI
The typical syntax of gcloud CLI is
gcloud [flags ] <group | command>. This is similar to AWS CLI, where it is
aws <command> <subcommand> [<subcommand> ...] [params].
Here is a compare and contrast to show configuration
bash-3.2$ gcloud config configurations list NAME IS_ACTIVE ACCOUNT PROJECT DEFAULT_ZONE DEFAULT_REGION default True email@example.com project-for-my-blog
bash-3.2$ aws configure list Name Value Type Location ---- ----- ---- -------- profile <not set> None None access_key <not set> None None secret_key <not set> None None region us-east-1 config-file ~/.aws/config
To list out components installed with the CLI, we can use
gcloud components list command. This is very different than AWS CLI.
bash-3.2$ gcloud components list Your current Cloud SDK version is: 155.0.0 The latest available version is: 155.0.0 ┌─────────────────────────────────────────────────────────────────────────────────────────────────────────────┐ │ Components │ ├───────────────┬──────────────────────────────────────────────────────┬──────────────────────────┬───────────┤ │ Status │ Name │ ID │ Size │ ├───────────────┼──────────────────────────────────────────────────────┼──────────────────────────┼───────────┤ │ Not Installed │ App Engine Go Extensions │ app-engine-go │ 96.6 MiB │ │ Not Installed │ Bigtable Command Line Tool │ cbt │ 3.9 MiB │ │ Not Installed │ Cloud Datalab Command Line Tool │ datalab │ < 1 MiB │ │ Not Installed │ Cloud Datastore Emulator │ cloud-datastore-emulator │ 15.4 MiB │ │ Not Installed │ Cloud Datastore Emulator (Legacy) │ gcd-emulator │ 38.1 MiB │ │ Not Installed │ Cloud Pub/Sub Emulator │ pubsub-emulator │ 21.0 MiB │ │ Not Installed │ Emulator Reverse Proxy │ emulator-reverse-proxy │ 14.5 MiB │ │ Not Installed │ Google Container Registry's Docker credential helper │ docker-credential-gcr │ 2.3 MiB │ │ Not Installed │ gcloud Alpha Commands │ alpha │ < 1 MiB │ │ Not Installed │ gcloud Beta Commands │ beta │ < 1 MiB │ │ Not Installed │ gcloud app Java Extensions │ app-engine-java │ 128.6 MiB │ │ Not Installed │ gcloud app PHP Extensions (Mac OS X) │ app-engine-php-darwin │ 21.9 MiB │ │ Not Installed │ gcloud app Python Extensions │ app-engine-python │ 6.1 MiB │ │ Not Installed │ kubectl │ kubectl │ 14.8 MiB │ │ Installed │ BigQuery Command Line Tool │ bq │ < 1 MiB │ │ Installed │ Cloud SDK Core Libraries │ core │ 6.0 MiB │ │ Installed │ Cloud Storage Command Line Tool │ gsutil │ 2.9 MiB │ │ Installed │ Default set of gcloud commands │ gcloud │ │ └───────────────┴──────────────────────────────────────────────────────┴──────────────────────────┴───────────┘ To install or remove components at your current SDK version [155.0.0], run: $ gcloud components install COMPONENT_ID $ gcloud components remove COMPONENT_ID To update your SDK installation to the latest version [155.0.0], run: $ gcloud components update
As we can see, we have the basic components pre-installed, and others can be installed/removed at any time.
gcloud Interactive Shell
Just like in AWS we can use the super awesome AWS Shell, for gcloud, the interactive shell is a part of the
Alpha component. It can be installed with
gcloud components install alpha, and can be run with
gcloud alpha shell.
bash-3.2$ gcloud alpha shell gcloud> ------------------------------------------------------------------------------- firstname.lastname@example.org | project-for-my-blog | ctrl-q: Quit | ctrl-t: Help ON
This has the contextual help as well as smart auto-complete/auto-suggest features similar to the aws-shell.
Creating a Project via gcloud CLI
This is where it gets really interesting, specially when compared to AWS. Before the project can be put to any use, it needs to have Cloud APIs enabled, and to do that, it needs to have billing enabled.
Verify the billing account, which has been initialized after authentication we did during
gcloud> alpha billing accounts list ID NAME OPEN XXXXXX-XXXXXX-XXXXXX My Billing Account True
Next, we associate, or
link our project,
project-for-my-blog with this
account-id, like so -
gcloud> alpha billing accounts projects link project-for-my-blog --account-id=XXXXXX-XXXXXX-XXXXXX billingAccountName: billingAccounts/XXXXXX-XXXXXX-XXXXXX billingEnabled: true name: projects/project-for-my-blog/billingInfo projectId: project-for-my-blog gcloud>
We’re not done yet. We need to associate services to this project. We can list the available services by using
list --available command.
gcloud> service-management list --available NAME TITLE picker.googleapis.com Google Picker API bigquery-json.googleapis.com BigQuery API chromewebstore.googleapis.com Chrome Web Store API tracing.googleapis.com Google Tracing API youtube.googleapis.com YouTube Data API v3 youtubeanalytics.googleapis.com YouTube Analytics API clouderrorreporting.googleapis.com Stackdriver Error Reporting API ... ...
Since we need the compute services enabled, we will enable it by using
service-management --enable <service-name>.
gcloud> service-management enable compute-component.googleapis.com Waiting for async operation operations/projectSettings.544ddc35-4780-414d-a814-XXXXXXXXX to complete... Operation finished successfully. The following command can describe the Operation details: gcloud service-management operations describe operations/projectSettings.544ddc35-4780-414d-a814-XXXXXXXX
Now that we got the billing and the service-management sorted out, we can issue
gcloud> compute instances list Listed 0 items. gcloud>
Creating an Instance
Next, we will create an instance. This is much easier compared to AWS, as gcloud defaults pretty much everything except for the name and zone of the instance.
gcloud> compute instances create first-gcloud-instance --zone us-west1-a Created [https://www.googleapis.com/compute/v1/projects/project-for-my-blog/zones/us-west1-a/instances/first-gcloud-instance]. NAME ZONE MACHINE_TYPE PREEMPTIBLE INTERNAL_IP EXTERNAL_IP STATUS first-gcloud-instance us-west1-a n1-standard-1 10.138.0.3 35.xxx.yyy.zz RUNNING
We can verify the default values by issuing a
describe command, just like AWS.
gcloud> compute instances describe first-gcloud-instance --zone us-west1-a canIpForward: false cpuPlatform: Intel Broadwell creationTimestamp: '2017-05-16T01:24:55.090-07:00' disks: - autoDelete: true boot: true deviceName: persistent-disk-0 index: 0 interface: SCSI kind: compute#attachedDisk licenses: - https://www.googleapis.com/compute/v1/projects/debian-cloud/global/licenses/debian-8-jessie mode: READ_WRITE source: https://www.googleapis.com/compute/v1/projects/project-for-my-blog/zones/us-west1-a/disks/first-gcloud-instance type: PERSISTENT id: '4643304423181191609' kind: compute#instance machineType: https://www.googleapis.com/compute/v1/projects/project-for-my-blog/zones/us-west1-a/machineTypes/n1-standard-1 metadata: fingerprint: CFAAAAAAA= kind: compute#metadata name: first-gcloud-instance networkInterfaces: - accessConfigs: - kind: compute#accessConfig name: external-nat natIP: 35.xxx.yyy.zz type: ONE_TO_ONE_NAT kind: compute#networkInterface name: nic0 network: https://www.googleapis.com/compute/v1/projects/project-for-my-blog/global/networks/default networkIP: 10.138.0.3 subnetwork: https://www.googleapis.com/compute/v1/projects/project-for-my-blog/regions/us-west1/subnetworks/default scheduling: automaticRestart: true onHostMaintenance: MIGRATE preemptible: false selfLink: https://www.googleapis.com/compute/v1/projects/project-for-my-blog/zones/us-west1-a/instances/first-gcloud-instance serviceAccounts: - email: email@example.com scopes: - https://www.googleapis.com/auth/cloud.useraccounts.readonly - https://www.googleapis.com/auth/devstorage.read_only - https://www.googleapis.com/auth/logging.write - https://www.googleapis.com/auth/monitoring.write - https://www.googleapis.com/auth/pubsub - https://www.googleapis.com/auth/service.management.readonly - https://www.googleapis.com/auth/servicecontrol - https://www.googleapis.com/auth/trace.append status: RUNNING tags: fingerprint: 42AAAAAAAAA= zone: https://www.googleapis.com/compute/v1/projects/project-for-my-blog/zones/us-west1-a gcloud>
To summarize, the instance is a Debian8 (Jessie) VM, type n1-standard-1 (1 vCPU, 3.75G RAM), has a Public IP, a 10GB Standard Persistent Disk (HDD).
In AWS terms, the AMI is
Debian GNU/Linux 8 (Jessie), instance type is
m3.medium, with a 10G HDD Root Volume, no EBS volume, in the
ssh into this instance. Note that we did not deal with any
keypair like we do with AWS. However, as we try to
ssh into this instance, gcloud will create a keypair for us.
gcloud> compute ssh first-gcloud-instance --zone us-west1-a WARNING: The public SSH key file for gcloud does not exist. WARNING: The private SSH key file for gcloud does not exist. WARNING: You do not have an SSH key for gcloud. WARNING: SSH keygen will be executed to generate a key. Generating public/private rsa key pair. Enter passphrase (empty for no passphrase): Enter same passphrase again: Your identification has been saved in /Users/mpandit/.ssh/google_compute_engine. Your public key has been saved in /Users/mpandit/.ssh/google_compute_engine.pub. The key fingerprint is: SHA256:************************************ mpandit@C02STG51GTFM The key's randomart image is: +---[RSA 2048]----+ .. .. .. .. +----[SHA256]-----+ Updating project ssh metadata...\Updated [https://www.googleapis.com/compute/v1/projects/project-for-my-blog]. Updating project ssh metadata...done. Waiting for SSH key to propagate. Warning: Permanently added 'compute.4643304423181191609' (ECDSA) to the list of known hosts. The programs included with the Debian GNU/Linux system are free software; the exact distribution terms for each program are described in the individual files in /usr/share/doc/*/copyright. Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent permitted by applicable law. mpandit@first-gcloud-instance:~$ mpandit@first-gcloud-instance:~$ uname -a Linux first-gcloud-instance 3.16.0-4-amd64 #1 SMP Debian 3.16.39-1+deb8u2 (2017-03-07) x86_64 GNU/Linux
Feel free to play around with the instance. Here is how to install Apache2.
mpandit@first-gcloud-instance:~$ sudo apt-get install apache2 Reading package lists... Done Building dependency tree Reading state information... Done The following extra packages will be installed: ... Processing triggers for libc-bin (2.19-18+deb8u7) ... Processing triggers for systemd (215-17+deb8u6) ... Processing triggers for sgml-base (1.26+nmu4) ... mpandit@first-gcloud-instance:~$ telnet localhost 80 Trying ::1... Trying 127.0.0.1... Connected to localhost. Escape character is '^]'.
However, if we try to use the public IP from our browser, the connection will fail. This is because only
ssh (port 22) access is allowed by default.
We can compare this to AWS where the Default VPC Security Group allows all traffic to all ports from
Deleting the Instance
Deleting the instance is also straightforward, needing only the name and the zone.
gcloud> compute instances list NAME ZONE MACHINE_TYPE PREEMPTIBLE INTERNAL_IP EXTERNAL_IP STATUS first-gcloud-instance us-west1-a n1-standard-1 10.138.0.3 35.xxx.yyy.zz RUNNING gcloud> compute instances delete my-gcloud-instance --zone us-west1-a The following instances will be deleted. Attached disks configured to be auto-deleted will be deleted unless they are attached to any other instances. Deleting a disk is irreversible and any data on the disk will be lost. - [my-gcloud-instance] in [us-west1-a] Do you want to continue (Y/n)? Y Deleted [https://www.googleapis.com/compute/v1/projects/project-for-my-blog/zones/us-west1-a/instances/first-gcloud-instance]. gcloud>
Deleting the Project
Finally, we can delete the project. This is not an instantaneous action though - as GCP lets us
undelete a project for up to a certain time (30 days?). Once the project is deleted, an email is sent informing the same. Deleting a project will delete all the resources associated with it.
gcloud> projects list PROJECT_ID NAME PROJECT_NUMBER project-for-my-blog project-for-my-blog 28XXXXXXXXXXXX gcloud> projects delete project-for-my-blog Your project will be deleted. Do you want to continue (Y/n)? Y Deleted [https://cloudresourcemanager.googleapis.com/v1/projects/project-for-my-blog]. You can undo this operation for a limited period by running: $ gcloud projects undelete project-for-my-blog gcloud>
While this is barely scrarching the surface of GCP, in this post we were able to get a brief introduction to using GCP via command line, and drive a very basic use case in an automated manner with no UX involvement. Please tweet/DM me with your feedback, and what would you like to see next.