In this article, I will show you how to setup Anthos on Google Cloud. Anthos setup requires that you have the following prerequisites in place:
- Google project with billing enabled
- Anthos APIs are enabled. It will allow you to use Anthos features
- The command line tool – gcloud. You will get this by installing Cloud SDK
As part of installation, you will setup the following:
- Setup a GKE Cluster
- Register the cluster for Anthos
- Setup Anthos Service Mesh (ASM)
- Setup Anthos Config Management (ACM)
Setting up a GKE Cluster
Creating Service Account
As a first step, you will create a service account that will be used by the cluster node VMs.
gcloud iam service-accounts create gke-anthos --project=${PROJECT_ID}
gcloud projects add-iam-policy-binding ${PROJECT_ID} \
--member="serviceAccount:gke-anthos@${PROJECT_ID}.iam.gserviceaccount.com" \
--role="roles/owner"
gcloud iam service-accounts keys create gke-anthos-key.json \
--iam-account=gke-anthos@${PROJECT_ID}.iam.gserviceaccount.com \
--project=${PROJECT_ID}
The above command creates a service account named gke-anthos and is assigned the owner role that grants full admin access to the said account. For our use case here, we are simplifying the access privilege by granting the owner permission. But in an ideal world, granting owner role is not recommended and the principle of least privilege should be followed when granting permissions to users or service accounts.
The second command will create and download the JSON key for the said service account.
Creating a GKE cluster
You will create a GKE cluster with two 4 vCPU nodes with the machines type as e2-standard-4.
gcloud container clusters create anthos-cluster \
--zone=asia-southeast1-a \
--machine-type=e2-standard-4 \
--num-nodes=2 \
--service-account=terraform-anthos@anthos-book-322415.iam.gserviceaccount.com \
--scopes=https://www.googleapis.com/auth/cloud-platform \
--workload-pool=anthos-book-322415.svc.id.goog
The above command will create the cluster named anthos-cluster in asia-southeast1-a (Singapore) zone that will use a regular release channel of GKE. The cluster has a workload identity enabled that will allow Kubernetes service account to represent the Google Cloud IAM service accounts.
Registering the cluster
The above created GKE cluster must be registered with the project fleet thereby allowing you to view and manage the cluster from the Anthos console dashboard.
gcloud container hub memberships register anthos --gke-cluster=asia-southeast1-a/anthos-cluster --service-account-key-file=
The above command registers the anthos-cluster cluster and creates a membership named anthos. It will install the Connect Agent on the cluster that will enable you to view and manage your cluster from the Anthos dashboard. The Connect Agent will authenticate to Google using the above created service account JSON key. Registering a cluster indicates that the cluster is now in the realm of Anthos ecosystem.
Setting up Anthos Service Mesh (ASM)
This section will demonstrate how to setup ASM. ASM setup requires that you have the following client tools as a prerequisites:
- git
- kpt
- kubectl
- jq
Download and Install
As a first step, you will download the ASM installation script. The script will validate the cluster so that it meets ASM requirements and automates all the steps that are required to install ASM.
curl https://storage.googleapis.com/csm-artifacts/asm/install_asm_1.10 > install_asm
At the time of writing this book, the latest ASM version is 1.10. The above command downloads the latest ASM installation script and redirects the content to the file named install_asm. The install_asm file is then made an executable script file. The executable script file is then used to install ASM
./install_asm \
--project_id PROJECT_ID \
--cluster_name anthos-cluster \
--cluster_location asia-southeast1-a \
--mode install \
--ca mesh_ca \
--output_dir \
--enable_all
The above command installs the ASM with default settings. The Certificate Authority used is Mesh CA. Mesh CA is a Google managed private certificate authority that issues certificates for mutual TLS authentication within the service mesh. The output_dir is where the script downloads the asm packages and Istio command line tool istioctl that can be used to customize the installation and also to debug and diagnose the mesh. The enable_all option enables the required Google APIs needed to install ASM and set IAM permissions.
Enable the Sidecar Proxy
Once the installation is complete, the control plane component istiod will contain a label containing ASM revision tag. This revision tag will be used to enable the automatic injection of sidecar proxy container. The below commands will show the ASM revision tag:
kubectl -n istio-system get pods -l app=istiod --show-labels
Output:
istiod-asm-1104-14-5696bb874-kqh59 1/1 Running 0 16h app=istiod,istio.io/rev=asm-1104-14
istiod-asm-1104-14-5696bb874-qc962 1/1 Running 0 16h app=istiod,istio.io/rev=asm-1104-14
As seen from the above output, the ASM revision tag is asm-1104-14.
ASM provides mesh functionality through the use of sidecar containers or envoy proxies or proxy containers. The sidecar container runs alongside your primary container in the same pod. As part of ASM installation, you have to enable the automatic injection of sidecar proxy. It effectively means you have to label your namespaces with the above ASM revision tag that will spin up proxy container alongside your main container as part of that namespace.
kubectl label namespace <namespace> istio-injection- istio.io/rev=<asm-revision> --overwrite
The above command will ensure that your pods will now have a mesh proxy container alongside its primary container. If the pods are already running, then you have to restart the pods to trigger the injection.
Setting up Anthos Config Management (ACM)
This section will demonstrate how to setup ACM. ACM setup requires that you have the following prerequisites in place:
- Any known source code repo like Git or Google Cloud Source Repository (this will be used by Config Sync to sync with clusters)
- nomos client tool
As part of installation, you will perform the following:
- Setup Config Sync
- Setup Policy Controller
Setting up Config Sync
Config Sync lets you deploy configurations and security policies consistently across multiple Kubernetes clusters and namespaces spanning hybrid and multi-cloud environments. The config files are stored as part of source repository like Git and its current state in the repo is synced with multiple clusters where Config Sync is enabled.
As a first step you will enable ACM APIs
gcloud beta container hub config-management enable
You will then download and apply the Custom Resource Definition (CRD) manifest that will represent the Config Sync operator resource.
gsutil cp gs://config-management-release/released/latest/config-management-operator.yaml config-management-operator.yaml
The above command will download the CRD manifest file. You will then apply the said manifest file.
kubectl apply -f config-management-operator.yaml
You will provide Config Sync operator with the read-only access to the source code repo. The way you do this is by setting up the authentication type. If your repo allows read-only access without any authentication then you do not have to do anything special and just specify ‘none’ as authentication type.
The following authentication types are supported:
- SSH key pair
- cookiefile (only supported by Google Cloud Source Repositories)
- Token based
- Google service account (only supported by Google Cloud Source Repositories)
The authentication type you choose will depend on the kind of source code repo that you have setup. Most repos will support SSH based authentication. It is universal and recommended way of authenticating with a repo. For our setup, you will use SSH based authentication in order for Config Sync operator to access the source code repo.
You will create a SSH key pair.
ssh-keygen -t rsa -b 4096 \ -C "<repo-user-name>" \ -N '' \ -f <path-to-key-file>
The above command creates 4096-bit RSA key. The user name is the one that will be used by Config Sync operator to authenticate with the repo. You will also specify the path where the key pair files will be generated. The key pair will include a private and a public key. Register the public key with your source code repo. Every repo will have its own way of recognising the public key. You can refer to the documentation of your source code repo to understand how to register the public key.
Note: Config Sync does not support SSH key passphrase
You will now create a Kubernetes Secret that will contain your private key. The Secret must be created in the config-management-system namespace and named as git-creds.
kubectl create ns config-management-system && \ kubectl create secret generic git-creds \ --namespace=config-management-system \ --from-file=ssh=<path-to-private-key>
The above command creates the secret named git-creds in the config-management-system namespace. You have to specify the path where your private key is stored.
You will now create a custom resource named ConfigManagement as defined by the above created Config Sync operator CRD and apply it to the cluster. This will allow us to tune or configure the behaviour of the Config Sync.
You can write the following manifest file.
# config-management.yaml apiVersion: configmanagement.gke.io/v1 kind: ConfigManagement metadata: name: config-management spec:
cluster: anthos-cluster
sourceFormat: unstructured git: syncRepo: <repo-url> secretType: ssh
The above configuration enables Config Sync for our cluster. You must specify the URL of the repo and the secret type as SSH as we used the same as our authentication type.
You can apply the above manifest to the cluster.
kubectl apply -f config-management.yaml
You can use the tool called nomos to verify the Config Sync installation.
nomos status
The above command will validate if the Config Sync operator is installed successfully or not. A status of PENDING or SYNCED indicates a successful installation.
Setting up Policy Controller
Policy Controller is a way to audit and enforce the compliance for your cluster through a well defined programmable policies. These policies are also called as guardrails that lay down the rules which guards the configuration of the resource against any changes or updates that may indicate or reflect a security violation.
It is very easy to setup the Policy Controller for your cluster. All you need is to change the above created config-management.yaml file to enable the Policy Controller.
apiVersion: configmanagement.gke.io/v1 kind: ConfigManagement metadata: name: config-management spec: cluster: anthos-cluster
sourceFormat: unstructured
policyController: enabled: true git: syncRepo: <repo-url> secretType: ssh
The above highlighted snippet enables the Policy Controller. Apply the manifest file and the Policy Controller will be installed.
You can also verify the installation.
gcloud beta container hub config-management status \
--project=PROJECT_ID
Output:
Name Status Last_Synced_Token Sync_Branch Last_Synced_Time Policy_Controller
anthos-cluster SYNCED a687c2c 1.0.0 2021-02-17T00:15:55Z INSTALLED
If the output has the value INSTALLED under the column Policy_Controller, it means Policy Controller is successfully enabled.