One of the advanced mesh constructs or topologies are multi mesh or cross mesh designs that aids in inter-cluster communication. With such a topology, each cluster will implement its own mesh runtime federating to form one logical mesh. You may setup mesh control plane in every cluster or you can have all the clusters communicate to one central mesh control plane. Anthos Service Mesh (ASM) supports federated mesh where every cluster will host both mesh control and data plane components.
This article will explain the multi cluster mesh model operating in a single and multiple VPC networks.
Multi cluster service mesh (single VPC network)
In this mesh network topology, two clusters will be part of same VPC and there will be direct connectivity between services in the mesh across clusters. Both the clusters must be setup to trust each other. This can be achieved by exchanging secrets (certificate derived from common root CA) of the other cluster thereby enabling access to Kubernetes (GKE) API servers. In effect, cluster one will be able to access API server and discover endpoints of the second cluster and vice versa.
As both the clusters are part of same VPC, there is no need of routing via ingress gateway. Service enpoints in both the clusters can directly communicate with each other.
Multi cluster service mesh (different VPC networks)
In this mesh network topology, two clusters will be set up in different VPC networks. Services in a mesh cannot communicate directly across clusters but have to make use of gateway to route the east-west traffic. To implement this model, you have to again set up trust between clusters, as discussed in the previous section, to enable endpoints discovery across clusters. You then have to setup an ingress gateway that will allow east-west traffic in both the clusters.
The gateway endpoint will be accessible over public internet but will expose only services with *.local domain. This will ensure services endpoints from both the clusters will be able to communicate via this gateway thereby enabling east-west traffic. The gateway will also ensure that only mTLS enabled services are able to communicate with each other thereby making sure that services are indeed part of the recognized clusters.