Platform-Specific Prerequisites

This document covers any platform- or environment-specific prerequisites for installing Istio in ambient mode.

Platform

Certain Kubernetes environments require you to set various Istio configuration options to support them.

Google Kubernetes Engine (GKE)

On GKE, Istio components with the system-node-critical priorityClassName can only be installed in namespaces that have a ResourceQuota defined. By default in GKE, only kube-system has a defined ResourceQuota for the node-critical class. The Istio CNI node agent and ztunnel both require the node-critical class, and so in GKE, both components must either:

  • Be installed into kube-system (not istio-system)
  • Be installed into another namespace (such as istio-system) in which a ResourceQuota has been manually created, for example:
apiVersion: v1
kind: ResourceQuota
metadata:
  name: gcp-critical-pods
  namespace: istio-system
spec:
  hard:
    pods: 1000
  scopeSelector:
    matchExpressions:
    - operator: In
      scopeName: PriorityClass
      values:
      - system-node-critical

k3d

When using k3d with the default Flannel CNI, you must append the correct platform value to your installation commands, as k3d uses nonstandard locations for CNI configuration and binaries which requires some Helm overrides.

  1. Create a cluster with Traefik disabled so it doesn’t conflict with Istio’s ingress gateways:

    $ k3d cluster create --api-port 6550 -p '9080:80@loadbalancer' -p '9443:443@loadbalancer' --agents 2 --k3s-arg '--disable=traefik@server:*'
    
  2. Set global.platform=k3d when installing Istio charts. For example:

    $ helm install istio-cni istio/cni -n istio-system --set profile=ambient --set global.platform=k3d --wait
    

K3s

When using K3s and one of its bundled CNIs, you must append the correct platform value to your installation commands, as K3s uses nonstandard locations for CNI configuration and binaries which requires some Helm overrides. For the default K3s paths, Istio provides built-in overrides based on the global.platform value.

$ helm install istio-cni istio/cni -n istio-system --set profile=ambient --set global.platform=k3s --wait

However, these locations may be overridden in K3s, according to K3s documentation. If you are using K3s with a custom, non-bundled CNI, you must manually specify the correct paths for those CNIs, e.g. /etc/cni/net.d - see the K3s docs for details. For example:

$ helm install istio-cni istio/cni -n istio-system --set profile=ambient --wait --set cniConfDir=/var/lib/rancher/k3s/agent/etc/cni/net.d --set cniBinDir=/var/lib/rancher/k3s/data/current/bin/

MicroK8s

If you are installing Istio on MicroK8s, you must append the correct platform value to your installation commands, as MicroK8s uses non-standard locations for CNI configuration and binaries. For example:

$ helm install istio-cni istio/cni -n istio-system --set profile=ambient --set global.platform=microk8s --wait

minikube

If you are using minikube with the Docker driver, you must append the correct platform value to your installation commands, as minikube with Docker uses a nonstandard bind mount path for containers. For example:

$ helm install istio-cni istio/cni -n istio-system --set profile=ambient --set global.platform=minikube --wait"

Red Hat OpenShift

OpenShift requires that ztunnel and istio-cni components are installed in the kube-system namespace, and that you set global.platform=openshift for all charts.

If you use helm, you can set the target namespace and global.platform values directly.

If you use istioctl, you must use a special profile named openshift-ambient to accomplish the same thing.

$ helm install istio-cni istio/cni -n kube-system --set profile=ambient --set global.platform=openshift --wait

CNI plugins

The following configurations apply to all platforms, when certain CNI plugins are used:

Cilium

  1. Cilium currently defaults to proactively deleting other CNI plugins and their config, and must be configured with cni.exclusive = false to properly support chaining. See the Cilium documentation for more details.

  2. Cilium’s BPF masquerading is currently disabled by default, and has issues with Istio’s use of link-local IPs for Kubernetes health checking. Enabling BPF masquerading via bpf.masquerade=true is not currently supported, and results in non-functional pod health checks in Istio ambient. Cilium’s default iptables masquerading implementation should continue to function correctly.

  3. Due to how Cilium manages node identity and internally allow-lists node-level health probes to pods, applying any default-DENY NetworkPolicy in a Cilium CNI install underlying Istio in ambient mode will cause kubelet health probes (which are by-default silently exempted from all policy enforcement by Cilium) to be blocked. This is because Istio uses a link-local SNAT address, which Cilium is not aware of, and Cilium does not have an option to exempt link-local addresses from policy enforcement.

    This can be resolved by applying the following CiliumClusterWideNetworkPolicy:

    apiVersion: "cilium.io/v2"
    kind: CiliumClusterwideNetworkPolicy
    metadata:
      name: "allow-ambient-hostprobes"
    spec:
      description: "Allows SNAT-ed kubelet health check probes into ambient pods"
      enableDefaultDeny:
        egress: false
        ingress: false
      endpointSelector: {}
      ingress:
      - fromCIDR:
        - "169.254.7.127/32"
    

    This policy override is not required unless you already have other default-deny NetworkPolicies or CiliumNetworkPolicies applied in your cluster.

    Please see issue #49277 and CiliumClusterWideNetworkPolicy for more details.

Was this information useful?
Do you have any suggestions for improvement?

Thanks for your feedback!