Secure Gateways
The Control Ingress Traffic task describes how to configure an ingress gateway to expose an HTTP service to external traffic. This task shows how to expose a secure HTTPS service using either simple or mutual TLS.
Before you begin
Setup Istio by following the instructions in the Installation guide.
Start the httpbin sample:
$ kubectl apply -f @samples/httpbin/httpbin.yaml@
For macOS users, verify that you use
curl
compiled with the LibreSSL library:$ curl --version | grep LibreSSL curl 7.54.0 (x86_64-apple-darwin17.0) libcurl/7.54.0 LibreSSL/2.0.20 zlib/1.2.11 nghttp2/1.24.0
If the previous command outputs a version of LibreSSL as shown, your
curl
command should work correctly with the instructions in this task. Otherwise, try a different implementation ofcurl
, for example on a Linux machine.
Generate client and server certificates and keys
This task requires several sets of certificates and keys which are used in the following examples. You can use your favorite tool to create them or use the commands below to generate them using openssl.
Create a root certificate and private key to sign the certificates for your services:
$ mkdir example_certs1 $ openssl req -x509 -sha256 -nodes -days 365 -newkey rsa:2048 -subj '/O=example Inc./CN=example.com' -keyout example_certs1/example.com.key -out example_certs1/example.com.crt
Generate a certificate and a private key for
httpbin.example.com
:$ openssl req -out example_certs1/httpbin.example.com.csr -newkey rsa:2048 -nodes -keyout example_certs1/httpbin.example.com.key -subj "/CN=httpbin.example.com/O=httpbin organization" $ openssl x509 -req -sha256 -days 365 -CA example_certs1/example.com.crt -CAkey example_certs1/example.com.key -set_serial 0 -in example_certs1/httpbin.example.com.csr -out example_certs1/httpbin.example.com.crt
Create a second set of the same kind of certificates and keys:
$ mkdir example_certs2 $ openssl req -x509 -sha256 -nodes -days 365 -newkey rsa:2048 -subj '/O=example Inc./CN=example.com' -keyout example_certs2/example.com.key -out example_certs2/example.com.crt $ openssl req -out example_certs2/httpbin.example.com.csr -newkey rsa:2048 -nodes -keyout example_certs2/httpbin.example.com.key -subj "/CN=httpbin.example.com/O=httpbin organization" $ openssl x509 -req -sha256 -days 365 -CA example_certs2/example.com.crt -CAkey example_certs2/example.com.key -set_serial 0 -in example_certs2/httpbin.example.com.csr -out example_certs2/httpbin.example.com.crt
Generate a certificate and a private key for
helloworld.example.com
:$ openssl req -out example_certs1/helloworld.example.com.csr -newkey rsa:2048 -nodes -keyout example_certs1/helloworld.example.com.key -subj "/CN=helloworld.example.com/O=helloworld organization" $ openssl x509 -req -sha256 -days 365 -CA example_certs1/example.com.crt -CAkey example_certs1/example.com.key -set_serial 1 -in example_certs1/helloworld.example.com.csr -out example_certs1/helloworld.example.com.crt
Generate a client certificate and private key:
$ openssl req -out example_certs1/client.example.com.csr -newkey rsa:2048 -nodes -keyout example_certs1/client.example.com.key -subj "/CN=client.example.com/O=client organization" $ openssl x509 -req -sha256 -days 365 -CA example_certs1/example.com.crt -CAkey example_certs1/example.com.key -set_serial 1 -in example_certs1/client.example.com.csr -out example_certs1/client.example.com.crt
Configure a TLS ingress gateway for a single host
Create a secret for the ingress gateway:
$ kubectl create -n istio-system secret tls httpbin-credential \ --key=example_certs1/httpbin.example.com.key \ --cert=example_certs1/httpbin.example.com.crt
Configure the ingress gateway:
First, define a gateway with a servers:
section for port 443, and specify values for
credentialName
to be httpbin-credential
. The values are the same as the
secret’s name. The TLS mode should have the value of SIMPLE
.
$ cat <<EOF | kubectl apply -f -
apiVersion: networking.istio.io/v1
kind: Gateway
metadata:
name: mygateway
spec:
selector:
istio: ingressgateway # use istio default ingress gateway
servers:
- port:
number: 443
name: https
protocol: HTTPS
tls:
mode: SIMPLE
credentialName: httpbin-credential # must be the same as secret
hosts:
- httpbin.example.com
EOF
Next, configure the gateway’s ingress traffic routes by defining a corresponding virtual service:
$ cat <<EOF | kubectl apply -f -
apiVersion: networking.istio.io/v1
kind: VirtualService
metadata:
name: httpbin
spec:
hosts:
- "httpbin.example.com"
gateways:
- mygateway
http:
- match:
- uri:
prefix: /status
- uri:
prefix: /delay
route:
- destination:
port:
number: 8000
host: httpbin
EOF
Finally, follow these instructions
to set the INGRESS_HOST
and SECURE_INGRESS_PORT
variables for accessing the gateway.
First, create a Kubernetes Gateway:
$ cat <<EOF | kubectl apply -f -
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
name: mygateway
namespace: istio-system
spec:
gatewayClassName: istio
listeners:
- name: https
hostname: "httpbin.example.com"
port: 443
protocol: HTTPS
tls:
mode: Terminate
certificateRefs:
- name: httpbin-credential
allowedRoutes:
namespaces:
from: Selector
selector:
matchLabels:
kubernetes.io/metadata.name: default
EOF
Next, configure the gateway’s ingress traffic routes by defining a corresponding HTTPRoute
:
$ cat <<EOF | kubectl apply -f -
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: httpbin
spec:
parentRefs:
- name: mygateway
namespace: istio-system
hostnames: ["httpbin.example.com"]
rules:
- matches:
- path:
type: PathPrefix
value: /status
- path:
type: PathPrefix
value: /delay
backendRefs:
- name: httpbin
port: 8000
EOF
Finally, get the gateway address and port from the Gateway
resource:
$ kubectl wait --for=condition=programmed gtw mygateway -n istio-system
$ export INGRESS_HOST=$(kubectl get gtw mygateway -n istio-system -o jsonpath='{.status.addresses[0].value}')
$ export SECURE_INGRESS_PORT=$(kubectl get gtw mygateway -n istio-system -o jsonpath='{.spec.listeners[?(@.name=="https")].port}')
Send an HTTPS request to access the
httpbin
service through HTTPS:$ curl -v -HHost:httpbin.example.com --resolve "httpbin.example.com:$SECURE_INGRESS_PORT:$INGRESS_HOST" \ --cacert example_certs1/example.com.crt "https://httpbin.example.com:$SECURE_INGRESS_PORT/status/418" ... HTTP/2 418 ... -=[ teapot ]=- _...._ .' _ _ `. | ."` ^ `". _, \_;`"---"`|// | ;/ \_ _/ `"""`
The
httpbin
service will return the 418 I’m a Teapot code.Change the gateway’s credentials by deleting the gateway’s secret and then recreating it using different certificates and keys:
$ kubectl -n istio-system delete secret httpbin-credential $ kubectl create -n istio-system secret tls httpbin-credential \ --key=example_certs2/httpbin.example.com.key \ --cert=example_certs2/httpbin.example.com.crt
Access the
httpbin
service withcurl
using the new certificate chain:$ curl -v -HHost:httpbin.example.com --resolve "httpbin.example.com:$SECURE_INGRESS_PORT:$INGRESS_HOST" \ --cacert example_certs2/example.com.crt "https://httpbin.example.com:$SECURE_INGRESS_PORT/status/418" ... HTTP/2 418 ... -=[ teapot ]=- _...._ .' _ _ `. | ."` ^ `". _, \_;`"---"`|// | ;/ \_ _/ `"""`
If you try to access
httpbin
using the previous certificate chain, the attempt now fails:$ curl -v -HHost:httpbin.example.com --resolve "httpbin.example.com:$SECURE_INGRESS_PORT:$INGRESS_HOST" \ --cacert example_certs1/example.com.crt "https://httpbin.example.com:$SECURE_INGRESS_PORT/status/418" ... * TLSv1.2 (OUT), TLS handshake, Client hello (1): * TLSv1.2 (IN), TLS handshake, Server hello (2): * TLSv1.2 (IN), TLS handshake, Certificate (11): * TLSv1.2 (OUT), TLS alert, Server hello (2): * curl: (35) error:04FFF06A:rsa routines:CRYPTO_internal:block type is not 01
Configure a TLS ingress gateway for multiple hosts
You can configure an ingress gateway for multiple hosts,
httpbin.example.com
and helloworld.example.com
, for example. The ingress gateway
is configured with unique credentials corresponding to each host.
Restore the
httpbin
credentials from the previous example by deleting and recreating the secret with the original certificates and keys:$ kubectl -n istio-system delete secret httpbin-credential $ kubectl create -n istio-system secret tls httpbin-credential \ --key=example_certs1/httpbin.example.com.key \ --cert=example_certs1/httpbin.example.com.crt
Start the
helloworld-v1
sample:$ kubectl apply -f @samples/helloworld/helloworld.yaml@ -l service=helloworld $ kubectl apply -f @samples/helloworld/helloworld.yaml@ -l version=v1
Create a
helloworld-credential
secret:$ kubectl create -n istio-system secret tls helloworld-credential \ --key=example_certs1/helloworld.example.com.key \ --cert=example_certs1/helloworld.example.com.crt
Configure the ingress gateway with hosts
httpbin.example.com
andhelloworld.example.com
:
Define a gateway with two server sections for port 443. Set the value of
credentialName
on each port to httpbin-credential
and helloworld-credential
respectively. Set TLS mode to SIMPLE
.
$ cat <<EOF | kubectl apply -f -
apiVersion: networking.istio.io/v1
kind: Gateway
metadata:
name: mygateway
spec:
selector:
istio: ingressgateway # use istio default ingress gateway
servers:
- port:
number: 443
name: https-httpbin
protocol: HTTPS
tls:
mode: SIMPLE
credentialName: httpbin-credential
hosts:
- httpbin.example.com
- port:
number: 443
name: https-helloworld
protocol: HTTPS
tls:
mode: SIMPLE
credentialName: helloworld-credential
hosts:
- helloworld.example.com
EOF
Configure the gateway’s traffic routes by defining a corresponding virtual service.
$ cat <<EOF | kubectl apply -f -
apiVersion: networking.istio.io/v1
kind: VirtualService
metadata:
name: helloworld
spec:
hosts:
- helloworld.example.com
gateways:
- mygateway
http:
- match:
- uri:
exact: /hello
route:
- destination:
host: helloworld
port:
number: 5000
EOF
Configure a Gateway
with two listeners for port 443. Set the value of
certificateRefs
on each listener to httpbin-credential
and helloworld-credential
respectively.
$ cat <<EOF | kubectl apply -f -
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
name: mygateway
namespace: istio-system
spec:
gatewayClassName: istio
listeners:
- name: https-httpbin
hostname: "httpbin.example.com"
port: 443
protocol: HTTPS
tls:
mode: Terminate
certificateRefs:
- name: httpbin-credential
allowedRoutes:
namespaces:
from: Selector
selector:
matchLabels:
kubernetes.io/metadata.name: default
- name: https-helloworld
hostname: "helloworld.example.com"
port: 443
protocol: HTTPS
tls:
mode: Terminate
certificateRefs:
- name: helloworld-credential
allowedRoutes:
namespaces:
from: Selector
selector:
matchLabels:
kubernetes.io/metadata.name: default
EOF
Configure the gateway’s traffic routes for the helloworld
service:
$ cat <<EOF | kubectl apply -f -
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: helloworld
spec:
parentRefs:
- name: mygateway
namespace: istio-system
hostnames: ["helloworld.example.com"]
rules:
- matches:
- path:
type: Exact
value: /hello
backendRefs:
- name: helloworld
port: 5000
EOF
Send an HTTPS request to
helloworld.example.com
:$ curl -v -HHost:helloworld.example.com --resolve "helloworld.example.com:$SECURE_INGRESS_PORT:$INGRESS_HOST" \ --cacert example_certs1/example.com.crt "https://helloworld.example.com:$SECURE_INGRESS_PORT/hello" ... HTTP/2 200 ...
Send an HTTPS request to
httpbin.example.com
and still get a teapot in return:$ curl -v -HHost:httpbin.example.com --resolve "httpbin.example.com:$SECURE_INGRESS_PORT:$INGRESS_HOST" \ --cacert example_certs1/example.com.crt "https://httpbin.example.com:$SECURE_INGRESS_PORT/status/418" ... -=[ teapot ]=- _...._ .' _ _ `. | ."` ^ `". _, \_;`"---"`|// | ;/ \_ _/ `"""`
Configure a mutual TLS ingress gateway
You can extend your gateway’s definition to support mutual TLS.
Change the credentials of the ingress gateway by deleting its secret and creating a new one. The server uses the CA certificate to verify its clients, and we must use the key
ca.crt
to hold the CA certificate.$ kubectl -n istio-system delete secret httpbin-credential $ kubectl create -n istio-system secret generic httpbin-credential \ --from-file=tls.key=example_certs1/httpbin.example.com.key \ --from-file=tls.crt=example_certs1/httpbin.example.com.crt \ --from-file=ca.crt=example_certs1/example.com.crt
Configure the ingress gateway:
Change the gateway’s definition to set the TLS mode to MUTUAL
.
$ cat <<EOF | kubectl apply -f -
apiVersion: networking.istio.io/v1
kind: Gateway
metadata:
name: mygateway
spec:
selector:
istio: ingressgateway # use istio default ingress gateway
servers:
- port:
number: 443
name: https
protocol: HTTPS
tls:
mode: MUTUAL
credentialName: httpbin-credential # must be the same as secret
hosts:
- httpbin.example.com
EOF
Because the Kubernetes Gateway API does not currently support mutual TLS termination in a
Gateway,
we use an Istio-specific option, gateway.istio.io/tls-terminate-mode: MUTUAL
,
to configure it:
$ cat <<EOF | kubectl apply -f -
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
name: mygateway
namespace: istio-system
spec:
gatewayClassName: istio
listeners:
- name: https
hostname: "httpbin.example.com"
port: 443
protocol: HTTPS
tls:
mode: Terminate
certificateRefs:
- name: httpbin-credential
options:
gateway.istio.io/tls-terminate-mode: MUTUAL
allowedRoutes:
namespaces:
from: Selector
selector:
matchLabels:
kubernetes.io/metadata.name: default
EOF
Attempt to send an HTTPS request using the prior approach and see how it fails:
$ curl -v -HHost:httpbin.example.com --resolve "httpbin.example.com:$SECURE_INGRESS_PORT:$INGRESS_HOST" \ --cacert example_certs1/example.com.crt "https://httpbin.example.com:$SECURE_INGRESS_PORT/status/418" * TLSv1.3 (OUT), TLS handshake, Client hello (1): * TLSv1.3 (IN), TLS handshake, Server hello (2): * TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8): * TLSv1.3 (IN), TLS handshake, Request CERT (13): * TLSv1.3 (IN), TLS handshake, Certificate (11): * TLSv1.3 (IN), TLS handshake, CERT verify (15): * TLSv1.3 (IN), TLS handshake, Finished (20): * TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1): * TLSv1.3 (OUT), TLS handshake, Certificate (11): * TLSv1.3 (OUT), TLS handshake, Finished (20): * TLSv1.3 (IN), TLS alert, unknown (628): * OpenSSL SSL_read: error:1409445C:SSL routines:ssl3_read_bytes:tlsv13 alert certificate required, errno 0
Pass a client certificate and private key to
curl
and resend the request. Pass your client’s certificate with the--cert
flag and your private key with the--key
flag tocurl
:$ curl -v -HHost:httpbin.example.com --resolve "httpbin.example.com:$SECURE_INGRESS_PORT:$INGRESS_HOST" \ --cacert example_certs1/example.com.crt --cert example_certs1/client.example.com.crt --key example_certs1/client.example.com.key \ "https://httpbin.example.com:$SECURE_INGRESS_PORT/status/418" ... -=[ teapot ]=- _...._ .' _ _ `. | ."` ^ `". _, \_;`"---"`|// | ;/ \_ _/ `"""`
More info
Key formats
Istio supports reading a few different Secret formats, to support integration with various tools such as cert-manager:
- A TLS Secret with keys
tls.key
andtls.crt
, as described above. For mutual TLS, aca.crt
key can be used. - A generic Secret with keys
key
andcert
. For mutual TLS, acacert
key can be used. - A generic Secret with keys
key
andcert
. For mutual TLS, a separate generic Secret named<secret>-cacert
, with acacert
key. For example,httpbin-credential
haskey
andcert
, andhttpbin-credential-cacert
hascacert
. - The
cacert
key value can be a CA bundle consisting of concatenated individual CA certificates.
SNI Routing
An HTTPS Gateway
will perform SNI matching against its configured host(s)
before forwarding a request, which may cause some requests to fail.
See configuring SNI routing for details.
Troubleshooting
Inspect the values of the
INGRESS_HOST
andSECURE_INGRESS_PORT
environment variables. Make sure they have valid values, according to the output of the following commands:$ kubectl get svc -n istio-system $ echo "INGRESS_HOST=$INGRESS_HOST, SECURE_INGRESS_PORT=$SECURE_INGRESS_PORT"
Make sure the value of
INGRESS_HOST
is an IP address. In some cloud platforms, e.g., AWS, you may get a domain name, instead. This task expects an IP address, so you will need to convert it with commands similar to the following:$ nslookup ab52747ba608744d8afd530ffd975cbf-330887905.us-east-1.elb.amazonaws.com $ export INGRESS_HOST=3.225.207.109
Check the log of the gateway controller for error messages:
$ kubectl logs -n istio-system <gateway-service-pod>
If using macOS, verify you are using
curl
compiled with the LibreSSL library, as described in the Before you begin section.Verify that the secrets are successfully created in the
istio-system
namespace:$ kubectl -n istio-system get secrets
httpbin-credential
andhelloworld-credential
should show in the secrets list.Check the logs to verify that the ingress gateway agent has pushed the key/certificate pair to the ingress gateway:
$ kubectl logs -n istio-system <gateway-service-pod>
The log should show that the
httpbin-credential
secret was added. If using mutual TLS, then thehttpbin-credential-cacert
secret should also appear. Verify the log shows that the gateway agent receives SDS requests from the ingress gateway, that the resource’s name ishttpbin-credential
, and that the ingress gateway obtained the key/certificate pair. If using mutual TLS, the log should show key/certificate was sent to the ingress gateway, that the gateway agent received the SDS request with thehttpbin-credential-cacert
resource name, and that the ingress gateway obtained the root certificate.
Cleanup
- Delete the gateway configuration and routes:
$ kubectl delete gateway mygateway
$ kubectl delete virtualservice httpbin helloworld
$ kubectl delete -n istio-system gtw mygateway
$ kubectl delete httproute httpbin helloworld
Delete the secrets, certificates and keys:
$ kubectl delete -n istio-system secret httpbin-credential helloworld-credential $ rm -rf ./example_certs1 ./example_certs2
Shutdown the
httpbin
andhelloworld
services:$ kubectl delete -f samples/httpbin/httpbin.yaml $ kubectl delete deployment helloworld-v1 $ kubectl delete service helloworld