Authenticated GRPC service on EKS with Istio

Abhishek Choudhary
3 min readMay 1, 2023

--

Adding basic authentication to a service hosted on Kubernetes

This post will describe hosting a GRPC service on AWS EKS cluster. The process is roughly going to be the same for every Kuberentes cluster — however, we had to do some specific settings on the AWS Load balancer for this to work.

Usecase of GRPC

The usecase was hosting Tensorflow models as APIs which accepted a payload of around 100MB in size. GRPC performs much better for larger payloads — so we exposed the GRPC port on port 5000.

Hosting the service

We hosted the service on Kubernetes using the Deployment YAML below:

apiVersion: apps/v1
kind: Deployment
metadata:
name: ml-api
namespace: ml-services
spec:
replicas: 1
selector:
matchLabels:
truefoundry.com/component: ml-api
template:
metadata:
labels:
truefoundry.com/application: ml-api
spec:
containers:
- name: ml-api
image: >-
XXXX.dkr.ecr.us-east-1.amazonaws.com/ml-services-ml-api:latest
ports:
- name: port-8500
containerPort: 8500
protocol: TCP
resources:
limits:
cpu: '4'
ephemeral-storage: 2G
memory: 4G
requests:
cpu: '1'
ephemeral-storage: 1G
memory: 500M
imagePullPolicy: IfNotPresent
restartPolicy: Always
terminationGracePeriodSeconds: 30
dnsPolicy: ClusterFirst
securityContext: {}
imagePullSecrets:
- name: ml-api-image-pull-secret
schedulerName: default-scheduler
strategy:
type: RollingUpdate
rollingUpdate:
maxUnavailable: 25%
maxSurge: 0

This will bring up the pod. We need to create the Service object using the YAML below:

apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
labels:
argocd.argoproj.io/instance: tfy-istio-ingress
name: tfy-wildcard
namespace: istio-system
spec:
selector:
istio: tfy-istio-ingress
servers:
- hosts:
- 'ml.example.com'
port:
name: http-tfy-wildcard
number: 80
protocol: HTTP
tls:
httpsRedirect: true
- hosts:
- 'ml.example.com'
port:
name: https-tfy-wildcard
number: 443
protocol: HTTP

Exposing the service

We are using Istio as the ingress layer in Kubernetes. Istio provisions a Load Balancer when the istio-ingress is installed. The load balancer configuration can be customized using annotations on the istio gateway. The spec for creating the Istio Gateway is as follows:

apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
labels:
argocd.argoproj.io/instance: tfy-istio-ingress
name: tfy-wildcard
namespace: istio-system
spec:
selector:
istio: tfy-istio-ingress
servers:
- hosts:
- 'ml.example.com'
port:
name: http-tfy-wildcard
number: 80
protocol: HTTP
tls:
httpsRedirect: true
- hosts:
- 'ml.example.com'
port:
name: https-tfy-wildcard
number: 443
protocol: HTTP

We are doing the SSL termination on the AWS Load Balancer. For this we have to attach the certificate to the Load Balancer. This can be achieved using the annotations below to the istio gateway chart (https://istio-release.storage.googleapis.com/charts).

"service.beta.kubernetes.io/aws-load-balancer-type": "nlb"
"service.beta.kubernetes.io/aws-load-balancer-backend-protocol": "tcp"
"service.beta.kubernetes.io/aws-load-balancer-ssl-cert": "<certificate-arn>"
"service.beta.kubernetes.io/aws-load-balancer-ssl-ports": "https"
"service.beta.kubernetes.io/aws-load-balancer-alpn-policy": "HTTP2Preferred"

The alpn-policy is important to specify to allow GRPC traffic. Our service ml-api can be exposed by creating a VirtualService pointing to the Kubernetes service. The YAML for the Virtual Service is as follows:

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
labels:
argocd.argoproj.io/instance: ml-services_ml-api
name: ml-apiport-8500-vs
namespace: ml-services
spec:
gateways:
- istio-system/tfy-wildcard
hosts:
- ml.example.com
http:
- route:
- destination:
host: ml-api
port:
number: 8500

Once the virtual service is exposed, we can make requests to our service at ml.example.com. We then wanted to add an authentication to the API so that everybody cannot call the API. We could have added the authentication in the code, but we decided to add it at the istio layer so that it can be a unified layer across all services.

To add authentication at the istio-ingress layer, we decided to go ahead with a IstioWasm Plugin ( https://istio.io/latest/docs/reference/config/proxy_extensions/wasm-plugin/). The yaml for the plugin looks something like:

apiVersion: extensions.istio.io/v1alpha1
kind: WasmPlugin
metadata:
name: ml-services-ml-api-0
namespace: istio-system
spec:
phase: AUTHN
pluginConfig:
basic_auth_rules:
- credentials:
- username:password
hosts:
- ml.example.com
prefix: /
request_methods:
- GET
- PUT
- POST
- PATCH
- DELETE
selector:
matchLabels:
istio: tfy-istio-ingress
url: oci://ghcr.io/istio-ecosystem/wasm-extensions/basic_auth:1.12.0

Once you have applied the above spec to the cluster, the app will ask for the username and password once you open it in the browser.

To make the above process much easier, we decide to make it really easy on the Truefoundry platform.

Originally published at https://blog.truefoundry.com on May 1, 2023.

--

--

Abhishek Choudhary

Enterprenuer | Ex-Facebook Hacker | Travel | Musician by aspirations