Istio Ingress

Configure Seldon ingress with Istio

Istio is a production-ready service mesh solution, which Seldon Deploy uses for routing of traffic. SImilarly, traffic can also be managed by Istio for dynamically created seldon models, which simplifies how users can send requests to a single endpoint.

You can use the istio CLI to install the istio containers into your cluster. Istio provides multiple installation configuration profiles, it is important to choose the most appropriate configuration, especially as Istio is often used for applications beyond just Seldon itself. You can read more about the profiles and configurations in the Istio documentation.

Installing Istio Pods

First, obtain istioctl CLI that will help in installation of Istio. We will be installing version 1.6.5 which is compatible with kubernetes clusters version 1.15-1.18.

export ISTIO_VERSION=1.6.5

Once you have downloaded the command line interface, you can install istio with the desired profile (as discussed above) with the command below. The default profile is a reasonable choice for a base production cluster, but it is suggested to optimise the configuration to the requirements of the cluster.

./istio-${ISTIO_VERSION}/bin/istioctl manifest apply --set profile=default

You can also verify that the installation has run correctly by running the following command:

./istio-${ISTIO_VERSION}/bin/istioctl verify-install

It’s worth emphasising that Istio in itself is quite a feature-rich open source project, which is often leveraged as central service mesh for clusters. If the use-cases are more specific, there are quite a lot of different configurations that can be chosen, examples covered above include the different installation profiles, but also there are instruction like the documentation in the istio section for KNative.

Installing Seldon with Istio Configuration Enabled

In order to install Seldon with Istio enabled there are a couple of requirements, namely:

  • An Istio gateway needs to be created
  • Seldon Core Helm Chart needs to be installed with the istio values
  • Seldon Deploy Helm Chart needs to be installed with istio values

Setting up Istio Gateway

Istio Gateway is required to access deployments and Seldon Deploy itself

kind: Gateway
  name: seldon-gateway
  namespace: istio-system
    app: istio-ingressgateway
    istio: ingressgateway
  - hosts:
    - '*'
      name: http
      number: 80
      protocol: HTTP

Example of https-enabled gateway can be found under


The Istio gateway can be created in the Istio namespace, but it can also be created in a different namespace, you will need to make sure that the configuration of Seldon Deploy and Core are also aligned to the namespace and gateway name.

Install Seldon Core with Istio Enabled

You can now add the following values in your core-values.yaml file. You need to make sure that the value for the gateway is <gatewayname>.<namespace>.svc.cluster.local.

  enabled: true
  gateway: "seldon-gateway.istio-system.svc.cluster.local"

Install Seldon Deploy with Istio Enabled

Similarly you would add following entries to your deploy-values.yaml file

  seldonIngressService: "istio-ingressgateway"
  kfServingIngressService: "istio-ingressgateway"
  ingressNamespace: "istio-system"

  create: true
    - seldon-gateway.istio-system.svc.cluster.local

Find address of your Seldon Deploy

Following script can help you find address of your Seldon Deploy instance running with Istio:

ISTIO_INGRESS=$(kubectl get svc -n istio-system istio-ingressgateway -o jsonpath='{.status.loadBalancer.ingress[0].ip}')
ISTIO_INGRESS+=$(kubectl get svc -n istio-system istio-ingressgateway -o jsonpath='{.status.loadBalancer.ingress[0].hostname}')

echo "seldon deploy: http://$ISTIO_INGRESS/seldon-deploy/"
Last modified September 3, 2020: fix missing leading slash in links (c4dcebc)