We use analytics and cookies to understand site traffic. Information about your use of our site is shared with Google for that purpose.You can read our privacy policies and terms of use etc by clicking here.
Istio Ingress
Istio is a production-ready service mesh solution, which Seldon Deploy uses for routing of traffic. SImilarly, traffic can also be managed by Istio for dynamically created seldon models, which simplifies how users can send requests to a single endpoint.
Note
Before starting the installation procedure, please download installation resources as explained here and make sure that all pre-requisites are satisfied.
This page also assumes that main Seldon components are installed.
You can use the istio CLI to install the istio containers into your cluster. Istio provides multiple installation configuration profiles, it is important to choose the most appropriate configuration, especially as Istio is often used for applications beyond just Seldon itself. You can read more about the profiles and configurations in the Istio documentation.
Installing Istio Pods
First, obtain istioctl
CLI that will help in installation of Istio. We will be installing version 1.6.5 which is compatible with kubernetes clusters version 1.15-1.18.
export ISTIO_VERSION=1.6.5
curl -L https://istio.io/downloadIstio | ISTIO_VERSION=${ISTIO_VERSION} sh -
Once you have downloaded the command line interface, you can install istio with the desired profile (as discussed above) with the command below. The default profile is a reasonable choice for a base production cluster, but it is suggested to optimise the configuration to the requirements of the cluster.
./istio-${ISTIO_VERSION}/bin/istioctl manifest apply --set profile=default
You can also verify that the installation has run correctly by running the following command:
./istio-${ISTIO_VERSION}/bin/istioctl verify-install
It’s worth emphasising that Istio in itself is quite a feature-rich open source project, which is often leveraged as central service mesh for clusters. If the use-cases are more specific, there are quite a lot of different configurations that can be chosen, examples covered above include the different installation profiles, but also there are instruction like the documentation in the istio section for KNative.
Installing Seldon with Istio Configuration Enabled
In order to install Seldon with Istio enabled there are a couple of requirements, namely:
- An Istio gateway needs to be created
- Seldon Core Helm Chart needs to be installed with the istio values
- Seldon Deploy Helm Chart needs to be installed with istio values
Setting up Istio Gateway
Istio Gateway is required to access deployments and Seldon Deploy itself
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
name: seldon-gateway
namespace: istio-system
spec:
selector:
app: istio-ingressgateway
istio: ingressgateway
servers:
- hosts:
- '*'
port:
name: http
number: 80
protocol: HTTP
Example of https-enabled gateway can be found under
seldon-deploy-install/prerequisites-setup/istio/seldon-gateway.yaml
The Istio gateway can be created in the Istio namespace, but it can also be created in a different namespace, you will need to make sure that the configuration of Seldon Deploy and Core are also aligned to the namespace and gateway name.
Install Seldon Core with Istio Enabled
You can now add the following values in your core-values.yaml
file. You need to make sure that the value for the gateway is <gatewayname>.<namespace>.svc.cluster.local
.
istio:
enabled: true
gateway: "seldon-gateway.istio-system.svc.cluster.local"
Install Seldon Deploy with Istio Enabled
Similarly you would add following entries to your deploy-values.yaml
file
ingressGateway:
seldonIngressService: "istio-ingressgateway"
kfServingIngressService: "istio-ingressgateway"
ingressNamespace: "istio-system"
virtualService:
create: true
gateways:
- seldon-gateway.istio-system.svc.cluster.local
Find address of your Seldon Deploy
Following script can help you find address of your Seldon Deploy instance running with Istio:
ISTIO_INGRESS=$(kubectl get svc -n istio-system istio-ingressgateway -o jsonpath='{.status.loadBalancer.ingress[0].ip}')
ISTIO_INGRESS+=$(kubectl get svc -n istio-system istio-ingressgateway -o jsonpath='{.status.loadBalancer.ingress[0].hostname}')
echo "seldon deploy: http://$ISTIO_INGRESS/seldon-deploy/"