We use analytics and cookies to understand site traffic. Information about your use of our site is shared with Google for that purpose.You can read our privacy policies and terms of use etc by clicking here.
KFserving Installation
This section will walk you through installation procedure of KFserving such that it is ready to be used with Seldon Deploy.
Note
Before starting the installation procedure, please download installation resources as explained here and make sure that all pre-requisites are satisfied.
This page also assumes that main Seldon components and Istio are installed.
Install KNative Serving
Install KNATIVE Serving
KNATIVE_SERVING_URL=https://github.com/knative/serving/releases/download
SERVING_VERSION=v0.17.0
kubectl apply -f ${KNATIVE_SERVING_URL}/${SERVING_VERSION}/serving-crds.yaml
kubectl apply -f ${KNATIVE_SERVING_URL}/${SERVING_VERSION}/serving-core.yaml
kubectl apply -f https://github.com/knative/net-istio/releases/download/${SERVING_VERSION}/release.yaml
Configure Cluster Local Gateway
KNative requires a Cluster Local Gateway to work properly. This can be added to the existing Istio installation by generating required manifests
cat << EOF > ./local-cluster-gateway.yaml
apiVersion: install.istio.io/v1alpha1
kind: IstioOperator
spec:
profile: empty
components:
ingressGateways:
- name: cluster-local-gateway
enabled: true
label:
istio: cluster-local-gateway
app: cluster-local-gateway
k8s:
service:
type: ClusterIP
ports:
- port: 15020
name: status-port
- port: 80
name: http2
- port: 443
name: https
values:
gateways:
istio-ingressgateway:
debug: error
EOF
istioctl manifest generate -f local-cluster-gateway.yaml > manifest.yaml
Note the profile: empty
line. This ensures that generated manifest only contain gateway related resources.
Read more about gateway configuration in Istio’s documentation.
Once manifest is generated, inspect them, and then use kubectl
to apply them
kubectl apply -f manifest.yaml
Install KFServing
Clone KFserving
repository and checkout the right branch
KFSERVING_VERSION = v0.4.0
KFSERVING_RESOURCE_DIR = kfserving-${KFSERVING_VERSION}
git -C ${KFSERVING_RESOURCE_DIR} fetch origin || git clone https://github.com/kubeflow/kfserving.git ${KFSERVING_RESOURCE_DIR}
git -C ${KFSERVING_RESOURCE_DIR} checkout ${KFSERVING_VERSION}
Install and configure KFserving
cd ${KFSERVING_RESOURCE_DIR}
kubectl apply -f ./install/${KFSERVING_VERSION}/kfserving.yaml || true
sleep 3
kubectl patch cm -n kfserving-system inferenceservice-config -p '{"data":{"explainers":"{\n \"alibi\":{\n \"image\" : \"docker.io/seldonio/kfserving-alibiexplainer\",\n \"defaultImageVersion\" : \"v0.3.0\"\n }\n}","ingress":"{\n \"ingressGateway\" : \"seldon-gateway.istio-system\",\n \"ingressService\" : \"istio-ingressgateway.istio-system.svc.cluster.local\"\n}","logger":"{\n \"image\" : \"gcr.io/kfserving/logger:v0.4.0\",\n \"memoryRequest\": \"100Mi\",\n \"memoryLimit\": \"1Gi\",\n \"cpuRequest\": \"100m\",\n \"cpuLimit\": \"1\",\n \"defaultUrl\": \"http://default-broker.seldon-logs\"\n}"}}'
# Create self-signed certs because we dont have cert-manager
./hack/self-signed-ca.sh
Configure Seldon Deploy
Enable KFserving in Seldon Deploy by adding a following section to your deploy-values.yaml
files
kfserving:
protocol: "http"
enabled: true