Kubernetes Setup Guide
This guide will help you to get started with the deployment of Private AI container in a Kubernetes cluster.
info
This guide is for V2 container versions only
Prerequisites
Install and setup kubectl
The Kubernetes command-line tool, kubectl
, allows you to run commands against Kubernetes clusters.
Find installation instructions for your OS here.
Setup your Kubernetes cluster
There are many flavours of Kubernetes available that you can choose from. Setup the one that best suits your needs. Here are few popular Kubernetes services and distributions.
Azure Kubernetes Services (AKS)
info
For recommendations on machine type, see our System Requirements Section.
Setup a container registry
Setup a container registry by creating a secret for Private AI’s private registry. Only after this step, you’ll be able to pull Private AI's private docker images.
kubectl create secret docker-registry regcred --docker-server="https://index.docker.io/v1/" --docker-username="paiuser" --docker-password=<your-password>
Replace <your-password>
with your access-token/password.
See this blog article for more details on pulling images from a private registry.
Deploying the deid application
Now that we have all the things in place, let’s create the manifest file deid-cpu.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: deid-deployment
spec:
replicas: 1
selector:
matchLabels:
app: deid
template:
metadata:
labels:
app: deid
spec:
affinity:
podAntiAffinity: # So that only one pod runs per node.
requiredDuringSchedulingIgnoredDuringExecution:
- labelSelector:
matchExpressions:
- key: app
operator: In
values:
- deid
topologyKey: "kubernetes.io/hostname"
imagePullSecrets:
- name: regcred
containers:
- name: deid
resources:
requests:
cpu: 2 # update with recommended requirements for your image / Instance Type
memory: 6Gi # update with recommended requirements for your image / Instance Type
limits:
cpu: 4 # update with recommended requirements for your image / Instance Type
memory: 8Gi # update with recommended requirements for your image / Instance Type
image: <private-ai-image>:<tag> # replace placeholders with appropriate image name and tag, example: privateai/deid:2.11
readinessProbe:
failureThreshold: 3
httpGet:
path: /healthz
port: 8080
scheme: HTTP
initialDelaySeconds: 30
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 10
livenessProbe:
failureThreshold: 3
httpGet:
path: /healthz
port: 8080
scheme: HTTP
initialDelaySeconds: 40
periodSeconds: 60
successThreshold: 1
timeoutSeconds: 10
terminationGracePeriodSeconds: 120
---
apiVersion: v1 # To see available service types https://kubernetes.io/docs/concepts/services-networking/service/
kind: Service
metadata:
name: deid-ip
spec:
type: LoadBalancer
selector:
app: deid
ports:
- name: http
port: 80
targetPort: 8080
Now create a deployment using this kubectl
command.
kubectl create -f deid-cpu.yaml
Post deployment
Checking the status of containers
Once deployed successfully, you’ll be able to check the status of pods with this command:
kubectl get pod
expected output
NAME READY STATUS RESTARTS AGE
<pod-name> 1/1 Running 0 1m
To check the logs, run this command with your pod name
kubectl logs <pod-name> # change <pod-name> with the name of pod from the command above
expected output
Log level is: info
Image Version: <version>
Using OX-E
Using 2 threads
Using OX-E
Using 2 threads
Using OX-E
Using 2 threads
INFO: Started server process [9]
INFO: Waiting for application startup.
INFO: Application startup complete.
INFO: Uvicorn running on http://ip:port (Press CTRL+C to quit)
INFO: ip:port - "GET /healthz HTTP/1.1" 200 OK
model time is 44.28 ms or 89.05 percent, rx time is 0.42 ms or 0.85 percent, total time: 49.73 ms
Auth call to Private AI servers took 154.39 ms
Got 100000 calls from PAI auth system
1 API calls used, 99999 remaining until next auth call. Total processing time is 0.05 secs, 19.91 API calls per sec.
INFO: ip:port - "POST /deidentify_text HTTP/1.1" 200 OK
The above deid.yaml
also creates a LoadBalancer service which exposes an IP address to access your application. To check the external IP, run this:
kubectl get svc
expected output
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
deid-ip LoadBalancer <cluster-ip> <external-ip> 80:30456/TCP 27m
Making requests
Your can use external-ip
(from the command above) of LoadBalancer service to make requests to deidentify text.
curl --location --request POST 'http://<external-ip>/deidentify_text' \
--header 'Content-Type: application/json' \
--data-raw '{"text": "Hi John, Grace this side. It'\''s been a while since we last met in Berlin.", "key": "<PUT-YOUR-API-KEY-HERE>"}'
You can expect a response like this:
{
"result": "Hi [NAME_1], [NAME_2] this side. It's been a while since we last met in [LOCATION_CITY_1].",
"result_fake": null,
"pii": [
{
"marker": "NAME_1",
"text": "John",
"best_label": "NAME",
"stt_idx": 3,
"end_idx": 7,
"labels": {
"NAME": 0.8446
}
},
{
"marker": "NAME_2",
"text": "Grace",
"best_label": "NAME",
"stt_idx": 9,
"end_idx": 14,
"labels": {
"NAME": 0.8399
}
},
{
"marker": "LOCATION_CITY_1",
"text": "Berlin",
"best_label": "LOCATION_CITY",
"stt_idx": 63,
"end_idx": 69,
"labels": {
"LOCATION_CITY": 0.8778,
"LOCATION": 0.8512
}
}
],
"api_calls_used": 1,
"output_checks_passed": true
}