Kubernetes Setup Guide

This guide will help you to get started with the deployment of Private AI container in a Kubernetes cluster.

info

This guide is for V2 container versions only

Prerequisites

Install and setup kubectl

The Kubernetes command-line tool, kubectl, allows you to run commands against Kubernetes clusters.

Find installation instructions for your OS here.

Setup your Kubernetes cluster

There are many flavours of Kubernetes available that you can choose from. Setup the one that best suits your needs. Here are few popular Kubernetes services and distributions.

Azure Kubernetes Services (AKS)

Amazon Web Services (EKS)

Google Cloud Platfrom (GKE)

Minikube

MicroK8s

info

For recommendations on machine type, see our System Requirements Section.

Setup a container registry

Setup a container registry by creating a secret for Private AI’s private registry. Only after this step, you’ll be able to pull Private AI's private docker images.

Copy
Copied
kubectl create secret docker-registry regcred --docker-server="https://index.docker.io/v1/" --docker-username="paiuser" --docker-password=<your-password>

Replace <your-password> with your access-token/password.

See this blog article for more details on pulling images from a private registry.

Deploying the deid application

Now that we have all the things in place, let’s create the manifest file deid-cpu.yaml

Copy
Copied
apiVersion: apps/v1
kind: Deployment
metadata:
  name: deid-deployment
spec:
  replicas: 1
  selector:
    matchLabels:
      app: deid
  template:
    metadata:
      labels:
        app: deid
    spec:
      affinity:
        podAntiAffinity:  # So that only one pod runs per node.
          requiredDuringSchedulingIgnoredDuringExecution:
            - labelSelector:
                matchExpressions:
                  - key: app
                    operator: In
                    values:
                      - deid
              topologyKey: "kubernetes.io/hostname"
      imagePullSecrets:
        - name: regcred
      containers:
        - name: deid
          resources:
            requests:
              cpu: 2 # update with recommended requirements for your image / Instance Type
              memory: 6Gi # update with recommended requirements for your image / Instance Type
            limits:
              cpu: 4 # update with recommended requirements for your image / Instance Type
              memory: 8Gi # update with recommended requirements for your image / Instance Type
          image: <private-ai-image>:<tag> # replace placeholders with appropriate image name and tag, example: privateai/deid:2.11
          readinessProbe:
            failureThreshold: 3
            httpGet:
              path: /healthz
              port: 8080
              scheme: HTTP
            initialDelaySeconds: 30
            periodSeconds: 10
            successThreshold: 1
            timeoutSeconds: 10
          livenessProbe:
            failureThreshold: 3
            httpGet:
              path: /healthz
              port: 8080
              scheme: HTTP
            initialDelaySeconds: 40
            periodSeconds: 60
            successThreshold: 1
            timeoutSeconds: 10
      terminationGracePeriodSeconds: 120
---
apiVersion: v1  # To see available service types https://kubernetes.io/docs/concepts/services-networking/service/
kind: Service
metadata:
  name: deid-ip
spec:
  type: LoadBalancer
  selector:
    app: deid
  ports:
    - name: http
      port: 80
      targetPort: 8080

Now create a deployment using this kubectl command.

Copy
Copied
kubectl create -f deid-cpu.yaml

Post deployment

Checking the status of containers

Once deployed successfully, you’ll be able to check the status of pods with this command:

Copy
Copied
kubectl get pod

expected output

NAME          READY   STATUS    RESTARTS   AGE
<pod-name>    1/1     Running   0          1m

To check the logs, run this command with your pod name

Copy
Copied
kubectl logs <pod-name>    # change <pod-name> with the name of pod from the command above

expected output

Log level is: info
Image Version: <version>
Using OX-E
Using 2 threads
Using OX-E
Using 2 threads
Using OX-E
Using 2 threads
INFO:     Started server process [9]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://ip:port (Press CTRL+C to quit)
INFO:     ip:port - "GET /healthz HTTP/1.1" 200 OK
 model time is 44.28 ms or 89.05 percent, rx time is 0.42 ms or 0.85 percent, total time: 49.73 ms
Auth call to Private AI servers took 154.39 ms
Got 100000 calls from PAI auth system
 1 API calls used, 99999 remaining until next auth call. Total processing time is 0.05 secs, 19.91 API calls per sec.
INFO:     ip:port - "POST /deidentify_text HTTP/1.1" 200 OK

The above deid.yaml also creates a LoadBalancer service which exposes an IP address to access your application. To check the external IP, run this:

Copy
Copied
kubectl get svc

expected output

NAME      TYPE           CLUSTER-IP    EXTERNAL-IP     PORT(S)        AGE
deid-ip   LoadBalancer   <cluster-ip>  <external-ip>   80:30456/TCP   27m

Making requests

Your can use external-ip (from the command above) of LoadBalancer service to make requests to deidentify text.

Copy
Copied
curl --location --request POST 'http://<external-ip>/deidentify_text' \
--header 'Content-Type: application/json' \
--data-raw '{"text": "Hi John, Grace this side. It'\''s been a while since we last met in Berlin.", "key": "<PUT-YOUR-API-KEY-HERE>"}'

You can expect a response like this:

Copy
Copied
{
        "result": "Hi [NAME_1], [NAME_2] this side. It's been a while since we last met in [LOCATION_CITY_1].",
        "result_fake": null,
        "pii": [
            {
                "marker": "NAME_1",
                "text": "John",
                "best_label": "NAME",
                "stt_idx": 3,
                "end_idx": 7,
                "labels": {
                    "NAME": 0.8446
                }
            },
            {
                "marker": "NAME_2",
                "text": "Grace",
                "best_label": "NAME",
                "stt_idx": 9,
                "end_idx": 14,
                "labels": {
                    "NAME": 0.8399
                }
            },
            {
                "marker": "LOCATION_CITY_1",
                "text": "Berlin",
                "best_label": "LOCATION_CITY",
                "stt_idx": 63,
                "end_idx": 69,
                "labels": {
                    "LOCATION_CITY": 0.8778,
                    "LOCATION": 0.8512
                }
            }
        ],
        "api_calls_used": 1,
        "output_checks_passed": true
    }

Additional Resources

© Copyright 2022, Private AI.