Installation
Private AI's privacy solution is primarily designed to be self-hosted by the user via a Docker image. This is to provide users with the best possible experience in terms of latency & security. It would be counter-productive to send sensitive data across the Internet to a 3rd party system for the purpose of improving privacy. It also ensures that Private AI never sees or handles customer data, unlike cloud APIs which retain a right to use any data passed through the system for service improvements and ML model development.
It is also possible to use a cloud version of the API, hosted by Private AI at the following endpoint: https://api.private-ai.com/deid/v1
Custom integrations that do not rely on Docker can also be delivered upon request.
System Requirements
The image comes in two different build flavours: a compact, CPU-only container that runs on any Intel or AMD CPU and a container with GPU acceleration. The CPU container is highly optimised for the majority of use cases, as the container uses hand-coded AVX2/AVX512/AVX512 VNNI instructions in conjunction with Neural Network compression techniques to deliver a ~25X speedup over a reference transformer-based system. The GPU container is designed for large-scale deployments making billions of API calls or processing terabytes of data per month.
The minimum & recommended system requirements for the Docker image are as follows:
Minimum | Recommended | Recommended Concurrency | |
---|---|---|---|
CPU | Any x86 (Intel or AMD) processor wit 6GB RAM | Intel Cascade Lake or newer CPUs supporting AVX512 VNNI with 8GB RAM | 1 |
GPU | Any x86 (Intel or AMD) processor with 28GB RAM. Nvidia GPU with compute capability 6.0 or higher (Pascal or newer) | Any x86 (Intel or AMD) processor with 32GB RAM & Nvidia Tesla T4 GPU | 32 |
The Private AI image can also run on the new Apple chips, such as the M1. Performance will be degraded however, due to the Rosetta 2 emulation of the AVX instructions. Native ARM CPU builds & builds requiring only 1-2GB RAM can also be delivered upon request.
While Private AI CPU-based container will run on any x86-compatible instance, the below cloud instance types give optimal throughput & latency per dollar:
Platform | Recommended Instance Type | Description |
---|---|---|
Azure | Standard_E2_v5 | 2x Intel Ice Lake vCPUs, 16GB RAM |
AWS | M5zn.large (2 vCPU, 8GB RAM) | 2x Intel Cascade Lake vCPUs, 8 GB RAM |
GCP | N2-Standard-2 (2 vCPU, 8GB RAM) | 2x Intel Cascade Lake or Ice Lake vCPUs, 8 GB RAM |
Note: In the event lower latency is required, the instance type should be scaled; e.g. using an M5zn.2xlarge in place of a M5zn.xlarge. While the Private AI docker solution can make use of all available CPU cores, it delivers best throughput per dollar using a single CPU core machine. Scaling CPU cores does not result in a linear increase in performance.
Similarly for the GPU-based image, Private AI recommends the following instance types:
Platform | Recommended Instance Type | Description |
---|---|---|
Azure | Standard_NC8as_T4_v3 | 8x AMD EPYC 7V12(Rome) vCPUs, 56GB RAM, Nvidia Tesla T4 GPU |
AWS | G4dn.2xlarge | 8x Intel Cascade Lake vCPUs, 32GB RAM, Nvidia Tesla T4 GPU |
GCP | N1-Standard-8 + Tesla T4 | 8x Intel Skylake vCPUs, 32GB RAM, Nvidia Tesla T4 GPU |
The aforementioned instance types were selected based on extensive benchmarking performed by Private AI. Please contact Private AI for a copy of the benchmark report.
Note: Please only run one container instance per CPU or GPU. Running multiple containers results in vastly reduced performance!
Prerequisites
The following Prerequisites are required to run the Docker container:
- Docker (can be installed using the official instructions )
- (GPU only) Nvidia Container Toolkit (can be installed using the following installation guide )
Docker Image Setup Instructions
The docker image can be pulled via two methods: from Docker Hub or via an encrypted Docker image export.
Docker Hub
- To pull the image from Docker Hub, please login to Private AI's customer Docker account using the access token provided by Private AI:
docker login -u paiuser -p <Access Token>
- Next, pull the appropriate version of the image:
docker pull privateai/deid:<version>
Docker Image Export
-
GnuPG and wget are required in order to download and decrypt the exported image:
- GnuPG (can be installed using the GnuPG website )
- wget (can be installed using the following installation instructions )
- Download the Docker image file using the following command:
wget <download link>
This will download the file to the current work directory. Please email us for the download link.
- Decrypt the Docker image file using the following command:
gpg private_ai_<version number>.tar.gpg
Enter the decryption password when prompted. This will create a .tar
file in the current working directory.
-
Load the
.tar
file to the Docker engine using the following command:
docker load -i private_ai_<version number>.tar
After setting up the image by following the steps above, the image can be found by the name deid
.
Run Instructions
The CPU container can be run with the following command:
docker run --rm -p 8080:8080 -it deid:<version number>
The command to run the GPU container required an additional --gpus
flag to specify the GPU ID to use:
docker run --gpus <GPU_ID, usually 0> --rm -p 8080:8080 -it deid:<version number>
Note that it is recommended to deploy the container on single GPU machines. For multi-GPU machines, please launch a container instance for each GPU and specify the GPU_ID
accordingly.
For cloud deployment, such as on Azure or AWS, the Private AI DevOps team can provide best practive guides on installation.
Environment Variables
The Private AI container supports a number of environment variables. The environment variables can be set in the Docker run command as follows:
docker run --rm -e <ENVIRONMENT_VARIABLE>=<VALUE> -p 8080:8080 -it deid:<version number>
Supported Environment Variables
Variable Name | Description |
---|---|
LOG_LEVEL |
Controls the verbosity of the container logging output. Allowed values are 'info', 'warning' or 'error'. Default is 'info' |
MARKER_FORMAT |
Allows for the redaction marker foramt to be set globally, instead of passing into each POST request. Please see Processing Text |
ALLOW_LIST |
Allows for the allow list to be set globally, instead of passing into each POST request. An example could be ALLOW_LIST='["John","Grace"]' . Please see Processing Text |
ENABLED_CLASSES |
Allows for the enabled classes to be set globally, instead of poassing it into each POST request. An example could be ENABLED_CLASSES="NAME," or ENABLED_CLASSES="NAME,AGE,ORGANIZATION" . Please see Processing Text |
To change the port used by the container, please set the host port as per the command below:
docker run --rm -p <host port>:8080 -it deid:<version number>
Authentication and External Communications
Private AI's de-identification suite is designed to run entirely on-device, on-premise, or in private cloud. The only outside communications made are for authentication & usage reporting with Private AI's servers. These communications do not contain any customer data - if training data is required, this must be given to Private AI separately. An authentication call is made upon the first API call after the Docker image is started, and again at pre-defined intervals based on your subscription.
An "airgapped" version of the container that doesn't require external communication can be delivered upon request.