Version 3.0 Migration Guide

Customer Portal

Version 3.0 is launching together with our new customer portal, instructions to get access will be shared by our Customer Success team. Inside the portal you can find usage information, a download for your license file and instructions on how to login to our container registry and pull the latest container image.

If you haven't received this information yet, please contact our customer support team at support@private-ai.com

Container Distribution

Starting with 3.0, we will be distributing our container exclusively through the Azure Container Registry. Login credentials and sample commands to download the container image can be found in the customer portal and will look like:

docker login -u INSERT_UNIQUE_CLIENT_ID -p INSERT_UNIQUE_CLIENT_PW crprivateaiprod.azurecr.io

Licensing Change

We have changed our licensing system from an API Key to a license file, which can be downloaded from the customer portal. In order to run the container with the license file, please mount the license file to /app/license/license.json like this:

docker run --rm -v "full path to license.json":/app/license/license.json \
-p 8080:8080 -it crprivateaiprod.azurecr.io/deid:<version>

Once you have the container up and running with the new license file, you can send the container a request like this:

{
    "text": [
        "Hello John"
    ]
}

curl --request POST --url http://localhost:8080/v3/process/text --header 'Content-Type: application/json' --data '{"text": ["Hello John"]}'

import requests

r = requests.post(url="http://localhost:8080/v3/process/text",                  
                  json={"text": ["Hello John"]})

results = r.json()

print(results)

from privateai_client import PAIClient
from privateai_client import request_objects

client = PAIClient(url="http://localhost:8080")

text_request = request_objects.process_text_obj(text=["Hello John"])
response = client.process_text(text_request)

print(response.processed_text)

New API Interface

Version 3.0 introduces many changes to the API, please see the new API Reference. Key changes:

• deidentify_text is now called /v3/process/text
• Endpoints in general now follow the standard of process/type/subtype
• text field is required to be a list by default, even with a single string
• key field has been removed from the body and is now an optional request header: X-API-KEY . It is only required when using our cloud API. When using an on-prem container, you do not need an API key.
• accuracy_mode is now called accuracy and can be found one layer down in the entity_detection dictionary settings
• return_entities parameter allows you to configure whether to include identified entities in the response
• unique_pii_markers has been removed. Instead, please set pattern inside the marker parameters to UNIQUE_NUMBERED_ENTITY_TYPE
• Entity is established in nomenclature to recognize PII, PHI, PCI

Example conversions from V2 request payload to V3:

###
Example with enabled_classes
###
2.x:
{"text": "Hello there John!", 
  "key":<My_api_key>, 
  "accuracy_mode":"high", 
  "enabled_classes":["NAME"]
}

3.0:
{"text": ["Hello there John! I live in Newark"],
 "entity_detection":
   {"accuracy": "high",
    "entity_types": [{"type": "ENABLE", "value":["NAME"]}]
  }
}

-----------------------------------------------------------------------------------------
###
Example with inclusion of all entity types in entity marker
###
2.0:
{"text": "Hello there Pieter!", 
"key":<My_api_key>,
"accuracy_mode":"standard",
"marker_format": "[ALL_CLASS_NAMES]"
}

3.0:
{"text": ["Hello there Pieter!"], 
  "entity_detection": {"accuracy": "standard"},
  "processed_text": {"type": "MARKER", "pattern": "[ALL_ENTITY_TYPES]"}
}


-----------------------------------------------------------------------------------------
###
Example with disabling unique_pii_markers through MARKER definition 
###
2.0:
{"text": "Hello there Paul!", 
"key":<My_api_key>,
"accuracy_mode":"high_multilingual",
"unique_pii_markers": false
}

3.0:
{"text": ["Hello there Paul!"], 
  "entity_detection": {"accuracy": "high_multilingual"},
  "processed_text": {"type": "MARKER", "pattern": "[BEST_ENTITY_TYPE]"}
}

File Support for PDFs / Images

3.0 supports file redaction using an unified endpoint, which works either with URIs or base64-encoded files: /v3/process/files/uri and /v3/process/files/base64. Please see the Quickstart Guide for details.

Environment Variables

All previous environment variables are now prefixed with “PAI” to better differentiate PAI specific variables. You can find the full list of environment variables in Environment Variables.

PII Metrics

In 3.0, non-airgapped users can enable PII metrics gathering for reporting purposes. In order to do this, add PAI_ENABLE_PII_COUNT_METERING=True as an environment variable. You'll be able to see the amount of PII captured by your license usage. In the future, we will be improving this feature to provide you with a granular view on entity types captured and other reporting features.

Please note that this feature is OFF by default and requires explicit configuration to gather this data. Any usage prior to enabling this feature is not captured and cannot be reported on retroactively.