Performance Tips

Concurrency and Batching

The following code examples outline how to increase throughput when using Private AI.

Please visit the recommended concurrency levels page to set the number of concurrent requests optimally.

Context

Private AI relies on Machine Learning to detect PII based on context, instead of pattern matching approaches such as regular expressions. Therefore, for best performance it is advisable to send text through in the largest possible chunks that still meet latency requirements. For example, the following chat log should be sent through in one call with link_batch enabled, as opposed to line-by-line:

"Hi John, how are you?"

"I'm good thanks"

"Great, hope Atlanta is treating you well"

The Batch Requests code example provided above shows how to implement this.

Similarly, text documents should be sent through in a single request, rather than by paragraph or sentence. In addition to improving accuracy, this will minimize the number of API calls made.

Capitalization

The PII detection models are optimised for normal English capitalization, e.g. "Robert is from Sydney, Australia. Muhab is from Wales". If this is not the case for your data, please contact Private AI so that we can provide you with the optimal model for your use case. Our solution will still work, but some performance will be lost.

This being said, Private AI is optimized for processing text with emojis and text containing ASR transcription errors.

ASR Transcripts

When processing audio transcripts, it is recommended to use the following input format:

"<speaker id>: <message>, <speaker id>: <message>,"

Model Tuning

Private AI's PII detection models generally perform well out of the box. However, model tuning may be beneficial in order to tailor our solution to unique use cases. The tuning process is outlined below.

1. Client-supplied Data Sample

Prepare a sample of at least 20 examples illustrating the problem to be addressed. See the table at the bottom of this page for examples. The sample should contain examples that are:

• anonymized such that all PII is manually replaced with synthetic entities
• long enough to provide enough context, ideally including ~30 words before and after the problematic entity
• diverse enough to be representative of the issues encountered

2. Secure Data Transfer

Data samples can be shared with Private AI via a secure transfer mechanism built on Microsoft Azure. Contact us for an access token.

3. Manual Data Anonymization

Private AI's data team will again manually anonymize the provided data, to ensure that no personal data is ever stored or used for training.

4. Model Tuning & Delivery

Using few-shot learning techniques, Private AI improves the PII Detection models by optimizing for your use case. Updated models are released via a new container version. Updates are usually delivered in the next regular release, but can be delivered via a patch release in as little as 72 hours, depending on the SLA and severity of the problem.

Example Data Samples

*Note that these examples are used only for illustration. Private AI correctly picks up the PII in each of these cases.