Websockets (Beta)

The Private AI container introduced a websocket endpoint in version 3.6.0 to allow users to send requests and receive responses with a single connection. Read more about Websockets here.

info

It is recommended to use the REST endpoints unless there are specific requirements for websocket.

Websockets are available on the Private AI container only.

How to use the websocket endpoint

The websocket endpoint can be used directly from the container if it is deployed as a single instance. The websocket endpoint requires additional setup and consideration in your infrastructure to be usable when the container is served behind a production load balancer.

The websocket endpoint is available in the container by sending a connection request to the /ws endpoint. This can be done with a websocket library such as websocket-client for Python, or simply by utilizing the Websockets API directly in Javascript.

Quickstart example

The following is a simple example written in Python to use the websocket endpoint:

Copy

Copied

import websocket
import json

ws = websocket.WebSocket()

ws.connect("ws://myprivateaicontainer:8000/ws")

payload = {"text": "John Smith lives at 123 Fake St."}
ws.send(json.dumps(payload))
print(ws.recv())
ws.close()

Output:

Copy

Copied

{"processed_text":"[NAME_1] lives at [LOCATION_ADDRESS_1].","entities":[{"processed_text":"NAME_1","text":"John Smith","location":{"stt_idx":0,"end_idx":10,"stt_idx_processed":0,"end_idx_processed":8},"best_label":"NAME","labels":{"NAME_GIVEN":0.4488,"NAME":0.8994,"NAME_FAMILY":0.4596}},{"processed_text":"LOCATION_ADDRESS_1","text":"123 Fake St","location":{"stt_idx":20,"end_idx":31,"stt_idx_processed":18,"end_idx_processed":38},"best_label":"LOCATION_ADDRESS","labels":{"LOCATION_ADDRESS":0.9486,"LOCATION":0.913}}],"entities_present":true,"characters_processed":32,"languages_detected":{"en":0.8259078860282898}}

Websocket Payloads

The websocket endpoint expects a JSON payload just like the REST API in exactly the same format with all the of the parameters from the REST API supported in the same way. The main difference with the websocket endpoint is that the text parameter does NOT support a list; we're expecting single request and response exchanges with the container!

Below are some examples of payloads and their expected outputs:

Copy

Copied

# Disabling the NAME entity type and subtypes resulting in no redaction of detected names
payload = {"text": "Hello, my name is Greg", 
            "entity_detection":
                {"entity_types":
                [{"type":"DISABLE", "value":["NAME", "NAME_GIVEN", "NAME_FAMILY"]}]}}

>>>>
{"processed_text": "Hello, my name is Greg" ...}

Copy

Copied

# Setting the processed text type to MASK and masking entities.
payload = {"text": "Hello, my name is John Smith.",
            "processed_text":
                {"type":"MASK", "mask_character":"$"}}

>>>>
{"processed_text": "Hello, my name is $$$$ $$$$$." ...}