Check text

POST /api/v1/check

Send the request to https://api.jp.flowmask.ai/api/v1/check.

Download the OpenAPI spec. The file describes POST /api/v1/check in the OpenAPI format. Import it into an API client or a code generator.

Request headersCopy Anchor Link

Request bodyCopy Anchor Link

The body is a JSON object with one field.

{ "text": "Contact Nguyễn Văn An at 0912 345 678 or [email protected]." }

Limits of the textCopy Anchor Link

• text must contain at least one character that is not white space. Unicode white space counts as white space, so a text of only U+3000 is empty.
• text can have up to 16,000 input tokens. A token is not a character. See Input tokens.
• The body can have up to 4,194,304 bytes, which is 4 MiB.
• The Check API does not change the text before it checks it. It does not normalize Unicode and does not trim white space.

A request that breaks a rule gets 400 validation_failed. Errors lists the field errors.

Input tokensCopy Anchor Link

Today the Check API counts the input tokens of text as the UTF-8 bytes of text.

• An ASCII character, such as an English letter, a digit, or a space, is 1 token.
• A Japanese character is usually 3 tokens.
• A Vietnamese letter with a diacritic, such as ă or ễ, is 2 or 3 tokens when it is one code point (NFC), and up to 5 when it is written with combining marks (NFD).

So the limit of 16,000 input tokens is about 16,000 ASCII characters, or about 5,300 Japanese characters.

The way the Check API counts can change. The inputTokens field of a 200 is the count that the Check API used, and the monthly quota counts the same input tokens. Rely on inputTokens, not on your own count.

To stay under the limit, split a long text into parts at paragraph or sentence boundaries, and send each part in its own request. The positions in each response count from the start of its part. Still handle text_too_long, because the count can change.

Unknown fieldsCopy Anchor Link

The Check API ignores every top-level field other than text. A 200 response then names the ignored fields in the X-Ignored-Fields header.

X-Ignored-Fields: lang, userId

• The header sorts the names and separates them with a comma and a space.
• The header holds at most 10 names. When the body has more, the header keeps the first 10 in sorted order.
• The header leaves out a name that is longer than 64 characters. It also leaves out a name with a character other than an ASCII letter (A to Z, a to z), a digit, _, ., or -.
• The header is absent when the body has no unknown field.

ResponseCopy Anchor Link

A 200 response has the fields data, code, and message. code is http.200 and message is OK.

Text with personal dataCopy Anchor Link

{
  "data": {
    "result": "UNSAFE",
    "confidence": 0.97,
    "findings": [
      {
        "text": "Nguyễn Văn An",
        "start": 8,
        "end": 21,
        "type": "person",
        "group": "BASIC"
      },
      {
        "text": "0912 345 678",
        "start": 25,
        "end": 37,
        "type": "phone_number",
        "group": "BASIC"
      },
      {
        "text": "[email protected]",
        "start": 41,
        "end": 62,
        "type": "email_address",
        "group": "BASIC"
      }
    ],
    "maskedText": "Contact [PERSON] at [PHONE] or [EMAIL].",
    "inputTokens": 66,
    "characters": 63,
    "requestId": "req_01M6JY9PD89FTH4BSM8NACAEYY",
    "region": "JP",
    "apiVersion": "v1"
  },
  "code": "http.200",
  "message": "OK"
}

Text without personal dataCopy Anchor Link

{
  "data": {
    "result": "SAFE",
    "confidence": 1,
    "findings": [],
    "maskedText": "Summarize the attached meeting notes in three bullet points.",
    "inputTokens": 60,
    "characters": 60,
    "requestId": "req_01M6JY9QCGVF0V9J80ZZJ8TNTV",
    "region": "JP",
    "apiVersion": "v1"
  },
  "code": "http.200",
  "message": "OK"
}

Fields of dataCopy Anchor Link

The Check API looks only for the data types that are on in your workspace. It reports no finding for data of a type that is off, and maskedText keeps that data as it is. A text whose personal data is all of types that are off gets SAFE.

Positions count code pointsCopy Anchor Link

start, end, and characters count Unicode code points. They do not count bytes or UTF-16 code units.

In Python, a string index counts code points, so text[start:end] returns the finding. In JavaScript, a string index counts UTF-16 code units. A character outside the Basic Multilingual Plane, such as an emoji, takes two code units. After such a character, text.slice(start, end) returns the wrong characters.

For example, in the text 😀 [email protected], the email address has start 2 and end 23. text.slice(2, 23) returns " [email protected]". Split the text into code points before you slice it.

const codePoints = Array.from(text);
for (const finding of data.findings) {
  const found = codePoints.slice(finding.start, finding.end).join('');
  console.log(finding.type, found);
}

Count positions on the exact string that you sent. A letter written as a base letter and combining marks is more than one code point. If your app normalizes the text after the check, the positions no longer match.

ErrorsCopy Anchor Link

This endpoint returns the errors in Errors. Rate limits and quota explains the two 429 errors.