Skip to content
APEX Document Management

Check and measure a form

Extract the data of one form

By the end of this lesson, you can build a test set, measure a form against it, add check expressions and field patterns, and read in the measurement whether a change helped.

Lesson 2 of 6~35 minScript: 02_seed_test_set.sql, 02_measure.sql, 02_report.sql, 02_add_checks.sql, 02_fix_line_check.sql, 02_add_patterns.sql, 02_try_checks.sql, 02_set_reasoning_level.sql

Lesson 1 left the profile pfx_supplier_forms with the document type invoice, the form HARBOUR_INVOICE and one result, for harbour-invoice-01-standard.pdf. This lesson starts from that state. It changes the document type and the form, and it adds a test set and measurements. It does not change the result of lesson 1.

Check a result without the correct values

Section titled “Check a result without the correct values”

ADM must decide for each new document whether a person must read it. For a new document, nobody knows the correct values. The checks use only the result itself. adm_ai_form_config_api.check_result runs two kinds of checks after the model answered:

  • Field patterns of the form. Each field in field_patterns must match its regular expression. A Harbour customer number looks like C-10233, so the pattern is ^C-[0-9]{5}$. A missing value fails the check too. A key like line_items.item_number checks that column in every line.
  • Check expressions of the document type. A check is a SQL condition on the fields. line_checks run once for each line item, and checks run once for the document. A condition that is false or null is an issue, with the name of the check.

A result with an issue gets the status REVIEW. Lesson 1 gave the Harbour invoice three field patterns and no check expressions.

The field patterns are checks, not part of the JSON schema. A pattern in the schema makes the model write every value in that form, also a value that it cannot read. A check after the answer finds a value in the wrong form instead.

Build the test set

Section titled “Build the test set”

A check decides what happens to one document. A measurement shows how good the checks and the model are on many documents. It needs a test set: documents whose correct values a person wrote down. A test case holds the document, its form, and the values and line items that are right. The test set belongs to the profile. A document that is no form is a test case too: its correct result is that it goes to a person.

Typing the values of 15 documents takes long, so the course gives you a script. Run @02_seed_test_set.sql. It writes one test case for each sample, with the source SAMPLE. A test case names its form, so the script skips the documents whose form does not exist yet. Lesson 3 runs it again.

test set: 15 document(s) written
skipped, because the form does not exist yet: 8 document(s)

The 15 test cases are the 13 Harbour invoice files, the image of the scan, and the letter harbour-letter-prices.pdf. The letter has no form.

Open the tab Test set of the profile. It lists the test cases with their form and their source. Choose the name of a document to see its values.

The tab Test set of the profile pfx_supplier_forms with the 15 test cases of the course
Select the image to open it in full size.

A test case page has the form and one field for each field of the document type, and a table for the line items. The credit note shows negative amounts.

The test case of harbour-invoice-09-credit-note.pdf with the form HARBOUR_INVOICE, its fields and one line item with a negative amount
Select the image to open it in full size.

You add a test case in three ways. Add document on the tab Test set lets you choose a document and type its values. Save as test case on the preview of Try a document keeps the values that the model read, and you correct them. The same button is on the result page of a document. Check every value that you keep: the model must not be the source of the correct answer.

Measure the form against the test set

Section titled “Measure the form against the test set”

A measurement runs the extraction for every selected test case and compares the result with the test case. It writes one row for each document to adm_ai_form_measurements, and nothing else: no result, no annotation, no tag. Each row gets one of five outcomes:

Outcome The values are The checks What it means for the extraction
RIGHT right passed The document needs no person.
FALSE_ALARM right failed A person reads a correct document.
CAUGHT wrong failed A person corrects the document.
MISSED wrong passed A wrong value goes into ADM, and nobody looks at it.
FAILED An error stopped the extraction.

MISSED is the number that decides whether people can trust the OK results. A MISSED result needs a new check. A CAUGHT result needs a better rule, or it stays a task for a person. A run has a label. The label names the run, and each run needs a new label. Use a label that says what you changed.

The first measurement uses the form HARBOUR_INVOICE and the documents whose name ends in .pdf. That leaves out the image of the scan, which lesson 4 measures. The name filter is a like pattern: % stands for any text. The Manager labels the field Name contains, but it does not add the % for you. A text without % matches only a document with exactly that name.

  1. Open the tab Measurements and choose Run measurement.
  2. Enter the Label baseline. Choose the Form HARBOUR_INVOICE. Enter %.pdf in the field Name contains. Leave the models, Prompt With rules and Reasoning Model default as they are.
The form Run a measurement with the label baseline, the form HARBOUR_INVOICE and the name filter %.pdf
Select the image to open it in full size.
  1. Choose Start. The measurement runs as a background job. The profile page shows its progress, and the run appears in the list when it starts. Choose the label of the run when it has finished.
The run baseline: 12 right, 1 caught, and the 13 documents with their outcomes
Select the image to open it in full size.

The run page shows the counts of each outcome, the tokens and the seconds. The table lists each document with its outcome, the form that was found, the wrong fields and the first issue.

 
DOCUMENT_NAME                              OUTCOME     SECONDS DETAILS
------------------------------------------ ----------- ------- ------------------------------------------------------------------------------------------------------------------------
harbour-invoice-12-scanned.pdf             CAUGHT         0.0  wrong: form none | issues: the document has no text (8 characters); a person must read it


DOCUMENTS  RIGHT FALSE_ALARM CAUGHT MISSED FAILED MEDIAN_S COMPLETION_TOKENS
--------- ------ ----------- ------ ------ ------ -------- -----------------
       13     12           0      1      0      0    1.8                1261

Twelve of the 13 files are RIGHT. The model read every invoice with text correctly, and the checks of lesson 1 passed. The thirteenth file, harbour-invoice-12-scanned.pdf, is CAUGHT. The PDF has no text. ADM calls no model for it and sends it to a person. Lesson 4 explains scans.

No result is MISSED. This is a statement about these 13 files and this model. It does not show that the checks work, because no value was wrong. The next sections add checks, and they test the checks in two ways: with a measurement for a check that fires when it must not, and with a wrong value that you enter yourself.

Add check expressions to the document type

Section titled “Add check expressions to the document type”

The amounts of an invoice depend on each other: the lines add up to the net amount, and the net amount plus the VAT gives the total. These are the rules BR-CO-13 and BR-CO-15 of EN 16931, the European standard for electronic invoices. A check that compares them finds a value that the model misread, because the model read each value from another place of the page.

The first version has one line check and three document checks:

{
  "line_checks": [
    {"name": "quantity x unit price = amount",
     "expr": "abs(quantity * unit_price - amount) <= 0.01"}
  ],
  "checks": [
    {"name": "sum of lines - discount = net amount",
     "expr": "abs(sum_amount - nvl(discount_amount, 0) - net_amount) <= 0.01"},
    {"name": "net amount + VAT = total",
     "expr": "abs(net_amount + vat_amount - total_amount) <= 0.01"},
    {"name": "net amount x VAT rate = VAT",
     "expr": "abs(round(net_amount * vat_rate / 100, 2) - vat_amount) <= 0.01"}
  ]
}

A line check sees the columns of one line item. A document check sees the fields of the document and the aggregates of the line items: line_count, and sum_<column>, min_<column> and max_<column> for each number column. sum_amount is the sum of the line amounts.

The expressions run as SQL in the schema of ADM. For this reason, the API accepts only a short list of tokens: the field names, numbers, strings, the operators + - * / ( ) , = <> != < <= > >=, the keywords and or not is null between in, and the functions abs round trunc sign nvl coalesce greatest least mod. It refuses every other token when you save the document type. Write check expressions in the guide has the full list.

  1. On the tab Document types & forms, choose the document type invoice.
  2. In the section Line checks, choose Add line check. Enter the name and the expression of the line check above.
  3. In the section Checks, choose Add check three times, and enter the name and the expression of each document check.
  4. Below each expression, the manager asks the server to check it while you type. A valid expression shows the SQL that runs. An expression with a token that is not allowed shows the reason.
The sections Checks and Line checks of the document type invoice. Each expression shows the SQL that ADM runs
Select the image to open it in full size.
  1. Choose Save.

Measure again with the label checks-v1. In the Manager, choose Run measurement again, enter the label and the same form and name filter. In SQL, run:

@02_measure.sql checks-v1 HARBOUR_INVOICE %.pdf rules N
DOCUMENT_NAME                              OUTCOME     SECONDS DETAILS
------------------------------------------ ----------- ------- ------------------------------------------------------------------------------------------------------------------------
harbour-invoice-09-credit-note.pdf         FALSE_ALARM    1.9  issues: line 1: quantity x unit price = amount
harbour-invoice-12-scanned.pdf             CAUGHT         0.0  wrong: form none | issues: the document has no text (8 characters); a person must read it


DOCUMENTS  RIGHT FALSE_ALARM CAUGHT MISSED FAILED MEDIAN_S COMPLETION_TOKENS
--------- ------ ----------- ------ ------ ------ -------- -----------------
       13     11           1      1      0      0    1.8                1340
The run checks-v1 in the Manager: 11 right, 1 false alarm and 1 caught, with the issue of the credit note in the table
Select the image to open it in full size.

The check now fires on a correct document. The credit note is FALSE_ALARM: the values are right and the check quantity x unit price = amount failed. A credit note prints a positive quantity and a positive unit price, and the amount is negative: 1 × 142.50 and -142.50. The check did not allow for that sign. Without the measurement, this check would send every credit note to a person, and nobody would know why.

Compare the amounts without their signs. The document check sum of lines - discount = net amount still uses the signs, so a wrong sign in the totals stays visible.

Open the document type invoice again. In the section Line checks, replace the expression with abs(abs(quantity * unit_price) - abs(amount)) <= 0.01 and choose Save.

The line check quantity x unit price = amount with the corrected expression and the SQL that ADM runs
Select the image to open it in full size.

Measure with the label checks:

@02_measure.sql checks HARBOUR_INVOICE %.pdf rules N
DOCUMENT_NAME                              OUTCOME     SECONDS DETAILS
------------------------------------------ ----------- ------- ------------------------------------------------------------------------------------------------------------------------
harbour-invoice-12-scanned.pdf             CAUGHT         0.0  wrong: form none | issues: the document has no text (8 characters); a person must read it


DOCUMENTS  RIGHT FALSE_ALARM CAUGHT MISSED FAILED MEDIAN_S COMPLETION_TOKENS
--------- ------ ----------- ------ ------ ------ -------- -----------------
       13     12           0      1      0      0    1.7                1353

FALSE_ALARM is 0 again, and the credit note is RIGHT. The checks accept all 12 correct invoices.

Add field patterns to the form

Section titled “Add field patterns to the form”

A field pattern checks the format of one value. Lesson 1 gave the form patterns for the invoice number, the customer number and the date. Two more cover values that arithmetic does not reach. A discount is a positive number, and the document prints it with a minus sign (-39,37). Every item number looks like a Harbour item number. The second pattern matters for the image of lesson 4, where the model reads the item numbers too.

Field Pattern
discount_amount ^[0-9.]+$
line_items.item_number ^[A-Z]{3}-[A-Z0-9-]+$
  1. On the tab Document types & forms, choose the form HARBOUR_INVOICE.
  2. In the section Field patterns, choose Add pattern. Choose the field discount_amount and enter its regular expression.
  3. Choose Add pattern again. Choose the field line_items.item_number and enter its regular expression.
The section Field patterns of the form HARBOUR_INVOICE with five patterns
Select the image to open it in full size.
  1. Choose Save.

Measure with the label patterns:

@02_measure.sql patterns HARBOUR_INVOICE %.pdf rules N
DOCUMENT_NAME                              OUTCOME     SECONDS DETAILS
------------------------------------------ ----------- ------- ------------------------------------------------------------------------------------------------------------------------
harbour-invoice-12-scanned.pdf             CAUGHT         0.0  wrong: form none | issues: the document has no text (8 characters); a person must read it


DOCUMENTS  RIGHT FALSE_ALARM CAUGHT MISSED FAILED MEDIAN_S COMPLETION_TOKENS
--------- ------ ----------- ------ ------ ------ -------- -----------------
       13     12           0      1      0      0    1.6                1378

The totals do not change: 12 RIGHT, 0 FALSE_ALARM, 0 MISSED. The new patterns accept every correct value of the test set.

Make a check refuse a wrong value

Section titled “Make a check refuse a wrong value”

On this test set, the model made no mistake that a check could catch. To see what the checks report, you enter a wrong value yourself. Approving a result is what a reviewer does in lesson 5. The reviewer can correct values, and ADM runs the checks on the corrected values. It refuses the approval and changes nothing when a check fails. Use this to try a wrong value on the result of lesson 1.

  1. On the tab Results, choose harbour-invoice-01-standard.pdf.
  2. In the review section, change the value of total_amount from 557.63 to 575.63, as if two digits had changed places. The number fields use the number format of your browser.
  3. Clear the check box Add to the test set, and choose Approve. The manager shows the issue and approves nothing.
The result page of harbour-invoice-01-standard.pdf with a changed total amount and the issue net amount + VAT = total after Approve
Select the image to open it in full size.

Reload the page to discard your change. Enter only a value that fails a check: ADM approves a value that passes all checks.

 
Corrected value: {"total_amount": 575.63}
  ORA-20619: The corrected values fail the checks: net amount + VAT = total
Corrected value: {"invoice_date": "14.03.2026"}
  ORA-20619: The corrected values fail the checks: invoice_date does not match ^20[0-9]{2}-[01][0-9]-[0-3][0-9]$: "14.03.2026"
Corrected value: {"discount_amount": -39.37}
  ORA-20619: The corrected values fail the checks: discount_amount does not match ^[0-9.]+$: -39.37; sum of lines - discount = net amount

Each issue names what failed. The total fails the check net amount + VAT = total. The date does not match the field pattern of lesson 1, and the issue shows the value that failed. The discount fails the new field pattern and the check sum of lines - discount = net amount. The pattern says what is wrong with the value. The check shows that the totals no longer add up. A result that the model reads wrong gets the same issues, and the status REVIEW.

Measure the form without its rules

Section titled “Measure the form without its rules”

The rules of a form cost work: somebody reads the documents and writes them. To see what they give on your documents, measure the prompt variant naive. It keeps the description of the form and the JSON schema, and it leaves out all rules of the profile, the document type and the form. The checks stay the same.

Choose Run measurement. Enter the label naive, the form HARBOUR_INVOICE and the name filter %.pdf. Choose the Prompt Naive and choose Start.

The form Run a measurement with the label naive and the prompt Naive
Select the image to open it in full size.
 
DOCUMENT_NAME                              OUTCOME     SECONDS DETAILS
------------------------------------------ ----------- ------- ------------------------------------------------------------------------------------------------------------------------
harbour-invoice-12-scanned.pdf             CAUGHT         0.0  wrong: form none | issues: the document has no text (8 characters); a person must read it


DOCUMENTS  RIGHT FALSE_ALARM CAUGHT MISSED FAILED MEDIAN_S COMPLETION_TOKENS
--------- ------ ----------- ------ ------ ------ -------- -----------------
       13     12           0      1      0      0    2.3                1677

All 12 invoices with text are RIGHT without rules. The completion tokens are 1,677 here and 1,378 with the rules (label patterns); the recordings do not show why. For gpt-5.6-luna, the description of the form and the schema are enough on these 13 files: the German number format, the dates and the BILL TO block need no rule. A weaker model, or a layout that is harder to read, can need them. This measurement does not show that.

The result changes what you do next. Do not add a rule for a field that the measurement shows as right. When a measurement shows a wrong value, add the rule at the lowest level where it is true, run the measurement again with a new label, and compare the two runs. A rule that states what a field means, for example that the customer is the BILL TO company, can stay: a test set of 13 documents cannot show every layout that the issuer prints.

Measure a model with reasoning

Section titled “Measure a model with reasoning”

Some models can reason before they answer: they write out their thinking first, and the answer comes after it. The profile does not turn reasoning on. The reasoning switch of a measurement turns it on or off for that run, and the reasoning level of the profile sets how much the model reasons. This measurement uses three invoices: harbour-invoice-%-d% matches the ambiguous dates, the other delivery address and the discount.

  1. Choose Run measurement. Enter the label reasoning-off, the form HARBOUR_INVOICE and the name filter harbour-invoice-%-d%. Choose Reasoning Off and Start.
  2. Run the measurement again with the label reasoning-on and Reasoning On.
  3. Open the profile with Edit. In the section How documents are read, open Advanced settings of the text model and choose the Reasoning level high. Choose Save.
The Advanced settings of the text model with the reasoning level high
Select the image to open it in full size.
  1. Run the measurement a third time with the label reasoning-high and Reasoning On.

All three runs read the three invoices right. The totals of 02_report.sql give these values:

Run Reasoning Reasoning level RIGHT Median seconds Completion tokens
reasoning-off Off – 3 of 3 2.3 317
reasoning-on On Default 3 of 3 1.6 317
reasoning-high On high 3 of 3 2.7 449

With the default level, the run with reasoning on has the same token count as the run with reasoning off, and the same result. With the level high, the model wrote 449 tokens for the three documents instead of 317, and the median time was 2.7 seconds. The median time of the run with reasoning on is shorter than that of the run with reasoning off. With three documents, this difference is within the spread of single calls: they took between 1.4 and 2.8 seconds in the three runs. Reasoning gave no better result on these invoices, because every invoice was already right. Measure it on your own forms before you turn it on.

After the measurements, set the level back. In the Manager, choose the Reasoning level Default in the profile. In SQL, run @02_set_reasoning_level.sql none.

Decide when a form is good enough

Section titled “Decide when a form is good enough”

Measure a form until both numbers are good enough for your process:

  • MISSED is 0 on the test set. Each MISSED result is a value that goes into ADM unchecked. Add a check for it, or accept the risk for that field.
  • The share of documents that need a person is low enough. That share is CAUGHT plus FALSE_ALARM. Lower it with rules and with better checks.

A form whose result stays wrong after a few rules is still useful: its documents go to a person, with the fields that the model read correctly as a start.

A test set of 13 documents is small. On these documents, gpt-5.6-luna made no mistake with the rules, and none without them. That does not prove that a form is always right. Measure again after each change of a rule, a check or the model, and add the documents that people correct (lesson 5).

Next: Add a formRead the text of a new form, write its markers, rules and patterns, add a document type, test the markers against all documents, and measure the forms.