Skip to content
APEX Document Management

Add a form

Check and measure a form

By the end of this lesson, you can add one of your own forms: read its text, write its markers, rules and patterns, add a document type with its checks, make sure that no two forms match one document, and measure the forms together.

Lesson 3 of 6~35 minScript: 03_show_text.sql, 03_add_forms.sql, 02_seed_test_set.sql, 03_test_markers.sql, 03_set_markers.sql, 02_measure.sql, 02_report.sql

Lesson 2 left the profile pfx_supplier_forms with the document type invoice (with its checks), the form HARBOUR_INVOICE, 15 test cases and the measurements of that lesson. This lesson starts from that state. It adds two forms and one document type: the freight invoice of Nordwind Logistics, and the delivery note of Harbour Office Supplies. At the end, the test set has 23 test cases, and the profile has the measurements nordwind, all-pdf, all-naive and weak-marker.

Steps to add a form

Section titled “Steps to add a form”

Do the same steps for each of your own forms:

  1. Collect sample documents of the form and write down their correct values.
  2. Read the text that ADM extracts from them.
  3. Choose markers that only this form matches.
  4. Decide whether the form uses an existing document type or needs a new one.
  5. Write the rules and the field patterns. Decide whether a line pattern or the model reads the line table.
  6. Test the markers of all forms against all documents.
  7. Measure the form, and correct it until it is good enough (lesson 2).

Collect sample documents

Section titled “Collect sample documents”

Collect 5 to 10 documents of the form. Choose documents that differ, not ten similar ones. Look for these cases:

  • A document with a value that is missing, for example no order number.
  • A credit note or a cancellation, with negative amounts.
  • A long document with several pages.
  • A document with a discount, a surcharge or a special VAT rule.
  • A document with numbers of other documents in its notes or in the payment text.
  • A long name or address that the layout wraps onto two lines.

Write down the correct values of each document. For the course, samples/pdf/expected.json holds them, and 02_seed_test_set.sql writes them into the test set. For your own forms, extract the documents, correct the results and approve them (lesson 5): every approved document goes into the test set.

Read the text of the new form

Section titled “Read the text of the new form”

The markers and the rules work on the text that ADM extracts from the PDF, so read it first.

  1. Open the profile and the tab Document types & forms, and choose Create form.
  2. Enter the Form code NORDWIND_INVOICE, choose the Document type invoice, and enter the Description a freight invoice of Nordwind Logistics B.V., Rotterdam.
  3. Go to the section Test the form on a document. The button Choose a document works after you have chosen a document type. Choose it, search for nordwind-invoice-01-standard.pdf, and choose Choose in its row.
  4. The section Text shows the text. Text the markers see is the text that the markers match against, and Raw text is the text as the extraction returned it.
The section Test the form on a document with nordwind-invoice-01-standard.pdf: no marker yet, and the first lines of the text
Select the image to open it in full size.

The form has no marker yet, so the test shows no marker result.

Both ways show the same text. A label and its value are often on separate lines: Invoice No: is on one line and NWL/26/10482 on the next. Compare the text with the fields of the document type invoice, and note each place where the model can go wrong:

  • The customer number is labeled Account:, and the order reference is labeled Your PO:.
  • Shipment: holds a number that looks like a reference, but it is a number of the carrier.
  • Numbers use the English format 1,234.56, and dates look like 14 Mar 2026.
  • The line table has the column Code instead of Item no.

Each of these points becomes a rule.

Choose markers that only this form matches

Section titled “Choose markers that only this form matches”

A marker is a regular expression. A form matches a document when all its markers match the text. A marker must match every document of the form and no document of another form. Good markers are values that only the issuer prints, and only on this kind of document:

  • The identity of the issuer, for example its VAT ID or registration number. A company name is weaker: other documents mention it too. The Harbour delivery notes name the carrier Nordwind Logistics B.V., but they do not contain its VAT ID.
  • The title of the document, as a whole line. Harbour prints invoices and delivery notes with the same letterhead. Only the title line INVOICE or DELIVERY NOTE tells them apart. The pattern ^DELIVERY NOTE$ matches only a line with exactly this text: ^ and $ match at the start and at the end of each line.

The two new forms use these markers:

NORDWIND_INVOICE        "markers": ["VAT NL854122736B01", "^FREIGHT INVOICE$"]
HARBOUR_DELIVERY_NOTE   "markers": ["VAT ID DE 812 334 905", "^DELIVERY NOTE$"]

Decide between a new form and a new document type

Section titled “Decide between a new form and a new document type”

A new issuer or a new layout always gets a new form, because the form holds the markers and the layout rules. The document type depends on the data that your application needs from the document:

  • If the document returns the same fields as an existing document type, the form uses that document type. It gets the JSON schema and the check expressions of that document type, and you write only the configuration of the form. The Nordwind invoice is such a form: it uses the document type invoice of lesson 1.
  • If the document returns other fields, create a new document type first, with its own JSON schema and checks. Then create the form for it. The Harbour delivery note is such a form: a delivery note has delivered quantities, not amounts and VAT.

A document type with one form is normal. A second form of the same document type costs less work, because the schema and the checks already exist.

Add a second layout of a document type

Section titled “Add a second layout of a document type”

The Nordwind invoice is a second layout of the document type invoice. It returns the same fields as the Harbour invoice, so it needs no new schema. The rules of the profile and of the document type apply to it, and so do the check expressions of lesson 2. The figure shows the same fields on both invoices. The Nordwind invoice prints them at other places, with other labels and with the English number and date format:

The fields of the document type invoice, marked on an invoice of Harbour Office Supplies and a freight invoice of Nordwind Logistics
Select the image to open it in full size.

The configuration of the form describes what is different on its page:

{
  "markers": ["VAT NL854122736B01", "^FREIGHT INVOICE$"],
  "field_patterns": {
    "invoice_number":         "^NWL/[0-9]{2}/[0-9]{5}$",
    "customer_number":        "^400-[0-9]{4}$",
    "invoice_date":           "^20[0-9]{2}-[01][0-9]-[0-3][0-9]$",
    "discount_amount":        "^[0-9.]+$",
    "line_items.item_number": "^[A-Z]{3,4}(-[A-Z]{3})?$"
  },
  "rules": [
    "The customer is the company under \"Invoice to:\".",
    "The customer number is the value of \"Account:\". The order reference is the value of \"Your PO:\"; \"n/a\" means there is none: return null.",
    "\"Shipment:\" is a shipment number of the carrier, not an order reference.",
    "Numbers use English format: \",\" groups thousands and \".\" is the decimal separator. \"1,234.56\" is 1234.56.",
    "Dates are printed like \"14 Mar 2026\".",
    "There is no discount: return 0.",
    "\"VAT 0% (reverse charge ...)\" means a VAT rate of 0."
  ]
}

Write a field pattern for each identifier, for each date and for the sign of each amount. The patterns catch a wrong value that the check expressions cannot catch. The form has no line_pattern, so the model reads its line table. A table of a few lines is short work for the model, and the check expressions check every line. Write a line pattern for a form whose tables can be long, as on the Harbour invoice: a pattern reads them faster and copies every value exactly.

  1. In the form that you started in the previous section, open the tab Advanced (JSON). Replace the text of the field Configuration with the JSON above, and choose Apply to form.
  2. Return to the tab Form. The sections Markers, Field patterns and Rules now hold the values of the JSON.
  3. Applying the JSON clears the document of the test. Choose nordwind-invoice-01-standard.pdf again. Both markers match. The line Forms that match now lists saved forms only, so it says None until you save this form.
The markers of the form NORDWIND_INVOICE and the test on nordwind-invoice-01-standard.pdf: both markers match
Select the image to open it in full size.
  1. Choose Create form.

Add a new document type

Section titled “Add a new document type”

The delivery note is a new document type. It gets its own JSON schema, with the fields of a delivery note and a line table with the delivered quantities. Its check expressions are different too: the delivered quantities must add up to the printed total. The figure marks the fields of the document type delivery_note on a sample delivery note. The line table has three quantities, and quantity in line_items is the value of the Delivered column:

The fields of the document type delivery_note, marked on a delivery note of Harbour Office Supplies
Select the image to open it in full size.

This is the configuration of the document type:

{
  "json_schema": {
    "type": "object",
    "properties": {
      "delivery_note_number": {"type": "string"},
      "delivery_date":        {"type": "string", "description": "Date of the delivery as YYYY-MM-DD"},
      "customer_number":      {"type": "string", "description": "The number of the customer at the issuer"},
      "customer_name":        {"type": "string", "description": "Company name of the customer who ordered the goods"},
      "order_reference":      {"type": ["string", "null"], "description": "Order number of the customer, null if there is none"},
      "total_quantity":       {"type": "number", "description": "The printed total of the delivered quantities"},
      "line_items": {
        "type": "array",
        "items": {
          "type": "object",
          "properties": {
            "item_number": {"type": "string"},
            "quantity":    {"type": "number", "description": "The delivered quantity"}
          },
          "required": ["item_number", "quantity"]
        }
      }
    },
    "required": ["delivery_note_number", "delivery_date", "customer_number", "customer_name",
                 "order_reference", "total_quantity", "line_items"]
  },
  "annotation_fields": ["delivery_note_number", "order_reference"],
  "line_checks": [
    {"name": "delivered quantity >= 0", "expr": "quantity >= 0"}
  ],
  "checks": [
    {"name": "sum of delivered quantities = total", "expr": "abs(sum_quantity - total_quantity) <= 0.01"}
  ]
}

The form HARBOUR_DELIVERY_NOTE belongs to this document type. One of its six rules names the column that holds the quantity: The quantity of a line is the Delivered column.

{
  "markers": ["VAT ID DE 812 334 905", "^DELIVERY NOTE$"],
  "field_patterns": {
    "delivery_note_number":   "^DN-[0-9]{4}-[0-9]{4}$",
    "customer_number":        "^C-[0-9]{5}$",
    "delivery_date":          "^20[0-9]{2}-[01][0-9]-[0-3][0-9]$",
    "line_items.item_number": "^[A-Z]{3}-[A-Z0-9-]+$"
  },
  "decimal_comma": true,
  "rules": [
    "The customer is the company in the BILL TO block. The DELIVER TO block is only the shipping address.",
    "Dates are printed as DD.MM.YYYY.",
    "\"Your order\" holds the order reference.",
    "The line table has three quantities: Ordered, Delivered and Open. The quantity of a line is the Delivered column.",
    "The total quantity is the value of \"Total delivered\".",
    "The carrier is not the customer."
  ]
}

A new document type and a new form need no code: the checks are part of the configuration.

  1. On the tab Document types & forms, choose Create document type. Enter the name delivery_note.
  2. Open the tab Advanced (JSON), replace the text with the document type JSON above, and choose Apply to form. The tab Form shows the six fields, the two line columns, the check and the line check.
The document type delivery_note after the JSON was applied: six fields, two line columns, one check and one line check
Select the image to open it in full size.
  1. Choose Create document type.
  2. Choose Create form. Enter the Form code HARBOUR_DELIVERY_NOTE, choose the Document type delivery_note, and enter the Description a delivery note of Harbour Office Supplies GmbH, Hamburg. Paste the form JSON above on the tab Advanced (JSON) and choose Apply to form.
  3. Test the markers on a document that is no delivery note: in the section Test the form on a document, choose harbour-letter-prices.pdf. The letter has the letterhead of Harbour, but no title line of a form. The first marker matches, the second does not, and the test says that the document does not belong to the form.
The markers of HARBOUR_DELIVERY_NOTE tested on the letter: the VAT ID matches, the title does not
Select the image to open it in full size.
  1. Choose Create form. The tab Document types & forms now lists two document types and three forms.
The tab Document types & forms with the document types delivery_note and invoice, and the forms HARBOUR_DELIVERY_NOTE, HARBOUR_INVOICE and NORDWIND_INVOICE
Select the image to open it in full size.

Write the test cases of the new forms

Section titled “Write the test cases of the new forms”

A test case names its form, and 02_seed_test_set.sql skipped the 8 documents of the new forms in lesson 2. Run it again now, in SQL, after the forms exist. It replaces the 15 test cases that exist and writes the 8 new ones:

@02_seed_test_set.sql
test set: 23 document(s) written

The 23 test cases are the 21 PDFs, the image of the scan and the photo of a Nordwind invoice. The two images belong to lesson 4.

Test the markers against all documents

Section titled “Test the markers against all documents”

Each document must match exactly one form, or no form when it is no form. This test calls no model.

The manager tests one document at a time. Open a form, and choose a document in the section Test the form on a document. The line Forms that match now lists every saved form whose markers all match, and the section Markers shows each marker of the open form. Test a Harbour invoice, a delivery note, a Nordwind invoice and the letter. The measurement of the section Measure all forms together tests every document in one run: its table lists the form that ADM found for each document.

 
harbour-delivery-01-complete.pdf            HARBOUR_DELIVERY_NOTE
harbour-delivery-02-partial.pdf             HARBOUR_DELIVERY_NOTE
harbour-delivery-03-backorder.pdf           HARBOUR_DELIVERY_NOTE
harbour-invoice-01-standard.pdf             HARBOUR_INVOICE
harbour-invoice-02-single-line.pdf          HARBOUR_INVOICE
harbour-invoice-03-day-month-ambiguous.pdf  HARBOUR_INVOICE
harbour-invoice-04-no-order-ref.pdf         HARBOUR_INVOICE
harbour-invoice-05-different-delivery.pdf   HARBOUR_INVOICE
harbour-invoice-06-notes-other-number.pdf   HARBOUR_INVOICE
harbour-invoice-07-large-amounts.pdf        HARBOUR_INVOICE
harbour-invoice-08-discount.pdf             HARBOUR_INVOICE
harbour-invoice-09-credit-note.pdf          HARBOUR_INVOICE
harbour-invoice-10-long-name.pdf            HARBOUR_INVOICE
harbour-invoice-11-two-pages.pdf            HARBOUR_INVOICE
harbour-invoice-12-scanned.pdf              -
harbour-invoice-13-five-pages.pdf           HARBOUR_INVOICE
harbour-letter-prices.pdf                   -
nordwind-invoice-01-standard.pdf            NORDWIND_INVOICE
nordwind-invoice-02-reverse-charge.pdf      NORDWIND_INVOICE
nordwind-invoice-03-no-po.pdf               NORDWIND_INVOICE
nordwind-invoice-04-large.pdf               NORDWIND_INVOICE

Every document matches the right form. The letter has the letterhead of Harbour, but no title line of a form, so it matches no form. The scan has no text.

Run this test after you add or change a form, against all documents of all forms. With many forms, two forms can match one document. A document that matches more than one form gets the status REVIEW: ADM never chooses between forms. The next sections show how a measurement reports this.

Measure the Nordwind invoices

Section titled “Measure the Nordwind invoices”

The first measurement uses the form NORDWIND_INVOICE and the documents whose name ends in .pdf. As in lesson 2, the name filter is a like pattern, so it needs the %.

  1. Open the tab Measurements and choose Run measurement.
  2. Enter the Label nordwind. Choose the Form NORDWIND_INVOICE. Enter %.pdf in the field Name contains.
The form Run a measurement with the label nordwind, the form NORDWIND_INVOICE and the name filter %.pdf
Select the image to open it in full size.
  1. Choose Start. The measurement runs as a background job. A job that the manager starts can take up to about two minutes to begin. Until then, the profile page shows Measuring (scheduled). Choose the label of the run when it has finished.
The profile page with the status line Measuring (scheduled) after Start
Select the image to open it in full size.
The run nordwind: 4 right, and the four Nordwind invoices with their outcomes
Select the image to open it in full size.
 
DOCUMENT_NAME                              OUTCOME     SECONDS DETAILS
------------------------------------------ ----------- ------- ------------------------------------------------------------------------------------------------------------------------
nordwind-invoice-01-standard.pdf           RIGHT          3.4
nordwind-invoice-02-reverse-charge.pdf     RIGHT          2.3
nordwind-invoice-03-no-po.pdf              RIGHT          2.0
nordwind-invoice-04-large.pdf              RIGHT          2.5


DOCUMENTS  RIGHT FALSE_ALARM CAUGHT MISSED FAILED MEDIAN_S COMPLETION_TOKENS
--------- ------ ----------- ------ ------ ------ -------- -----------------
        4      4           0      0      0      0    2.4                 821

All four invoices are RIGHT. This includes the line table, where the column is called Code on the page and item_number in the schema: the model matched them without a rule for it, and the field pattern line_items.item_number accepted every item number. The model wrote about 205 completion tokens for each Nordwind invoice (821 for four). For comparison, the run patterns of lesson 2 wrote 1,378 tokens for 13 Harbour invoices, about 106 each: those lines come from the line pattern, not from the model.

A measurement that finds no error does not show that the rules are needed. The section Measure the forms without their rules tests that.

Measure all forms together

Section titled “Measure all forms together”

Measure all PDFs of the test set. This run includes the delivery notes and the letter. Leave the Form at All forms, and use the name filter %.pdf.

Choose Run measurement, enter the Label all-pdf, enter %.pdf in Name contains, and choose Start. The table of the run lists the expected form and the found form of each document.

The run all-pdf: 20 right and 1 caught, 21 documents, with the expected form and the found form of each
Select the image to open it in full size.
 
DOCUMENT_NAME                              OUTCOME     SECONDS DETAILS
------------------------------------------ ----------- ------- ------------------------------------------------------------------------------------------------------------------------
harbour-invoice-12-scanned.pdf             CAUGHT         0.0  wrong: form none | issues: the document has no text (8 characters); a person must read it


--------- ------ ----------- ------ ------ ------ -------- -----------------
       21     20           0      1      0      0    1.8                2423

Twenty of the 21 PDFs are RIGHT, and nothing is MISSED. The remaining CAUGHT is the scan without text, which lesson 4 explains.

The letter is RIGHT: its correct result is “no form”, and it went to a person. For a text document without a form, ADM asks the model which form the document is, and writes the answer into the issues as a hint for the person. The answer never chooses the form of a text document. Here the model answered unknown.

Measure the forms without their rules

Section titled “Measure the forms without their rules”

Lesson 2 measured the prompt Naive for one form. Do the same for all forms, to see whether the rules of the two new forms change any outcome.

Choose Run measurement. Enter the Label all-naive and the name filter %.pdf. Choose the Prompt Naive and choose Start.

 
DOCUMENT_NAME                              OUTCOME     SECONDS DETAILS
------------------------------------------ ----------- ------- ------------------------------------------------------------------------------------------------------------------------
harbour-invoice-12-scanned.pdf             CAUGHT         0.0  wrong: form none | issues: the document has no text (8 characters); a person must read it


--------- ------ ----------- ------ ------ ------ -------- -----------------
       21     20           0      1      0      0    2.0                2883

The outcomes are the same as with the rules: 20 RIGHT, the scan CAUGHT, nothing MISSED. The tokens differ. The run all-pdf used 21,184 prompt tokens and 2,423 completion tokens, and all-naive used 17,657 and 2,883. The rules are part of the prompt, which explains the shorter prompt. The recordings do not show why the answers are longer without them.

On these 21 documents, gpt-5.6-luna reads the Nordwind invoices and the delivery notes right without any rule. Lesson 2 gives the way to decide: do not add a rule for a field that the measurement shows as right, and add a rule where a measurement shows a wrong value. A rule that states what a field means, as in the Nordwind form ("Shipment:" is not the order reference), costs one line and protects against a layout that this small test set does not contain. Measure again when you change the model: a weaker model can need them.

Correct a form whose markers are too weak

Section titled “Correct a form whose markers are too weak”

The measurements above found no wrong value. The markers can still fail, and a measurement shows it. To see how, remove the title marker ^DELIVERY NOTE$ from HARBOUR_DELIVERY_NOTE. The form then matches every document that carries the VAT ID of Harbour: the invoices, the delivery notes and the letter.

  1. On the tab Document types & forms, choose HARBOUR_DELIVERY_NOTE.
  2. In the section Markers, choose the trash icon of the second marker, and choose Save.
  3. Open the form again. In the section Test the form on a document, choose harbour-invoice-01-standard.pdf. The only marker matches, and Forms that match now lists two forms.
The form HARBOUR_DELIVERY_NOTE with one marker, tested on harbour-invoice-01-standard.pdf: the marker matches, and two forms match the document
Select the image to open it in full size.
 
harbour-invoice-01-standard.pdf             MORE THAN ONE: HARBOUR_DELIVERY_NOTE, HARBOUR_INVOICE
harbour-invoice-02-single-line.pdf          MORE THAN ONE: HARBOUR_DELIVERY_NOTE, HARBOUR_INVOICE
...
harbour-invoice-11-two-pages.pdf            MORE THAN ONE: HARBOUR_DELIVERY_NOTE, HARBOUR_INVOICE
harbour-invoice-12-scanned.pdf              -
harbour-invoice-13-five-pages.pdf           MORE THAN ONE: HARBOUR_DELIVERY_NOTE, HARBOUR_INVOICE
harbour-letter-prices.pdf                   HARBOUR_DELIVERY_NOTE

The test finds the problem before any model call. A measurement of the Harbour invoices shows what it means for the extraction. Use the label weak-marker, no form, and the name filter harbour-invoice-0%:

Choose Run measurement. Enter the Label weak-marker and the name filter harbour-invoice-0%. Leave the Form at All forms and choose Start.

The run weak-marker: 9 caught, with the issue that more than one form matches
Select the image to open it in full size.
 
DOCUMENT_NAME                              OUTCOME     SECONDS DETAILS
------------------------------------------ ----------- ------- ------------------------------------------------------------------------------------------------------------------------
harbour-invoice-01-standard.pdf            CAUGHT         0.0  wrong: form none | issues: more than one form matches: HARBOUR_DELIVERY_NOTE, HARBOUR_INVOICE
harbour-invoice-02-single-line.pdf         CAUGHT         0.0  wrong: form none | issues: more than one form matches: HARBOUR_DELIVERY_NOTE, HARBOUR_INVOICE
harbour-invoice-03-day-month-ambiguous.pdf CAUGHT         0.0  wrong: form none | issues: more than one form matches: HARBOUR_DELIVERY_NOTE, HARBOUR_INVOICE
...
harbour-invoice-09-credit-note.pdf         CAUGHT         0.0  wrong: form none | issues: more than one form matches: HARBOUR_DELIVERY_NOTE, HARBOUR_INVOICE


DOCUMENTS  RIGHT FALSE_ALARM CAUGHT MISSED FAILED MEDIAN_S COMPLETION_TOKENS
--------- ------ ----------- ------ ------ ------ -------- -----------------
        9      0           0      9      0      0                          0

ADM calls no model for these documents: the tokens are 0. Each document is CAUGHT with the issue more than one form matches, and goes to a person. The values would have been right, but nothing reads them. The fix is in the marker, not in a rule. Put the title marker back.

Open HARBOUR_DELIVERY_NOTE again, choose Add marker, enter ^DELIVERY NOTE$ in the new field, and choose Save.

The output of 03_test_markers.sql is again the output of the section Test the markers against all documents: every document matches one form, or none. The run weak-marker stays in the list of measurements as a record. You can delete it with Delete run on its page.

Choose the next step from the outcome

Section titled “Choose the next step from the outcome”

The runs of lessons 2 and 3 show these outcomes, and what each one means for the next change:

What the measurement shows Cause in the recordings What to do
CAUGHT with more than one form matches. A marker that other forms of the same issuer match too (weak-marker). Add a marker that only this form matches, and run 03_test_markers.sql.
CAUGHT with the document has no text. The PDF is a scan (harbour-invoice-12-scanned.pdf). Read it with OCR, or as an image with a vision model (lesson 4).
FALSE_ALARM. A check that does not fit a correct document (checks-v1, the credit note). Change the check, and measure again (lesson 2).
CAUGHT or MISSED with named wrong fields. The model read a value wrong. This did not happen with the forms of this lesson. Quote the printed label in a rule, or add a field pattern or a check (lesson 2).
The model writes many tokens for the line table. The form has no line pattern: about 205 completion tokens for each Nordwind invoice, and about 106 for each Harbour invoice, whose lines a pattern reads. Write a line pattern for a form with long tables.

A form whose result stays wrong after a few rules is still useful: its documents go to a person, with the fields that the model read correctly as a start.

Next: Read images and choose the modelsAdd a vision model to the profile, keep scans without text away from the text model, and measure candidate models.