# LLM extraction comparison: WDF vs PDF (T5.3)

**Method.** The same document — the example _Decree no. 87/2026_ (a fictional
public-administration decree) — exists in two forms produced from the same source:
`examples/municipal-decree/comparison.pdf` (printed via headless Chrome, a clean born-digital
PDF: deliberately a _best case_ for PDF, not a strawman) and `municipal-decree.wdf` (canonical
package). The same task was given to an LLM (Claude, August 2026) against each input:

> _«Extract the committed amounts, by chapter and year, and cite the source of every value so
> that an auditor can verify it.»_

The PDF was ingested as a document attachment (standard multimodal ingestion); the WDF side used
the output of `wdf extract` (`ai/content.md`, whose fidelity to the rendered document is
machine-verified, spec §7.1) plus the manifest id for citations.

## Answer from the PDF

> The committed amounts are (table on page 2):
>
> - chapter "1042.3 — IT services", year 2026: 14640 euros
> - chapter "1042.3 — IT services", year 2027: 9760 euros
> - chapter "1015.1 — Staff training", year 2026: 1830 euros
>
> **Source:** table "Spending commitments by chapter and year", **page 2 of the PDF**. Note: in
> the extracted text the table rows arrive as flat text (e.g. `1042.3 — IT services 2026 14640
2026-12-31`), so the attribution of values to columns was _inferred_ from their order; there is
> no stable identifier to cite, only the page number, and nothing guarantees that the extracted
> text coincides with what the document displays.

## Answer from WDF (`wdf extract`)

> The committed amounts, from the typed dataset bound to table `tbl-commitments`, are:
>
> | chapter                 | year | amount (euros) | source                                                              |
> | ----------------------- | ---- | -------------- | ------------------------------------------------------------------- |
> | 1042.3 — IT services    | 2026 | 14640          | `wdf:urn:uuid:7d444840-9dc0-5d1c-b745-1a56c4e5f6a7#tbl-commitments` |
> | 1042.3 — IT services    | 2027 | 9760           | `wdf:urn:uuid:7d444840-9dc0-5d1c-b745-1a56c4e5f6a7#tbl-commitments` |
> | 1015.1 — Staff training | 2026 | 1830           | `wdf:urn:uuid:7d444840-9dc0-5d1c-b745-1a56c4e5f6a7#tbl-commitments` |
>
> The total consideration of 24400 euros is citable as `wdf:…#li-0001`; the training share as
> `wdf:…#li-0003`. Every citation resolves to the exact element, and the package is verifiable:
> hashes cover every file and `ai/content.md` is _provably_ the canonical derivation of the
> rendered document.

## Honest assessment

On a clean, born-digital PDF like this one, a modern multimodal LLM reads the raw text and values
correctly — the difference is **not** raw text access. The differences that matter:

1. **Column semantics.** The PDF text layer serializes each table row as one undifferentiated
   line; column boundaries and types are inferred, and inference fails silently on merged cells,
   wrapped rows, or scanned documents. In WDF the table _is_ a typed dataset (`integer`, `number`,
   `date`), and validation proves the rendered cells equal the data.
2. **Citations.** The PDF answer can only cite "page 2". The WDF answer cites
   `wdf:<document-id>#tbl-commitments` — a stable identifier that survives re-pagination and
   resolves to the exact element in the human view, the agent view, and the outline.
3. **Verifiability.** Nothing ties the PDF's extracted text to its rendering. A WDF consumer
   re-runs the canonical extraction and compares byte-for-byte (spec §8.2): if human and agent
   views diverged, verification would fail.
4. **Degradation path.** Real-world PA documents are often scanned or generated by conversion
   chains; PDF extraction quality then collapses to OCR guesswork, while a WDF package either
   validates or is rejected.

**Reproduce it:** `pnpm demo`, open `_site/examples/municipal-decree.pdf` and attach it to any
LLM with the prompt above; then run
`node packages/cli/dist/index.js extract _site/examples/municipal-decree.wdf` and give the model
the output. (This file records a run performed with Claude in August 2026, on the English
example; an earlier run on the original Italian version, July 2026, gave equivalent results.)