24 real documents, and exactly what happened to each.
Anyone can claim their tool works. This is a published sample of our testing: documents from public sources, run through the exact pipeline customers use, validated by veraPDF — the open-source reference validator auditors themselves run. We test continuously and this is one run, but it is published whole: no row was dropped for looking bad, and every row carries one of three statuses and a fixed reason code rather than a story written afterwards.
The three statuses
- PASS
- Fixed, validated, delivered. Charged only at this point, never before.
- REFUSED
- We attempted it and our own gates rejected the result. Nothing delivered, nothing charged — this is the guarantee firing.
- OUT-OF-SCOPE
- Not work this service does — a scan with no text, a file we can't open, a document already tagged by someone else. Routed before any processing, never charged.
Reason codes
| Code | Status | Meaning |
|---|---|---|
OK | PASS | Delivered, PDF/UA-1 certified |
R-VALIDATION | REFUSED | Did not pass PDF/UA-1 validation |
R-EXPOSURE | REFUSED | Too little text reachable by a screen reader |
R-FONTS | REFUSED | Source fonts damaged — unfixable without altering appearance |
S-OCR | OUT-OF-SCOPE | Scan with no text layer — needs OCR first |
S-TAGGED | OUT-OF-SCOPE | Already tagged — priced as repair, never overwritten |
S-PAGES | OUT-OF-SCOPE | Over the 150-page limit |
S-FILE | OUT-OF-SCOPE | File unreadable — corrupt or password-protected |
PASS — 15 documents
Each passed all seven structural checks, passed full PDF/UA-1 validation, and kept at least 91% of its text reachable by a screen reader. That last number is ours, not the standard's: a document can pass ISO validation with most of its text hidden from assistive technology, so we measure exposure separately and refuse to deliver a file that reads as mostly empty.
| Source document | Status | Code | Text exposed |
|---|---|---|---|
| arXiv 1706.03762 — Attention Is All You Need | PASS | OK | 99% |
| arXiv 1409.0473 (2014 LaTeX) | PASS | OK | 99% |
| arXiv 2310.06825 — Mistral 7B | PASS | OK | 99% |
| Berkshire Hathaway 2023 shareholder letter | PASS | OK | 98% |
| Bitcoin whitepaper | PASS | OK | 100% |
| Control fixture: business report with table | PASS | OK | 100% |
| Control fixture: embedded-font document | PASS | OK | 100% |
| Constructed: German business letter | PASS | OK | 100% |
| Constructed: service invoice with table | PASS | OK | 100% |
| arXiv quant-ph/9708016 (1997 LaTeX) | PASS | OK | 100% |
| arXiv 2106.09685 — LoRA | PASS | OK | 99% |
| arXiv 2005.11401 — RAG | PASS | OK | 99% |
| arXiv 1512.03385 — ResNet | PASS | OK | 100% |
| Mozilla pdf.js test corpus (TCPDF output) | PASS | OK | 91% |
| W3C WAI test file | PASS | OK | 100% |
REFUSED — 1 document
Attempted, rejected by our own gates, not delivered, not charged.
| Source document | Status | Code | Reason |
|---|---|---|---|
| UN Universal Declaration of Human Rights | REFUSED | R-FONTS | Source fonts damaged — unfixable without altering appearance |
OUT-OF-SCOPE — 8 documents
Recognised and routed rather than attempted. A scan has no text to tag; an already-tagged file needs repair, not rebuilding, and we never overwrite someone's structure blindly; an unopenable file is unopenable. The free checker tells you which of these you have before you upload anything.
| Source document | Status | Code | Reason |
|---|---|---|---|
| Constructed: file truncated mid-body | OUT-OF-SCOPE | S-FILE | File unreadable — corrupt or password-protected |
| Constructed: password-protected file | OUT-OF-SCOPE | S-FILE | File unreadable — corrupt or password-protected |
| IRS Form 1040 (author-tagged) | OUT-OF-SCOPE | S-TAGGED | Already tagged — priced as repair, never overwritten |
| NIST SP 800-63-3 | OUT-OF-SCOPE | S-TAGGED | Already tagged — priced as repair, never overwritten |
| Constructed: 160-page document (over the 150 cap) | OUT-OF-SCOPE | S-PAGES | Over the 150-page limit |
| Constructed: image-only scan (no text layer) | OUT-OF-SCOPE | S-OCR | Scan with no text layer — needs OCR first |
| Federal Reserve FOMC minutes | OUT-OF-SCOPE | S-TAGGED | Already tagged — priced as repair, never overwritten |
| IRS Publication 17, 142 pages (author-tagged) | OUT-OF-SCOPE | S-TAGGED | Already tagged — priced as repair, never overwritten |
Method
Each document runs through the identical pipeline customers get, then is validated with veraPDF, the open-source ISO 14289-1 reference validator. Documents that fail any gate are refused and free — refusals below are the guarantee working, not failures of it.
- Documents come from public sources — arXiv, the IRS, NIST, the UN, the Federal Reserve, Berkshire Hathaway — plus constructed edge cases (a scan with no text layer, a truncated file, a password-protected file, an oversized document) that exist to test refusals rather than successes.
- Every file runs through the same code path a paying customer's document takes, with every gate enabled. No hand-tuning, no per-document exceptions.
- Validation is performed by veraPDF, which we don't control, and the version is recorded above. The same validator is what compliance auditors use.
- Statuses and codes are assigned by the same script that reads the run's raw results, so this table cannot say something the test run didn't measure.
- This is a sample, not a complete history — we run new documents regularly and publish a run once, in full. If you want a class of document tested that isn't represented here, send it and we'll add the result whichever way it goes.
Two honest limits: conformance to a named validator is not a legal opinion, and no machine can judge whether an image description is well written — validators check that alt text exists, not that it's good. We don't claim otherwise anywhere.
Test your own document
The checker runs in your browser and uploads nothing. If your file can be fixed, it's $3 for up to 10 pages — and only after it passes validation.