pdf-odin-llm/docs/corpus-notes.md

# What real PDFs actually use

Feature counts across the 51 PDFs on this machine (39 in ~/Downloads, 12 ISO spec
documents) — contracts, invoices, standards, scans, exports from Word, InDesign
and web tooling. Percentages are of files that contain the marker at all.

| Feature | Files | Note |
|---|---:|---|
| FlateDecode | 51 (100%) | the only filter that must be fast |
| DCTDecode | 19 (37%) | own JPEG decoder is not optional |
| ASCII85Decode | 3 (6%) | |
| CCITTFaxDecode | 1 (2%) | long tail |
| LZWDecode | 1 (2%) | long tail |
| JPXDecode / JBIG2Decode | 0 | never seen; pass through |
| xref streams + object streams | 26 (51%) | half the corpus is PDF 1.5+ structure |
| Linearized | 13 (25%) | |
| Encrypted | 3 (6%) | |
| AcroForm | 13 (25%) | |
| Annots | 27 (53%) | appearance streams matter for rendering |
| ExtGState | 44 (86%) | |
| SMask | 30 (59%) | **transparency is mainstream, not exotic** |
| Group (transparency) | 25 (49%) | |
| BM (blend mode) | 23 (45%) | |
| Type0 (composite) fonts | 27 (53%) | |
| CIDFontType2 | 29 (57%) | |
| FontFile2 (TrueType) | 31 (61%) | |
| FontFile3 (CFF) | 11 (22%) | |
| ToUnicode | 32 (63%) | text extraction has a real chance |
| Type3 fonts | 1 (2%) | long tail |
| ICCBased | 12 (24%) | render via /Alternate |
| Indexed | 8 (16%) | |
| Separation | 3 (6%) | |
| Lab | 3 (6%) | |
| Pattern | 7 (14%) | |
| Shading | 3 (6%) | |
| DeviceN, CalRGB | 0 | implement, but do not tune for |

Reproduce with the loop in the project history; it is a plain grep for each
marker across every PDF, which works because stream dictionaries are never
themselves inside an object stream.

**What this ordered:** Flate and DCT decode quality, TrueType and CFF glyph
outlines, Type0/CID mapping, `/ToUnicode`, and the transparency model (soft
masks, groups, blend modes) are the load-bearing work. CCITT, LZW, JPX, JBIG2,
Type3, DeviceN and the exotic shading types are correctness obligations, not
performance ones.