initial commit

This commit is contained in:
2026-09-25 23:32:29 +02:00
commit fff8b677f8
114 changed files with 84233 additions and 0 deletions
+155
View File
@@ -0,0 +1,155 @@
# testdata — the golden PDFs
Fourteen committed files, one per structural case, asserted by
`golden_test.odin`. They are deliberately tiny (370 B to 1.4 KB) and, wherever
the format allows, plain text: open one in an editor and the whole of it is
visible at once.
## Why these exist at all
The rest of the suite builds its PDFs in memory with the `pb_*` builders in
`xref_test.odin`, and should keep doing so — the builder records byte offsets as
it writes, so a test never asserts a hand-counted number. But a builder is part
of the same codebase as the reader, and there are three things it therefore
cannot show:
1. that `open_file` works on a real file on disk, not just on a byte slice;
2. that byte offsets written by *some other tool* are the ones we parse — a
committed file's offsets cannot silently follow a change in how we emit them;
3. that a refactor did not quietly change what the library accepts by changing
the builder at the same time. These bytes are frozen; the builder is not.
They also give the render layer, when it arrives, fourteen pages it can draw
without inventing its own fixtures.
## What every file has in common
- One page (two in `page-tree-deep.pdf`) whose content stream draws a filled
blue rectangle and a red stroked diagonal:
```
% <file name>
0.20 0.40 0.90 rg
20 20 160 100 re
f
0.90 0.10 0.10 RG
4 w
20 20 m 180 180 l
S
```
The first line names the file, so a test that decodes a content stream can say
*which* file's stream it got. `golden_test.odin` asserts the marker line on
page 0 and `160 100 re` on every page.
- `/MediaBox [0 0 200 200]`, except where the file's whole point is a different
box: `incremental-update.pdf` (300×300 after the update) and
`page-tree-deep.pdf` (300×400, inherited).
- No `/Filter` on the content stream. Compression is exercised by the filter
unit tests, which have the format's own vectors; a golden file that had to be
inflated before it could be read would be a worse golden file.
## The files
| File | Exercises | Opens with |
|---|---|---|
| `classic-xref.pdf` | the minimal well-formed document: four objects, one `xref` subsection, `trailer`, `startxref` | no repairs |
| `xref-stream.pdf` | PDF 1.5 `/Type /XRef` stream, `/W [1 4 2]`, FlateDecode + `/Predictor 12 /Columns 7` | no repairs |
| `object-stream.pdf` | catalog, page-tree root and page living inside a `/Type /ObjStm`; xref stream with type-2 entries | no repairs |
| `incremental-update.pdf` | two generations; the second rewrites object 3 and adds object 5, with a three-subsection table and `/Prev` | no repairs |
| `junk-before-header.pdf` | `%PDF-` at offset 70, every offset in the file measured from the header rather than from byte zero | no repairs |
| `broken-xref.pdf` | `startxref` pointing into a run of garbage; recovery is by scanning for `N G obj` | `Bad_Startxref`, `Rebuilt_Xref` |
| `bad-length.pdf` | a content stream whose `/Length` says 12 and whose body is 92 bytes; `endstream` has to be found by scanning | `Bad_Stream_Length` |
| `no-xref.pdf` | no table, no `trailer`, no `startxref` at all — objects and `%%EOF`, nothing else | `Bad_Startxref`, `Rebuilt_Xref`, `Missing_Root` |
| `page-tree-broken.pdf` | a `/Kids` cycle (2 → 3 → 2) with a real page hanging off the far side of it | `Cyclic_Page_Tree` |
| `encrypted-rc4-40.pdf` | standard security handler, `/V 1 /R 2`, RC4 40-bit | no repairs |
| `encrypted-rc4-128.pdf` | `/V 2 /R 3 /Length 128`, RC4 128-bit | no repairs |
| `encrypted-aes128.pdf` | `/V 4 /R 4`, crypt filter `/StdCF` with `/CFM /AESV2` | no repairs |
| `encrypted-aes256.pdf` | `/V 5 /R 6`, `/CFM /AESV3`, with `/UE`, `/OE` and `/Perms` | no repairs |
| `page-tree-deep.pdf` | three nested `/Pages` levels; `/MediaBox` and `/Rotate` declared only on the root, `/Resources` only on the middle node; page 2 overrides `/Rotate` alone | no repairs |
"Opens with" is the *exact* set of `Repair_Kind`s the file must record once every
page's content stream has been read — `golden_test.odin` compares the set, not a
subset. A file that starts opening more cleanly than this is as much a
regression as one that starts opening less cleanly, and both fail here.
Two details worth knowing before editing one of these:
- `broken-xref.pdf`'s garbage run is bare keywords (`garbagegarbage…`), not a
`%` comment. A comment is skipped by the lexer, which would carry the probe
straight on to the real `xref` table and make the bad offset work by accident.
It is also longer than 2 × `XREF_LOCATE_WINDOW` so that the ±64-byte search
around the bad offset cannot reach an object header either.
- `bad-length.pdf`'s repair is recorded when the *stream* is parsed, which is
lazy. A test that reads `doc.repairs` straight after `open_file` sees nothing.
## Encryption
All four encrypted files share one arrangement:
- **user password empty** — the file opens without prompting;
- **owner password `owner`** — so the owner branch of the key derivation is
exercised by a real file and not only by a unit vector;
- `/P -44`: bits 3 (modify) and 5 (annotate) clear, everything else set. R2 only
ever defined four permission bits, so `encrypted-rc4-40.pdf` reports
`{Print, Copy}` where the others report six;
- a `/Info` dictionary whose `/Title` is the encrypted string `Golden File`, so
that string decryption is covered as well as stream decryption;
- a fixed `/ID` of `9A1C2B3D4E5F60718293A4B5C6D7E8F9` (both halves), which is an
input to the R2–R4 key derivation and therefore may not be changed on its own;
- a classic `xref` table rather than a cross-reference stream, because an xref
stream is read before the file key exists and is never encrypted — which would
make the file exercise less, not more.
`encrypted-aes256.pdf` additionally fixes the file encryption key itself (it is
wrapped in `/UE` and `/OE` rather than derived), the two 8-byte salts in `/U` and
the two in `/O`. `/Perms` is the 16 bytes `P` (little-endian) ‖ `FF FF FF FF` ‖
`'T'` ‖ `"adb"` ‖ `"odin"`, AES-256-ECB encrypted with the file key.
AES payloads are `IV ‖ CBC(PKCS#7(plaintext))`. The IVs are fixed, not random:
`iv[i] = obj_num XOR (i*17 + 3)`. Nothing checks them, and a golden file whose
bytes changed on every regeneration would not be golden.
## Regenerating one
There is deliberately **no generator in the repository**. The point of these
files is that the bytes are fixed: a generator that ran at test time would be
one more piece of the same codebase, and would answer none of the three
questions at the top of this file. Editing a file by hand in a text editor is
the expected way to change one — every offset that matters is in the `xref`
table at the end, and `golden_test.odin` will tell you immediately if you got
one wrong.
If you do need to rebuild one from scratch, the recipe that produced these was a
throwaway `@(test)` procedure in the package (so that it could use the
`@(private)` `pb_*` builders and the `crypt.odin` primitives), which then wrote
the result with `os.write_entire_file`. The parts that are not obvious:
- **Structure.** `pb_make`, `pb_obj`, `pb_classic_xref`, `pb_trailer` and
`pb_startxref` from `xref_test.odin` do the whole of `classic-xref.pdf`. The
content-stream objects were written with a local variant of `pb_stream_obj`
that omits the double space an empty dictionary head leaves behind.
- **Cross-reference streams.** `xref_rows_encode(rows, 1, 4, 2)` lays out the
rows; `png_up_encode(raw, 7)` applies the PNG `Up` filter for `/Predictor 12`;
`zlib_stored` wraps the result in stored DEFLATE blocks, which is a valid zlib
stream and keeps the output byte-stable without needing a compressor.
- **Object streams.** `objstm_payload(nums, bodies)` returns the payload and the
`/First` the dictionary must declare. The one here is left unfiltered so that
the pair table and the three objects stay legible.
- **`/O` and `/U` for R2–R4.** The library only ever needs the *inverse* of
Algorithm 3 (`crypt_owner_password`, which decrypts `/O` back to the user
password), so the encrypting half has to be written out: pad the owner
password, MD5 it, iterate MD5 50 times over the full 16-byte digest when
R ≥ 3, take the first `n` bytes as an RC4 key, RC4 the padded *user* password,
and for R ≥ 3 repeat 19 more times with the key XORed by `i` for `i` = 1…19 —
ascending, where the decrypting direction descends. `/U` is then just
`crypt_key_r234` followed by `crypt_user_value`, both of which the library
already has.
- **R6.** `crypt_hash_r56` is Algorithm 2.B and is used as-is for all four
hashes. `/UE` and `/OE` are `aes_cbc_encrypt` of the 32-byte file key under
the key-salt hash with a zero IV and no padding.
After regenerating, update the row in `golden_test.odin`'s `GOLDEN` table if the
page size, page count, marker line or repair set changed. Nothing else in the
suite reads these files.
+37
View File
@@ -0,0 +1,37 @@
%PDF-1.4
%âãÏÓ
1 0 obj
<< /Type /Catalog /Pages 2 0 R >>
endobj
2 0 obj
<< /Type /Pages /Kids [3 0 R] /Count 1 >>
endobj
3 0 obj
<< /Type /Page /Parent 2 0 R /MediaBox [0 0 200 200] /Resources << >> /Contents 4 0 R >>
endobj
4 0 obj
<< /Length 12 >>
stream
% bad-length
0.20 0.40 0.90 rg
20 20 160 100 re
f
0.90 0.10 0.10 RG
4 w
20 20 m 180 180 l
S
endstream
endobj
xref
0 5
0000000000 65535 f
0000000015 00000 n
0000000064 00000 n
0000000121 00000 n
0000000225 00000 n
trailer
<< /Size 5 /Root 1 0 R >>
startxref
367
%%EOF
+45
View File
@@ -0,0 +1,45 @@
%PDF-1.4
%âãÏÓ
1 0 obj
<< /Type /Catalog /Pages 2 0 R >>
endobj
2 0 obj
<< /Type /Pages /Kids [3 0 R] /Count 1 >>
endobj
3 0 obj
<< /Type /Page /Parent 2 0 R /MediaBox [0 0 200 200] /Resources << >> /Contents 4 0 R >>
endobj
4 0 obj
<< /Length 93 >>
stream
% broken-xref
0.20 0.40 0.90 rg
20 20 160 100 re
f
0.90 0.10 0.10 RG
4 w
20 20 m 180 180 l
S
endstream
endobj
garbagegarbagegarbage garbagegarbagegarbage garbagegarbage
garbagegarbagegarbage garbagegarbagegarbage garbagegarbage
garbagegarbagegarbage garbagegarbagegarbage garbagegarbage
garbagegarbagegarbage garbagegarbagegarbage garbagegarbage
garbagegarbagegarbage garbagegarbagegarbage garbagegarbage
garbagegarbagegarbage garbagegarbagegarbage garbagegarbage
garbagegarbagegarbage garbagegarbagegarbage garbagegarbage
garbagegarbagegarbage garbagegarbagegarbage garbagegarbage
xref
0 5
0000000000 65535 f
0000000015 00000 n
0000000064 00000 n
0000000121 00000 n
0000000225 00000 n
trailer
<< /Size 5 /Root 1 0 R >>
startxref
568
%%EOF
+37
View File
@@ -0,0 +1,37 @@
%PDF-1.4
%âãÏÓ
1 0 obj
<< /Type /Catalog /Pages 2 0 R >>
endobj
2 0 obj
<< /Type /Pages /Kids [3 0 R] /Count 1 >>
endobj
3 0 obj
<< /Type /Page /Parent 2 0 R /MediaBox [0 0 200 200] /Resources << >> /Contents 4 0 R >>
endobj
4 0 obj
<< /Length 94 >>
stream
% classic-xref
0.20 0.40 0.90 rg
20 20 160 100 re
f
0.90 0.10 0.10 RG
4 w
20 20 m 180 180 l
S
endstream
endobj
xref
0 5
0000000000 65535 f
0000000015 00000 n
0000000064 00000 n
0000000121 00000 n
0000000225 00000 n
trailer
<< /Size 5 /Root 1 0 R >>
startxref
369
%%EOF
+37
View File
@@ -0,0 +1,37 @@
%PDF-1.6
%âãÏÓ
1 0 obj
<< /Type /Catalog /Pages 2 0 R >>
endobj
2 0 obj
<< /Type /Pages /Kids [3 0 R] /Count 1 >>
endobj
3 0 obj
<< /Type /Page /Parent 2 0 R /MediaBox [0 0 200 200] /Resources << >> /Contents 4 0 R >>
endobj
4 0 obj
<< /Length 128 >>
stream
!2C\m~�˜©ºËäõ»8Îá­Y~mR’‡CÇ®$í®†6:T¼á‡$L³¸¸dägVÛê[ûVe‡ødå3™B¼ƒR]ÚRxHäkšÏ%ðç’w_‹�úôoë~^lÞôªо轀nX¼‰l_2F í.=âö¹õ4*œ
endstream
endobj
5 0 obj
<< /Title <06112033425D6C7F8E99A8BBCAE5F40791B0DFC20181D644F6FA19EB2BF92931> >>
endobj
6 0 obj
<< /Filter /Standard /V 4 /R 4 /Length 128 /CF << /StdCF << /CFM /AESV2 /AuthEvent /DocOpen /Length 16 >> >> /StmF /StdCF /StrF /StdCF /EncryptMetadata true /O <566FA873EE33C797CD3B904FDADF814AFA34DF9A38F6ED41B984E2C6DA2AA6F5> /U <04E69B49575902CD47A461F90975799100000000000000000000000000000000> /P -44 >>
endobj
xref
0 7
0000000000 65535 f
0000000015 00000 n
0000000064 00000 n
0000000121 00000 n
0000000225 00000 n
0000000404 00000 n
0000000499 00000 n
trailer
<< /Size 7 /Root 1 0 R /Info 5 0 R /Encrypt 6 0 R /ID [<9A1C2B3D4E5F60718293A4B5C6D7E8F9> <9A1C2B3D4E5F60718293A4B5C6D7E8F9>] >>
startxref
821
%%EOF
+38
View File
@@ -0,0 +1,38 @@
%PDF-2.0
%âãÏÓ
1 0 obj
<< /Type /Catalog /Pages 2 0 R >>
endobj
2 0 obj
<< /Type /Pages /Kids [3 0 R] /Count 1 >>
endobj
3 0 obj
<< /Type /Page /Parent 2 0 R /MediaBox [0 0 200 200] /Resources << >> /Contents 4 0 R >>
endobj
4 0 obj
<< /Length 128 >>
stream
!2C\m~�˜©ºËäõ¯�›ÇZkÚ␍1CŠœÖÙ)J¢JŽÚõ»Ç¿ÿ,�<R:óà¯S{Ò�¯‚“ÌOzB·.oÚ“(¾U)ðõõãtµ<zù�…}çÔaŒWòØ4޲C­wC+e
$E–çDÛ®q³’9ì 2/HÀ Ž�@
endstream
endobj
5 0 obj
<< /Title <06112033425D6C7F8E99A8BBCAE5F407EE911981738BAAD67B804D5E3B44F870> >>
endobj
6 0 obj
<< /Filter /Standard /V 5 /R 6 /Length 256 /CF << /StdCF << /CFM /AESV3 /AuthEvent /DocOpen /Length 32 >> >> /StmF /StdCF /StrF /StdCF /EncryptMetadata true /O <363382C35280D1B1F38BD862F8A55B4B76CCB803C384613CBE26F2062B67C6B811223344556677888877665544332211> /U <1E500E81EF817EB3BECC79AA210AE9A4B51CAB5B51D1CC3772D51B1E8807AF570123456789ABCDEFFEDCBA9876543210> /OE <3544CF567AA1F49774347ABCDC9603E4461596DF24C8F6A8429E9EFC24FD40E8> /UE <5BCA59428091C155EA83D8F0B8C3567316081A3614AA3084A5508F72CAE98140> /Perms <E87095B67CCC2A23DB8F3C647CC22CD4> /P -44 >>
endobj
xref
0 7
0000000000 65535 f
0000000015 00000 n
0000000064 00000 n
0000000121 00000 n
0000000225 00000 n
0000000404 00000 n
0000000499 00000 n
trailer
<< /Size 7 /Root 1 0 R /Info 5 0 R /Encrypt 6 0 R /ID [<9A1C2B3D4E5F60718293A4B5C6D7E8F9> <9A1C2B3D4E5F60718293A4B5C6D7E8F9>] >>
startxref
1069
%%EOF
BIN
View File
Binary file not shown.
+37
View File
@@ -0,0 +1,37 @@
%PDF-1.6
%âãÏÓ
1 0 obj
<< /Type /Catalog /Pages 2 0 R >>
endobj
2 0 obj
<< /Type /Pages /Kids [3 0 R] /Count 1 >>
endobj
3 0 obj
<< /Type /Page /Parent 2 0 R /MediaBox [0 0 200 200] /Resources << >> /Contents 4 0 R >>
endobj
4 0 obj
<< /Length 98 >>
stream
pj¢›(Q[㦤OK*bÝ¢Ñìõ(%÷Ç=º*-žò›jóùð`ýÛü 䥪�pʪ‰Ä°.Wi–|„ ¸à"±]œ&MæÍÏe ü~d$¢“,ÇñÁb$OkR
endstream
endobj
5 0 obj
<< /Title <5CC0D3FC3ED8A04994CBA4> >>
endobj
6 0 obj
<< /Filter /Standard /V 1 /R 2 /O <C92422687FACEE686E373F10B5C7D04738053152F7E2EE30E11C69EC442576AB> /U <E7923A69A56C6231A75D35F63FD78FEF3BB4056F69A207811E9E93A5BDBF3A14> /P -44 >>
endobj
xref
0 7
0000000000 65535 f
0000000015 00000 n
0000000064 00000 n
0000000121 00000 n
0000000225 00000 n
0000000373 00000 n
0000000426 00000 n
trailer
<< /Size 7 /Root 1 0 R /Info 5 0 R /Encrypt 6 0 R /ID [<9A1C2B3D4E5F60718293A4B5C6D7E8F9> <9A1C2B3D4E5F60718293A4B5C6D7E8F9>] >>
startxref
622
%%EOF
+66
View File
@@ -0,0 +1,66 @@
%PDF-1.4
%âãÏÓ
1 0 obj
<< /Type /Catalog /Pages 2 0 R >>
endobj
2 0 obj
<< /Type /Pages /Kids [3 0 R] /Count 1 >>
endobj
3 0 obj
<< /Type /Page /Parent 2 0 R /MediaBox [0 0 200 200] /Resources << >> /Contents 4 0 R >>
endobj
4 0 obj
<< /Length 113 >>
stream
% incremental-update generation 1
0.20 0.40 0.90 rg
20 20 160 100 re
f
0.90 0.10 0.10 RG
4 w
20 20 m 180 180 l
S
endstream
endobj
xref
0 5
0000000000 65535 f
0000000015 00000 n
0000000064 00000 n
0000000121 00000 n
0000000225 00000 n
trailer
<< /Size 5 /Root 1 0 R /ID [<0102030405060708090A0B0C0D0E0F10> <0102030405060708090A0B0C0D0E0F10>] >>
startxref
389
%%EOF
3 0 obj
<< /Type /Page /Parent 2 0 R /MediaBox [0 0 300 300] /Resources << >> /Contents 5 0 R >>
endobj
5 0 obj
<< /Length 113 >>
stream
% incremental-update generation 2
0.20 0.40 0.90 rg
20 20 160 100 re
f
0.90 0.10 0.10 RG
4 w
20 20 m 180 180 l
S
endstream
endobj
xref
0 1
0000000000 65535 f
3 1
0000000628 00000 n
5 1
0000000732 00000 n
trailer
<< /Size 6 /Root 1 0 R /Prev 389 /ID [<0102030405060708090A0B0C0D0E0F10> <1112131415161718191A1B1C1D1E1F20>] >>
startxref
896
%%EOF
+41
View File
@@ -0,0 +1,41 @@
HTTP/1.1 200 OK
Content-Type: application/pdf
Content-Length: 999
%PDF-1.4
%âãÏÓ
1 0 obj
<< /Type /Catalog /Pages 2 0 R >>
endobj
2 0 obj
<< /Type /Pages /Kids [3 0 R] /Count 1 >>
endobj
3 0 obj
<< /Type /Page /Parent 2 0 R /MediaBox [0 0 200 200] /Resources << >> /Contents 4 0 R >>
endobj
4 0 obj
<< /Length 100 >>
stream
% junk-before-header
0.20 0.40 0.90 rg
20 20 160 100 re
f
0.90 0.10 0.10 RG
4 w
20 20 m 180 180 l
S
endstream
endobj
xref
0 5
0000000000 65535 f
0000000015 00000 n
0000000064 00000 n
0000000121 00000 n
0000000225 00000 n
trailer
<< /Size 5 /Root 1 0 R >>
startxref
376
%%EOF
+26
View File
@@ -0,0 +1,26 @@
%PDF-1.4
%âãÏÓ
1 0 obj
<< /Type /Catalog /Pages 2 0 R >>
endobj
2 0 obj
<< /Type /Pages /Kids [3 0 R] /Count 1 >>
endobj
3 0 obj
<< /Type /Page /Parent 2 0 R /MediaBox [0 0 200 200] /Resources << >> /Contents 4 0 R >>
endobj
4 0 obj
<< /Length 89 >>
stream
% no-xref
0.20 0.40 0.90 rg
20 20 160 100 re
f
0.90 0.10 0.10 RG
4 w
20 20 m 180 180 l
S
endstream
endobj
%%EOF
BIN
View File
Binary file not shown.
+41
View File
@@ -0,0 +1,41 @@
%PDF-1.4
%âãÏÓ
1 0 obj
<< /Type /Catalog /Pages 2 0 R >>
endobj
2 0 obj
<< /Type /Pages /Kids [3 0 R] /Count 1 >>
endobj
3 0 obj
<< /Type /Pages /Parent 2 0 R /Kids [2 0 R 4 0 R] /Count 1 >>
endobj
4 0 obj
<< /Type /Page /Parent 3 0 R /MediaBox [0 0 200 200] /Resources << >> /Contents 5 0 R >>
endobj
5 0 obj
<< /Length 98 >>
stream
% page-tree-broken
0.20 0.40 0.90 rg
20 20 160 100 re
f
0.90 0.10 0.10 RG
4 w
20 20 m 180 180 l
S
endstream
endobj
xref
0 6
0000000000 65535 f
0000000015 00000 n
0000000064 00000 n
0000000121 00000 n
0000000198 00000 n
0000000302 00000 n
trailer
<< /Size 6 /Root 1 0 R >>
startxref
450
%%EOF
+64
View File
@@ -0,0 +1,64 @@
%PDF-1.4
%âãÏÓ
1 0 obj
<< /Type /Catalog /Pages 2 0 R >>
endobj
2 0 obj
<< /Type /Pages /Kids [3 0 R] /Count 2 /MediaBox [0 0 300 400] /Rotate 90 >>
endobj
3 0 obj
<< /Type /Pages /Parent 2 0 R /Kids [4 0 R] /Count 2 /Resources << >> >>
endobj
4 0 obj
<< /Type /Pages /Parent 3 0 R /Kids [5 0 R 6 0 R] /Count 2 >>
endobj
5 0 obj
<< /Type /Page /Parent 4 0 R /Contents 7 0 R >>
endobj
6 0 obj
<< /Type /Page /Parent 4 0 R /Rotate 180 /Contents 8 0 R >>
endobj
7 0 obj
<< /Length 103 >>
stream
% page-tree-deep page 1
0.20 0.40 0.90 rg
20 20 160 100 re
f
0.90 0.10 0.10 RG
4 w
20 20 m 180 180 l
S
endstream
endobj
8 0 obj
<< /Length 103 >>
stream
% page-tree-deep page 2
0.20 0.40 0.90 rg
20 20 160 100 re
f
0.90 0.10 0.10 RG
4 w
20 20 m 180 180 l
S
endstream
endobj
xref
0 9
0000000000 65535 f
0000000015 00000 n
0000000064 00000 n
0000000156 00000 n
0000000244 00000 n
0000000321 00000 n
0000000384 00000 n
0000000459 00000 n
0000000613 00000 n
trailer
<< /Size 9 /Root 1 0 R >>
startxref
767
%%EOF
BIN
View File
Binary file not shown.