156 lines
8.8 KiB
Markdown
156 lines
8.8 KiB
Markdown
# testdata — the golden PDFs
|
||
|
||
Fourteen committed files, one per structural case, asserted by
|
||
`golden_test.odin`. They are deliberately tiny (370 B to 1.4 KB) and, wherever
|
||
the format allows, plain text: open one in an editor and the whole of it is
|
||
visible at once.
|
||
|
||
## Why these exist at all
|
||
|
||
The rest of the suite builds its PDFs in memory with the `pb_*` builders in
|
||
`xref_test.odin`, and should keep doing so — the builder records byte offsets as
|
||
it writes, so a test never asserts a hand-counted number. But a builder is part
|
||
of the same codebase as the reader, and there are three things it therefore
|
||
cannot show:
|
||
|
||
1. that `open_file` works on a real file on disk, not just on a byte slice;
|
||
2. that byte offsets written by *some other tool* are the ones we parse — a
|
||
committed file's offsets cannot silently follow a change in how we emit them;
|
||
3. that a refactor did not quietly change what the library accepts by changing
|
||
the builder at the same time. These bytes are frozen; the builder is not.
|
||
|
||
They also give the render layer, when it arrives, fourteen pages it can draw
|
||
without inventing its own fixtures.
|
||
|
||
## What every file has in common
|
||
|
||
- One page (two in `page-tree-deep.pdf`) whose content stream draws a filled
|
||
blue rectangle and a red stroked diagonal:
|
||
|
||
```
|
||
% <file name>
|
||
0.20 0.40 0.90 rg
|
||
20 20 160 100 re
|
||
f
|
||
0.90 0.10 0.10 RG
|
||
4 w
|
||
20 20 m 180 180 l
|
||
S
|
||
```
|
||
|
||
The first line names the file, so a test that decodes a content stream can say
|
||
*which* file's stream it got. `golden_test.odin` asserts the marker line on
|
||
page 0 and `160 100 re` on every page.
|
||
|
||
- `/MediaBox [0 0 200 200]`, except where the file's whole point is a different
|
||
box: `incremental-update.pdf` (300×300 after the update) and
|
||
`page-tree-deep.pdf` (300×400, inherited).
|
||
|
||
- No `/Filter` on the content stream. Compression is exercised by the filter
|
||
unit tests, which have the format's own vectors; a golden file that had to be
|
||
inflated before it could be read would be a worse golden file.
|
||
|
||
## The files
|
||
|
||
| File | Exercises | Opens with |
|
||
|---|---|---|
|
||
| `classic-xref.pdf` | the minimal well-formed document: four objects, one `xref` subsection, `trailer`, `startxref` | no repairs |
|
||
| `xref-stream.pdf` | PDF 1.5 `/Type /XRef` stream, `/W [1 4 2]`, FlateDecode + `/Predictor 12 /Columns 7` | no repairs |
|
||
| `object-stream.pdf` | catalog, page-tree root and page living inside a `/Type /ObjStm`; xref stream with type-2 entries | no repairs |
|
||
| `incremental-update.pdf` | two generations; the second rewrites object 3 and adds object 5, with a three-subsection table and `/Prev` | no repairs |
|
||
| `junk-before-header.pdf` | `%PDF-` at offset 70, every offset in the file measured from the header rather than from byte zero | no repairs |
|
||
| `broken-xref.pdf` | `startxref` pointing into a run of garbage; recovery is by scanning for `N G obj` | `Bad_Startxref`, `Rebuilt_Xref` |
|
||
| `bad-length.pdf` | a content stream whose `/Length` says 12 and whose body is 92 bytes; `endstream` has to be found by scanning | `Bad_Stream_Length` |
|
||
| `no-xref.pdf` | no table, no `trailer`, no `startxref` at all — objects and `%%EOF`, nothing else | `Bad_Startxref`, `Rebuilt_Xref`, `Missing_Root` |
|
||
| `page-tree-broken.pdf` | a `/Kids` cycle (2 → 3 → 2) with a real page hanging off the far side of it | `Cyclic_Page_Tree` |
|
||
| `encrypted-rc4-40.pdf` | standard security handler, `/V 1 /R 2`, RC4 40-bit | no repairs |
|
||
| `encrypted-rc4-128.pdf` | `/V 2 /R 3 /Length 128`, RC4 128-bit | no repairs |
|
||
| `encrypted-aes128.pdf` | `/V 4 /R 4`, crypt filter `/StdCF` with `/CFM /AESV2` | no repairs |
|
||
| `encrypted-aes256.pdf` | `/V 5 /R 6`, `/CFM /AESV3`, with `/UE`, `/OE` and `/Perms` | no repairs |
|
||
| `page-tree-deep.pdf` | three nested `/Pages` levels; `/MediaBox` and `/Rotate` declared only on the root, `/Resources` only on the middle node; page 2 overrides `/Rotate` alone | no repairs |
|
||
|
||
"Opens with" is the *exact* set of `Repair_Kind`s the file must record once every
|
||
page's content stream has been read — `golden_test.odin` compares the set, not a
|
||
subset. A file that starts opening more cleanly than this is as much a
|
||
regression as one that starts opening less cleanly, and both fail here.
|
||
|
||
Two details worth knowing before editing one of these:
|
||
|
||
- `broken-xref.pdf`'s garbage run is bare keywords (`garbagegarbage…`), not a
|
||
`%` comment. A comment is skipped by the lexer, which would carry the probe
|
||
straight on to the real `xref` table and make the bad offset work by accident.
|
||
It is also longer than 2 × `XREF_LOCATE_WINDOW` so that the ±64-byte search
|
||
around the bad offset cannot reach an object header either.
|
||
- `bad-length.pdf`'s repair is recorded when the *stream* is parsed, which is
|
||
lazy. A test that reads `doc.repairs` straight after `open_file` sees nothing.
|
||
|
||
## Encryption
|
||
|
||
All four encrypted files share one arrangement:
|
||
|
||
- **user password empty** — the file opens without prompting;
|
||
- **owner password `owner`** — so the owner branch of the key derivation is
|
||
exercised by a real file and not only by a unit vector;
|
||
- `/P -44`: bits 3 (modify) and 5 (annotate) clear, everything else set. R2 only
|
||
ever defined four permission bits, so `encrypted-rc4-40.pdf` reports
|
||
`{Print, Copy}` where the others report six;
|
||
- a `/Info` dictionary whose `/Title` is the encrypted string `Golden File`, so
|
||
that string decryption is covered as well as stream decryption;
|
||
- a fixed `/ID` of `9A1C2B3D4E5F60718293A4B5C6D7E8F9` (both halves), which is an
|
||
input to the R2–R4 key derivation and therefore may not be changed on its own;
|
||
- a classic `xref` table rather than a cross-reference stream, because an xref
|
||
stream is read before the file key exists and is never encrypted — which would
|
||
make the file exercise less, not more.
|
||
|
||
`encrypted-aes256.pdf` additionally fixes the file encryption key itself (it is
|
||
wrapped in `/UE` and `/OE` rather than derived), the two 8-byte salts in `/U` and
|
||
the two in `/O`. `/Perms` is the 16 bytes `P` (little-endian) ‖ `FF FF FF FF` ‖
|
||
`'T'` ‖ `"adb"` ‖ `"odin"`, AES-256-ECB encrypted with the file key.
|
||
|
||
AES payloads are `IV ‖ CBC(PKCS#7(plaintext))`. The IVs are fixed, not random:
|
||
`iv[i] = obj_num XOR (i*17 + 3)`. Nothing checks them, and a golden file whose
|
||
bytes changed on every regeneration would not be golden.
|
||
|
||
## Regenerating one
|
||
|
||
There is deliberately **no generator in the repository**. The point of these
|
||
files is that the bytes are fixed: a generator that ran at test time would be
|
||
one more piece of the same codebase, and would answer none of the three
|
||
questions at the top of this file. Editing a file by hand in a text editor is
|
||
the expected way to change one — every offset that matters is in the `xref`
|
||
table at the end, and `golden_test.odin` will tell you immediately if you got
|
||
one wrong.
|
||
|
||
If you do need to rebuild one from scratch, the recipe that produced these was a
|
||
throwaway `@(test)` procedure in the package (so that it could use the
|
||
`@(private)` `pb_*` builders and the `crypt.odin` primitives), which then wrote
|
||
the result with `os.write_entire_file`. The parts that are not obvious:
|
||
|
||
- **Structure.** `pb_make`, `pb_obj`, `pb_classic_xref`, `pb_trailer` and
|
||
`pb_startxref` from `xref_test.odin` do the whole of `classic-xref.pdf`. The
|
||
content-stream objects were written with a local variant of `pb_stream_obj`
|
||
that omits the double space an empty dictionary head leaves behind.
|
||
- **Cross-reference streams.** `xref_rows_encode(rows, 1, 4, 2)` lays out the
|
||
rows; `png_up_encode(raw, 7)` applies the PNG `Up` filter for `/Predictor 12`;
|
||
`zlib_stored` wraps the result in stored DEFLATE blocks, which is a valid zlib
|
||
stream and keeps the output byte-stable without needing a compressor.
|
||
- **Object streams.** `objstm_payload(nums, bodies)` returns the payload and the
|
||
`/First` the dictionary must declare. The one here is left unfiltered so that
|
||
the pair table and the three objects stay legible.
|
||
- **`/O` and `/U` for R2–R4.** The library only ever needs the *inverse* of
|
||
Algorithm 3 (`crypt_owner_password`, which decrypts `/O` back to the user
|
||
password), so the encrypting half has to be written out: pad the owner
|
||
password, MD5 it, iterate MD5 50 times over the full 16-byte digest when
|
||
R ≥ 3, take the first `n` bytes as an RC4 key, RC4 the padded *user* password,
|
||
and for R ≥ 3 repeat 19 more times with the key XORed by `i` for `i` = 1…19 —
|
||
ascending, where the decrypting direction descends. `/U` is then just
|
||
`crypt_key_r234` followed by `crypt_user_value`, both of which the library
|
||
already has.
|
||
- **R6.** `crypt_hash_r56` is Algorithm 2.B and is used as-is for all four
|
||
hashes. `/UE` and `/OE` are `aes_cbc_encrypt` of the 32-byte file key under
|
||
the key-salt hash with a zero IV and no padding.
|
||
|
||
After regenerating, update the row in `golden_test.odin`'s `GOLDEN` table if the
|
||
page size, page count, marker line or repair set changed. Nothing else in the
|
||
suite reads these files.
|