Guide

Snapshot-testing generated PDFs in CI with a hash instead of a diff

· ~5 min read

A headless-browser renderer stamps a fresh /CreationDate into every PDF it produces, reorders internal objects between runs, and can re-hint fonts slightly differently depending on the machine that ran it. A snapshot test built on "diff these bytes" fails on every single run against that kind of renderer, whether or not the invoice actually changed. PDFops fills the same template and field values into byte-identical output every time, so a hash-based snapshot test can catch what actually changed: the template, or the data, instead of drowning in render noise.

Why a raw PDF diff doesn't work

Open two PDFs produced a minute apart by the same rendering pipeline and run a byte-level diff, and you'll get hits even when the visible content is identical:

None of that is a bug in the renderer. It's just what "render HTML to a page" costs you: the output is a fresh artifact each time, built from a pipeline with several points where non-content bytes can legitimately differ. A snapshot test needs the opposite guarantee: same input, same bytes, always.

What determinism changes

POST /api/fill-form only writes into AcroForm fields that already exist in the template you upload. It doesn't re-render the page, re-flow layout, or touch anything outside the fields you named. Send the same template and the same field values twice and the response bytes match, down to the last one. That's the whole property a snapshot test needs: hash the output once, commit the hash, and every later run either matches it or tells you something real happened: the template changed, the field values changed, or the request itself is wrong.

The workflow

Keep a fixture template and a fixed set of field values in the repo, call fill-form with them, hash the response, and compare against a committed golden hash. This script does the comparing part: 

// scripts/verify-pdf-snapshot.mjs
import { createHash } from 'node:crypto';
import { readFile } from 'node:fs/promises';

const GOLDEN_HASH = (await readFile('fixtures/invoice.sha256', 'utf8')).trim();
const FIELDS = { customer_name: 'Acme Co', total: '$1,250.00' };

const template = await readFile('fixtures/invoice-template.pdf');
const form = new FormData();
form.append('pdf', new Blob([template]), 'invoice-template.pdf');
form.append('fields', JSON.stringify(FIELDS));

const res = await fetch('https://pdfops.dev/api/fill-form', {
  method: 'POST',
  headers: { 'X-API-Key': process.env.PDFOPS_API_KEY },
  body: form,
});

if (!res.ok) {
  const { error, details } = await res.json();
  throw new Error(`fill-form failed (${error}): ${details}`);
}

const bytes = new Uint8Array(await res.arrayBuffer());
const hash = createHash('sha256').update(bytes).digest('hex');

if (hash !== GOLDEN_HASH) {
  console.error(`snapshot mismatch\n  got:      ${hash}\n  expected: ${GOLDEN_HASH}`);
  process.exit(1);
}
console.log('PDF snapshot matches.');

Wire it into a pull-request workflow the same way you'd wire up any other check: 

# .github/workflows/pdf-snapshot.yml
name: pdf-snapshot
on: [pull_request]
jobs:
  verify:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with: { node-version: 22 }
      - run: node scripts/verify-pdf-snapshot.mjs
        env:
          PDFOPS_API_KEY: ${{ secrets.PDFOPS_API_KEY }}

A pull request that touches the invoice template but leaves the field values alone now fails this job with a clear reason, instead of a green check that never looked.

Updating the golden hash on purpose

When the template changes for real, regenerate the fixture hash and commit it as its own reviewable line, so the person reviewing the pull request sees exactly what triggered it: 

node scripts/verify-pdf-snapshot.mjs --update
git diff fixtures/invoice.sha256

A one-line hash change next to a template edit reads as expected. The same one-line hash change with no template edit in the diff is the exact signal a snapshot test exists to raise.

Getting a key for CI

GitHub-hosted runners draw from a shared pool of Azure IP ranges used by many unrelated jobs at once, and PDFops' anonymous trial meters by IP address. A CI job running keyless can share its 100-requests-per-month bucket with other repositories' traffic on the same address, which makes for a confusing rate-limit failure that has nothing to do with your own test volume. POST /api/signup with an email gets a free key delivered to your inbox: 250 requests/month, no card. 

curl -X POST https://pdfops.dev/api/signup \
  -H "Content-Type: application/json" \
  -d '{"email":"you@example.com"}'

Store it as a repository secret (PDFOPS_API_KEY in the workflow above) and the job's quota is yours alone to watch, with GET /api/usage available if you want to check it from a script instead of the dashboard.

Related

← All posts · PDFops home