Team Cereustooling wiki

Skip unchanged steps and make .dcx files comparable

Partly checkedagainst the code on 2026-11-12 11:43

Two helpers for stage builds. Checked against the code; neither was run for this page.

EB means py -3.12 -I -B S:/eclipse-build/eb.py.

eb cache run: skip a step whose inputs did not change

Use it for an expensive step whose inputs rarely change, for example the animation stage of a DS2 port.

EB cache run --in <inputs...> --out <outputs...> [--env NAME...] [--py EXE...] [--mem GB] [--no-cache] -- <command ...>
Flag Meaning
--in Every file or folder the command reads, the code included.
--out Every file or folder the command writes. Required.
--env Environment variables that change the result. Their values go into the key.
--py Python interpreters the command runs. Their path, exact version and installed packages go into the key.
--mem Peak memory in GB for job admission. Default 2.
--no-cache Run anyway and check that the outputs equal the cached ones.

How it works:

  • The key is made from the command, the content of every input, the listed environment variables and the --py interpreters. Input and output paths inside the command are replaced by placeholders, so the same step on the same content in another work folder still hits.
  • Hit: the outputs are deleted, restored from the store and re-hashed. It prints eb cache: hit <key> (N files restored in X s, the run took Y s).
  • Miss: the outputs are deleted first, the command runs as an eb job, and its outputs are copied into the store. It prints eb cache: miss <key>: ran in X s, N output files cached. A command that fails is not cached, and its exit code is returned.
  • --no-cache with an existing entry prints outputs identical to the cached run, or OUTPUTS DIFFER from the cached run: an input is missing from --in, or the step is not deterministic and exits 3.

Things to know:

  • The outputs belong to the step. They are deleted before every run and every restore. Never name a folder in --out that holds anything else.
  • An input you forget in --in gives a stale hit. List the code too.
  • Cache entries are in S:/eclipse-build-out/cache, the bytes in the store.
  • eb cache: restored ... does not match its hash; delete <entry> and run again means a store blob changed. Delete the named entry file and run again.

DS2 port builds already cache their animation stage this way. EB_NO_CACHE=1 turns that off (per docs/AGENTS-GUIDE.md; the switch is read in the souls-script-kt build script, not in eclipse-build, and was not checked for this page).

eb dcx normalize and eb dcx check: canonical Kraken encoding

Two builds of the same content can write different .dcx bytes, because the Kraken compressor's output varies between runs. normalize re-encodes each file so equal content gives equal bytes. The content never changes.

EB dcx normalize <files or folders...>
EB dcx check <files or folders...>
Mode What it does
normalize Re-encodes every Kraken .dcx file and rewrites it when the bytes differ.
check Reports the files that would be re-encoded. Writes nothing.
  • A folder is searched for *.dcx in all subfolders. A path that does not exist is skipped silently.
  • Each file is decompressed and compressed twice. The two results must agree and must decompress to the same content.
  • A file that was or would be changed prints dcxnorm <file>: re-encoded (... -> ... bytes payload, content ... unchanged). The last line is dcxnorm: N dcx files checked.
  • check exits 0 even when it lists files. Read the output, not the exit code.
  • It runs as a 2 GB eb job and works on 4 files at a time. EB_DCX_JOBS=1 makes it one at a time.

When to use which:

  • eb stage make already normalizes the files it ships. You do not need to run it for that.
  • Run normalize on your own build folders before you compare or hash them yourself.
  • Run check when you only want to know whether a folder is canonical.

What goes wrong:

  • <file>: two compressions differ or canonical payload does not round-trip: stop and report it. The file is left as it was.
  • <file>: payload differs from the canonical one and the file tail is not zero padding: the file has data after the compressed payload. It is not rewritten.
Source files this page documents (2)
  • eb:eb.py
  • eb:tools/release-py/dcxnorm.py

Changed one of these? Update this page: build/cache-and-dcx.md (how).