Keep files in the store · Team Cereus tooling wiki

The store keeps every input a shipped file was built from, without putting big files in git. It is S:/eclipse-build-store/objects/<first two hex digits>/<sha256>: one file ("blob") per content. A named list of files is a "set". frozen/INDEX.tsv in the repo has one line per set, and that file is the only part of the store that is in git.

Checked against the code. No store command was run for this page.

EB means py -3.12 -I -B S:/eclipse-build/eb.py.

What goes where

Goes in git	Goes in the store
eclipse-build's own code and docs	logs and dumps
`frozen/INDEX.tsv`	reports, copies of other repos' files, tool copies
	build output
	anything over 5 MiB

Never in git and never in the store: secrets. store ingest and rescue leave out files named discord-webhook.txt, discord-posted.txt or .env, any file with webhook in its name, files ending in .pfx .p12 .pem .key .snk .cer, and every signing folder.

Put a folder in the store

EB store ingest <path> --set NAME [--root DIR] [--copy-below BYTES] [--note TEXT]

Hashes every file under <path>. Junctions and symlinks are never followed. The folders bin obj .git __pycache__ node_modules target .vs .idea .gradle are skipped.
A file on S: of at least --copy-below bytes (default 4 MiB) becomes a hardlink in the store, which costs no disk space. Smaller files and files on another drive are copied. A hardlink that fails falls back to a copy.
--root is the folder the recorded paths are relative to. Default: <path>.
Running it again changes nothing for blobs the store already has.
It prints set NAME: N files, X MiB (new link .., new copy .., known .., indexed only ..) in S s -> list <hash> and updates frozen/INDEX.tsv.
It refuses a copy that would leave under 6 GB free on the store's drive: STOP: copying ... would leave under 6.0 GB free.

Rescue a folder

EB rescue <src> [--set NAME] [--note TEXT] [--copy-below BYTES] [--keep-bin] [--files LIST] [--index-other-volume]

The same as ingest, for freezing a work folder as it is.

Flag	Meaning
`--set`	Set name. Default: `rescue_` plus the path with `/` turned into `_`.
`--keep-bin`	Also freeze `bin/` folders (frozen tool builds).
`--files LIST`	A text file with paths relative to `<src>`, one per line. Only those are rescued.
`--index-other-volume`	For big files on another drive than the store: record the hash only, do not copy the bytes.
`--copy-below`	As for ingest. Default 4 MiB.

It prints how many secret-looking files it left out.

Check, restore, measure

Command	What it does
`EB store verify [--set NAME ...]`	Re-hashes the blobs of the named sets, or of every set. Prints `MISSING` and `CHANGED` lines and one summary line per set. Exit code 1 if anything is missing or changed.
`EB store restore --set NAME --to DIR`	Copies a set's files out to `DIR`. Never links.
`EB store stat`	Blob count, total size, how many blobs are shared by hardlink, free space on `S:`.

CHANGED ... (a linked file was rewritten in place) means a tool rewrote an original file that the store shares by hardlink, so the stored bytes changed with it. Report it; the set no longer holds what it recorded.

Freeze stage folders

EB freeze-stages <stage.json ...>

Freezes each stage folder into the store, copies it out to S:/eclipse-build-out/sets/stage-<hash>, hash-checks the copy, and prints the path of each copy's stage.json. The folders work, scratch, tmp and _work inside a stage are left out. eb release check and eb release build do this for you (check-and-build-a-release.md); you rarely need it by hand. It runs the provenance gate first.

Rules

Never delete or move originals because they are in the store: S:/souls-script-kt/.tmp, stage folders, S:/_modding, S:\Smithbox-havok-pin, release trees. Deleting needs Stan's explicit OK.
Never set the read-only attribute on blobs. It is per file on disk, hardlinks share it, and it would break the release build's delete-then-copy.
Never hardlink a release tree into the store. The release build treats link count 1 as "changed in this release".

The pre-commit hook

hooks/pre-commit refuses a commit that:

adds a line that looks like a secret (a Discord webhook URL, a GitHub token, a private key, an AWS key, an sk- key): pre-commit: a staged change looks like a secret;
adds or changes a file over 5 MiB (other than frozen/*.tsv): pre-commit: files over 5 MiB belong in the store;
adds logs, dumps, backups (.log .dmp .bak .orig .tmp), bin/, obj/, __pycache__/, rescue/20* or a frozen/*.tsv other than INDEX.tsv: pre-commit: logs, backups, build output and snapshots go to the store, not git.

Install it once per clone:

git -C S:/eclipse-build config core.hooksPath hooks

Do not bypass it. .gitignore lists the same file kinds.