Team Cereustooling wiki

Run a heavy step without freezing the machine

Checkedagainst the code on 2026-11-12 11:43

Anything that uses real memory or CPU runs as an eb job: builds, port builds, plan.py, extraction, big hashing. A job says how much memory it needs at its peak and starts when the machine has that much free.

EB means py -3.12 -I -B S:/eclipse-build/eb.py.

Steps

  1. Run the command through eb run. --mem is required: the peak memory in GB.

    EB run --mem 4 -- py -3.12 build.py ...
    

    Everything after -- is the command. eb run waits until the job is admitted, runs the command at BelowNormal priority, and exits with the command's exit code.

  2. While it waits it prints why, for example waiting (...): memory: 9.1 GB commit free, 2.0 GB promised to starting jobs, need 4.0 + 6 reserve. When it starts after a wait it prints admitted after N s.

  3. See what runs and what waits:

    EB jobs
    

    The first line is commit free X of Y GB; old lock: <holder or free>. Then one line per job: run or wait, pid, declared GB, minutes, the reason text, and for a waiting job why it waits.

  4. Before writing big outputs, check disk space:

    EB free
    

    It prints the free space on S:/ and C:/.

Options of eb run

Flag Meaning
--mem GB Required. Peak memory the command needs.
--exclusive Also take the old shared lock S:/eclipse-0083-stage/.heavy-lock.
--reason TEXT The text shown in eb jobs. Default: the first 120 characters of the command.

Admission rules

A job starts when all of these hold:

  • Fewer than 4 eb jobs are running.
  • Free commit, minus the memory declared by jobs that started less than 120 seconds ago, covers the job's memory plus a 6 GB reserve.
  • For a job of more than 1 GB: no older job of more than 1 GB is waiting for memory. Big jobs go first come, first served. Jobs of 1 GB or less never queue behind big ones.
  • For a job of 8 GB or more, or with --exclusive: the old lock folder S:/eclipse-0083-stage/.heavy-lock is free and no .heavy-lock-priority exists beside it. The job takes the lock and writes owner.txt in it.

A job that could never fit (its memory plus the reserve is more than the commit limit) is refused at once: job needs ... the commit limit is ... GB: it can never run.

Crashed jobs do not block anyone: entries of dead processes are dropped on every check, and an old lock left by a dead eb job is removed. A lock folder made by hand is never touched.

Peaks to declare

Declare the real peak. An under-declared job can still run the machine out of memory.

Step Peak
plan.py (castle-variants) about 2 GB since the compact-table fix (older notes say 17 GB)
DS2 port build about 4 GB
dotnet build 1 GB
eb tool build 1 GB (declared by eb itself)
eb dcx 2 GB (declared by eb itself)
eb cache run --mem, default 2 GB
eb release build 4 GB (declared by eb itself)

The first three numbers come from AGENTS.md and docs/AGENTS-GUIDE.md, not from a measurement made for this page.

Also keep using dotnet -m:1 or -m:2 and cargo -j 2.

What goes wrong

  • bash cannot see S:/ paths. eb run -- bash script.sh finds WSL's bash first, and WSL does not see S:/. Name Git Bash in full:

    EB run --mem 1 -- "C:/Program Files/Git/usr/bin/bash.exe" script.sh
    
  • eb run: no command. Nothing came after --.

  • It keeps waiting on old lock held: .... Another session holds .heavy-lock. EB jobs names the holder. Do not mkdir or remove the lock folder by hand.

  • It waits on old lock: .heavy-lock-priority exists. Someone reserved the old lock. Wait for them to remove it.

Source files this page documents (3)
  • eb:ebjobs.py
  • eb:eb.py
  • eb:AGENTS.md

Changed one of these? Update this page: build/run-heavy-jobs.md (how).