Skip to content

Parallel Execution & Thread Budget

Speed up your QA runs by distributing work across multiple CPU cores.

Enable parallel execution

Set processes in your flow options:

'flows' => [
    'options' => [
        'processes' => 4,  // total CPU cores budget
    ],
    'qa' => ['jobs' => ['phpcs-src', 'phpstan_src', 'phpmd_src', 'parallel_lint']],
],

Or override from the CLI:

githooks flow qa --processes=4

How thread budgeting works

The processes value is the total CPU cores available, not the number of parallel jobs. GitHooks distributes threads across jobs that support internal parallelism:

Tool Internal parallelism flag
phpcs / phpcbf --parallel
parallel-lint -j
psalm --threads
paratest --processes
phpstan Worker count from .neon config (read-only)

For example, with processes: 4 and two threadable jobs (phpcs and parallel-lint), each gets approximately 2 threads.

Reserve cores explicitly (cores)

Sometimes you want a specific job to always get a fixed amount of cores regardless of what the rest of the flow does — typically for paratest workers, phpstan with many workers declared in .neon, or custom jobs running their own parallel runner. Declare cores: N on the job:

'jobs' => [
    'phpcs-src' => [
        'type'  => 'phpcs',
        'paths' => ['src'],
        'cores' => 2,   // reserves 2 cores + passes --parallel=2 to phpcs
    ],
    'psalm-src' => [
        'type'  => 'psalm',
        'paths' => ['src'],
        'cores' => 4,   // reserves 4 cores + passes --threads=4 to psalm
    ],
],

The allocator reserves the declared amount and, for tools with controllable threading, passes the right flag (--parallel, --threads, -j, --processes) automatically. You don't need to remember each tool's specific option.

Running paratest inside a flow

Paratest is a parallel driver for PHPUnit. It reuses the same CLI — --filter, --group, -c — and adds --processes=N to control worker count. GitHooks ships with a dedicated type: paratest:

'jobs' => [
    'paratest_all' => [
        'type'          => 'paratest',
        'configuration' => 'phpunit.xml',
        'cores'         => 4,   // reserves 4 cores + passes --processes=4
    ],
],
'flows' => [
    'options' => ['processes' => 8],
    'qa'     => ['jobs' => ['phpcs-src', 'phpstan_src', 'paratest_all']],
],

With processes: 8 and paratest declaring cores: 4, the allocator leaves 4 cores for the remaining jobs in the flow. See Paratest for the full keyword reference.

Monitor thread usage

Use --monitor to see the actual thread usage after execution:

githooks flow qa --processes=4 --monitor

Check your system

githooks system:info

Shows detected CPU count and current processes configuration. Warns if processes exceeds available CPUs.

Tips

  • Start with processes equal to your CPU core count.
  • PHPStan worker count is configured in the .neon file (maximumNumberOfProcesses), not via GitHooks. It is accounted for in the budget but not adjustable at runtime.
  • Use --monitor to verify that the budget is being distributed as expected.

Memory budget and 2D scheduling

When processes alone is not enough — typical of monolith QA where phpstan and phpunit can together breach a 6 GB CI runner — declare a flow memory-budget and per-job memory reservations. The allocator then admits jobs by both axes (cores AND memory), so two heavy analyzers do not start simultaneously even when cores are free.

'flows' => [
    'options' => [
        'processes'     => 10,
        'memory-budget' => ['warn-above' => 5500, 'fail-above' => 6000],
        'allocator'     => 'greedy',
    ],
    'qa' => ['jobs' => ['phpstan-src', 'phpunit', 'phpcs', 'phpmd-src']],
],
'jobs' => [
    'phpstan-src' => ['type' => 'phpstan', 'cores' => 2, 'memory' => 2000],
    'phpunit'     => ['type' => 'phpunit', 'cores' => 4, 'memory' => 1500],
    'phpcs'       => ['type' => 'phpcs',   'cores' => 1, 'memory' => 256],
    'phpmd-src'   => ['type' => 'phpmd',   'cores' => 2, 'memory' => 800],
],

When the simultaneous RSS sum crosses fail-above, the runtime kills the jobs in flight (process->stop) and skips the queued ones. The flow exits 1 even if every individual job had passed up to that point — that is the conceptual key of the feature.

Calibrating with --stats

Run any flow with --stats first without thresholds to discover real peaks. The canonical 5-column table prints after the Results: line:

githooks flow qa --stats

Then declare conservative warn-above/fail-above based on the table.

The status column is colour-coded when the output is decorated (TTY or under CI — see CI/CD → ANSI colour in CI logs):

Cell Colour Meaning
OK default Job passed cleanly.
OK ⚠ yellow Job passed but crossed a warn-above / warn-after threshold.
KO red Job failed (tool exit code ≠ 0, or fail-above / fail-after crossed).
blue Job was skipped (no input files match, fail-fast cancellation, killed by flow guard).
TOTAL ✔ green Flow passed.
TOTAL ✗ red Flow failed (any job KO, or flow-level budget exceeded).

FIFO vs greedy

Strategy When to pick
fifo (default) Predictable order. Use it when CI parity matters or when most jobs are similar in cost.
greedy A heavy job declared late blocks lighter ones in FIFO; greedy lets them slip in while the heavy one waits for resources.

The strategy applies in both 1D mode (cores only) and 2D mode. See Memory budget for the full configuration reference.

Platform support

Platform Sampler Notes
Linux /proc/<PID>/status walked across the process tree Native, lowest overhead.
macOS ps -o pid=,ppid=,rss= -ax once per sample Single subprocess invocation per tick (~ms).
Windows not available in v3.3 Runtime emits one stderr warning and disables thresholds. The 2D allocator still schedules using declared memory: reservations and --stats still reports cores. Sampler is reserved for a future iteration.

In every platform the RSS values reported reflect the entire process tree rooted at the job's PID — Symfony spawns commands under a shell wrapper, and the heavy analyzers (php phpstan, php phpunit) run as child processes whose memory must be summed for the figure to make sense.