Skip to main content

State machine

States​

StateReported by workerCommandable by CoreMeaning
idleYesYesNot running. As a desired state: no command outstanding.
runningYesYesExecuting now.
stoppingYesYesStop requested, not yet finished.
failedYesNoTerminated abnormally. Observed only.

Transitions​

FromEventTo
idlestart command appliedrunning
runningstop command appliedstopping
stoppingprocess returnsidle
runningunhandled errorfailed
stoppingunhandled errorfailed
runningprogressive process completesidle
failedstart command appliedrunning

Command lifecycle​

StepActorEffect
1OperatorPUT /api/processes/{id}/desired-state
2CoreSets desiredState, increments desiredToken
3WorkerReads response, sees desiredToken > ackToken
4WorkerApplies command, sets ackToken = desiredToken
5WorkerReports new ackToken on next heartbeat
6CoreClears an acknowledged start, or an acknowledged stop once the process is terminal; resets desiredState to idle
7CoreKeeps runId while the execution is active; closes and retains the execution when the worker reports idle or failed

Command completion and execution completion are different. A start command is complete once accepted, while its execution and runId remain active until the process finishes or stops.

If a new start arrives while the previous run is still stopping, it has not yet been accepted. The worker leaves its token unacknowledged and preserves the old run id. Core redelivers the command until the old run is terminal; the worker then starts the new execution and acknowledges it.

Invariants​

  • ackToken never decreases.
  • A command with desiredToken <= ackToken is a duplicate and must be ignored.
  • failed is never sent as a desired state.
  • idle as a desired state means no command; it is never an instruction to stop.
  • A deferred start never advances ackToken or replaces the active runId.
  • A heartbeat is rejected only when no process in it is readable.