Build log markers

A project image build does a lot in one docker build. It installs curl, runs each devcontainer feature, installs each VS Code extension and finalizes ownership, and around that the control plane pulls the base image and may bake a Docker-in-Docker seed. From the raw log alone, a UI can only say something is happening.

The protocol is the cheapest mechanism that works end to end. The build prints a marker line around each block, and the control plane parses those markers back out of the log stream it already forwards. There is no extra channel and no change to the transport: the hook is a line of output. Anything that can print to stdout inside or beside the build can therefore declare a step.

A marker is a contract between three programs that cannot import each other: the shell script that emits it, the control plane that parses it, and the UI that renders the result. Module reference: steps.

The bytesCopy Anchor Link

A marker is one line, starting at column 0, with nothing else on it:

::spunto:step:start:<id>
::spunto:step:end:<id>

The parser is this regular expression, applied to each complete line after one trailing \r is removed:

const MARKER_PREFIX = "::spunto:step:"
const MARKER_RE = /^::spunto:step:(start|end):(.+?)[ \t]*$/

Because of ^, a marker that does not start the line (for example, one quoted inside another line, or prefixed by a timestamp) is not a marker. Whatever forwards the log must hand the tracker the build's output lines unprefixed.

EmittersCopy Anchor Link

There are two emitters, and they produce the same bytes. Both are in the package so that no one retypes the literal.

/** Shell line a build script emits to declare that a block starts. */
export function stepStartLine(id: string): string {
  return `echo '${MARKER_PREFIX}start:${id}'`
}

/** Shell line a build script emits to declare that a block finished successfully. */
export function stepEndLine(id: string): string {
  return `echo '${MARKER_PREFIX}end:${id}'`
}

/** The same two markers, as plain strings — for emitters that aren't shell (the agent). */
export const stepMarker = {
  start: (id: string) => `${MARKER_PREFIX}start:${id}\n`,
  end: (id: string) => `${MARKER_PREFIX}end:${id}\n`,
}

• stepStartLine / stepEndLine return shell source, a line to put in a generated script. The id is wrapped in single quotes and not escaped, so it must not contain '. The ids the package generates never do.
• stepMarker.start / .end return the output itself, newline included. This is for code that runs beside the build rather than in it. Spunto Cloud's node agent uses it to declare the dind-seed block it runs after docker build, and appends the returned line to the stored log as well as streaming it.

The planCopy Anchor Link

The protocol only lights up steps that are already in a list. A BuildStep is:

type BuildStep = {
  id: string                  // what the markers carry
  label: string
  kind: "image" | "runtime" | "feature" | "extensions" | "finalize" // drives the icon
  detail?: string             // secondary line: image ref, OCI ref, "3 extensions"…
  state: "pending" | "running" | "done" | "error" | "skipped"
  startedAt?: string          // ISO, stamped by the control plane when the marker is seen
  completedAt?: string
}

buildImageScript returns the plan from the same call that generates the script, so the list a UI shows greyed out up front cannot drift from the blocks the script actually runs:

A control plane adds the blocks that it runs around the script. Spunto Cloud prepends { id: "base-image", kind: "image", state: "running" } (the pull, which has no end marker of its own) and, when it bakes prewarm images, appends dind-seed. Persist the plan next to the build, and a finished build replays with its blocks and timings.

How the tracker parses a streamCopy Anchor Link

import { BuildStepTracker } from "@spunto/build/steps"

const tracker = new BuildStepTracker(plan)
socket.on("data", (chunk) => {
  const { text, changed } = tracker.ingest(chunk)  // text: what to show; changed: re-render the list
  terminal.write(normalize(text))
  if (changed) saveSteps(tracker.steps)
})
// when docker build exits:
tracker.finish(exitCode === 0 ? "ready" : "error")

ingest(chunk, now?) handles the stream line by line:

1. Line buffering. The chunk is appended to whatever was held from the last call and split on \n. Complete lines are processed. The trailing partial line is held back only if it could still become a marker, meaning it is a prefix of ::spunto:step: or starts with it. Anything else is forwarded immediately. Pull-progress redraws (\x1b[2K\rdownloading 42%) carry cursor codes and no newline, and they must not stall behind a line buffer. A marker split across two chunks (::spunto:step:sta + rt:code-server\n) is still recognized.
2. Ordinary lines pass through verbatim, \r included, with their \n restored.
3. A marker for an unknown id is swallowed. Raw ::spunto:step: lines never leak into the terminal.
4. start:<id> first closes every other step still running: each becomes done with completedAt = now, and its end banner is emitted. That is how the base-image pull, which has no end marker, ends when the script starts speaking. Then the step becomes running with startedAt = now, and its start banner is emitted.
5. end:<id> sets the step to done with completedAt = now, and emits its end banner.

changed is true if any marker was applied in this chunk.

finish(state, now?) closes the list when the build itself ends. It flushes any held partial line, then:

BannersCopy Anchor Link

The tracker replaces a marker with a human-readable banner, so that the raw log can be read block by block. The log is still what a human scrolls through when a build fails. These are the exact bytes:

// start: newline, orange ▸, bold label
`\n\x1b[38;5;208m▸\x1b[0m \x1b[1m${step.label}\x1b[0m\n`

// end: dim "  ✓ label — duration"
`\x1b[2m  ✓ ${step.label}${durationLabel(step)}\x1b[0m\n`

The duration is " — 450ms" under a second, " — 12.3s" (one decimal) above, and empty when either timestamp is missing or the difference is not positive.

Newlines in banners are bare \n. The caller is expected to pass the text through the same xterm normalization it already applies to the rest of the log (\n → \r\n). Emitting \r\n here would double up for callers that normalize.

Replaying a stored logCopy Anchor Link

Store the raw log, markers included, and the steps with their timestamps. To show a finished build, use:

import { renderStoredBuildLog } from "@spunto/build/steps"

terminal.write(normalize(renderStoredBuildLog(build.logs, build.steps)))

• With a non-empty steps, each marker line becomes the same banner as in the live stream, using the recorded startedAt/completedAt for the duration. The banner's trailing \n is dropped because the line keeps its own. Markers for unknown ids are removed.
• With steps empty, null or undefined (for builds stored before steps existed), every line starting with ::spunto:step: is simply removed.

Watching a build and reopening a finished one therefore look the same.

ExampleCopy Anchor Link

What the build script's output looks like before parsing (the [build] lines come from the script itself):

::spunto:step:start:bootstrap
::spunto:step:end:bootstrap
::spunto:step:start:feature:common-utils
[build] Installing feature: common-utils...
…
::spunto:step:end:feature:common-utils
::spunto:step:start:feature:spunto-pack
[build] Installing feature: spunto-pack...
…
::spunto:step:end:feature:spunto-pack
::spunto:step:start:extensions
[build] Installing VS Code extensions to /opt/mp-extensions...
[build] Installing extension: esbenp.prettier-vscode...
[build] Extensions installed
::spunto:step:end:extensions
::spunto:step:start:finalize
[build] Image build complete
::spunto:step:end:finalize

And what the tracker forwards for the extensions block, with escape codes shown:

\n\x1b[38;5;208m▸\x1b[0m \x1b[1mVS Code extensions\x1b[0m\n
[build] Installing VS Code extensions to /opt/mp-extensions...\n
[build] Installing extension: esbenp.prettier-vscode...\n
[build] Extensions installed\n
\x1b[2m  ✓ VS Code extensions — 8.4s\x1b[0m\n

CompatibilityCopy Anchor Link

• Changing the prefix, or the start/end words, breaks every stored log. Replay would show raw markers, or drop banners. It also breaks every image script generated by an older package version that is still running. Do not do it without a reader that accepts both.
• Renaming a step id is safe for new builds (the plan and the markers come from the same call), but stored builds keep their old ids in both their log and their persisted steps, so replay still works.
• Adding a step is free. A tracker swallows markers it has no step for, and a plan entry that never gets a marker is closed by finish.