Chapter 0 · agent-neuron · Lab

The idea in one alert

A burst of inputs, a cheap check on each, and one costly step. The idea is that the rule for when that step runs sits on the neuron that runs it, not in a controller.

The problem

Picture a burst. In the recording, forty invented alerts arrive for five services in under six seconds.

Cheap checks look at every alert. One step is costly: in the recording it is a stand-in that waits 500 ms, where a call to a model would go. You want it to run only when the cheap checks agree that something is wrong.

Then the burst keeps going. A newer alert for the same service makes the older work pointless, because its answer would be about input that has already been replaced. So the rule has two parts: when to act, and when to stop.

The common answer

A common answer is a controller in the middle. Every detector reports to it. It counts the flags for each alert, keeps a timer for each, starts the costly step when enough have agreed, and cancels the step when something newer arrives.

It works. The cost is that every rule sits in one place that has to know about every detector, every timer and every cancellation. Changing a rule, or adding a detector, means changing the controller.

// a sketch, invented for this page
const flags = new Map();   // alert -> detectors that flagged it
const timers = new Map();  // alert -> its expiry timer
const running = new Map(); // service -> the call in flight

function onFlag(alert, detector) {
  const seen = flags.get(alert.id) ?? new Set();
  seen.add(detector);
  flags.set(alert.id, seen);
  if (!timers.has(alert.id)) {
    timers.set(alert.id, setTimeout(() => forget(alert.id), WITHIN_MS));
  }
  if (seen.size >= THRESHOLD) {
    clearTimeout(timers.get(alert.id));
    cancel(running.get(alert.service));
    running.set(alert.service, startCostlyStep(alert));
  }
}

function onAlert(alert) {
  dropFlagsFor(alert.service);        // older flags are stale
  cancel(running.get(alert.service)); // older work is stale
}

Sketch, inventedFigure 1. A sketch of a central controller, invented for this page. It is not code from the recordings.

The rule on the box

The alternative I’m testing puts the rule on the box that acts. The costly neuron’s wrapper carries it: how many different senders must agree, how long a flag may wait, and that newer input for the same key makes older work stale.

A network is a small JSON file: a list of neurons and a list of links between them.

On the costly neuron the rule is two lines: a threshold of 2 and a window of 500 ms. The key that says what makes work stale, called replaces, is not in the file. The source sets it on every alert, here the service name, and it travels with the signal.

The detectors know nothing of this. Each looks at an input, then answers or stays quiet. The counting, the waiting and the cutting all happen at the costly neuron.

alertssource

logsshell script

deploysshell script

metricsJavaScript

hypothesismodel stand-in

acts when two different detectors agree
within 500 ms
a newer alert for the same service makes older work stale

diagnosiscomposed

Illustration, invented dataFigure 2. The network behind the recordings: a source, three cheap detectors, the costly neuron with its rule written on it, and a composed neuron that gives one verdict. Signals run left to right, or top to bottom on a phone. The alerts are invented.

{
  "neurons": [
    …
    {
      "id": "hypothesis",
      "run": [ … ],
      "threshold": 2,
      "within": 500
    },
    …
  ],
  "links": [
    …
    { "from": "logs", "to": "hypothesis" },
    { "from": "deploys", "to": "hypothesis" },
    { "from": "metrics", "to": "hypothesis" },
    …
  ]
}

ExcerptFigure 3. The costly neuron as the network file writes it, shortened, with the links that reach it. The numbers are read from the recording of invented alerts.

One alert, step by step

Start with alert 1, for gateway, which opens the recording.

At 2 ms it goes to all three detectors at once.
Logs stays quiet at 14 ms, deploys flags it at 17 ms and metrics flags it at 27 ms.
The earlier flag waits. The later one, from metrics at 27 ms, makes two different detectors agree, so the costly step starts.
The step takes 500 ms. At 152 ms alert 2 arrives, also for gateway, so the work for alert 1 is stale.
The recording has the call cut at 178 ms, 151 ms after it started, and no answer from it is passed on.
Alert 2 goes through the same steps. Two calls in this burst were cut. The call for alert 3, with no newer gateway alert behind it, finished at 830 ms, and only its result goes on to the diagnosis neuron.

What it did

Across forty alerts the costly step started 17 times, not once for every flag. 10 of those calls were cut as stale and 7 finished.

With a threshold of 1, where one flag is enough, the same alerts started 52 calls. With a threshold of 3 they started 0.

Hold that against three things. The alerts are invented and arrive in bursts, so the waste is built in.

The recordings that reach a verdict reach the same one, by construction, so they show when work starts and stops, not whether an answer is right.

And each recording is one run on one machine, played back.

Agreement, the next chapter, plays all three recordings on one clock and goes through what they do not show.

Each recording is a separate run at its own threshold, on the same invented alerts.
Threshold	Alerts	Calls started	Cut as stale	Finished
1	40	52	31	21
2	40	17	10	7
3	40	0	0	0

Recording, invented dataFigure 4. The same forty invented alerts, recorded three times, once at each threshold.

Any program fits

The contract is small. The wrapper hands a program one JSON value on its input and reads one JSON value from its output. Printing nothing means the program has nothing to say, and nothing is passed on.

In the recordings, two of the detectors are shell scripts and one is a few lines of JavaScript. Any other program that can read and write JSON would do, in whatever language it is written.

A neuron with a threshold above one is handed a little more: the input that crossed it, and the inputs that agreed with it.

import json, sys

alert = json.load(sys.stdin)
if alert.get("severity") == "crit":
    print(json.dumps({"service": alert["service"]}))
# printing nothing means quiet

Sketch, inventedFigure 5. A detector in Python, invented for this page, in the shape the contract describes.

Why neuron, and what is not built

Why the name neuron? It is a loose name, not a model of the brain. Here it stands for connective tissue between programs: a box with many inputs that acts when its own rule says so.

The name hints at two things that are not built: connections that strengthen and weaken, and a network that works out which ones matter. In every recording here each detector counts the same.

Unix pipes join programs that read and write text too, but in a line, one into the next. Here they join in a graph, many into one and one into many, and the rule for when to act is set on each box.

That is the question I’m testing: whether the rule for when to act belongs on each neuron’s wrapper, not in a central controller.

To try it, run the same alert storm yourself with npm run demo:network. It needs Node and no Rust, and it makes no network requests. The code is public:

overfocus/agent-neuron