vps-health-api/agent/README.md

# Workstation agent

Collects one snapshot of the machine it runs on and pushes it to
`POST /hosts/<id>` on the health API. Cron runs it every 5 minutes; the
dashboard reads the result through `GET /hosts`.

Zero dependencies — `node:http` / `node:https` and the `metrics.js` already
shared with the server. Node 22+.

| File | Role |
|------|------|
| `push-metrics.js` | Collect, build the payload, POST once, exit |
| `run-push.sh` | Cron wrapper: load the env file, check the install, exec node |

This directory is **not** part of the server image: the `Dockerfile` copies
`package.json index.js metrics.js` one by one, and a test pins that it stays
that way. The agent ships by copying files onto a workstation, never by deploy.

## What it deliberately does not do

**No queue, no replay.** One run is one attempt. A push that fails is dropped,
logged, and the process exits non-zero — the next beat is five minutes away and
a heartbeat from five minutes ago describes a machine that no longer exists.
Nothing is spooled to disk, so nothing can ever be replayed to make the
dashboard show a past that is no longer true.

**No secret in the logs.** `run-push.sh` is meant to be piped into `logger`, so
everything the agent prints ends up in syslog and journald permanently. The
failure path therefore reports **only** `err.code` and the HTTP status — never
an error object, never the request options, never a response body, and never a
header. That is also why `run-push.sh` must never gain a `set -x`: the shell
would echo the token as it sources the env file.

The cost is real and accepted: a `400` tells you the server rejected the
payload, not why. The reason is in the server's logs, and `--dry-run` (below)
shows you the exact payload that was sent.

## Install on a new workstation

### 1. Freeze a copy

The cron must point at a copy, **never at the git working tree**. The server is
deployed by a manual trigger, so the checked-out repo drifts freely: a branch
checkout in `~/claude-code/vps-health-api` would silently change — or break —
the heartbeat of the workstation, and nothing would say so.

`push-metrics.js` loads `metrics.js` from one directory up, so the copy keeps
the repo layout:

```
mkdir -p ~/.local/share/maximus-host-agent/agent
cp metrics.js ~/.local/share/maximus-host-agent/
cp agent/push-metrics.js agent/run-push.sh ~/.local/share/maximus-host-agent/agent/
chmod +x ~/.local/share/maximus-host-agent/agent/run-push.sh
```

Re-run those three `cp` after any change to the agent — that copy step *is* the
deployment.

### 2. Write the env file

`~/.config/maximus-host-agent.env`, mode `600` (the agent warns on any other
mode). Use `printf`, not a heredoc: a trailing newline or space inside the token
value is the most common install failure.

```
HOSTS_API_URL=https://health.lacompagniemaximus.com
HOSTS_INGEST_TOKEN=<the ingest token>
HOST_ID=thinkpad
# NODE_BIN=/usr/bin/node          # only if cron cannot find node (see below)
# HOST_AGENT_TIMEOUT_MS=10000     # push timeout, wall-clock; default 10000
```

```
chmod 600 ~/.config/maximus-host-agent.env
```

`HOSTS_INGEST_TOKEN` is the write-only token: it can push snapshots for an
allowlisted id and nothing else. It is deliberately *not* `HEALTH_TOKEN`, so
that the two can be rotated independently and so a workstation that only pushes
never needs the read token.

Do not read more into that separation than it gives you. On the ThinkPad it
buys no containment at all: that machine already stores `HEALTH_TOKEN` in
cleartext for the `defenseur-auto` cron, so losing the laptop compromises both
tokens. **Rotate them together** — see `secret-rotation-ops.md`.

`HOST_ID` must match `^[a-z0-9][a-z0-9-]{0,31}$` and be listed in the server's
`HOSTS_ALLOWED_IDS`, otherwise the push comes back `403`.

### 3. Dry run

Collect and print the payload without touching the network or needing any
config:

```
node ~/.local/share/maximus-host-agent/agent/push-metrics.js --dry-run
```

Then one real push, by hand, before installing the cron:

```
~/.local/share/maximus-host-agent/agent/run-push.sh
# host-agent: ok id=thinkpad status=204 hostname=thinkpad-x1
```

Confirm the server side (from a machine that holds `HEALTH_TOKEN`):

```
curl -H "Authorization: Bearer $HEALTH_TOKEN" \
  https://health.lacompagniemaximus.com/hosts
```

The entry for your id should show `online: true` and a small `ageSeconds`.

### 4. Install the cron

```
crontab -e
```

```
*/5 * * * * $HOME/.local/share/maximus-host-agent/agent/run-push.sh 2>&1 | logger -t host-agent
```

Read what it did:

```
journalctl -t host-agent --since -1h
```

If the line works by hand but produces nothing under cron, it is almost always
`node`: cron's `PATH` is `/usr/bin:/bin`, and a node installed by nvm or under
`/usr/local` is not on it. Set `NODE_BIN` to an absolute path in the env file —
the wrapper says so explicitly rather than failing silently.

## One id per workstation

The server keeps **one file per host id** and the last write wins. Two
workstations configured with the same `HOST_ID` therefore overwrite each other
every five minutes, and nothing anywhere reports an error: the dashboard shows
one host that looks perfectly healthy while its numbers come from whichever
machine pushed last.

The signal to watch for is the `hostname` field in `GET /hosts` **changing
between polls** while the id stays the same — plus CPU and memory that jump
around without pattern. If you see that, two machines are sharing an id.

Give every workstation its own id and add it to `HOSTS_ALLOWED_IDS` on the
server before installing the agent on it.

## Exit codes

| Code | Meaning |
|------|---------|
| 0 | Snapshot accepted (server answered 2xx, normally `204`) |
| 1 | Install problem: env file missing, a variable unset, malformed `HOST_ID`, node not found, `metrics.js` not found next to the agent |
| 2 | The push did not land: network error (`code=…`) or a non-2xx answer (`HTTP …`) |

A `1` needs a human on the workstation. A `2` usually fixes itself on the next
beat; a `2` that repeats for an hour means the API or the link is down.

## Payload

Exactly the `GET /health` body minus `logto` and `timestamp`:

```
{ hostname, uptime,
  cpu:    { model, cores, loadAvg, usagePercent },
  memory: { totalGB, usedGB, freeGB, usagePercent },
  disk:   { totalGB, usedGB, freeGB, usagePercent } }
```

`loadAvg` is exactly three numbers (1, 5 and 15 minutes), as `os.loadavg()`
returns them; the server answers `400` to any other length.

Sharing `metrics.js` with the server keeps the two in step, but it is not the
guarantee — the frozen copy can lag behind a deployed server. The real contract
is the server's own validation, and `__tests__/agent.test.js` checks the agent's
payload against it (including a real snapshot through the real handler).

## claude-usage agent (`agent/claude-usage/`)

A second, independent agent on the ThinkPad. Every 15 minutes it reads the
Claude plan limits, aggregates the local Claude Code transcripts over the last
30 Toronto days, and pushes one `schemaVersion: 1` snapshot to
`POST /claude-usage`; the admin dashboard reads it through `GET /claude-usage`.

It lives in this repo, next to the server, because the snapshot contract and
its sanitizer (`claude-usage.js`) are tested together:
`__tests__/agent-claude-usage.test.js` builds a real snapshot from fixtures and
checks that `sanitizeClaudeUsage()` keeps it unchanged. Like the host agent, it
never enters the server image.

| File | Role |
|------|------|
| `aggregate.js` | Read `~/.claude/projects/**/*.jsonl`, build `days` and `projects` |
| `limits.js` | One GET to Anthropic's OAuth usage endpoint, mapped to `limits.items` |
| `state.js` | Local cache: frozen days and last good limits |
| `push-claude-usage.js` | Entry point: aggregate, limits, write the cache, push once, exit |
| `run-claude-usage.sh` | Cron wrapper: lock, load the env file, check the install, exec node |

### What it deliberately does not do

**It never refreshes the OAuth token and never writes
`~/.claude/.credentials.json`.** Refresh tokens are single use: refreshing from
here would log Claude Code out. The file is only read; an expired token gives
`limits.status: "token_expired"` and no call at all, until Claude Code itself
refreshes it.

**The Anthropic URL is pinned.** It is a constant in `limits.js`, no
environment variable changes it, and redirects are refused: the same process
holds the OAuth bearer and a configurable push URL, and the bearer must only
ever reach `https://api.anthropic.com`. `CLAUDE_USAGE_API_URL` must be
`https://` (the agent exits 1 otherwise).

**One call, one push, no retry.** The usage endpoint rate-limits hard and sends
no `Retry-After`. When it fails, the snapshot carries the failure status
(`http_error`, `unavailable`, ...) with the last good items and their
`fetchedAt`, so the dashboard can show how old they are.

**No secret in the logs.** Same rule as the host agent: the output goes to
`logger -t claude-usage`, so failures print fixed templates plus an HTTP status
or an error code — never an error object, a response body, or a header — and
`run-claude-usage.sh` must never gain a `set -x`.

### Install

Freeze a copy — never point the cron at the git working tree:

```
mkdir -p ~/.local/share/maximus-claude-usage
cp agent/claude-usage/aggregate.js agent/claude-usage/limits.js agent/claude-usage/state.js agent/claude-usage/push-claude-usage.js agent/claude-usage/run-claude-usage.sh ~/.local/share/maximus-claude-usage/
chmod +x ~/.local/share/maximus-claude-usage/run-claude-usage.sh
```

Re-run that `cp` after any change to the agent: the copy step *is* the
deployment. The code lives in `~/.local/share/`, the cache in
`~/.local/state/maximus-claude-usage/` (`state.json` and `run.lock`, created on
the first run): wiping the copy never loses the cache, and vice versa.

Env file `~/.config/maximus-claude-usage.env`, mode `600` (the wrapper warns on
any other mode). Write it with `printf`, not a heredoc: a trailing newline in
the token is refused with exit 1.

```
CLAUDE_USAGE_API_URL=https://health.lacompagniemaximus.com
CLAUDE_USAGE_INGEST_TOKEN=<the claude-usage ingest token>
# CLAUDE_USAGE_LIMITS=off        # skip the Anthropic call entirely
# NODE_BIN=/usr/bin/node         # only if cron cannot find node
```

```
chmod 600 ~/.config/maximus-claude-usage.env
```

`CLAUDE_USAGE_INGEST_TOKEN` is its own token: neither `HOSTS_INGEST_TOKEN` nor
`HEALTH_TOKEN` works on `POST /claude-usage`, and it opens nothing else.

Dry run — prints the snapshot, pushes nothing, leaves the cache untouched, needs
no env file (it still calls the Anthropic endpoint once unless
`CLAUDE_USAGE_LIMITS=off`):

```
CLAUDE_USAGE_LIMITS=off node ~/.local/share/maximus-claude-usage/push-claude-usage.js --dry-run
```

Then one real run by hand, before installing the cron (the first one scans every
transcript of the window and can take a while; the next ones only read files
written since yesterday):

```
~/.local/share/maximus-claude-usage/run-claude-usage.sh
# claude-usage: ok status=204 limits=ok bytes=...
```

Crontab line:

```
*/15 * * * * $HOME/.local/share/maximus-claude-usage/run-claude-usage.sh 2>&1 | logger -t claude-usage
```

```
journalctl -t claude-usage --since -1h
```

A beat that finds the previous one still running (`flock` on
`~/.local/state/maximus-claude-usage/run.lock`) logs
`a run is already in progress, skipping` and exits 0.

### Exit codes

| Code | Meaning |
|------|---------|
| 0 | Snapshot accepted (2xx), or a skipped beat (lock held) |
| 1 | Install problem: env file missing, a variable unset, URL not `https://`, token with an invalid character, node or flock not found |
| 2 | The push did not land (network or non-2xx), or the snapshot exceeded 64 KiB (`payload_too_large`, nothing sent) |

The limits failing is **not** an exit code: the snapshot is still pushed, with
`limits.status` saying why.

### Unreadable transcripts

A day is frozen in `state.json` two days after it ends, and never recomputed
afterwards. A beat that cannot read part of `~/.claude/projects` (a directory it
cannot list, a transcript removed between the walk and its `stat`, a stream
error) therefore freezes **no** new day: it still pushes its snapshot, exits as
usual, and logs only a count:

```
claude-usage: warning: read_errors=N, no day frozen this beat
```

The next clean beat freezes the days that were waiting. A file that stays
unreadable (left at `chmod 000`, for instance) blocks freezing on every beat:
each run then rescans the whole 30-day window, slower but correct, and this
log line is the only sign. Fix the file's permissions. To force a full
recount after the fact, `rm ~/.local/state/maximus-claude-usage/state.json`.