300 lines
12 KiB
Markdown
300 lines
12 KiB
Markdown
# Workstation agent
|
|
|
|
Collects one snapshot of the machine it runs on and pushes it to
|
|
`POST /hosts/<id>` on the health API. Cron runs it every 5 minutes; the
|
|
dashboard reads the result through `GET /hosts`.
|
|
|
|
Zero dependencies — `node:http` / `node:https` and the `metrics.js` already
|
|
shared with the server. Node 22+.
|
|
|
|
| File | Role |
|
|
|------|------|
|
|
| `push-metrics.js` | Collect, build the payload, POST once, exit |
|
|
| `run-push.sh` | Cron wrapper: load the env file, check the install, exec node |
|
|
|
|
This directory is **not** part of the server image: the `Dockerfile` copies
|
|
`package.json index.js metrics.js` one by one, and a test pins that it stays
|
|
that way. The agent ships by copying files onto a workstation, never by deploy.
|
|
|
|
## What it deliberately does not do
|
|
|
|
**No queue, no replay.** One run is one attempt. A push that fails is dropped,
|
|
logged, and the process exits non-zero — the next beat is five minutes away and
|
|
a heartbeat from five minutes ago describes a machine that no longer exists.
|
|
Nothing is spooled to disk, so nothing can ever be replayed to make the
|
|
dashboard show a past that is no longer true.
|
|
|
|
**No secret in the logs.** `run-push.sh` is meant to be piped into `logger`, so
|
|
everything the agent prints ends up in syslog and journald permanently. The
|
|
failure path therefore reports **only** `err.code` and the HTTP status — never
|
|
an error object, never the request options, never a response body, and never a
|
|
header. That is also why `run-push.sh` must never gain a `set -x`: the shell
|
|
would echo the token as it sources the env file.
|
|
|
|
The cost is real and accepted: a `400` tells you the server rejected the
|
|
payload, not why. The reason is in the server's logs, and `--dry-run` (below)
|
|
shows you the exact payload that was sent.
|
|
|
|
## Install on a new workstation
|
|
|
|
### 1. Freeze a copy
|
|
|
|
The cron must point at a copy, **never at the git working tree**. The server is
|
|
deployed by a manual trigger, so the checked-out repo drifts freely: a branch
|
|
checkout in `~/claude-code/vps-health-api` would silently change — or break —
|
|
the heartbeat of the workstation, and nothing would say so.
|
|
|
|
`push-metrics.js` loads `metrics.js` from one directory up, so the copy keeps
|
|
the repo layout:
|
|
|
|
```
|
|
mkdir -p ~/.local/share/maximus-host-agent/agent
|
|
cp metrics.js ~/.local/share/maximus-host-agent/
|
|
cp agent/push-metrics.js agent/run-push.sh ~/.local/share/maximus-host-agent/agent/
|
|
chmod +x ~/.local/share/maximus-host-agent/agent/run-push.sh
|
|
```
|
|
|
|
Re-run those three `cp` after any change to the agent — that copy step *is* the
|
|
deployment.
|
|
|
|
### 2. Write the env file
|
|
|
|
`~/.config/maximus-host-agent.env`, mode `600` (the agent warns on any other
|
|
mode). Use `printf`, not a heredoc: a trailing newline or space inside the token
|
|
value is the most common install failure.
|
|
|
|
```
|
|
HOSTS_API_URL=https://health.lacompagniemaximus.com
|
|
HOSTS_INGEST_TOKEN=<the ingest token>
|
|
HOST_ID=thinkpad
|
|
# NODE_BIN=/usr/bin/node # only if cron cannot find node (see below)
|
|
# HOST_AGENT_TIMEOUT_MS=10000 # push timeout, wall-clock; default 10000
|
|
```
|
|
|
|
```
|
|
chmod 600 ~/.config/maximus-host-agent.env
|
|
```
|
|
|
|
`HOSTS_INGEST_TOKEN` is the write-only token: it can push snapshots for an
|
|
allowlisted id and nothing else. It is deliberately *not* `HEALTH_TOKEN`, so
|
|
that the two can be rotated independently and so a workstation that only pushes
|
|
never needs the read token.
|
|
|
|
Do not read more into that separation than it gives you. On the ThinkPad it
|
|
buys no containment at all: that machine already stores `HEALTH_TOKEN` in
|
|
cleartext for the `defenseur-auto` cron, so losing the laptop compromises both
|
|
tokens. **Rotate them together** — see `secret-rotation-ops.md`.
|
|
|
|
`HOST_ID` must match `^[a-z0-9][a-z0-9-]{0,31}$` and be listed in the server's
|
|
`HOSTS_ALLOWED_IDS`, otherwise the push comes back `403`.
|
|
|
|
### 3. Dry run
|
|
|
|
Collect and print the payload without touching the network or needing any
|
|
config:
|
|
|
|
```
|
|
node ~/.local/share/maximus-host-agent/agent/push-metrics.js --dry-run
|
|
```
|
|
|
|
Then one real push, by hand, before installing the cron:
|
|
|
|
```
|
|
~/.local/share/maximus-host-agent/agent/run-push.sh
|
|
# host-agent: ok id=thinkpad status=204 hostname=thinkpad-x1
|
|
```
|
|
|
|
Confirm the server side (from a machine that holds `HEALTH_TOKEN`):
|
|
|
|
```
|
|
curl -H "Authorization: Bearer $HEALTH_TOKEN" \
|
|
https://health.lacompagniemaximus.com/hosts
|
|
```
|
|
|
|
The entry for your id should show `online: true` and a small `ageSeconds`.
|
|
|
|
### 4. Install the cron
|
|
|
|
```
|
|
crontab -e
|
|
```
|
|
|
|
```
|
|
*/5 * * * * $HOME/.local/share/maximus-host-agent/agent/run-push.sh 2>&1 | logger -t host-agent
|
|
```
|
|
|
|
Read what it did:
|
|
|
|
```
|
|
journalctl -t host-agent --since -1h
|
|
```
|
|
|
|
If the line works by hand but produces nothing under cron, it is almost always
|
|
`node`: cron's `PATH` is `/usr/bin:/bin`, and a node installed by nvm or under
|
|
`/usr/local` is not on it. Set `NODE_BIN` to an absolute path in the env file —
|
|
the wrapper says so explicitly rather than failing silently.
|
|
|
|
## One id per workstation
|
|
|
|
The server keeps **one file per host id** and the last write wins. Two
|
|
workstations configured with the same `HOST_ID` therefore overwrite each other
|
|
every five minutes, and nothing anywhere reports an error: the dashboard shows
|
|
one host that looks perfectly healthy while its numbers come from whichever
|
|
machine pushed last.
|
|
|
|
The signal to watch for is the `hostname` field in `GET /hosts` **changing
|
|
between polls** while the id stays the same — plus CPU and memory that jump
|
|
around without pattern. If you see that, two machines are sharing an id.
|
|
|
|
Give every workstation its own id and add it to `HOSTS_ALLOWED_IDS` on the
|
|
server before installing the agent on it.
|
|
|
|
## Exit codes
|
|
|
|
| Code | Meaning |
|
|
|------|---------|
|
|
| 0 | Snapshot accepted (server answered 2xx, normally `204`) |
|
|
| 1 | Install problem: env file missing, a variable unset, malformed `HOST_ID`, node not found, `metrics.js` not found next to the agent |
|
|
| 2 | The push did not land: network error (`code=…`) or a non-2xx answer (`HTTP …`) |
|
|
|
|
A `1` needs a human on the workstation. A `2` usually fixes itself on the next
|
|
beat; a `2` that repeats for an hour means the API or the link is down.
|
|
|
|
## Payload
|
|
|
|
Exactly the `GET /health` body minus `logto` and `timestamp`:
|
|
|
|
```
|
|
{ hostname, uptime,
|
|
cpu: { model, cores, loadAvg, usagePercent },
|
|
memory: { totalGB, usedGB, freeGB, usagePercent },
|
|
disk: { totalGB, usedGB, freeGB, usagePercent } }
|
|
```
|
|
|
|
`loadAvg` is exactly three numbers (1, 5 and 15 minutes), as `os.loadavg()`
|
|
returns them; the server answers `400` to any other length.
|
|
|
|
Sharing `metrics.js` with the server keeps the two in step, but it is not the
|
|
guarantee — the frozen copy can lag behind a deployed server. The real contract
|
|
is the server's own validation, and `__tests__/agent.test.js` checks the agent's
|
|
payload against it (including a real snapshot through the real handler).
|
|
|
|
## claude-usage agent (`agent/claude-usage/`)
|
|
|
|
A second, independent agent on the ThinkPad. Every 15 minutes it reads the
|
|
Claude plan limits, aggregates the local Claude Code transcripts over the last
|
|
30 Toronto days, and pushes one `schemaVersion: 1` snapshot to
|
|
`POST /claude-usage`; the admin dashboard reads it through `GET /claude-usage`.
|
|
|
|
It lives in this repo, next to the server, because the snapshot contract and
|
|
its sanitizer (`claude-usage.js`) are tested together:
|
|
`__tests__/agent-claude-usage.test.js` builds a real snapshot from fixtures and
|
|
checks that `sanitizeClaudeUsage()` keeps it unchanged. Like the host agent, it
|
|
never enters the server image.
|
|
|
|
| File | Role |
|
|
|------|------|
|
|
| `aggregate.js` | Read `~/.claude/projects/**/*.jsonl`, build `days` and `projects` |
|
|
| `limits.js` | One GET to Anthropic's OAuth usage endpoint, mapped to `limits.items` |
|
|
| `state.js` | Local cache: frozen days and last good limits |
|
|
| `push-claude-usage.js` | Entry point: aggregate, limits, write the cache, push once, exit |
|
|
| `run-claude-usage.sh` | Cron wrapper: lock, load the env file, check the install, exec node |
|
|
|
|
### What it deliberately does not do
|
|
|
|
**It never refreshes the OAuth token and never writes
|
|
`~/.claude/.credentials.json`.** Refresh tokens are single use: refreshing from
|
|
here would log Claude Code out. The file is only read; an expired token gives
|
|
`limits.status: "token_expired"` and no call at all, until Claude Code itself
|
|
refreshes it.
|
|
|
|
**The Anthropic URL is pinned.** It is a constant in `limits.js`, no
|
|
environment variable changes it, and redirects are refused: the same process
|
|
holds the OAuth bearer and a configurable push URL, and the bearer must only
|
|
ever reach `https://api.anthropic.com`. `CLAUDE_USAGE_API_URL` must be
|
|
`https://` (the agent exits 1 otherwise).
|
|
|
|
**One call, one push, no retry.** The usage endpoint rate-limits hard and sends
|
|
no `Retry-After`. When it fails, the snapshot carries the failure status
|
|
(`http_error`, `unavailable`, ...) with the last good items and their
|
|
`fetchedAt`, so the dashboard can show how old they are.
|
|
|
|
**No secret in the logs.** Same rule as the host agent: the output goes to
|
|
`logger -t claude-usage`, so failures print fixed templates plus an HTTP status
|
|
or an error code — never an error object, a response body, or a header — and
|
|
`run-claude-usage.sh` must never gain a `set -x`.
|
|
|
|
### Install
|
|
|
|
Freeze a copy — never point the cron at the git working tree:
|
|
|
|
```
|
|
mkdir -p ~/.local/share/maximus-claude-usage
|
|
cp agent/claude-usage/aggregate.js agent/claude-usage/limits.js agent/claude-usage/state.js agent/claude-usage/push-claude-usage.js agent/claude-usage/run-claude-usage.sh ~/.local/share/maximus-claude-usage/
|
|
chmod +x ~/.local/share/maximus-claude-usage/run-claude-usage.sh
|
|
```
|
|
|
|
Re-run that `cp` after any change to the agent: the copy step *is* the
|
|
deployment. The code lives in `~/.local/share/`, the cache in
|
|
`~/.local/state/maximus-claude-usage/` (`state.json` and `run.lock`, created on
|
|
the first run): wiping the copy never loses the cache, and vice versa.
|
|
|
|
Env file `~/.config/maximus-claude-usage.env`, mode `600` (the wrapper warns on
|
|
any other mode). Write it with `printf`, not a heredoc: a trailing newline in
|
|
the token is refused with exit 1.
|
|
|
|
```
|
|
CLAUDE_USAGE_API_URL=https://health.lacompagniemaximus.com
|
|
CLAUDE_USAGE_INGEST_TOKEN=<the claude-usage ingest token>
|
|
# CLAUDE_USAGE_LIMITS=off # skip the Anthropic call entirely
|
|
# NODE_BIN=/usr/bin/node # only if cron cannot find node
|
|
```
|
|
|
|
```
|
|
chmod 600 ~/.config/maximus-claude-usage.env
|
|
```
|
|
|
|
`CLAUDE_USAGE_INGEST_TOKEN` is its own token: neither `HOSTS_INGEST_TOKEN` nor
|
|
`HEALTH_TOKEN` works on `POST /claude-usage`, and it opens nothing else.
|
|
|
|
Dry run — prints the snapshot, pushes nothing, leaves the cache untouched, needs
|
|
no env file (it still calls the Anthropic endpoint once unless
|
|
`CLAUDE_USAGE_LIMITS=off`):
|
|
|
|
```
|
|
CLAUDE_USAGE_LIMITS=off node ~/.local/share/maximus-claude-usage/push-claude-usage.js --dry-run
|
|
```
|
|
|
|
Then one real run by hand, before installing the cron (the first one scans every
|
|
transcript of the window and can take a while; the next ones only read files
|
|
written since yesterday):
|
|
|
|
```
|
|
~/.local/share/maximus-claude-usage/run-claude-usage.sh
|
|
# claude-usage: ok status=204 limits=ok bytes=...
|
|
```
|
|
|
|
Crontab line:
|
|
|
|
```
|
|
*/15 * * * * $HOME/.local/share/maximus-claude-usage/run-claude-usage.sh 2>&1 | logger -t claude-usage
|
|
```
|
|
|
|
```
|
|
journalctl -t claude-usage --since -1h
|
|
```
|
|
|
|
A beat that finds the previous one still running (`flock` on
|
|
`~/.local/state/maximus-claude-usage/run.lock`) logs
|
|
`a run is already in progress, skipping` and exits 0.
|
|
|
|
### Exit codes
|
|
|
|
| Code | Meaning |
|
|
|------|---------|
|
|
| 0 | Snapshot accepted (2xx), or a skipped beat (lock held) |
|
|
| 1 | Install problem: env file missing, a variable unset, URL not `https://`, token with an invalid character, node or flock not found |
|
|
| 2 | The push did not land (network or non-2xx), or the snapshot exceeded 64 KiB (`payload_too_large`, nothing sent) |
|
|
|
|
The limits failing is **not** an exit code: the snapshot is still pushed, with
|
|
`limits.status` saying why.
|