diff --git a/CLAUDE.md b/CLAUDE.md index 71c5954..5d94928 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -56,17 +56,19 @@ curl -H "Authorization: Bearer $(cat ~/.coolify-token)" \ ## Tests -- `npm test` (vitest) — 61 cas +- `npm test` (vitest) — 244 cas - `__tests__/findings.test.js` — `/defenseurs/findings` (14 cas : auth, validation, filtres severity/category, asymetrie INFO, scan clean vs no_data, JSON corrompu) - `__tests__/health.test.js` — module `metrics.js` + `/health` (6 cas : exports, payload champ pour champ, garde de latence) - `__tests__/auth.test.js` — filet de securite auth sur les routes de lecture (19 cas : matrice route x mode d'echec, en-tetes malformes, fail-closed, et **2 cas `404 (not 401)` qui figent l'ordre routage -> auth**) - - `__tests__/hosts.test.js` — smoke des endpoints postes (22 cas : regex de route comme controle anti-traversee, 401 avant 403, cloisonnement des deux tokens, 503 fail-closed, reconstruction par liste blanche, 413, `neverSeen`, fraicheur serveur, entree corrompue degradee). La matrice exhaustive est l'issue #14. + - `__tests__/hosts.test.js` — matrice exhaustive des endpoints postes (155 cas : regex de route comme controle anti-traversee, 401 avant 403, cloisonnement des deux tokens, 503 fail-closed, reconstruction par liste blanche, 413, `neverSeen`, fraicheur serveur sous horloge figee, degradation de lecture) + - `__tests__/agent.test.js` — agent poste (50 cas : contrat de payload verifie contre `sanitizeSnapshot()`, une seule tentative sans rejeu, timeout, codes de sortie, wrapper shell, et **le token absent de toute sortie d'echec, fragments de 5 caracteres compris**) - Runtime reste 0-dep ; vitest en devDep uniquement ## Gotchas - Pas d'Express — HTTP natif Node.js uniquement -- Le `COPY` du Dockerfile liste les fichiers un par un (`package.json index.js metrics.js`) : tout nouveau module runtime doit y etre ajoute, sinon le conteneur plante au demarrage sur un `MODULE_NOT_FOUND` que les tests locaux ne voient pas. +- Le `COPY` du Dockerfile liste les fichiers un par un (`package.json index.js metrics.js`) : tout nouveau module runtime doit y etre ajoute, sinon le conteneur plante au demarrage sur un `MODULE_NOT_FOUND` que les tests locaux ne voient pas. Corollaire : `agent/` n'entre PAS dans l'image — c'est du code de poste, livre par copie de fichiers (voir `agent/README.md`), et un test epingle que le `COPY` ne le mentionne jamais. +- `agent/` tourne sur le ThinkPad, pas sur le VPS, et sa sortie part dans `logger -t host-agent` : tout ce qu'il imprime finit dans `/var/log/syslog` et journald pour de bon. Le chemin d'erreur ne rend donc que `err.code` et le code HTTP — jamais un objet d'erreur, jamais les options de requete, jamais un en-tete. Et jamais de `set -x` dans `run-push.sh` : le shell echoerait le token en sourcant le fichier d'env. - `getHealth()` garde `Promise.all([collectMetrics(), getLogtoHealth()])` : l'echantillon CPU de 500 ms et le check Logto (timeout 3 s) sont deliberement concurrents. Les serialiser ferait monter le p99 de `/health` a ~3,5 s. Garde de non-regression : le test de latence dans `__tests__/health.test.js`. - L'API n'est plus read-only depuis l'issue #13. `status.json`, `agents-map.json` et les rapports de scan restent ecrits par le Sergent defenseurs et lus seulement ici ; en revanche `POST /hosts/` ecrit dans `HOSTS_DIR`. Consequence : ce montage-la doit etre en lecture-ecriture pour l'uid 1000, contrairement aux montages `/data/defenseurs`. - L'ecriture des snapshots passe par un fichier temporaire dans le **meme** repertoire puis `renameSync` — `rename` est atomique dans un systeme de fichiers, donc un `GET /hosts` concurrent voit l'ancien snapshot ou le nouveau, jamais un JSON tronque. Ne pas « simplifier » en `writeFileSync` direct. diff --git a/README.md b/README.md index 90a6149..b005f36 100644 --- a/README.md +++ b/README.md @@ -121,6 +121,13 @@ Read-write — the API is the writer: - `/data/hosts/` -> `/data/hosts/` (`HOSTS_DIR`). Must be writable by uid 1000, the `node` user the container runs as, otherwise every ingest 500s. +## Workstation agent + +`agent/` holds the zero-dependency collector that runs from cron on a +workstation and feeds `POST /hosts/`. It is not part of the server image — +it is installed by copying files onto the machine it measures. Install steps, +crontab line and failure modes: [`agent/README.md`](agent/README.md). + ## Tests ``` diff --git a/__tests__/agent.test.js b/__tests__/agent.test.js new file mode 100644 index 0000000..e2b116d --- /dev/null +++ b/__tests__/agent.test.js @@ -0,0 +1,597 @@ +const http = require("node:http"); +const fs = require("node:fs"); +const os = require("node:os"); +const path = require("node:path"); +const { execFile } = require("node:child_process"); +const { promisify } = require("node:util"); + +const execFileAsync = promisify(execFile); + +const { + buildSnapshot, + buildTargetUrl, + postSnapshot, + readConfig, + DEFAULT_TIMEOUT_MS, + EXIT_CONFIG, + EXIT_PUSH, +} = require("../agent/push-metrics.js"); +const { collectMetrics } = require("../metrics.js"); +const { sanitizeSnapshot } = require("../index.js"); + +// Coverage for the local workstation agent (issue #15). +// +// Three things are worth stating up front, because they explain why this file +// spends most of its length on failure paths rather than on the happy one. +// +// * THE TOKEN MUST NOT SURVIVE INTO A LOG LINE. run-push.sh pipes the agent's +// output into `logger -t host-agent`; anything printed lands in syslog and +// journald for good. Every failure path here is therefore checked against +// every 5-character-or-longer fragment of the token — message, stack, JSON +// rendering — and the two spawned-process cases check the real stdout and +// stderr of a real run, which is what `logger` would actually swallow. +// * NO QUEUE, NO REPLAY. A failed push is dropped, not retried: the +// assertions count the requests the server saw and expect exactly one. +// * The payload contract is pinned against the server's own +// sanitizeSnapshot(), and one case pushes a REAL collectMetrics() snapshot +// through the REAL ingestion handler. A drift between agent and server +// shows up as a red test rather than as a 400 at 3 a.m. on the ThinkPad. + +const AGENT_PATH = path.join(__dirname, "..", "agent", "push-metrics.js"); +const RUNNER_PATH = path.join(__dirname, "..", "agent", "run-push.sh"); + +// Mixed letters and digits, no dictionary word and no run of five digits, so a +// port number or a timestamp in the output can never look like a leak. +const TOKEN = "7f2a9c4e1b6d8035a5e0c3d1"; +const HOST_ID = "thinkpad"; + +const servers = []; +let tmpDir; + +function fakeMetrics() { + return { + cpu: { + model: "Intel(R) Core(TM) i7-8650U CPU @ 1.90GHz", + cores: 8, + loadAvg: [0.42, 0.31, 0.25], + usagePercent: 12, + }, + memory: { totalGB: 32, usedGB: 12.5, freeGB: 19.5, usagePercent: 39 }, + disk: { totalGB: 500, usedGB: 220, freeGB: 280, usagePercent: 44 }, + }; +} + +async function startServer(onRequest) { + const server = http.createServer(onRequest); + servers.push(server); + await new Promise((resolve) => server.listen(0, "127.0.0.1", resolve)); + return { server, url: `http://127.0.0.1:${server.address().port}` }; +} + +// A recording server that answers with the given status. Returns the log of +// what it saw, so "exactly one request" and "the Authorization header was sent" +// are assertions rather than assumptions. +async function startRecordingServer(status = 204) { + const seen = []; + const { url, server } = await startServer((req, res) => { + const chunks = []; + req.on("data", (c) => chunks.push(c)); + req.on("end", () => { + seen.push({ + method: req.method, + url: req.url, + headers: req.headers, + body: Buffer.concat(chunks).toString("utf-8"), + }); + res.writeHead(status); + res.end(); + }); + }); + return { url, server, seen }; +} + +async function closeServers() { + while (servers.length) { + const server = servers.pop(); + server.closeAllConnections?.(); + await new Promise((resolve) => server.close(resolve)); + } +} + +// Every substring of the token from `min` characters up. A failure message that +// contains none of them cannot contain the token, nor a truncated half of it. +function tokenFragments(token, min = 5) { + const out = []; + for (let len = min; len <= token.length; len++) { + for (let i = 0; i + len <= token.length; i++) out.push(token.slice(i, i + len)); + } + return out; +} + +function expectNoTokenLeak(text, token = TOKEN) { + const haystack = String(text); + const leaked = tokenFragments(token).filter((frag) => haystack.includes(frag)); + // Report the longest match only: every shorter fragment of it also matches, + // and dumping the 200-odd of them buries the failure it is reporting. + const worst = leaked.reduce((a, b) => (b.length > a.length ? b : a), ""); + expect( + leaked.length, + `token leaked into the output — longest fragment found: "${worst}"`, + ).toBe(0); +} + +// Everything a careless `console.error(err)` would print. +function renderError(err) { + return [ + err.message, + String(err), + err.stack, + JSON.stringify(err), + JSON.stringify(err, Object.getOwnPropertyNames(err)), + ].join("\n"); +} + +async function pushTo(url, overrides = {}) { + return postSnapshot({ + apiUrl: url, + token: TOKEN, + hostId: HOST_ID, + snapshot: buildSnapshot(fakeMetrics()), + timeoutMs: 2000, + ...overrides, + }); +} + +beforeEach(() => { + tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), "agent-test-")); +}); + +afterEach(async () => { + await closeServers(); + fs.rmSync(tmpDir, { recursive: true, force: true }); +}); + +describe("payload — matches the POST /hosts/ contract", () => { + test("the built snapshot passes the server's sanitizeSnapshot() unchanged", () => { + const snapshot = buildSnapshot(fakeMetrics()); + const result = sanitizeSnapshot(snapshot); + expect(result.error).toBeUndefined(); + expect(result.snapshot).toEqual(snapshot); + }); + + test("exactly the documented keys, nothing more", () => { + const snapshot = buildSnapshot(fakeMetrics()); + expect(Object.keys(snapshot).sort()).toEqual([ + "cpu", + "disk", + "hostname", + "memory", + "uptime", + ]); + expect(Object.keys(snapshot.cpu).sort()).toEqual([ + "cores", + "loadAvg", + "model", + "usagePercent", + ]); + for (const key of ["memory", "disk"]) { + expect(Object.keys(snapshot[key]).sort()).toEqual([ + "freeGB", + "totalGB", + "usagePercent", + "usedGB", + ]); + } + }); + + test("an extra field added to collectMetrics() does not leave the workstation", () => { + const metrics = fakeMetrics(); + metrics.temperatureC = 61; + metrics.cpu.serial = "PF0X1234"; + const snapshot = buildSnapshot(metrics); + expect(snapshot.temperatureC).toBeUndefined(); + expect(snapshot.cpu.serial).toBeUndefined(); + }); + + test("hostname and uptime come from the host, uptime floored like getHealth()", () => { + const snapshot = buildSnapshot(fakeMetrics()); + expect(snapshot.hostname).toBe(os.hostname()); + expect(Number.isInteger(snapshot.uptime)).toBe(true); + expect(snapshot.uptime).toBeGreaterThan(0); + }); + + test("a real collectMetrics() snapshot is accepted by sanitizeSnapshot()", async () => { + const snapshot = buildSnapshot(await collectMetrics()); + const result = sanitizeSnapshot(snapshot); + expect(result.error).toBeUndefined(); + expect(result.snapshot).toEqual(snapshot); + }); +}); + +describe("delivery — one attempt, no queue", () => { + test("POSTs to /hosts/ with the bearer token and resolves on 204", async () => { + const { url, seen } = await startRecordingServer(204); + // Build ONCE and compare against that object. Two calls to buildSnapshot() + // straddling a second boundary differ by 1 on `uptime`, which turns this + // assertion into a one-in-several-runs failure. + const snapshot = buildSnapshot(fakeMetrics()); + const result = await pushTo(url, { snapshot }); + + expect(result.statusCode).toBe(204); + expect(seen).toHaveLength(1); + expect(seen[0].method).toBe("POST"); + expect(seen[0].url).toBe(`/hosts/${HOST_ID}`); + expect(seen[0].headers.authorization).toBe(`Bearer ${TOKEN}`); + expect(seen[0].headers["content-type"]).toBe("application/json"); + expect(JSON.parse(seen[0].body)).toEqual(snapshot); + }); + + test("any 2xx counts as delivered", async () => { + const { url } = await startRecordingServer(200); + await expect(pushTo(url)).resolves.toEqual({ statusCode: 200 }); + }); + + test.each([400, 401, 403, 413, 500, 503])("rejects on HTTP %i", async (status) => { + const { url } = await startRecordingServer(status); + await expect(pushTo(url)).rejects.toThrow(`push failed: HTTP ${status}`); + }); + + test("a rejected push is dropped, never retried", async () => { + const { url, seen } = await startRecordingServer(500); + await expect(pushTo(url)).rejects.toThrow(); + await new Promise((resolve) => setTimeout(resolve, 150)); + expect(seen).toHaveLength(1); + }); + + test("reports only err.code when the connection is refused", async () => { + const { url, server } = await startRecordingServer(); + await new Promise((resolve) => server.close(resolve)); + servers.pop(); + await expect(pushTo(url)).rejects.toThrow("push failed: request error (code=ECONNREFUSED)"); + }); + + test("gives up on a server that never answers, as ETIMEDOUT", async () => { + const { url } = await startServer(() => { + /* never responds */ + }); + const started = Date.now(); + await expect(pushTo(url, { timeoutMs: 200 })).rejects.toThrow( + "push failed: request error (code=ETIMEDOUT)", + ); + expect(Date.now() - started).toBeLessThan(2000); + }); + + test("the default timeout is 10s", () => { + expect(DEFAULT_TIMEOUT_MS).toBe(10000); + }); + + test("a trailing slash on HOSTS_API_URL does not double up", () => { + expect(buildTargetUrl("https://health.example.com/", "thinkpad").href).toBe( + "https://health.example.com/hosts/thinkpad", + ); + expect(buildTargetUrl("https://health.example.com", "thinkpad").href).toBe( + "https://health.example.com/hosts/thinkpad", + ); + }); + + test.each([ + ["ftp://health.example.com", "HOSTS_API_URL must be http:// or https://"], + ["not a url", "HOSTS_API_URL is not a valid URL"], + ])("refuses %s before opening a socket", async (apiUrl, message) => { + await expect(pushTo(apiUrl)).rejects.toThrow(message); + }); +}); + +describe("the failure path never carries the token", () => { + test("HTTP failure: message, stack and JSON rendering are all clean", async () => { + const { url } = await startRecordingServer(500); + const err = await pushTo(url).catch((e) => e); + expect(err.message).toBe("push failed: HTTP 500"); + expectNoTokenLeak(renderError(err)); + }); + + test("transport failure: message, stack and JSON rendering are all clean", async () => { + const { url, server } = await startRecordingServer(); + await new Promise((resolve) => server.close(resolve)); + servers.pop(); + const err = await pushTo(url).catch((e) => e); + expectNoTokenLeak(renderError(err)); + }); + + test("timeout failure: message, stack and JSON rendering are all clean", async () => { + const { url } = await startServer(() => {}); + const err = await pushTo(url, { timeoutMs: 200 }).catch((e) => e); + expectNoTokenLeak(renderError(err)); + }); + + test("a token with a trailing newline fails without quoting itself", async () => { + const { url } = await startRecordingServer(204); + const err = await pushTo(url, { token: `${TOKEN}\n` }).catch((e) => e); + expect(err.message).toContain("HOSTS_INGEST_TOKEN contains an invalid character"); + expectNoTokenLeak(renderError(err)); + }); + + test("the real process output of a failing run contains no fragment of the token", async () => { + const { url } = await startRecordingServer(500); + const result = await execFileAsync(process.execPath, [AGENT_PATH], { + env: { + PATH: process.env.PATH, + HOSTS_API_URL: url, + HOSTS_INGEST_TOKEN: TOKEN, + HOST_ID, + }, + }).catch((e) => e); + + expect(result.code).toBe(EXIT_PUSH); + expect(result.stderr).toContain("host-agent: push failed: HTTP 500"); + expectNoTokenLeak(`${result.stdout}${result.stderr}`); + }); + + test("the real process output of an unreachable server contains no fragment of the token", async () => { + const { url, server } = await startRecordingServer(); + await new Promise((resolve) => server.close(resolve)); + servers.pop(); + + const result = await execFileAsync(process.execPath, [AGENT_PATH], { + env: { + PATH: process.env.PATH, + HOSTS_API_URL: url, + HOSTS_INGEST_TOKEN: TOKEN, + HOST_ID, + }, + }).catch((e) => e); + + expect(result.code).toBe(EXIT_PUSH); + expect(result.stderr).toContain("code=ECONNREFUSED"); + expectNoTokenLeak(`${result.stdout}${result.stderr}`); + }); +}); + +describe("configuration", () => { + test.each([ + ["HOSTS_API_URL", "HOSTS_API_URL is not set"], + ["HOSTS_INGEST_TOKEN", "HOSTS_INGEST_TOKEN is not set"], + ["HOST_ID", "HOST_ID is not set"], + ])("a missing %s is named, not guessed", (missing, message) => { + const env = { HOSTS_API_URL: "https://x.test", HOSTS_INGEST_TOKEN: TOKEN, HOST_ID }; + delete env[missing]; + const { error, config } = readConfig(env); + expect(config).toBeUndefined(); + expect(error.message).toContain(message); + expectNoTokenLeak(renderError(error)); + }); + + test.each(["THINKPAD", "-thinkpad", "think pad", "a".repeat(33), "../thinkpad"])( + "rejects the malformed HOST_ID %j client-side", + (hostId) => { + const { error } = readConfig({ + HOSTS_API_URL: "https://x.test", + HOSTS_INGEST_TOKEN: TOKEN, + HOST_ID: hostId, + }); + expect(error.message).toContain("HOST_ID must match"); + }, + ); + + test.each([ + ["ftp://health.example.com", "HOSTS_API_URL must be http:// or https://"], + ["not a url", "HOSTS_API_URL is not a valid URL"], + ])("a broken HOSTS_API_URL (%j) is an install error, not a push error", (apiUrl, message) => { + const { error, config } = readConfig({ + HOSTS_API_URL: apiUrl, + HOSTS_INGEST_TOKEN: TOKEN, + HOST_ID, + }); + expect(config).toBeUndefined(); + expect(error.message).toContain(message); + }); + + test("accepts a well-formed config and defaults the timeout", () => { + const { config } = readConfig({ + HOSTS_API_URL: "https://health.example.com", + HOSTS_INGEST_TOKEN: TOKEN, + HOST_ID, + }); + expect(config).toEqual({ + apiUrl: "https://health.example.com", + token: TOKEN, + hostId: HOST_ID, + timeoutMs: DEFAULT_TIMEOUT_MS, + }); + }); + + test("--dry-run prints the payload and touches no network", async () => { + const { stdout } = await execFileAsync(process.execPath, [AGENT_PATH, "--dry-run"], { + env: { PATH: process.env.PATH }, + }); + const snapshot = JSON.parse(stdout); + expect(sanitizeSnapshot(snapshot).error).toBeUndefined(); + expect(snapshot.hostname).toBe(os.hostname()); + }); +}); + +describe("run-push.sh", () => { + function writeEnvFile(lines, mode = 0o600) { + const file = path.join(tmpDir, "maximus-host-agent.env"); + fs.writeFileSync(file, `${lines.join("\n")}\n`, { mode }); + return file; + } + + async function runWrapper(envFile) { + return execFileAsync(RUNNER_PATH, [], { + env: { PATH: process.env.PATH, HOME: tmpDir, HOST_AGENT_ENV_FILE: envFile }, + }).catch((e) => e); + } + + test("delivers the snapshot end to end", async () => { + const { url, seen } = await startRecordingServer(204); + const envFile = writeEnvFile([ + `HOSTS_API_URL=${url}`, + `HOSTS_INGEST_TOKEN=${TOKEN}`, + `HOST_ID=${HOST_ID}`, + `NODE_BIN=${process.execPath}`, + ]); + + const result = await runWrapper(envFile); + + expect(result.stderr).toBe(""); + expect(result.stdout).toContain(`host-agent: ok id=${HOST_ID} status=204`); + expect(seen).toHaveLength(1); + expect(seen[0].headers.authorization).toBe(`Bearer ${TOKEN}`); + expect(sanitizeSnapshot(JSON.parse(seen[0].body)).error).toBeUndefined(); + expectNoTokenLeak(`${result.stdout}${result.stderr}`); + }); + + test.each([ + [["HOSTS_INGEST_TOKEN", "HOST_ID"], "HOSTS_API_URL is not set"], + [["HOSTS_API_URL", "HOST_ID"], "HOSTS_INGEST_TOKEN is not set"], + [["HOSTS_API_URL", "HOSTS_INGEST_TOKEN"], "HOST_ID is not set"], + ])("stops when a variable is missing (%j)", async (present, message) => { + const values = { + HOSTS_API_URL: "https://health.example.com", + HOSTS_INGEST_TOKEN: TOKEN, + HOST_ID, + }; + const envFile = writeEnvFile(present.map((key) => `${key}=${values[key]}`)); + + const result = await runWrapper(envFile); + + expect(result.code).toBe(1); + expect(result.stderr).toContain(message); + expectNoTokenLeak(`${result.stdout}${result.stderr}`); + }); + + test("stops when the env file is absent", async () => { + const missing = path.join(tmpDir, "nope.env"); + const result = await runWrapper(missing); + expect(result.code).toBe(1); + expect(result.stderr).toContain(`host-agent: env file not found: ${missing}`); + }); + + test("warns about a world-readable env file but still runs", async () => { + const { url, seen } = await startRecordingServer(204); + const envFile = writeEnvFile( + [ + `HOSTS_API_URL=${url}`, + `HOSTS_INGEST_TOKEN=${TOKEN}`, + `HOST_ID=${HOST_ID}`, + `NODE_BIN=${process.execPath}`, + ], + 0o644, + ); + + const result = await runWrapper(envFile); + + expect(result.stderr).toContain("is mode 644, expected 600"); + expect(seen).toHaveLength(1); + expectNoTokenLeak(`${result.stdout}${result.stderr}`); + }); + + test("never enables shell tracing — `set -x` would echo the token into syslog", () => { + const script = fs.readFileSync(RUNNER_PATH, "utf-8"); + const traced = script + .split("\n") + .filter((line) => !line.trim().startsWith("#")) + .filter((line) => /\bset\b[^#\n]*-[a-z]*x/.test(line)); + expect(traced).toEqual([]); + }); + + test("is executable", () => { + expect(fs.statSync(RUNNER_PATH).mode & 0o111).toBeGreaterThan(0); + }); +}); + +describe("the agent stays out of the server image", () => { + test("the Dockerfile copies files one by one and none of them is the agent", () => { + const dockerfile = fs.readFileSync(path.join(__dirname, "..", "Dockerfile"), "utf-8"); + const copies = dockerfile + .split("\n") + .filter((line) => /^\s*(COPY|ADD)\b/i.test(line)); + + expect(copies).toHaveLength(1); + expect(copies[0]).not.toMatch(/agent/); + // A bare `COPY . .` would drag agent/ in without ever naming it. + expect(copies[0]).not.toMatch(/^\s*COPY\s+\.\s/i); + }); +}); + +describe("against the real ingestion handler", () => { + const savedEnv = {}; + const ENV_KEYS = ["HOSTS_INGEST_TOKEN", "HOSTS_DIR", "HOSTS_ALLOWED_IDS", "HEALTH_TOKEN"]; + + beforeEach(() => { + for (const key of ENV_KEYS) savedEnv[key] = process.env[key]; + process.env.HOSTS_INGEST_TOKEN = TOKEN; + process.env.HOSTS_DIR = path.join(tmpDir, "hosts"); + process.env.HOSTS_ALLOWED_IDS = HOST_ID; + process.env.HEALTH_TOKEN = "read-token"; + }); + + afterEach(() => { + for (const key of ENV_KEYS) { + if (savedEnv[key] === undefined) delete process.env[key]; + else process.env[key] = savedEnv[key]; + } + }); + + test("a real snapshot is accepted with 204 and persisted field for field", async () => { + // Fresh import so index.js picks up the env above. + delete require.cache[require.resolve("../index.js")]; + const { handler } = require("../index.js"); + const { url } = await startServer(handler); + + const snapshot = buildSnapshot(await collectMetrics()); + const result = await postSnapshot({ + apiUrl: url, + token: TOKEN, + hostId: HOST_ID, + snapshot, + timeoutMs: 5000, + }); + + expect(result.statusCode).toBe(204); + + const written = JSON.parse( + fs.readFileSync(path.join(process.env.HOSTS_DIR, `${HOST_ID}.json`), "utf-8"), + ); + expect(written.id).toBe(HOST_ID); + expect(typeof written.receivedAt).toBe("string"); + delete written.id; + delete written.receivedAt; + expect(written).toEqual(snapshot); + }); + + test("a wrong ingest token is rejected with 401 and no fragment leaks", async () => { + delete require.cache[require.resolve("../index.js")]; + const { handler } = require("../index.js"); + const { url } = await startServer(handler); + + const err = await postSnapshot({ + apiUrl: url, + token: "wrong-token", + hostId: HOST_ID, + snapshot: buildSnapshot(fakeMetrics()), + timeoutMs: 5000, + }).catch((e) => e); + + expect(err.message).toBe("push failed: HTTP 401"); + expectNoTokenLeak(renderError(err)); + }); + + test("a host id outside the allowlist surfaces as HTTP 403", async () => { + process.env.HOSTS_ALLOWED_IDS = "someone-else"; + delete require.cache[require.resolve("../index.js")]; + const { handler } = require("../index.js"); + const { url } = await startServer(handler); + + await expect( + postSnapshot({ + apiUrl: url, + token: TOKEN, + hostId: HOST_ID, + snapshot: buildSnapshot(fakeMetrics()), + timeoutMs: 5000, + }), + ).rejects.toThrow("push failed: HTTP 403"); + }); +}); diff --git a/agent/README.md b/agent/README.md new file mode 100644 index 0000000..3a20d1c --- /dev/null +++ b/agent/README.md @@ -0,0 +1,170 @@ +# Workstation agent + +Collects one snapshot of the machine it runs on and pushes it to +`POST /hosts/` on the health API. Cron runs it every 5 minutes; the +dashboard reads the result through `GET /hosts`. + +Zero dependencies — `node:http` / `node:https` and the `metrics.js` already +shared with the server. Node 22+. + +| File | Role | +|------|------| +| `push-metrics.js` | Collect, build the payload, POST once, exit | +| `run-push.sh` | Cron wrapper: load the env file, check the install, exec node | + +This directory is **not** part of the server image: the `Dockerfile` copies +`package.json index.js metrics.js` one by one, and a test pins that it stays +that way. The agent ships by copying files onto a workstation, never by deploy. + +## What it deliberately does not do + +**No queue, no replay.** One run is one attempt. A push that fails is dropped, +logged, and the process exits non-zero — the next beat is five minutes away and +a heartbeat from five minutes ago describes a machine that no longer exists. +Nothing is spooled to disk, so nothing can ever be replayed to make the +dashboard show a past that is no longer true. + +**No secret in the logs.** `run-push.sh` is meant to be piped into `logger`, so +everything the agent prints ends up in syslog and journald permanently. The +failure path therefore reports **only** `err.code` and the HTTP status — never +an error object, never the request options, never a response body, and never a +header. That is also why `run-push.sh` must never gain a `set -x`: the shell +would echo the token as it sources the env file. + +The cost is real and accepted: a `400` tells you the server rejected the +payload, not why. The reason is in the server's logs, and `--dry-run` (below) +shows you the exact payload that was sent. + +## Install on a new workstation + +### 1. Freeze a copy + +The cron must point at a copy, **never at the git working tree**. The server is +deployed by a manual trigger, so the checked-out repo drifts freely: a branch +checkout in `~/claude-code/vps-health-api` would silently change — or break — +the heartbeat of the workstation, and nothing would say so. + +`push-metrics.js` loads `metrics.js` from one directory up, so the copy keeps +the repo layout: + +``` +mkdir -p ~/.local/share/maximus-host-agent/agent +cp metrics.js ~/.local/share/maximus-host-agent/ +cp agent/push-metrics.js agent/run-push.sh ~/.local/share/maximus-host-agent/agent/ +chmod +x ~/.local/share/maximus-host-agent/agent/run-push.sh +``` + +Re-run those three `cp` after any change to the agent — that copy step *is* the +deployment. + +### 2. Write the env file + +`~/.config/maximus-host-agent.env`, mode `600` (the agent warns on any other +mode). Use `printf`, not a heredoc: a trailing newline or space inside the token +value is the most common install failure. + +``` +HOSTS_API_URL=https://health.lacompagniemaximus.com +HOSTS_INGEST_TOKEN= +HOST_ID=thinkpad +# NODE_BIN=/usr/bin/node # only if cron cannot find node (see below) +``` + +``` +chmod 600 ~/.config/maximus-host-agent.env +``` + +`HOSTS_INGEST_TOKEN` is the write-only token: it can push snapshots and nothing +else. It is deliberately *not* `HEALTH_TOKEN` — a stolen workstation must not +grant read access to the Defenseurs reports. + +`HOST_ID` must match `^[a-z0-9][a-z0-9-]{0,31}$` and be listed in the server's +`HOSTS_ALLOWED_IDS`, otherwise the push comes back `403`. + +### 3. Dry run + +Collect and print the payload without touching the network or needing any +config: + +``` +node ~/.local/share/maximus-host-agent/agent/push-metrics.js --dry-run +``` + +Then one real push, by hand, before installing the cron: + +``` +~/.local/share/maximus-host-agent/agent/run-push.sh +# host-agent: ok id=thinkpad status=204 hostname=thinkpad-x1 +``` + +Confirm the server side (from a machine that holds `HEALTH_TOKEN`): + +``` +curl -H "Authorization: Bearer $HEALTH_TOKEN" \ + https://health.lacompagniemaximus.com/hosts +``` + +The entry for your id should show `online: true` and a small `ageSeconds`. + +### 4. Install the cron + +``` +crontab -e +``` + +``` +*/5 * * * * $HOME/.local/share/maximus-host-agent/agent/run-push.sh 2>&1 | logger -t host-agent +``` + +Read what it did: + +``` +journalctl -t host-agent --since -1h +``` + +If the line works by hand but produces nothing under cron, it is almost always +`node`: cron's `PATH` is `/usr/bin:/bin`, and a node installed by nvm or under +`/usr/local` is not on it. Set `NODE_BIN` to an absolute path in the env file — +the wrapper says so explicitly rather than failing silently. + +## One id per workstation + +The server keeps **one file per host id** and the last write wins. Two +workstations configured with the same `HOST_ID` therefore overwrite each other +every five minutes, and nothing anywhere reports an error: the dashboard shows +one host that looks perfectly healthy while its numbers come from whichever +machine pushed last. + +The signal to watch for is the `hostname` field in `GET /hosts` **changing +between polls** while the id stays the same — plus CPU and memory that jump +around without pattern. If you see that, two machines are sharing an id. + +Give every workstation its own id and add it to `HOSTS_ALLOWED_IDS` on the +server before installing the agent on it. + +## Exit codes + +| Code | Meaning | +|------|---------| +| 0 | Snapshot accepted (server answered 2xx, normally `204`) | +| 1 | Install problem: env file missing, a variable unset, malformed `HOST_ID`, node not found, `metrics.js` not found next to the agent | +| 2 | The push did not land: network error (`code=…`) or a non-2xx answer (`HTTP …`) | + +A `1` needs a human on the workstation. A `2` usually fixes itself on the next +beat; a `2` that repeats for an hour means the API or the link is down. + +## Payload + +Exactly the `GET /health` body minus `logto` and `timestamp`: + +``` +{ hostname, uptime, + cpu: { model, cores, loadAvg, usagePercent }, + memory: { totalGB, usedGB, freeGB, usagePercent }, + disk: { totalGB, usedGB, freeGB, usagePercent } } +``` + +Sharing `metrics.js` with the server keeps the two in step, but it is not the +guarantee — the frozen copy can lag behind a deployed server. The real contract +is the server's own validation, and `__tests__/agent.test.js` checks the agent's +payload against it (including a real snapshot through the real handler). diff --git a/agent/push-metrics.js b/agent/push-metrics.js new file mode 100644 index 0000000..8a5241c --- /dev/null +++ b/agent/push-metrics.js @@ -0,0 +1,309 @@ +#!/usr/bin/env node + +// Local workstation agent: collect one snapshot and POST it to /hosts/. +// Runs from cron every 5 minutes on the ThinkPad. Zero runtime dependencies — +// node:http / node:https only, same posture as the server it talks to. +// +// Two properties are load-bearing and easy to break by accident. +// +// * NO QUEUE, NO REPLAY. One run = one attempt. A missed beat is replaced by +// the next one five minutes later; a stale heartbeat has no value, and a +// spool file would only ever let the dashboard display a past that is no +// longer true. Every failure exits non-zero and stops there. +// * THE FAILURE PATH NEVER PRINTS A REQUEST OBJECT. run-push.sh pipes this +// process' output into `logger -t host-agent`, so anything written here +// lands in /var/log/syslog and journald permanently. An error object or an +// options bag carrying the Authorization header would persist the ingest +// token on disk, in cleartext, forever. Failures are therefore rebuilt from +// a fixed template plus err.code or the HTTP status — nothing else. +// __tests__/agent.test.js asserts no fragment of the token survives into +// the process output. + +const os = require("node:os"); +const http = require("node:http"); +const https = require("node:https"); +const path = require("node:path"); + +const DEFAULT_TIMEOUT_MS = 10000; + +// Exit codes are the only signal cron has. 1 means "the install is wrong" +// (fix the env file), 2 means "the push did not land" (network or server). +const EXIT_CONFIG = 1; +const EXIT_PUSH = 2; + +// Mirrors HOST_ID_PATTERN in index.js. Checked client-side so a typo in the env +// file fails with a readable message instead of an opaque 404 from the server. +const HOST_ID_RE = /^[a-z0-9][a-z0-9-]{0,31}$/; + +// --- Payload ---------------------------------------------------------------- + +function pickUsage(value) { + return { + totalGB: value.totalGB, + usedGB: value.usedGB, + freeGB: value.freeGB, + usagePercent: value.usagePercent, + }; +} + +// Build the POST /hosts/ body: the GET /health payload minus `logto` and +// `timestamp`. Fields are copied one by one rather than spread, so a future +// addition to collectMetrics() cannot silently widen what leaves the +// workstation. `uptime` is floored exactly like getHealth() does. +function buildSnapshot(metrics) { + return { + hostname: os.hostname(), + uptime: Math.floor(os.uptime()), + cpu: { + model: metrics.cpu.model, + cores: metrics.cpu.cores, + loadAvg: metrics.cpu.loadAvg, + usagePercent: metrics.cpu.usagePercent, + }, + memory: pickUsage(metrics.memory), + disk: pickUsage(metrics.disk), + }; +} + +// --- Failure messages ------------------------------------------------------- +// +// The ONLY three places a failure string is built. Keep it that way: every +// caller below reports through one of these, none of them accepts an error +// object, and none of them is given the token or the request options. + +function configFailure(reason) { + return new Error(`configuration error: ${reason}`); +} + +function transportFailure(code) { + return new Error(`push failed: request error (code=${code || "UNKNOWN"})`); +} + +function httpFailure(statusCode) { + return new Error(`push failed: HTTP ${statusCode}`); +} + +// --- Config ----------------------------------------------------------------- + +function readConfig(env) { + const apiUrl = (env.HOSTS_API_URL || "").trim(); + const token = env.HOSTS_INGEST_TOKEN || ""; + const hostId = (env.HOST_ID || "").trim(); + + if (!apiUrl) return { error: configFailure("HOSTS_API_URL is not set") }; + if (!token) return { error: configFailure("HOSTS_INGEST_TOKEN is not set") }; + if (!hostId) return { error: configFailure("HOST_ID is not set") }; + if (!HOST_ID_RE.test(hostId)) { + return { + error: configFailure("HOST_ID must match ^[a-z0-9][a-z0-9-]{0,31}$"), + }; + } + + // Shape-check the URL here too, so a broken env file exits 1 ("fix the + // install") rather than 2 ("the network is having a bad day"). postSnapshot + // repeats the check because it is also called directly by the tests. + let target; + try { + target = buildTargetUrl(apiUrl, hostId); + } catch { + return { error: configFailure("HOSTS_API_URL is not a valid URL") }; + } + if (target.protocol !== "http:" && target.protocol !== "https:") { + return { error: configFailure("HOSTS_API_URL must be http:// or https://") }; + } + + const timeoutRaw = parseInt(env.HOST_AGENT_TIMEOUT_MS || "", 10); + const timeoutMs = + Number.isFinite(timeoutRaw) && timeoutRaw > 0 ? timeoutRaw : DEFAULT_TIMEOUT_MS; + + return { config: { apiUrl, token, hostId, timeoutMs } }; +} + +// --- Delivery --------------------------------------------------------------- + +function buildTargetUrl(apiUrl, hostId) { + const base = String(apiUrl).replace(/\/+$/, ""); + return new URL(`${base}/hosts/${hostId}`); +} + +// One attempt, hard-bounded in wall-clock time. Resolves to { statusCode } on +// 2xx, rejects with a template message otherwise. No retry lives here or above. +function postSnapshot({ apiUrl, token, hostId, snapshot, timeoutMs = DEFAULT_TIMEOUT_MS }) { + return new Promise((resolve, reject) => { + let target; + try { + target = buildTargetUrl(apiUrl, hostId); + } catch { + reject(configFailure("HOSTS_API_URL is not a valid URL")); + return; + } + + const client = + target.protocol === "https:" ? https : target.protocol === "http:" ? http : null; + if (!client) { + reject(configFailure("HOSTS_API_URL must be http:// or https://")); + return; + } + + const body = Buffer.from(JSON.stringify(snapshot), "utf-8"); + let settled = false; + let req = null; + + const finish = (fn, arg) => { + if (settled) return; + settled = true; + clearTimeout(timer); + fn(arg); + }; + + // Wall-clock ceiling over the whole exchange: DNS, connect, TLS, response. + // The `timeout` request option only covers socket inactivity, which can add + // up past the budget on a flaky link. destroy() surfaces as an 'error' with + // the code below, so the timeout reports as ETIMEDOUT and nothing else. + const timer = setTimeout(() => { + if (settled || !req) return; + req.destroy(Object.assign(new Error("timeout"), { code: "ETIMEDOUT" })); + }, timeoutMs); + + try { + req = client.request( + target, + { + method: "POST", + headers: { + "Content-Type": "application/json", + "Content-Length": body.length, + Authorization: `Bearer ${token}`, + }, + }, + (res) => { + const { statusCode } = res; + // Drain without reading: the response body is never inspected and + // never logged. A 400 says the payload was rejected; the reason is in + // the server's logs, where it can be read without a token nearby. + res.resume(); + res.on("end", () => { + if (statusCode >= 200 && statusCode < 300) { + finish(resolve, { statusCode }); + } else { + finish(reject, httpFailure(statusCode)); + } + }); + res.on("error", (err) => finish(reject, transportFailure(err && err.code))); + }, + ); + } catch (err) { + // request() throws synchronously on an unusable header — in practice a + // token with a trailing newline, the classic env-file mistake. Node's + // ERR_INVALID_CHAR names the header but never quotes its value; it is + // still converted here rather than propagated, so no error object from + // the HTTP stack can ever reach a console call. + if (err && err.code === "ERR_INVALID_CHAR") { + finish( + reject, + configFailure( + "HOSTS_INGEST_TOKEN contains an invalid character (a trailing newline in the env file is the usual cause)", + ), + ); + } else { + finish(reject, transportFailure(err && err.code)); + } + return; + } + + // err.code and NOTHING else. err.message can carry the resolved address, + // and the error object dragged into a console call would carry the whole + // request — headers included. + req.on("error", (err) => finish(reject, transportFailure(err && err.code))); + + req.end(body); + }); +} + +// --- Entry point ------------------------------------------------------------ + +// metrics.js is shared with the server and lives one directory up. The frozen +// install copy must keep that layout (see README) — when it does not, say so in +// one line rather than let cron mail a MODULE_NOT_FOUND stack. +function loadCollectMetrics() { + try { + return require(path.join(__dirname, "..", "metrics.js")).collectMetrics; + } catch (err) { + if (err && err.code === "MODULE_NOT_FOUND") { + throw configFailure( + "metrics.js not found one level above the agent — the install copy must keep the repo layout (see agent/README.md)", + ); + } + throw err; + } +} + +async function main(argv = process.argv.slice(2), env = process.env) { + // Dry run: collect and print the payload, touch no network, need no config. + // This is what commissioning runs first, before the env file even exists. + const dryRun = argv.includes("--dry-run"); + + // Config first: the CPU sample below costs 500ms, and there is no point + // spending it to then discover HOST_ID is a typo. + let config = null; + if (!dryRun) { + const result = readConfig(env); + if (result.error) { + console.error(`host-agent: ${result.error.message}`); + return EXIT_CONFIG; + } + config = result.config; + } + + let snapshot; + try { + const collectMetrics = loadCollectMetrics(); + snapshot = buildSnapshot(await collectMetrics()); + } catch (err) { + console.error(`host-agent: ${err.message}`); + return EXIT_CONFIG; + } + + if (dryRun) { + console.log(JSON.stringify(snapshot, null, 2)); + return 0; + } + + try { + const { statusCode } = await postSnapshot({ ...config, snapshot }); + console.log( + `host-agent: ok id=${config.hostId} status=${statusCode} hostname=${snapshot.hostname}`, + ); + return 0; + } catch (err) { + console.error(`host-agent: ${err.message}`); + return EXIT_PUSH; + } +} + +if (require.main === module) { + main().then( + (code) => { + process.exitCode = code; + }, + (err) => { + // Nothing should reach here — every path above is handled. Print the + // message only, never the error object: a stack rendered by console.error + // is fine, but an object argument is what leaks headers. + console.error(`host-agent: unexpected failure: ${err && err.message}`); + process.exitCode = EXIT_PUSH; + }, + ); +} + +module.exports = { + buildSnapshot, + buildTargetUrl, + postSnapshot, + readConfig, + main, + DEFAULT_TIMEOUT_MS, + EXIT_CONFIG, + EXIT_PUSH, + HOST_ID_RE, +}; diff --git a/agent/run-push.sh b/agent/run-push.sh new file mode 100755 index 0000000..b5ec568 --- /dev/null +++ b/agent/run-push.sh @@ -0,0 +1,67 @@ +#!/bin/sh +# +# Cron wrapper for push-metrics.js: load the secrets, check the install, hand +# over to node. Every five minutes, forever, unattended. +# +# NEVER add `set -x`. This script is meant to be piped into `logger`, and the +# shell would echo the `. "$ENV_FILE"` expansion — the ingest token — straight +# into /var/log/syslog and journald, where it would stay. Same reason nothing +# below ever echoes a variable that holds a secret: the checks print the NAME of +# what is missing, never its value. +# +# Suggested crontab line (see README.md): +# */5 * * * * $HOME/.local/share/maximus-host-agent/agent/run-push.sh 2>&1 | logger -t host-agent + +set -eu + +SCRIPT_DIR=$(CDPATH= cd -- "$(dirname -- "$0")" && pwd) +AGENT="$SCRIPT_DIR/push-metrics.js" + +# Overridable for testing only; cron uses the default. +ENV_FILE="${HOST_AGENT_ENV_FILE:-$HOME/.config/maximus-host-agent.env}" + +if [ ! -f "$ENV_FILE" ]; then + echo "host-agent: env file not found: $ENV_FILE" >&2 + exit 1 +fi + +# A warning, not a refusal: a wrong mode bit is worth shouting about, but it is +# not worth silencing the heartbeat of the whole workstation over. +FILE_MODE=$(stat -c "%a" "$ENV_FILE" 2>/dev/null || echo "") +if [ -n "$FILE_MODE" ] && [ "$FILE_MODE" != "600" ]; then + echo "host-agent: warning: $ENV_FILE is mode $FILE_MODE, expected 600" >&2 +fi + +# shellcheck source=/dev/null +. "$ENV_FILE" + +if [ -z "${HOSTS_API_URL:-}" ]; then + echo "host-agent: HOSTS_API_URL is not set in $ENV_FILE" >&2 + exit 1 +fi +if [ -z "${HOSTS_INGEST_TOKEN:-}" ]; then + echo "host-agent: HOSTS_INGEST_TOKEN is not set in $ENV_FILE" >&2 + exit 1 +fi +if [ -z "${HOST_ID:-}" ]; then + echo "host-agent: HOST_ID is not set in $ENV_FILE" >&2 + exit 1 +fi + +if [ ! -f "$AGENT" ]; then + echo "host-agent: agent not found: $AGENT" >&2 + exit 1 +fi + +# cron's PATH is famously short (/usr/bin:/bin), and a node installed through +# nvm or /usr/local lives outside it. Set NODE_BIN in the env file when +# `command -v node` comes up empty under cron but works in a login shell. +NODE_BIN="${NODE_BIN:-node}" +if ! command -v "$NODE_BIN" >/dev/null 2>&1; then + echo "host-agent: node not found (NODE_BIN=$NODE_BIN); set NODE_BIN in $ENV_FILE" >&2 + exit 1 +fi + +export HOSTS_API_URL HOSTS_INGEST_TOKEN HOST_ID + +exec "$NODE_BIN" "$AGENT" "$@"