Compare commits

..

3 commits

Author SHA1 Message Date
le king fu
ae27c7d504 docs: record the known holes of the column-0 require pin (#35)
All checks were successful
CI / verification (pull_request) Successful in 21s
The PR #34 review listed two forms that pass the "require at column 0"
heuristic: a function written whole on a column-0 line, and a require(
wrapped onto a column-0 line inside a function. Both stay open: neither
fits the style of the repository, and the second has the very shape of
the legitimate `} = require("node:fs");` that closes index.js's
multi-line import, so only a parser could tell them apart. They are now
written next to the pin and in CLAUDE.md, rather than only in a closed
PR and its follow-up issue.

CLAUDE.md also records the proxy bypass of boot-from-copy.sh, the wider
continue-on-error pin and its self-check, and the new counts (ci.test.js
12 -> 18 cases, vitest 340 -> 346).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 20:54:06 -04:00
le king fu
32db0f81d5 test(ci): catch continue-on-error in any YAML spelling (#35)
The pin was anchored at the start of the line, so a step written as a
flow mapping, `- { name: x, run: false, continue-on-error: true }`,
passed it: 12/12 green with that step appended to ci.yml. A quoted key,
`"continue-on-error": true`, escaped it the same way.

The detector is now unanchored and accepts a closing quote before the
colon. It still runs on `ciCode`, so full comment lines may quote the
key; anything else that quotes it (a run: script, a step name, a
trailing comment) now fails the pin, loudly, by design.

The detector is checked on its own: through the same comment stripping
as `ciCode`, it must catch a block key, a list item, a flow mapping and
a quoted key, and ignore a full comment line. Proven by mutation: the
flow-style and quoted-key steps appended to ci.yml each turn the pin
red; the old anchored regex turns the flow-mapping and quoted-key rows
red; dropping the quote class turns the quoted-key row red; disabling
the comment stripping turns the comment row red.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 20:53:13 -04:00
le king fu
fef9f69fc0 fix(ci): keep every curl of the boot check off environment proxies (#35)
Only the port pre-check of scripts/boot-from-copy.sh carried --noproxy.
The wait loop and the two assertions still honoured http_proxy, so on a
workstation with a proxy configured the script talked to the proxy
instead of the server it had just started, and failed loudly.

Export NO_PROXY='*' and no_proxy='*' at the top of the script, which
covers every curl call, and drop the now-redundant --noproxy.

The new test boots the script for real behind a fake proxy that answers
everything and records what reaches it. It first proves the proxy is
live (a bare curl with the same environment reaches it), then requires
that nothing of the boot check reaches it and that the script passes.
Red on the previous script (the wait loop and the first assertion went
through the proxy), red again when the export is removed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 20:51:57 -04:00
3 changed files with 106 additions and 21 deletions

View file

@ -62,17 +62,17 @@ curl -H "Authorization: Bearer $(cat ~/.coolify-token)" \
## Tests
- **CI** : `.forgejo/workflows/ci.yml` (issue #24), sur toute PR (pas de filtre `branches:`, pour que les maillons d'une pile soient batis) et sur `push` vers `main`. Un seul job en `container: node:22-alpine`, la base du Dockerfile : `npm ci`, `npm test`, `bash test-curl.sh`, puis `sh scripts/boot-from-copy.sh`. Verdict a lire sur le statut de commit (`GET /repos/maximus/vps-health-api/commits/<sha>/status`), l'API Actions repondant 404 sur cette instance.
- `npm test` (vitest) — 340 cas
- `npm test` (vitest) — 346 cas
- `__tests__/findings.test.js` — `/defenseurs/findings` (14 cas : auth, validation, filtres severity/category, asymetrie INFO, scan clean vs no_data, JSON corrompu)
- `__tests__/state.test.js` — `/defenseurs/state` + non-regression `/defenseurs` apres extraction de `serveJsonFile()` (9 cas : corps identique octet pour octet sur un JSON non canonique, no_data, isolation des deux chemins, 401 sans token / mauvais token / token d'ingestion, 404 sur POST)
- `__tests__/health.test.js` — module `metrics.js` + `/health` (6 cas : exports, payload champ pour champ, garde de latence)
- `__tests__/auth.test.js` — filet de securite auth sur les routes de lecture (67 cas : matrice route x mode d'echec, en-tetes malformes (dont le separateur `Bearer` autre qu'un espace), quasi-rates du token sur les 6 routes de lecture, fail-closed, **2 cas `404 (not 401)` qui figent l'ordre routage -> auth**, **un espion sur `timingSafeEqual`** qui exige deux empreintes de 32 octets a chaque requete, et **des epingles sur le source** : aucune egalite contre une chaine `Bearer`, un seul `tokenMatches()` et un seul `return null` dans `checkAuth()`, egalites limitees a une liste blanche, `tokenMatches()` qui hache les deux cotes sans test de longueur)
- `__tests__/hosts.test.js` — matrice exhaustive des endpoints postes (171 cas : regex de route comme controle anti-traversee, 401 avant 403, cloisonnement des deux tokens, 503 fail-closed, reconstruction par liste blanche, `loadAvg` a exactement 3 elements, 413, `neverSeen`, fraicheur serveur sous horloge figee, degradation de lecture, **balayage des `.tmp` orphelins** au demarrage, prouve sur le vrai point d'entree)
- `__tests__/agent.test.js` — agent poste (50 cas : contrat de payload verifie contre `sanitizeSnapshot()`, une seule tentative sans rejeu, timeout, codes de sortie, wrapper shell, et **le token absent de toute sortie d'echec, fragments de 5 caracteres compris**)
- `__tests__/ci.test.js` — epingles du workflow (12 cas : image du `container:` identique au `FROM` du Dockerfile, les quatre etapes presentes, aucun filtre `branches:` sous `pull_request`, `push` limite a `main`, ni `defaults:` ni `working-directory` sur une etape `uses:`, **aucun `continue-on-error`** (une etape rouge passerait au vert en silence), ce que le boot check suppose du Dockerfile : `USER node`, `CMD` en forme exec, le **refus d'un port deja occupe** (voisin HTTP reel, script lance en `spawn` asynchrone), et **des `require` en colonne 0 seulement, sans `import()`**, dans chaque `.js` du `COPY`)
- `__tests__/ci.test.js` — epingles du workflow (18 cas : image du `container:` identique au `FROM` du Dockerfile, les quatre etapes presentes, aucun filtre `branches:` sous `pull_request`, `push` limite a `main`, ni `defaults:` ni `working-directory` sur une etape `uses:`, **aucun `continue-on-error`, quelle qu'en soit l'ecriture** (une etape rouge passerait au vert en silence ; detecteur non ancre, lui-meme verifie sur la cle en bloc, en element de liste, en style flow et entre guillemets ; seules les lignes de commentaire entieres en sont exemptees), ce que le boot check suppose du Dockerfile : `USER node`, `CMD` en forme exec, le **refus d'un port deja occupe** (voisin HTTP reel, script lance en `spawn` asynchrone), **aucune requete par un proxy de l'environnement** (boot complet derriere un faux proxy qui repond a tout, sa vivacite prouvee d'abord par un `curl` nu), et **des `require` en colonne 0 seulement, sans `import()`**, dans chaque `.js` du `COPY`. Cette derniere epingle est une heuristique a deux trous connus, laisses ouverts (#35) : une fonction ecrite entiere sur une ligne en colonne 0 (`const lazy = () => require("x");`), et un `require(` renvoye a la ligne en colonne 0 dans une fonction, de meme forme que le `} = require("node:fs");` legitime d'`index.js`)
- `__tests__/writes.test.js` — seule surface d'ecriture : `HOSTS_DIR` (11 cas : espions sur toutes les fonctions d'ecriture de `node:fs`, formes callback, Sync et `fs.promises`, et sur `child_process`, poses avant un `require` frais apres purge du cache de **tous** les modules runtime (un `metrics.js` reste en cache garderait les espions restaures d'un test precedent) ; demarrage par le balayage des `.tmp` ; les six routes GET n'ecrivent rien, meme une macrotache apres la reponse, contenu reel exige ; `POST /hosts/<id>` n'ecrit que sous `HOSTS_DIR` ; **chemins degrades** : rejets 401/403/404, entrees 400/413, `agents-map.json` corrompu, fichiers Defenseurs absents, snapshot corrompu, avec le contenu degrade exige (`no_data`, `neverSeen`) ; **commandes shell** figees sur `head -1 /proc/stat` et `df -k /`, `/health` servant de temoin ; chaque cas prouve que ses espions sont vivants ; **epingle sur le bloc `require.main`**, ensemble exact des appels, puisqu'il ne tourne jamais dans le processus de test. Limites : un chemin qu'aucun cas n'emprunte, et une ecriture differee au-dela de 20 ms)
- Runtime reste 0-dep ; vitest en devDep uniquement
- `sh scripts/boot-from-copy.sh` — demarre le `CMD` depuis les seules sources du `COPY`, copiees dans un dossier vide, et exige 401 sans token, 200 avec, et des metriques disque/memoire non nulles. Tient lieu de `docker build` en CI : un vrai build demanderait le socket Docker dans le runner d'instance, donc root sur le VPS de prod pour tout workflow de tout depot. Il n'accepte qu'une ligne `COPY` en forme shell sans option, et un `CMD` en forme exec ; toute autre forme echoue bruyamment plutot que de verifier du vide. Port 3097 par defaut (`BOOT_PORT` pour en changer) ; il refuse de demarrer si le port est deja pris (tout code de sortie curl autre que 7, « connexion refusee »), sans quoi ses assertions pourraient interroger un voisin. Il n'appelle que `/health` : il ne prouve que les modules charges au demarrage, d'ou l'epingle « `require` en colonne 0 » dans `ci.test.js`.
- `sh scripts/boot-from-copy.sh` — demarre le `CMD` depuis les seules sources du `COPY`, copiees dans un dossier vide, et exige 401 sans token, 200 avec, et des metriques disque/memoire non nulles. Tient lieu de `docker build` en CI : un vrai build demanderait le socket Docker dans le runner d'instance, donc root sur le VPS de prod pour tout workflow de tout depot. Il n'accepte qu'une ligne `COPY` en forme shell sans option, et un `CMD` en forme exec ; toute autre forme echoue bruyamment plutot que de verifier du vide. Port 3097 par defaut (`BOOT_PORT` pour en changer) ; il refuse de demarrer si le port est deja pris (tout code de sortie curl autre que 7, « connexion refusee »), sans quoi ses assertions pourraient interroger un voisin. Il exporte `NO_PROXY='*'` et `no_proxy='*'` en tete : chaque `curl` vise la boucle locale, et un `http_proxy` de l'environnement repondrait a la place du port comme du serveur demarre. Il n'appelle que `/health` : il ne prouve que les modules charges au demarrage, d'ou l'epingle « `require` en colonne 0 » dans `ci.test.js`.
- `bash test-curl.sh` — passe au niveau socket, complementaire et non redondante (45 cas). Vitest couvre la logique ; ce script couvre ce que seuls un vrai client et un vrai TCP peuvent montrer : le 413 rendu **avant** que la connexion soit coupee, les deux tokens qui se refusent mutuellement leurs routes, le fichier servi octet pour octet par `/defenseurs/state`, et le 503 d'une instance demarree sans `HOSTS_INGEST_TOKEN` (elle tourne sur un second port, le temps du script). Il boote ses propres serveurs sur 3099 et 3098 — aucun service exterieur requis.
## Gotchas

View file

@ -15,10 +15,13 @@ const dockerfile = fs.readFileSync(path.join(ROOT, "Dockerfile"), "utf-8");
// Comment lines stripped, so an explanation that quotes a key cannot satisfy
// (or break) a pin.
const ciCode = ci
function stripCommentLines(text) {
return text
.split("\n")
.filter((line) => !/^\s*#/.test(line))
.join("\n");
}
const ciCode = stripCommentLines(ci);
// Lines of a top-level-indented block (` pull_request:`), up to the next key at
// the same or a lower indentation.
@ -77,10 +80,32 @@ describe("CI workflow", () => {
if (/^\s*uses:|\n\s*uses:/.test(step)) expect(step).not.toMatch(/working-directory:/);
}
});
test("never lets a failing step pass", () => {
// `continue-on-error: true` turns a red step green without a trace in the
// commit status. Anchored on the key position, so only a real key counts.
expect(ciCode).not.toMatch(/^\s*(- )?continue-on-error\s*:/m);
// commit status. Any spelling of the key counts: a block key or a list item,
// but also a flow mapping and a quoted key, which a pin anchored at the start
// of the line let through (#35). Unanchored on purpose: the key quoted in a
// `run:` script, a step name or a trailing comment fails the pin too, loudly.
// Only full comment lines are exempt, being stripped from `ciCode`.
const CONTINUE_ON_ERROR = /continue-on-error["']?\s*:/;
test("never lets a failing step pass", () => {
expect(ciCode).not.toMatch(CONTINUE_ON_ERROR);
});
// The detector itself, through the same comment stripping as `ciCode`. Anchor
// it at the start of the line again and the last two spellings turn red.
test.each([
["a block key", " - name: x\n run: npm test\n continue-on-error: true"],
["a list item", " - continue-on-error: true\n run: npm test"],
["a flow mapping", " - { name: x, run: false, continue-on-error: true }"],
["a quoted key", ' - name: x\n "continue-on-error": true'],
])("the continue-on-error pin catches %s", (_, step) => {
expect(stripCommentLines(step)).toMatch(CONTINUE_ON_ERROR);
});
test("the continue-on-error pin ignores a full comment line quoting the key", () => {
expect(stripCommentLines(" # never add continue-on-error: true here")).not.toMatch(CONTINUE_ON_ERROR);
});
});
@ -99,15 +124,12 @@ describe("what the boot check assumes of the Dockerfile", () => {
});
});
// Runs the script with a listener already on its port: without the pre-check,
// the wait loop could reach this neighbour and assert on a server it never
// started. Async spawn, so the listener keeps answering while the script runs.
function runBootScript(env) {
// Spawns a command from the repository root with `env` over the test's own
// environment; a key set to undefined is dropped (spawn ignores undefined
// values). Async, so a listener in this process keeps answering meanwhile.
function run(command, args, env) {
return new Promise((resolve, reject) => {
const child = spawn("sh", ["scripts/boot-from-copy.sh"], {
cwd: ROOT,
env: { ...process.env, ...env },
});
const child = spawn(command, args, { cwd: ROOT, env: { ...process.env, ...env } });
let stderr = "";
child.stderr.on("data", (d) => (stderr += d));
child.on("error", reject);
@ -115,7 +137,22 @@ function runBootScript(env) {
});
}
const runBootScript = (env) => run("sh", ["scripts/boot-from-copy.sh"], env);
// A port nothing listens on. The probe closes before the script starts: should
// something take the port in between, the script's own check refuses it and
// the test fails loudly.
async function freePort() {
const probe = http.createServer();
await new Promise((resolve) => probe.listen(0, "127.0.0.1", resolve));
const { port } = probe.address();
await new Promise((resolve) => probe.close(resolve));
return port;
}
describe("boot-from-copy.sh", () => {
// A listener already on the port: without the pre-check, the wait loop could
// reach this neighbour and assert on a server it never started.
test("refuses a port something else already holds", async () => {
const neighbour = http.createServer((req, res) => res.end("not the API"));
await new Promise((resolve) => neighbour.listen(0, "127.0.0.1", resolve));
@ -129,6 +166,39 @@ describe("boot-from-copy.sh", () => {
await new Promise((resolve) => neighbour.close(resolve));
}
}, 15000);
// Every curl of the script targets the loopback, and a proxy from the
// environment would answer in the server's place. Before #35 only the port
// check had --noproxy: the wait loop and the assertions reached the proxy.
test("never sends a request through a proxy from the environment", async () => {
// Answers everything, as a real proxy would, and records what reaches it.
const proxied = [];
const proxy = http.createServer((req, res) => {
proxied.push(`${req.method} ${req.url}`);
res.end("answered by the proxy");
});
await new Promise((resolve) => proxy.listen(0, "127.0.0.1", resolve));
try {
const proxyUrl = `http://127.0.0.1:${proxy.address().port}`;
// ALL_PROXY as well, so that unsetting http_proxy alone would not pass.
// An inherited NO_PROXY would hide the defect, hence dropped.
const env = { http_proxy: proxyUrl, ALL_PROXY: proxyUrl, NO_PROXY: undefined, no_proxy: undefined };
const port = await freePort();
// The proxy is live: a bare curl with the same environment reaches it.
// Without this, "nothing reached the proxy" could pass for nothing.
await run("curl", ["-s", "-o", "/dev/null", "--max-time", "5", `http://127.0.0.1:${port}/`], env);
expect(proxied).toEqual([`GET http://127.0.0.1:${port}/`]);
proxied.length = 0;
const { code, stderr } = await runBootScript({ ...env, BOOT_PORT: String(port) });
expect(proxied).toEqual([]);
expect(code, stderr).toBe(0);
} finally {
await new Promise((resolve) => proxy.close(resolve));
}
}, 20000);
});
// The boot check only calls /health, so it proves the existence of the modules
@ -137,6 +207,14 @@ describe("boot-from-copy.sh", () => {
// startup": every require starts at column 0 (a top-level statement), and there
// is no dynamic import. Comment lines are skipped, so an explanation quoting
// require( cannot break the pin.
//
// Known holes, left open (#35). A function written whole on a column-0 line
// passes: `const lazy = () => require("x");`, or the same as a one-line
// `function lazy() { return require("x"); }`. So does a require( wrapped onto a
// column-0 line inside a function. Neither fits the style of this repository,
// and the second has the very shape of the top-level `} = require("node:fs");`
// that closes index.js's multi-line import: telling them apart takes a parser,
// not a line regex.
describe("runtime modules are all loaded at startup", () => {
const copyLine = dockerfile.match(/^\s*COPY\s+(.+)$/m);
const sources = copyLine ? copyLine[1].trim().split(/\s+/).slice(0, -1) : [];

View file

@ -20,6 +20,13 @@
set -eu
# Every curl below targets this machine's loopback, and a proxy from the
# environment (http_proxy, ALL_PROXY) would answer in its place: the port check
# would probe the proxy instead of the port, the wait loop and the assertions
# would talk to the proxy instead of the server started here. Both spellings,
# since curl reads the lowercase one first and an inherited no_proxy would win.
export NO_PROXY='*' no_proxy='*'
PORT="${BOOT_PORT:-3097}"
TOKEN="boot-from-copy-token"
DOCKERFILE="Dockerfile"
@ -65,11 +72,11 @@ CMD_SH=$(printf '%s' "$CMD_LINE" | node -e '
# check a server we never started. So refuse up front: the port counts as free
# only when curl gets "connection refused" (exit 7). Any other exit — an HTTP
# answer (0), a listener that accepts and stalls (28), a non-HTTP service
# (52, 56) — means it is taken. --noproxy keeps an http_proxy from answering
# in the port's place.
# (52, 56) — means it is taken. No proxy answers in the port's place: NO_PROXY
# is exported at the top.
command -v curl >/dev/null 2>&1 || fail "curl is required"
RC=0
curl -s -o /dev/null --noproxy '*' --max-time 2 "http://127.0.0.1:$PORT/" || RC=$?
curl -s -o /dev/null --max-time 2 "http://127.0.0.1:$PORT/" || RC=$?
[ "$RC" = "7" ] || fail "port $PORT is already taken (curl exit $RC); set BOOT_PORT to a free port"
# --- Build the image's file set ---------------------------------------------