diff --git a/.forgejo/workflows/ci.yml b/.forgejo/workflows/ci.yml new file mode 100644 index 0000000..60696fe --- /dev/null +++ b/.forgejo/workflows/ci.yml @@ -0,0 +1,67 @@ +name: CI + +# Until issue #24 nothing verified this repository: every stack was checked by +# hand, by whoever remembered to run the right command. Modelled on +# `roadtrip/.forgejo/workflows/ci.yml` and `simpl-sante/.forgejo/workflows/ci.yml`, +# the workflows known to run on this runner (Forgejo 8.0.3, runner v12.7.0). +# Two forms are avoided because the runner rejects the whole job before its +# first command, the error being readable only in the Forgejo database on the VPS: +# +# - `working-directory` on a `uses:` step (allowed on `run:` only) +# - `defaults:` at job level +# +# No `concurrency:` key: it is not inert on this instance, and a second push +# cancels the first run — whose missing status then reads like a runner failure. +# +# See `la-compagnie-maximus/docs/forgejo-ops.md` and the `diag-forgejo-ci` skill. + +on: + # No `branches:` filter, on purpose. In a stacked run each PR targets the + # previous link's branch and only the root one targets `main`: with + # `branches: [main]` every intermediate link would get no status at all, + # indistinguishable from a runner that never saw the commit. + pull_request: + # Merges land as fast-forwards pushed to `main`: a PR trigger alone would miss + # what actually reaches production. + push: + branches: [main] + +jobs: + verification: + runs-on: ubuntu + # The Dockerfile's own base image (a vitest pin keeps the two aligned): the + # tests, the socket pass and the boot check all see the musl libc and busybox + # tools production runs on. The job runs as root in this container, unlike + # the image (`USER node`) — a future test relying on an unreadable file + # would be weakened here without failing. + container: node:22-alpine + + steps: + - name: Tooling + # Before any `uses:` step: checkout needs git, test-curl.sh needs bash + # and curl. The image already carries node, which runs checkout@v4. + run: | + apk add --no-cache git bash curl + node --version + npm --version + + - name: Checkout + uses: https://github.com/actions/checkout@v4 + + - name: Dependencies + run: npm ci + + - name: Unit tests + run: npm test + + - name: Socket-level pass + # Boots its own servers on 3099 and 3098, no outside service needed. + run: bash test-curl.sh + + - name: Boot from the COPY file set + # Stands in for a `docker build`: a real build would need the Docker + # socket inside the job, which on this instance-scoped runner makes every + # workflow of every repository root on the production VPS. The script + # starts the CMD from the COPY sources alone and catches the + # MODULE_NOT_FOUND that #11 had to fix by hand. + run: sh scripts/boot-from-copy.sh diff --git a/CLAUDE.md b/CLAUDE.md index 5a8815a..11ade2e 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -59,20 +59,23 @@ curl -H "Authorization: Bearer $(cat ~/.coolify-token)" \ ## Tests -- `npm test` (vitest) — 301 cas +- **CI** : `.forgejo/workflows/ci.yml` (issue #24), sur toute PR (pas de filtre `branches:`, pour que les maillons d'une pile soient batis) et sur `push` vers `main`. Un seul job en `container: node:22-alpine`, la base du Dockerfile : `npm ci`, `npm test`, `bash test-curl.sh`, puis `sh scripts/boot-from-copy.sh`. Verdict a lire sur le statut de commit (`GET /repos/maximus/vps-health-api/commits//status`), l'API Actions repondant 404 sur cette instance. +- `npm test` (vitest) — 308 cas - `__tests__/findings.test.js` — `/defenseurs/findings` (14 cas : auth, validation, filtres severity/category, asymetrie INFO, scan clean vs no_data, JSON corrompu) - `__tests__/state.test.js` — `/defenseurs/state` + non-regression `/defenseurs` apres extraction de `serveJsonFile()` (9 cas : corps identique octet pour octet sur un JSON non canonique, no_data, isolation des deux chemins, 401 sans token / mauvais token / token d'ingestion, 404 sur POST) - `__tests__/health.test.js` — module `metrics.js` + `/health` (6 cas : exports, payload champ pour champ, garde de latence) - `__tests__/auth.test.js` — filet de securite auth sur les routes de lecture (67 cas : matrice route x mode d'echec, en-tetes malformes (dont le separateur `Bearer` autre qu'un espace), quasi-rates du token sur les 6 routes de lecture, fail-closed, **2 cas `404 (not 401)` qui figent l'ordre routage -> auth**, **un espion sur `timingSafeEqual`** qui exige deux empreintes de 32 octets a chaque requete, et **des epingles sur le source** : aucune egalite contre une chaine `Bearer`, un seul `tokenMatches()` et un seul `return null` dans `checkAuth()`, egalites limitees a une liste blanche, `tokenMatches()` qui hache les deux cotes sans test de longueur) - `__tests__/hosts.test.js` — matrice exhaustive des endpoints postes (155 cas : regex de route comme controle anti-traversee, 401 avant 403, cloisonnement des deux tokens, 503 fail-closed, reconstruction par liste blanche, 413, `neverSeen`, fraicheur serveur sous horloge figee, degradation de lecture) - `__tests__/agent.test.js` — agent poste (50 cas : contrat de payload verifie contre `sanitizeSnapshot()`, une seule tentative sans rejeu, timeout, codes de sortie, wrapper shell, et **le token absent de toute sortie d'echec, fragments de 5 caracteres compris**) + - `__tests__/ci.test.js` — epingles du workflow (7 cas : image du `container:` identique au `FROM` du Dockerfile, les quatre etapes presentes, aucun filtre `branches:` sous `pull_request`, `push` limite a `main`, ni `defaults:` ni `working-directory` sur une etape `uses:`, et ce que le boot check suppose du Dockerfile : `USER node`, `CMD` en forme exec) - Runtime reste 0-dep ; vitest en devDep uniquement +- `sh scripts/boot-from-copy.sh` — demarre le `CMD` depuis les seules sources du `COPY`, copiees dans un dossier vide, et exige 401 sans token, 200 avec, et des metriques disque/memoire non nulles. Tient lieu de `docker build` en CI : un vrai build demanderait le socket Docker dans le runner d'instance, donc root sur le VPS de prod pour tout workflow de tout depot. Il n'accepte qu'une ligne `COPY` en forme shell sans option, et un `CMD` en forme exec ; toute autre forme echoue bruyamment plutot que de verifier du vide. - `bash test-curl.sh` — passe au niveau socket, complementaire et non redondante (45 cas). Vitest couvre la logique ; ce script couvre ce que seuls un vrai client et un vrai TCP peuvent montrer : le 413 rendu **avant** que la connexion soit coupee, les deux tokens qui se refusent mutuellement leurs routes, le fichier servi octet pour octet par `/defenseurs/state`, et le 503 d'une instance demarree sans `HOSTS_INGEST_TOKEN` (elle tourne sur un second port, le temps du script). Il boote ses propres serveurs sur 3099 et 3098 — aucun service exterieur requis. ## Gotchas - Pas d'Express — HTTP natif Node.js uniquement -- Le `COPY` du Dockerfile liste les fichiers un par un (`package.json index.js metrics.js`) : tout nouveau module runtime doit y etre ajoute, sinon le conteneur plante au demarrage sur un `MODULE_NOT_FOUND` que les tests locaux ne voient pas. Corollaire : `agent/` n'entre PAS dans l'image — c'est du code de poste, livre par copie de fichiers (voir `agent/README.md`), et un test epingle que le `COPY` ne le mentionne jamais. +- Le `COPY` du Dockerfile liste les fichiers un par un (`package.json index.js metrics.js`) : tout nouveau module runtime doit y etre ajoute, sinon le conteneur plante au demarrage sur un `MODULE_NOT_FOUND` que `npm test` ne voit pas (seul `scripts/boot-from-copy.sh`, en CI, le rattrape). Corollaire : `agent/` n'entre PAS dans l'image — c'est du code de poste, livre par copie de fichiers (voir `agent/README.md`), et un test epingle que le `COPY` ne le mentionne jamais. - `agent/` tourne sur le ThinkPad, pas sur le VPS, et sa sortie part dans `logger -t host-agent` : tout ce qu'il imprime finit dans `/var/log/syslog` et journald pour de bon. Le chemin d'erreur ne rend donc que `err.code` et le code HTTP — jamais un objet d'erreur, jamais les options de requete, jamais un en-tete. Et jamais de `set -x` dans `run-push.sh` : le shell echoerait le token en sourcant le fichier d'env. - `getHealth()` garde `Promise.all([collectMetrics(), getLogtoHealth()])` : l'echantillon CPU de 500 ms et le check Logto (timeout 3 s) sont deliberement concurrents. Les serialiser ferait monter le p99 de `/health` a ~3,5 s. Garde de non-regression : le test de latence dans `__tests__/health.test.js`. - L'API n'est plus read-only depuis l'issue #13. `status.json`, `agents-map.json` et les rapports de scan restent ecrits par le Sergent defenseurs et lus seulement ici ; en revanche `POST /hosts/` ecrit dans `HOSTS_DIR`. Consequence : ce montage-la doit etre en lecture-ecriture pour l'uid 1000, contrairement aux montages `/data/defenseurs`. diff --git a/__tests__/ci.test.js b/__tests__/ci.test.js new file mode 100644 index 0000000..5d34d06 --- /dev/null +++ b/__tests__/ci.test.js @@ -0,0 +1,93 @@ +const fs = require("node:fs"); +const path = require("node:path"); + +// Pins on the CI workflow and on what the boot check assumes of the Dockerfile. +// The runtime stays 0-dependency and vitest ships no YAML parser, so these read +// the files with narrow regexes: each one looks at a single key, never at the +// YAML structure as a whole. + +const ROOT = path.join(__dirname, ".."); +const CI_PATH = path.join(ROOT, ".forgejo", "workflows", "ci.yml"); +const ci = fs.readFileSync(CI_PATH, "utf-8"); +const dockerfile = fs.readFileSync(path.join(ROOT, "Dockerfile"), "utf-8"); + +// Comment lines stripped, so an explanation that quotes a key cannot satisfy +// (or break) a pin. +const ciCode = ci + .split("\n") + .filter((line) => !/^\s*#/.test(line)) + .join("\n"); + +// Lines of a top-level-indented block (` pull_request:`), up to the next key at +// the same or a lower indentation. +function block(text, key) { + const lines = text.split("\n"); + const start = lines.findIndex((l) => new RegExp(`^(\\s*)${key}:`).test(l)); + if (start === -1) return null; + const indent = lines[start].match(/^(\s*)/)[1].length; + const out = [lines[start]]; + for (const line of lines.slice(start + 1)) { + if (line.trim() === "") continue; + if (line.match(/^(\s*)/)[1].length <= indent) break; + out.push(line); + } + return out.join("\n"); +} + +describe("CI workflow", () => { + test("runs in the Dockerfile's own base image", () => { + const from = dockerfile.match(/^\s*FROM\s+(\S+)/m); + const container = ciCode.match(/^\s*container:\s*["']?([^"'\s]+)["']?\s*$/m); + + expect(from).not.toBeNull(); + expect(container).not.toBeNull(); + expect(container[1]).toBe(from[1]); + }); + + test("runs the unit tests, the socket pass and the boot check", () => { + expect(ciCode).toMatch(/^\s*run:\s*npm ci\s*$/m); + expect(ciCode).toMatch(/^\s*run:\s*npm test\s*$/m); + expect(ciCode).toMatch(/^\s*run:\s*bash test-curl\.sh\s*$/m); + expect(ciCode).toMatch(/^\s*run:\s*sh scripts\/boot-from-copy\.sh\s*$/m); + }); + + test("builds every pull request, whatever branch it targets", () => { + // A `branches: [main]` here would leave the intermediate links of a stack + // without any status. + const pr = block(ciCode, "pull_request"); + + expect(pr).not.toBeNull(); + expect(pr).not.toMatch(/branches(-ignore)?:/); + }); + + test("builds pushes to main", () => { + const push = block(ciCode, "push"); + + expect(push).not.toBeNull(); + expect(push).toMatch(/branches:\s*\[\s*main\s*\]/); + }); + + test("avoids the two forms the runner rejects before the first command", () => { + // `defaults:` anywhere, and `working-directory` on a `uses:` step. + expect(ciCode).not.toMatch(/^\s*defaults:/m); + const steps = ciCode.split(/^\s*- (?=name:|uses:|run:)/m); + for (const step of steps) { + if (/^\s*uses:|\n\s*uses:/.test(step)) expect(step).not.toMatch(/working-directory:/); + } + }); +}); + +describe("what the boot check assumes of the Dockerfile", () => { + test("the image still drops to the node user, as the script does", () => { + expect(dockerfile).toMatch(/^\s*USER\s+node\s*$/m); + }); + + test("CMD is in exec form, the only one the script parses", () => { + const cmd = dockerfile.match(/^\s*CMD\s+(.+)$/m); + + expect(cmd).not.toBeNull(); + const argv = JSON.parse(cmd[1]); + expect(Array.isArray(argv)).toBe(true); + expect(argv.length).toBeGreaterThan(0); + }); +}); diff --git a/scripts/boot-from-copy.sh b/scripts/boot-from-copy.sh new file mode 100644 index 0000000..3ddac84 --- /dev/null +++ b/scripts/boot-from-copy.sh @@ -0,0 +1,140 @@ +#!/bin/sh +# boot-from-copy.sh — start the API from exactly the files the image would hold. +# +# The Dockerfile COPY lists runtime files one by one. A module missing from that +# list passes every test (the tests run from the full checkout) and only crashes +# the container at redeploy, with MODULE_NOT_FOUND. That is what #11 had to fix +# by hand for metrics.js. +# +# A real `docker build` would need the Docker socket inside the CI job, which on +# the shared instance-scoped runner makes every workflow of every repository +# root on the production VPS. So this script rebuilds the image's file set +# instead: it copies the COPY sources into an empty directory, runs the CMD +# there, and asserts the server answers. In CI it runs in `node:22-alpine`, the +# image's own base. +# +# Usage : sh scripts/boot-from-copy.sh (from the repository root) +# Exit 0 if the server boots and answers, 1 otherwise. + +set -eu + +PORT="${BOOT_PORT:-3097}" +TOKEN="boot-from-copy-token" +DOCKERFILE="Dockerfile" + +fail() { + echo "FAIL: $*" >&2 + exit 1 +} + +[ -f "$DOCKERFILE" ] || fail "no $DOCKERFILE here — run from the repository root" + +# --- Parse COPY ------------------------------------------------------------- +# Only the plain shell form is understood. Anything else (flags, JSON array, +# several COPY lines) fails loudly rather than being half-parsed into a +# check that verifies nothing. +COPY_LINES=$(grep -cE '^[[:space:]]*COPY[[:space:]]' "$DOCKERFILE" || true) +[ "$COPY_LINES" = "1" ] || fail "expected exactly one COPY line, found $COPY_LINES" +COPY_ARGS=$(grep -E '^[[:space:]]*COPY[[:space:]]' "$DOCKERFILE" | sed -E 's/^[[:space:]]*COPY[[:space:]]+//') +case "$COPY_ARGS" in + --*) fail "COPY flags are not supported by this check: $COPY_ARGS" ;; + \[*) fail "COPY JSON form is not supported by this check: $COPY_ARGS" ;; +esac +# shellcheck disable=SC2086 # word splitting is the point: one word per source +set -- $COPY_ARGS +[ "$#" -ge 2 ] || fail "COPY needs at least one source and a destination: $COPY_ARGS" + +# --- Parse CMD -------------------------------------------------------------- +# Taken from the Dockerfile rather than hardcoded, so the check follows the +# entrypoint if it ever changes. Exec (JSON array) form only. +CMD_LINE=$(grep -E '^[[:space:]]*CMD[[:space:]]' "$DOCKERFILE" | tail -1 | sed -E 's/^[[:space:]]*CMD[[:space:]]+//') +CMD_SH=$(printf '%s' "$CMD_LINE" | node -e ' + let s = ""; process.stdin.on("data", (d) => (s += d)).on("end", () => { + let a; + try { a = JSON.parse(s); } catch { process.exit(1); } + if (!Array.isArray(a) || !a.length || !a.every((x) => /^[A-Za-z0-9._\/-]+$/.test(x))) process.exit(1); + process.stdout.write(a.join(" ")); + }); +') || fail "CMD must be an exec-form JSON array of plain words: $CMD_LINE" + +# --- Build the image's file set --------------------------------------------- +APP_DIR=$(mktemp -d) +LOG="$APP_DIR.log" +SERVER_PID="" +trap '[ -n "$SERVER_PID" ] && kill "$SERVER_PID" 2>/dev/null; rm -rf "$APP_DIR" "$LOG"; true' EXIT + +N=0 +while [ "$#" -gt 1 ]; do + [ -f "$1" ] || fail "COPY source $1 does not exist in the repository" + cp "$1" "$APP_DIR/" + N=$((N + 1)) + shift +done +[ "$N" -gt 0 ] || fail "no COPY source found" +echo "Image file set ($N): $(ls "$APP_DIR" | tr '\n' ' ')" + +# mktemp creates the directory 0700: the `node` user could not even enter it +# and the failure would read EACCES instead of the MODULE_NOT_FOUND we look for. +chmod 755 "$APP_DIR" +chmod 644 "$APP_DIR"/* + +# --- Start ------------------------------------------------------------------ +# The image runs as `node`. In the CI container the job is root, so drop to +# `node` like the image does; on a workstation, run as whoever we are. +# LOGTO_HEALTH_URL points at a closed local port: no external network needed, +# the Logto check reports "down" and /health still answers 200. +# busybox `su` cannot preserve the environment, hence the inline variables. +ENV_INLINE="PORT=$PORT HEALTH_TOKEN=$TOKEN LOGTO_HEALTH_URL=http://127.0.0.1:9/" +if [ "$(id -u)" = "0" ]; then + id node >/dev/null 2>&1 || fail "running as root but no 'node' user to drop to" + su node -s /bin/sh -c "cd '$APP_DIR' && exec env $ENV_INLINE $CMD_SH" >"$LOG" 2>&1 & +else + (cd "$APP_DIR" && exec env $ENV_INLINE $CMD_SH) >"$LOG" 2>&1 & +fi +SERVER_PID=$! + +UP="" +i=0 +while [ "$i" -lt 50 ]; do + if ! kill -0 "$SERVER_PID" 2>/dev/null; then + echo "--- server log ---" >&2 + cat "$LOG" >&2 + fail "the server died at startup (see its log above)" + fi + if curl -s -o /dev/null "http://127.0.0.1:$PORT/health"; then + UP=yes + break + fi + sleep 0.2 + i=$((i + 1)) +done +if [ -z "$UP" ]; then + echo "--- server log ---" >&2 + cat "$LOG" >&2 + fail "the server did not answer on :$PORT within 10 s" +fi + +# --- Assert ----------------------------------------------------------------- +CODE=$(curl -s -o /dev/null -w '%{http_code}' "http://127.0.0.1:$PORT/health") +[ "$CODE" = "401" ] || fail "GET /health without token: expected 401, got $CODE" +echo "ok GET /health without token -> 401" + +BODY=$(curl -s -w '\n%{http_code}' -H "Authorization: Bearer $TOKEN" "http://127.0.0.1:$PORT/health") +CODE=$(printf '%s' "$BODY" | tail -1) +[ "$CODE" = "200" ] || fail "GET /health with token: expected 200, got $CODE" +echo "ok GET /health with token -> 200" + +# metrics.js swallows a failed `df` parse and reports zeros. Production runs on +# Alpine, whose busybox df is what this catches. In CI the figure is the job +# container's overlay root, not the production disk: a failure here means the +# parsing broke, never that a disk is full. +printf '%s' "$BODY" | sed '$d' | node -e ' + let s = ""; process.stdin.on("data", (d) => (s += d)).on("end", () => { + const b = JSON.parse(s); + if (!(b.disk && b.disk.totalGB > 0)) { console.error("disk:", JSON.stringify(b.disk)); process.exit(1); } + if (!(b.memory && b.memory.totalGB > 0)) { console.error("memory:", JSON.stringify(b.memory)); process.exit(1); } + }); +' || fail "/health reported empty disk or memory metrics" +echo "ok /health disk and memory metrics are non-zero" + +echo "boot-from-copy: PASS"