ci: add a Forgejo Actions workflow with a boot-from-COPY check (#24)
Nothing verified this repository until now. The workflow runs on every pull request (no branches filter, so intermediate stack links get a status) and on pushes to main, in node:22-alpine, the Dockerfile's own base: npm ci, npm test, the socket-level pass, then scripts/boot-from-copy.sh. The script stands in for a docker build, which would need the Docker socket in the instance-scoped runner and so root on the production VPS for every workflow. It starts the CMD from the COPY sources alone and fails on the MODULE_NOT_FOUND that #11 had to fix by hand. __tests__/ci.test.js pins the workflow and what the script assumes of the Dockerfile. Refs #24 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
parent
2b35b78a0b
commit
4e7879da6c
4 changed files with 305 additions and 2 deletions
67
.forgejo/workflows/ci.yml
Normal file
67
.forgejo/workflows/ci.yml
Normal file
|
|
@ -0,0 +1,67 @@
|
|||
name: CI
|
||||
|
||||
# Until issue #24 nothing verified this repository: every stack was checked by
|
||||
# hand, by whoever remembered to run the right command. Modelled on
|
||||
# `roadtrip/.forgejo/workflows/ci.yml` and `simpl-sante/.forgejo/workflows/ci.yml`,
|
||||
# the workflows known to run on this runner (Forgejo 8.0.3, runner v12.7.0).
|
||||
# Two forms are avoided because the runner rejects the whole job before its
|
||||
# first command, the error being readable only in the Forgejo database on the VPS:
|
||||
#
|
||||
# - `working-directory` on a `uses:` step (allowed on `run:` only)
|
||||
# - `defaults:` at job level
|
||||
#
|
||||
# No `concurrency:` key: it is not inert on this instance, and a second push
|
||||
# cancels the first run — whose missing status then reads like a runner failure.
|
||||
#
|
||||
# See `la-compagnie-maximus/docs/forgejo-ops.md` and the `diag-forgejo-ci` skill.
|
||||
|
||||
on:
|
||||
# No `branches:` filter, on purpose. In a stacked run each PR targets the
|
||||
# previous link's branch and only the root one targets `main`: with
|
||||
# `branches: [main]` every intermediate link would get no status at all,
|
||||
# indistinguishable from a runner that never saw the commit.
|
||||
pull_request:
|
||||
# Merges land as fast-forwards pushed to `main`: a PR trigger alone would miss
|
||||
# what actually reaches production.
|
||||
push:
|
||||
branches: [main]
|
||||
|
||||
jobs:
|
||||
verification:
|
||||
runs-on: ubuntu
|
||||
# The Dockerfile's own base image (a vitest pin keeps the two aligned): the
|
||||
# tests, the socket pass and the boot check all see the musl libc and busybox
|
||||
# tools production runs on. The job runs as root in this container, unlike
|
||||
# the image (`USER node`) — a future test relying on an unreadable file
|
||||
# would be weakened here without failing.
|
||||
container: node:22-alpine
|
||||
|
||||
steps:
|
||||
- name: Tooling
|
||||
# Before any `uses:` step: checkout needs git, test-curl.sh needs bash
|
||||
# and curl. The image already carries node, which runs checkout@v4.
|
||||
run: |
|
||||
apk add --no-cache git bash curl
|
||||
node --version
|
||||
npm --version
|
||||
|
||||
- name: Checkout
|
||||
uses: https://github.com/actions/checkout@v4
|
||||
|
||||
- name: Dependencies
|
||||
run: npm ci
|
||||
|
||||
- name: Unit tests
|
||||
run: npm test
|
||||
|
||||
- name: Socket-level pass
|
||||
# Boots its own servers on 3099 and 3098, no outside service needed.
|
||||
run: bash test-curl.sh
|
||||
|
||||
- name: Boot from the COPY file set
|
||||
# Stands in for a `docker build`: a real build would need the Docker
|
||||
# socket inside the job, which on this instance-scoped runner makes every
|
||||
# workflow of every repository root on the production VPS. The script
|
||||
# starts the CMD from the COPY sources alone and catches the
|
||||
# MODULE_NOT_FOUND that #11 had to fix by hand.
|
||||
run: sh scripts/boot-from-copy.sh
|
||||
|
|
@ -59,20 +59,23 @@ curl -H "Authorization: Bearer $(cat ~/.coolify-token)" \
|
|||
|
||||
## Tests
|
||||
|
||||
- `npm test` (vitest) — 301 cas
|
||||
- **CI** : `.forgejo/workflows/ci.yml` (issue #24), sur toute PR (pas de filtre `branches:`, pour que les maillons d'une pile soient batis) et sur `push` vers `main`. Un seul job en `container: node:22-alpine`, la base du Dockerfile : `npm ci`, `npm test`, `bash test-curl.sh`, puis `sh scripts/boot-from-copy.sh`. Verdict a lire sur le statut de commit (`GET /repos/maximus/vps-health-api/commits/<sha>/status`), l'API Actions repondant 404 sur cette instance.
|
||||
- `npm test` (vitest) — 308 cas
|
||||
- `__tests__/findings.test.js` — `/defenseurs/findings` (14 cas : auth, validation, filtres severity/category, asymetrie INFO, scan clean vs no_data, JSON corrompu)
|
||||
- `__tests__/state.test.js` — `/defenseurs/state` + non-regression `/defenseurs` apres extraction de `serveJsonFile()` (9 cas : corps identique octet pour octet sur un JSON non canonique, no_data, isolation des deux chemins, 401 sans token / mauvais token / token d'ingestion, 404 sur POST)
|
||||
- `__tests__/health.test.js` — module `metrics.js` + `/health` (6 cas : exports, payload champ pour champ, garde de latence)
|
||||
- `__tests__/auth.test.js` — filet de securite auth sur les routes de lecture (67 cas : matrice route x mode d'echec, en-tetes malformes (dont le separateur `Bearer` autre qu'un espace), quasi-rates du token sur les 6 routes de lecture, fail-closed, **2 cas `404 (not 401)` qui figent l'ordre routage -> auth**, **un espion sur `timingSafeEqual`** qui exige deux empreintes de 32 octets a chaque requete, et **des epingles sur le source** : aucune egalite contre une chaine `Bearer`, un seul `tokenMatches()` et un seul `return null` dans `checkAuth()`, egalites limitees a une liste blanche, `tokenMatches()` qui hache les deux cotes sans test de longueur)
|
||||
- `__tests__/hosts.test.js` — matrice exhaustive des endpoints postes (155 cas : regex de route comme controle anti-traversee, 401 avant 403, cloisonnement des deux tokens, 503 fail-closed, reconstruction par liste blanche, 413, `neverSeen`, fraicheur serveur sous horloge figee, degradation de lecture)
|
||||
- `__tests__/agent.test.js` — agent poste (50 cas : contrat de payload verifie contre `sanitizeSnapshot()`, une seule tentative sans rejeu, timeout, codes de sortie, wrapper shell, et **le token absent de toute sortie d'echec, fragments de 5 caracteres compris**)
|
||||
- `__tests__/ci.test.js` — epingles du workflow (7 cas : image du `container:` identique au `FROM` du Dockerfile, les quatre etapes presentes, aucun filtre `branches:` sous `pull_request`, `push` limite a `main`, ni `defaults:` ni `working-directory` sur une etape `uses:`, et ce que le boot check suppose du Dockerfile : `USER node`, `CMD` en forme exec)
|
||||
- Runtime reste 0-dep ; vitest en devDep uniquement
|
||||
- `sh scripts/boot-from-copy.sh` — demarre le `CMD` depuis les seules sources du `COPY`, copiees dans un dossier vide, et exige 401 sans token, 200 avec, et des metriques disque/memoire non nulles. Tient lieu de `docker build` en CI : un vrai build demanderait le socket Docker dans le runner d'instance, donc root sur le VPS de prod pour tout workflow de tout depot. Il n'accepte qu'une ligne `COPY` en forme shell sans option, et un `CMD` en forme exec ; toute autre forme echoue bruyamment plutot que de verifier du vide.
|
||||
- `bash test-curl.sh` — passe au niveau socket, complementaire et non redondante (45 cas). Vitest couvre la logique ; ce script couvre ce que seuls un vrai client et un vrai TCP peuvent montrer : le 413 rendu **avant** que la connexion soit coupee, les deux tokens qui se refusent mutuellement leurs routes, le fichier servi octet pour octet par `/defenseurs/state`, et le 503 d'une instance demarree sans `HOSTS_INGEST_TOKEN` (elle tourne sur un second port, le temps du script). Il boote ses propres serveurs sur 3099 et 3098 — aucun service exterieur requis.
|
||||
|
||||
## Gotchas
|
||||
|
||||
- Pas d'Express — HTTP natif Node.js uniquement
|
||||
- Le `COPY` du Dockerfile liste les fichiers un par un (`package.json index.js metrics.js`) : tout nouveau module runtime doit y etre ajoute, sinon le conteneur plante au demarrage sur un `MODULE_NOT_FOUND` que les tests locaux ne voient pas. Corollaire : `agent/` n'entre PAS dans l'image — c'est du code de poste, livre par copie de fichiers (voir `agent/README.md`), et un test epingle que le `COPY` ne le mentionne jamais.
|
||||
- Le `COPY` du Dockerfile liste les fichiers un par un (`package.json index.js metrics.js`) : tout nouveau module runtime doit y etre ajoute, sinon le conteneur plante au demarrage sur un `MODULE_NOT_FOUND` que `npm test` ne voit pas (seul `scripts/boot-from-copy.sh`, en CI, le rattrape). Corollaire : `agent/` n'entre PAS dans l'image — c'est du code de poste, livre par copie de fichiers (voir `agent/README.md`), et un test epingle que le `COPY` ne le mentionne jamais.
|
||||
- `agent/` tourne sur le ThinkPad, pas sur le VPS, et sa sortie part dans `logger -t host-agent` : tout ce qu'il imprime finit dans `/var/log/syslog` et journald pour de bon. Le chemin d'erreur ne rend donc que `err.code` et le code HTTP — jamais un objet d'erreur, jamais les options de requete, jamais un en-tete. Et jamais de `set -x` dans `run-push.sh` : le shell echoerait le token en sourcant le fichier d'env.
|
||||
- `getHealth()` garde `Promise.all([collectMetrics(), getLogtoHealth()])` : l'echantillon CPU de 500 ms et le check Logto (timeout 3 s) sont deliberement concurrents. Les serialiser ferait monter le p99 de `/health` a ~3,5 s. Garde de non-regression : le test de latence dans `__tests__/health.test.js`.
|
||||
- L'API n'est plus read-only depuis l'issue #13. `status.json`, `agents-map.json` et les rapports de scan restent ecrits par le Sergent defenseurs et lus seulement ici ; en revanche `POST /hosts/<id>` ecrit dans `HOSTS_DIR`. Consequence : ce montage-la doit etre en lecture-ecriture pour l'uid 1000, contrairement aux montages `/data/defenseurs`.
|
||||
|
|
|
|||
93
__tests__/ci.test.js
Normal file
93
__tests__/ci.test.js
Normal file
|
|
@ -0,0 +1,93 @@
|
|||
const fs = require("node:fs");
|
||||
const path = require("node:path");
|
||||
|
||||
// Pins on the CI workflow and on what the boot check assumes of the Dockerfile.
|
||||
// The runtime stays 0-dependency and vitest ships no YAML parser, so these read
|
||||
// the files with narrow regexes: each one looks at a single key, never at the
|
||||
// YAML structure as a whole.
|
||||
|
||||
const ROOT = path.join(__dirname, "..");
|
||||
const CI_PATH = path.join(ROOT, ".forgejo", "workflows", "ci.yml");
|
||||
const ci = fs.readFileSync(CI_PATH, "utf-8");
|
||||
const dockerfile = fs.readFileSync(path.join(ROOT, "Dockerfile"), "utf-8");
|
||||
|
||||
// Comment lines stripped, so an explanation that quotes a key cannot satisfy
|
||||
// (or break) a pin.
|
||||
const ciCode = ci
|
||||
.split("\n")
|
||||
.filter((line) => !/^\s*#/.test(line))
|
||||
.join("\n");
|
||||
|
||||
// Lines of a top-level-indented block (` pull_request:`), up to the next key at
|
||||
// the same or a lower indentation.
|
||||
function block(text, key) {
|
||||
const lines = text.split("\n");
|
||||
const start = lines.findIndex((l) => new RegExp(`^(\\s*)${key}:`).test(l));
|
||||
if (start === -1) return null;
|
||||
const indent = lines[start].match(/^(\s*)/)[1].length;
|
||||
const out = [lines[start]];
|
||||
for (const line of lines.slice(start + 1)) {
|
||||
if (line.trim() === "") continue;
|
||||
if (line.match(/^(\s*)/)[1].length <= indent) break;
|
||||
out.push(line);
|
||||
}
|
||||
return out.join("\n");
|
||||
}
|
||||
|
||||
describe("CI workflow", () => {
|
||||
test("runs in the Dockerfile's own base image", () => {
|
||||
const from = dockerfile.match(/^\s*FROM\s+(\S+)/m);
|
||||
const container = ciCode.match(/^\s*container:\s*["']?([^"'\s]+)["']?\s*$/m);
|
||||
|
||||
expect(from).not.toBeNull();
|
||||
expect(container).not.toBeNull();
|
||||
expect(container[1]).toBe(from[1]);
|
||||
});
|
||||
|
||||
test("runs the unit tests, the socket pass and the boot check", () => {
|
||||
expect(ciCode).toMatch(/^\s*run:\s*npm ci\s*$/m);
|
||||
expect(ciCode).toMatch(/^\s*run:\s*npm test\s*$/m);
|
||||
expect(ciCode).toMatch(/^\s*run:\s*bash test-curl\.sh\s*$/m);
|
||||
expect(ciCode).toMatch(/^\s*run:\s*sh scripts\/boot-from-copy\.sh\s*$/m);
|
||||
});
|
||||
|
||||
test("builds every pull request, whatever branch it targets", () => {
|
||||
// A `branches: [main]` here would leave the intermediate links of a stack
|
||||
// without any status.
|
||||
const pr = block(ciCode, "pull_request");
|
||||
|
||||
expect(pr).not.toBeNull();
|
||||
expect(pr).not.toMatch(/branches(-ignore)?:/);
|
||||
});
|
||||
|
||||
test("builds pushes to main", () => {
|
||||
const push = block(ciCode, "push");
|
||||
|
||||
expect(push).not.toBeNull();
|
||||
expect(push).toMatch(/branches:\s*\[\s*main\s*\]/);
|
||||
});
|
||||
|
||||
test("avoids the two forms the runner rejects before the first command", () => {
|
||||
// `defaults:` anywhere, and `working-directory` on a `uses:` step.
|
||||
expect(ciCode).not.toMatch(/^\s*defaults:/m);
|
||||
const steps = ciCode.split(/^\s*- (?=name:|uses:|run:)/m);
|
||||
for (const step of steps) {
|
||||
if (/^\s*uses:|\n\s*uses:/.test(step)) expect(step).not.toMatch(/working-directory:/);
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
describe("what the boot check assumes of the Dockerfile", () => {
|
||||
test("the image still drops to the node user, as the script does", () => {
|
||||
expect(dockerfile).toMatch(/^\s*USER\s+node\s*$/m);
|
||||
});
|
||||
|
||||
test("CMD is in exec form, the only one the script parses", () => {
|
||||
const cmd = dockerfile.match(/^\s*CMD\s+(.+)$/m);
|
||||
|
||||
expect(cmd).not.toBeNull();
|
||||
const argv = JSON.parse(cmd[1]);
|
||||
expect(Array.isArray(argv)).toBe(true);
|
||||
expect(argv.length).toBeGreaterThan(0);
|
||||
});
|
||||
});
|
||||
140
scripts/boot-from-copy.sh
Normal file
140
scripts/boot-from-copy.sh
Normal file
|
|
@ -0,0 +1,140 @@
|
|||
#!/bin/sh
|
||||
# boot-from-copy.sh — start the API from exactly the files the image would hold.
|
||||
#
|
||||
# The Dockerfile COPY lists runtime files one by one. A module missing from that
|
||||
# list passes every test (the tests run from the full checkout) and only crashes
|
||||
# the container at redeploy, with MODULE_NOT_FOUND. That is what #11 had to fix
|
||||
# by hand for metrics.js.
|
||||
#
|
||||
# A real `docker build` would need the Docker socket inside the CI job, which on
|
||||
# the shared instance-scoped runner makes every workflow of every repository
|
||||
# root on the production VPS. So this script rebuilds the image's file set
|
||||
# instead: it copies the COPY sources into an empty directory, runs the CMD
|
||||
# there, and asserts the server answers. In CI it runs in `node:22-alpine`, the
|
||||
# image's own base.
|
||||
#
|
||||
# Usage : sh scripts/boot-from-copy.sh (from the repository root)
|
||||
# Exit 0 if the server boots and answers, 1 otherwise.
|
||||
|
||||
set -eu
|
||||
|
||||
PORT="${BOOT_PORT:-3097}"
|
||||
TOKEN="boot-from-copy-token"
|
||||
DOCKERFILE="Dockerfile"
|
||||
|
||||
fail() {
|
||||
echo "FAIL: $*" >&2
|
||||
exit 1
|
||||
}
|
||||
|
||||
[ -f "$DOCKERFILE" ] || fail "no $DOCKERFILE here — run from the repository root"
|
||||
|
||||
# --- Parse COPY -------------------------------------------------------------
|
||||
# Only the plain shell form is understood. Anything else (flags, JSON array,
|
||||
# several COPY lines) fails loudly rather than being half-parsed into a
|
||||
# check that verifies nothing.
|
||||
COPY_LINES=$(grep -cE '^[[:space:]]*COPY[[:space:]]' "$DOCKERFILE" || true)
|
||||
[ "$COPY_LINES" = "1" ] || fail "expected exactly one COPY line, found $COPY_LINES"
|
||||
COPY_ARGS=$(grep -E '^[[:space:]]*COPY[[:space:]]' "$DOCKERFILE" | sed -E 's/^[[:space:]]*COPY[[:space:]]+//')
|
||||
case "$COPY_ARGS" in
|
||||
--*) fail "COPY flags are not supported by this check: $COPY_ARGS" ;;
|
||||
\[*) fail "COPY JSON form is not supported by this check: $COPY_ARGS" ;;
|
||||
esac
|
||||
# shellcheck disable=SC2086 # word splitting is the point: one word per source
|
||||
set -- $COPY_ARGS
|
||||
[ "$#" -ge 2 ] || fail "COPY needs at least one source and a destination: $COPY_ARGS"
|
||||
|
||||
# --- Parse CMD --------------------------------------------------------------
|
||||
# Taken from the Dockerfile rather than hardcoded, so the check follows the
|
||||
# entrypoint if it ever changes. Exec (JSON array) form only.
|
||||
CMD_LINE=$(grep -E '^[[:space:]]*CMD[[:space:]]' "$DOCKERFILE" | tail -1 | sed -E 's/^[[:space:]]*CMD[[:space:]]+//')
|
||||
CMD_SH=$(printf '%s' "$CMD_LINE" | node -e '
|
||||
let s = ""; process.stdin.on("data", (d) => (s += d)).on("end", () => {
|
||||
let a;
|
||||
try { a = JSON.parse(s); } catch { process.exit(1); }
|
||||
if (!Array.isArray(a) || !a.length || !a.every((x) => /^[A-Za-z0-9._\/-]+$/.test(x))) process.exit(1);
|
||||
process.stdout.write(a.join(" "));
|
||||
});
|
||||
') || fail "CMD must be an exec-form JSON array of plain words: $CMD_LINE"
|
||||
|
||||
# --- Build the image's file set ---------------------------------------------
|
||||
APP_DIR=$(mktemp -d)
|
||||
LOG="$APP_DIR.log"
|
||||
SERVER_PID=""
|
||||
trap '[ -n "$SERVER_PID" ] && kill "$SERVER_PID" 2>/dev/null; rm -rf "$APP_DIR" "$LOG"; true' EXIT
|
||||
|
||||
N=0
|
||||
while [ "$#" -gt 1 ]; do
|
||||
[ -f "$1" ] || fail "COPY source $1 does not exist in the repository"
|
||||
cp "$1" "$APP_DIR/"
|
||||
N=$((N + 1))
|
||||
shift
|
||||
done
|
||||
[ "$N" -gt 0 ] || fail "no COPY source found"
|
||||
echo "Image file set ($N): $(ls "$APP_DIR" | tr '\n' ' ')"
|
||||
|
||||
# mktemp creates the directory 0700: the `node` user could not even enter it
|
||||
# and the failure would read EACCES instead of the MODULE_NOT_FOUND we look for.
|
||||
chmod 755 "$APP_DIR"
|
||||
chmod 644 "$APP_DIR"/*
|
||||
|
||||
# --- Start ------------------------------------------------------------------
|
||||
# The image runs as `node`. In the CI container the job is root, so drop to
|
||||
# `node` like the image does; on a workstation, run as whoever we are.
|
||||
# LOGTO_HEALTH_URL points at a closed local port: no external network needed,
|
||||
# the Logto check reports "down" and /health still answers 200.
|
||||
# busybox `su` cannot preserve the environment, hence the inline variables.
|
||||
ENV_INLINE="PORT=$PORT HEALTH_TOKEN=$TOKEN LOGTO_HEALTH_URL=http://127.0.0.1:9/"
|
||||
if [ "$(id -u)" = "0" ]; then
|
||||
id node >/dev/null 2>&1 || fail "running as root but no 'node' user to drop to"
|
||||
su node -s /bin/sh -c "cd '$APP_DIR' && exec env $ENV_INLINE $CMD_SH" >"$LOG" 2>&1 &
|
||||
else
|
||||
(cd "$APP_DIR" && exec env $ENV_INLINE $CMD_SH) >"$LOG" 2>&1 &
|
||||
fi
|
||||
SERVER_PID=$!
|
||||
|
||||
UP=""
|
||||
i=0
|
||||
while [ "$i" -lt 50 ]; do
|
||||
if ! kill -0 "$SERVER_PID" 2>/dev/null; then
|
||||
echo "--- server log ---" >&2
|
||||
cat "$LOG" >&2
|
||||
fail "the server died at startup (see its log above)"
|
||||
fi
|
||||
if curl -s -o /dev/null "http://127.0.0.1:$PORT/health"; then
|
||||
UP=yes
|
||||
break
|
||||
fi
|
||||
sleep 0.2
|
||||
i=$((i + 1))
|
||||
done
|
||||
if [ -z "$UP" ]; then
|
||||
echo "--- server log ---" >&2
|
||||
cat "$LOG" >&2
|
||||
fail "the server did not answer on :$PORT within 10 s"
|
||||
fi
|
||||
|
||||
# --- Assert -----------------------------------------------------------------
|
||||
CODE=$(curl -s -o /dev/null -w '%{http_code}' "http://127.0.0.1:$PORT/health")
|
||||
[ "$CODE" = "401" ] || fail "GET /health without token: expected 401, got $CODE"
|
||||
echo "ok GET /health without token -> 401"
|
||||
|
||||
BODY=$(curl -s -w '\n%{http_code}' -H "Authorization: Bearer $TOKEN" "http://127.0.0.1:$PORT/health")
|
||||
CODE=$(printf '%s' "$BODY" | tail -1)
|
||||
[ "$CODE" = "200" ] || fail "GET /health with token: expected 200, got $CODE"
|
||||
echo "ok GET /health with token -> 200"
|
||||
|
||||
# metrics.js swallows a failed `df` parse and reports zeros. Production runs on
|
||||
# Alpine, whose busybox df is what this catches. In CI the figure is the job
|
||||
# container's overlay root, not the production disk: a failure here means the
|
||||
# parsing broke, never that a disk is full.
|
||||
printf '%s' "$BODY" | sed '$d' | node -e '
|
||||
let s = ""; process.stdin.on("data", (d) => (s += d)).on("end", () => {
|
||||
const b = JSON.parse(s);
|
||||
if (!(b.disk && b.disk.totalGB > 0)) { console.error("disk:", JSON.stringify(b.disk)); process.exit(1); }
|
||||
if (!(b.memory && b.memory.totalGB > 0)) { console.error("memory:", JSON.stringify(b.memory)); process.exit(1); }
|
||||
});
|
||||
' || fail "/health reported empty disk or memory metrics"
|
||||
echo "ok /health disk and memory metrics are non-zero"
|
||||
|
||||
echo "boot-from-copy: PASS"
|
||||
Loading…
Reference in a new issue