schema: migration v17 — the full import format on import_sources #334
No reviewers
Labels
No labels
autopilot:pending-human
source:analyste
source:defenseur
source:human
source:medic
status:approved
status:blocked
status:in-progress
status:needs-clarification
status:needs-fix
status:ready
status:review
status:triage
type:bug
type:feature
type:infra
type:refactor
type:schema
type:security
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference: maximus/Simpl-Resultat#334
Loading…
Reference in a new issue
No description provided.
Delete branch "issue-323-migration-v17"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Link 2 of a 10-link linear stack (
spec-plan-import-csv-format.md). Based onissue-326-corpus-fixtures-csv(link 1, PR #333) — not onmain.Resolves #323
What this does
import_sourcescarried only the mechanical CSV settings (delimiter, encoding, date format, column mapping). The two fields that decide how an amount is read —amount_modeandsign_convention— existed only onimport_config_templates. That asymmetry is the root bug of this chantier: restoring a saved source re-inferred the mode from the mapping and hardcodedsignConvention: "negative_expense"(useImportWizard.ts:321-323), so a source configured with positive expenses silently flipped back on its second import.Migration v17 puts the full format on the source. After it, both tables carry the same eight format fields and the asymmetry is gone structurally. No observable behaviour changes — the backfill reproduces exactly what the code did on the fly.
Migration v17 — strictly additive
v1tov16are untouched; the diff is pure insertion (523 lines added, 0 removed), which is what "checksums intact" reduces to here — no checksum harness exists in this crate (per the/review-specrevision).amount_modeTEXT NOT NULL DEFAULT 'single'+CHECKabsolute_indicatorfrom the start so the third amount mode ships without another migration, while the DB still refuses a corrupted value todaysign_conventionTEXT NOT NULL DEFAULT 'negative_expense'+CHECKheader_signatureTEXTtemplate_idINTEGER REFERENCES import_config_templates(id) ON DELETE SET NULLThe
CHECKs follow the revision block, which overrides the original body's "pas de contrainteCHECK". Same pattern as v15 onbalance_accounts.kind.Backfill —
UPDATE import_sources SET amount_mode = 'debit_credit' WHERE column_mapping LIKE '%debitAmount%', reproducinguseImportWizard.ts:321exactly.LIKErather thanjson_extractso the migration depends on no JSON1 extension in the bundled SQLite.sign_conventionis deliberately not backfilled: itsDEFAULTrestores precisely the value the code hardcoded at:323, the only past convention that can be inferred. Guessing anything else would silently rewrite meaning.Consolidated mirror
The four columns are mirrored into
consolidated_schema.sql, placed beforecreated_at/updated_atwith a comment naming the migration — the house convention (cf.balance_accountsv12/v15).Per the revision block, the rationale is not that new profiles need them there: that script runs after every migration and only uses
CREATE TABLE IF NOT EXISTS, so those columns are inert on the production path and a new profile receives them from v17. The mirror exists so the file stays the tested reference definition — the parity test is what gives it teeth.Tests — 5 new (111 Rust total, was 106)
migration_v17_applies_on_a_populated_v16_db— a configured source, a template and animported_fileschild row; every pre-existing field survives, new columns land on their defaults, the FK holds.migration_v17_backfill_reproduces_the_wizard_rule— 5 mapping shapes (both columns, debit only, single amount, credit only, minimal).migration_v17_checks_reject_unknown_enum_values—absolute_indicatoraccepted, garbage refused.migration_v17_template_id_is_a_nullable_provenance_tag— editing a template changes no linked source (the spec's acceptance criterion); deleting one nulls the tag and leaves the format intact; a dangling tag is refused.consolidated_schema_matches_v17_chain_on_import_sources_at_parity— the parity test the revision asked for, on the model ofconsolidated_schema_has_holdings_tables_and_kind_at_parity: 15 columns compared name/type/NOT NULL/DEFAULT, the FK compared with itsON DELETEaction, and theCHECKs proven behaviourally on both definitions (pragma does not expose them).The parity test was mutation-checked, not trusted on a first-try green: drifting the consolidated
sign_conventionDEFAULT made the column comparison fail with a readable diff, and narrowing theamount_modeCHECK made the behavioural half fail. A guard that cannot fail is not a guard.Verification
cargo checkclean, no warningscargo test111 passed (106 baseline + 5)npm test924 passed — unchanged. A schema migration changes no parsing behaviour, so none of link 1's 8KNOWN DEFECTblocks flipped, as expectednpm run buildgreenReviewer notes
V17_SQLis a hand-kept copy of the inline migration string (the V13/V15/V16 shape), not aninclude_str!shared constant (the V14 shape, which would have zero drift risk). Chosen because the issue asks for "le patternV13_SQLaV16_SQL", v15 is the closest analogue and is inline, and V14 uses a file only because it is a whole multi-table schema. The parity test is the drift mitigation.db_pre_v17()applies v1 + v3 + v5 only, not the full v1→v16 chain. Those are exactly the three migrations that shapeimport_sourcesandimport_config_templates; v2, v4 and v6-v16 never touch them. The claim is not taken on faith — the parity test would fail if any other migration shaped these tables, sinceconsolidated_schema.sqlis the full post-v16 truth. Replaying all 16 would mean hand-copying V1_SQL..V9_SQL constants no test needs, multiplying the very drift risk the pattern already carries.docs/architecture.md, noCLAUDE.mdin this PR. This issue changes no observable behaviour, and the spec plan centralizes both in Issue 10 (last link) on purpose, to avoid rebase drift across a 10-link stack (spec-plan line 224).CLAUDE.mdstill says "16 migrations inline (v1→v16)" and will need the bump there.Generated autonomously by /autopilot run of 2026-08-13
/pr-review— APPROVEMigration additive propre, dont chaque affirmation du corps a ete verifiee plutot que crue. La v17 s'applique reellement sur une base v1+v3+v5 avec
PRAGMA foreign_keys = ON(la condition runtime de sqlx), la parite consolidated/chaine tient a 15 colonnes et une FK identiques, et le garde de parite est genuinement sensible a la mutation — je l'ai re-teste independamment.Verifications faites (pas de blocage)
Diff pur ajout — confirme.
git diff --numstatsur la base reelle (issue-326-corpus-fixtures-csv) donne16/0+507/0= 523 insertions, 0 suppression. Aucune chaine SQL v1→v16 n'apparait au diff. J'ai aussi verifie independamment par grep de toutes les migrations delib.rsque seules v1, v3 et v5 faconnentimport_sources/import_config_templates(la ligne v6 qui les mentionne creeimported_files_new, elle n'altere pasimport_sources) — la premisse dedb_pre_v17()tient.SQL rejoue hors du harnais. J'ai reconstruit la chaine (schema v1 + v3 + v5) et applique
V17_SQLtel quel,foreign_keysactives :ALTERpassent —ADD COLUMN ... NOT NULL DEFAULT ... CHECKa deja le precedent v15 en production (balance_accounts.kind), etADD COLUMN ... REFERENCES ... ON DELETE SET NULLsatisfait la contrainte SQLite (« defaut NULL obligatoire quand les FK sont actives ») puisque la colonne n'a pas deDEFAULT;debitAmount→debit_credit, sans →single,sign_convention=negative_expensepartout ;CHECKmordent :absolute_indicatoraccepte, valeur inconnue refusee ;ON DELETE SET NULLse declenche bien (tag a NULL,sign_conventionintact) et une etiquette pendante est refusee.Parite reproduite independamment : 15 colonnes identiques (nom/type/NOT NULL/DEFAULT), FK identique
(import_config_templates, template_id, id, SET NULL).Le garde de parite a ete re-mute de mon cote, pas seulement pris au mot :
DEFAULTdesign_conventiondans le consolide → la comparaison de colonnes echoue ;absolute_indicatorduCHECKconsolide → invisible au pragma, mais rattrape par la moitie comportementale.C'est exactement la separation que le corps annonce, et elle est reelle. Un garde qui ne peut pas echouer n'est pas un garde — celui-ci echoue.
La regle du backfill correspond au runtime.
useImportWizard.ts:321=mapping.debitAmount !== undefined ? "debit_credit" : "single",:323=signConvention: "negative_expense"en dur. Ce bloc est la seule voie qui relit une source stockee, donc leDEFAULTrestaure bien la valeur que le code appliquait : le point 4 des contraintes tient au challenge.Le rationale du miroir consolide est le bon, pas l'inverse. Le commentaire dit « runs AFTER every migration / only uses CREATE TABLE IF NOT EXISTS / inert on the production path » — conforme a
profile_commands.rs:116-123(« tauri-plugin-sql applies every declared migration onDatabase.load; the consolidated script then... »).Surface d'effet de bord fermee : l'export/import SREF ne fait pas de round-trip sur
import_sources(dataExportService.ts:265-318purge puis insere une source de suivi synthetique), donc les 4 colonnes n'ouvrent aucune perte de donnees a la restauration.Reste : zero secret, zero injection (chaine de migration statique ; les
format!des tests interpolent des litteraux de test), zero#[ignore]/TODO/dead code, commentaires en anglais.Suggestions non bloquantes
LIKE '%debitAmount%'est non ancre et insensible a la casse (LIKE SQLite, ASCII, par defaut). Verifie : une cle ecriteDEBITAMOUNTpart endebit_creditalors que la regle JS repondraitsingle. Aucune source reelle ne peut declencher ca — les cles sont ecrites par le code et les valeurs deColumnMappingsont des entiers (src/shared/types/index.ts:205-211) — donc ce n'est pas un blocage. Mais une migration est a coup unique et irreparable a posteriori :LIKE '%"debitAmount":%'fermerait la porte pour 8 caracteres.lib.rs:3488(et les 5 cas demigration_v17_backfill_reproduces_the_wizard_rule) : les fixtures utilisent des valeurs chaines ({"date":"Date","debitAmount":"Débit"}) alors queColumnMappingporte des index numeriques. La forme testee n'existe pas en production. Le risque n'est pas le test d'aujourd'hui mais celui de demain : quelqu'un qui durcirait leLIKEen se fiant a ces fixtures ecrirait un motif qui ne matche plus les vraies donnees. Des index entiers coutent la meme chose.db_pre_v17()saute la v6, doncimported_filesest dans sa forme v1 (UNIQUE(source_id, file_hash),ON DELETE CASCADE) et non v16. Sans consequence — la v17 n'y touche pas — mais le docstring « this IS their v16 shape » n'est exact que des deux tables que la v17 faconne ; une demi-phrase leve l'ambiguite."Deux colonnes","Débit seul","Modèle A") dans un module de test integralement anglais, et interpoles dans les messages d'assertion → sortie d'echec bilingue. Cosmetique.CLAUDE.md:122. Il faut aussidocs/architecture.md:144-161, qui porte un tableau par migration s'arretant a v16 et qui reclamera sa ligne v17. Au passage,CLAUDE.md:73dit encore « 7 migrations inline » (perime bien avant cette PR) — a balayer en meme temps.Review adversariale — verdict base sur rejeu du SQL et mutation independante du garde de parite, pas sur le corps de la PR.
Mergée dans
mainen fast-forward avec le reste de la pile (tip37b832e).Forgejo ne détecte pas un merge local comme merged — la PR est donc fermée à la main, et l'issue liée s'est fermée automatiquement via son
Resolves #N.Tip cumulé validé avant push : 1181 vitest, 111 tests Rust, build tsc + vite. La CI ne tourne pas sur push
main, cette validation locale était donc le seul filet.Pull request closed