diff --git a/docs/changelog-claude.md b/docs/changelog-claude.md index 37ce791..656b7f4 100644 --- a/docs/changelog-claude.md +++ b/docs/changelog-claude.md @@ -7,6 +7,9 @@ This file documents all changes made to this project via Claude Code. ### Added - Added internal technical reference document `docs/technical-notes-codex.md` with repository structure, application architecture, processing flow, parser system rules, ticketing/Autotask constraints, feedback attachment notes, deployment/build workflow, and operational attention points +### Changed +- Changed `docs/technical-notes-codex.md` language from Dutch to English to align with project language rules for documentation + ### Fixed - Fixed Autotask tickets and internal tickets not being linked to missed runs by calling `link_open_internal_tickets_to_run` after creating missed JobRun records in `_ensure_missed_runs_for_job` (both weekly and monthly schedules), ensuring missed runs now receive the same ticket propagation as email-based runs - Fixed checkboxes being automatically re-selected after delete actions on Inbox and Run Checks pages by adding `autocomplete="off"` attribute to all checkboxes, preventing browser from restoring previous checkbox states after page reload diff --git a/docs/technical-notes-codex.md b/docs/technical-notes-codex.md index a7dbd2a..172222c 100644 --- a/docs/technical-notes-codex.md +++ b/docs/technical-notes-codex.md @@ -1,57 +1,57 @@ -# Technische Notities (intern) +# Technical Notes (Internal) -Laatste update: 2026-02-13 +Last updated: 2026-02-13 -## Doel -Interne technische momentopname van de repository `backupchecks` voor snelle onboarding, troubleshooting en wijzigingsimpact. +## Purpose +Internal technical snapshot of the `backupchecks` repository for faster onboarding, troubleshooting, and change impact analysis. -## Repository-overzicht -- Applicatie: Flask webapp met SQLAlchemy + Flask-Migrate. -- Runtime: containerized (Docker), deploy via Docker Compose stack. -- Primaire map met broncode: `containers/backupchecks/src`. -- Project bevat daarnaast veel functionele/documentatiebestanden in `docs/` en meerdere TODO-roadmaps op rootniveau. +## Repository Overview +- Application: Flask web app with SQLAlchemy and Flask-Migrate. +- Runtime: Containerized (Docker), deployed via Docker Compose stack. +- Primary source code location: `containers/backupchecks/src`. +- The project also contains extensive functional documentation in `docs/` and multiple roadmap TODO files at repository root. -## Belangrijkste structuur -- `containers/backupchecks/Dockerfile`: Python 3.12-slim image, `gunicorn` start met `backend.app:create_app()`. -- `containers/backupchecks/requirements.txt`: Flask stack + Postgres driver + reporting libs (`reportlab`, `Markdown`). -- `containers/backupchecks/src/backend/app`: backend domeinlogica, routes, parsers, modellen, migraties. -- `containers/backupchecks/src/templates`: Jinja templates voor auth/main/documentation pagina's. +## Main Structure +- `containers/backupchecks/Dockerfile`: Python 3.12-slim image, starts `gunicorn` with `backend.app:create_app()`. +- `containers/backupchecks/requirements.txt`: Flask stack + PostgreSQL driver + reporting libraries (`reportlab`, `Markdown`). +- `containers/backupchecks/src/backend/app`: backend domain logic, routes, parsers, models, migrations. +- `containers/backupchecks/src/templates`: Jinja templates for auth/main/documentation pages. - `containers/backupchecks/src/static`: CSS, images, favicon. -- `deploy/backupchecks-stack.yml`: compose stack met `backupchecks`, `postgres`, `adminer`. -- `build-and-push.sh`: release/test buildscript met versiebump, tags en docker push. -- `docs/`: functioneel ontwerp, changelog, migraties, API-notities. +- `deploy/backupchecks-stack.yml`: compose stack with `backupchecks`, `postgres`, `adminer`. +- `build-and-push.sh`: release/test build script with version bumping, tags, and image push. +- `docs/`: functional design, changelogs, migration notes, API notes. -## Applicatie-architectuur (huidige observatie) -- Factory patroon: `create_app()` in `containers/backupchecks/src/backend/app/__init__.py`. +## Application Architecture (Current Observation) +- Factory pattern: `create_app()` in `containers/backupchecks/src/backend/app/__init__.py`. - Blueprints: - - `auth_bp` voor authenticatie. - - `main_bp` voor kernfunctionaliteit. - - `doc_bp` voor interne documentatiepagina's. -- DB-init bij startup: + - `auth_bp` for authentication. + - `main_bp` for core functionality. + - `doc_bp` for internal documentation pages. +- Database initialization at startup: - `db.create_all()` - `run_migrations()` -- Achtergrondtaak: - - `start_auto_importer(app)` start automatische mailimport thread. +- Background task: + - `start_auto_importer(app)` starts the automatic mail importer thread. - Health endpoint: - - `GET /health` geeft `{ "status": "ok" }`. + - `GET /health` returns `{ "status": "ok" }`. -## Functionele verwerkingsflow +## Functional Processing Flow - Import: - - Mail wordt opgehaald via Microsoft Graph API. + - Email is fetched via Microsoft Graph API. - Parse: - - Parser-selectie via registry + parserimplementaties per leverancier. + - Parser selection through registry + software-specific parser implementations. - Approve: - - Nieuwe jobs landen eerst in Inbox voor eerste klantkoppeling. + - New jobs first appear in Inbox for initial customer assignment. - Auto-process: - - Vervolgmails voor bekende jobs maken automatisch `JobRun` records. + - Subsequent emails for known jobs automatically create `JobRun` records. - Monitor: - - Runs verschijnen in Daily Jobs en Run Checks. + - Runs appear in Daily Jobs and Run Checks. - Review: - - Handmatige review verwijdert items uit ongereviewde werklijst. + - Manual review removes items from the unreviewed operational queue. -## Configuratie en runtime -- Config wordt uit environment opgebouwd in `containers/backupchecks/src/backend/app/config.py`. -- Belangrijke variabelen: +## Configuration and Runtime +- Config is built from environment variables in `containers/backupchecks/src/backend/app/config.py`. +- Important variables: - `APP_SECRET_KEY` - `APP_ENV` - `APP_PORT` @@ -60,106 +60,106 @@ Interne technische momentopname van de repository `backupchecks` voor snelle onb - `POSTGRES_PASSWORD` - `DB_HOST` - `DB_PORT` -- DB URI patroon: +- Database URI pattern: - `postgresql+psycopg2://:@:/` - Default timezone in config: `Europe/Amsterdam`. -## Datamodel (globaal) -Bestand: `containers/backupchecks/src/backend/app/models.py` -- Auth/gebruikers: - - `User` met role(s), actieve rol in sessie. -- Systeeminstellingen: - - `SystemSettings` met Graph/mailinstellingen, importinstellingen, UI-timezone, dashboard-policy, sandbox vlag. - - Autotask configuratie en cachevelden zijn aanwezig. +## Data Model (High-level) +File: `containers/backupchecks/src/backend/app/models.py` +- Auth/users: + - `User` with role(s), active role in session. +- System settings: + - `SystemSettings` with Graph/mail settings, import settings, UI timezone, dashboard policy, sandbox flag. + - Autotask configuration and cache fields are present. - Logging: - `AuditLog` (legacy alias `AdminLog`). -- Domein: - - `Customer`, `Job`, `Override` en veel run/ticket-gerelateerde logica in andere modules/routes. - - Kernentiteiten uit systeemkennis: `JobRun`, `MailMessage`, ticket/remark-koppeltabellen, feedback-tabellen. +- Domain: + - `Customer`, `Job`, `Override`, plus extensive run/ticket logic in other modules/routes. + - Core entities from system knowledge include `JobRun`, `MailMessage`, ticket/remark link tables, and feedback tables. -## Parserarchitectuur -- Map: `containers/backupchecks/src/backend/app/parsers/` -- Twee lagen: +## Parser Architecture +- Folder: `containers/backupchecks/src/backend/app/parsers/` +- Two layers: - `registry.py`: - - matching/documentatie/zichtbaarheid op `/parsers`. - - voorbeelden moeten generiek blijven (geen klantnamen). - - parserbestanden (`veeam.py`, `synology.py`, etc.): - - echte detectie en parsing. - - leveren gestructureerde output: software, type, jobnaam, status, objecten. -- Praktijkregel: - - patronen uitbreiden door toe te voegen, niet te vervangen (backward compatibility). + - matching/documentation/visibility on `/parsers`. + - examples must stay generic (no customer names). + - parser files (`veeam.py`, `synology.py`, etc.): + - actual detection and parsing logic. + - return structured output: software, type, job name, status, objects. +- Practical rule: + - extend patterns by adding, not replacing (backward compatibility). -## Ticketing en Autotask (kritische regels) -- Twee ticketsporen: - - interne tickets (`tickets`, `ticket_scopes`, `ticket_job_runs`). - - Autotask-ticketgegevens op `job_runs` + synchronisatie met intern ticket. -- Propagatie naar nieuwe runs gebeurt via `link_open_internal_tickets_to_run`. -- Deze propagatie moet worden aangeroepen bij: - - email-gebaseerde runcreatie (importflow), - - gemiste-runs generatie in `routes_run_checks.py`. -- Weergavelogica: - - link-gebaseerd via expliciete JOINs. - - resolved tickets (`resolved_at` gezet) mogen niet meer aan nieuwe runs gekoppeld worden. - - historische links blijven zichtbaar voor audit trail. -- Anti-patterns (niet gebruiken): - - resolved-logica op basis van datumvergelijkingen zoals `resolved_at >= run_date`. +## Ticketing and Autotask (Critical Rules) +- Two ticket tracks: + - internal tickets (`tickets`, `ticket_scopes`, `ticket_job_runs`). + - Autotask ticket fields on `job_runs` + synchronization with internal ticket records. +- Propagation to new runs is handled by `link_open_internal_tickets_to_run`. +- This propagation must be called for: + - email-based run creation (import flow), + - missed-run generation in `routes_run_checks.py`. +- Display logic: + - link-based via explicit JOIN queries. + - resolved tickets (`resolved_at` set) must no longer be linked to new runs. + - historical links remain visible for audit trail. +- Anti-patterns (do not use): + - date-based resolved logic like `resolved_at >= run_date`. -## UI en UX-notities -- Navbar is fixed-top met dynamische padding-correctie in de main container. -- Status-badges gebruiken semantische kleurcodes (success/warning/error/override/reviewed). -- Ticket-copy knop heeft drievoudige fallback: +## UI and UX Notes +- Navbar is fixed-top with dynamic main container padding correction. +- Status badges use semantic color coding (success/warning/error/override/reviewed). +- Ticket copy button uses a three-level fallback: - Clipboard API, - `execCommand('copy')`, - `prompt()` fallback. -## Feedbackmodule met screenshots -- Modellen: `FeedbackItem`, `FeedbackVote`, `FeedbackReply`, `FeedbackAttachment`. -- Bijlagen: - - meerdere uploads, typevalidatie, limiet per bestand, opslag in database (BYTEA). -- Delete-strategie: - - soft delete als standaard, - - permanente delete alleen admin en alleen na soft delete. +## Feedback Module with Screenshots +- Models: `FeedbackItem`, `FeedbackVote`, `FeedbackReply`, `FeedbackAttachment`. +- Attachments: + - multiple uploads, type validation, per-file size limits, storage in database (BYTEA). +- Delete strategy: + - soft delete by default, + - permanent delete only for admins and only after soft delete. -## Deployment en operations -- Stackbestand exposeert: - - app op `8080` - - adminer op `8081` -- Postgres persistent volume: +## Deployment and Operations +- Stack exposes: + - app on `8080` + - adminer on `8081` +- PostgreSQL persistent volume: - `/docker/appdata/backupchecks/backupchecks-postgres:/var/lib/postgresql/data` -- In `deploy/backupchecks-stack.yml` staan onderaan ook voorbeeld `.env` variabelen opgenomen. +- `deploy/backupchecks-stack.yml` also contains example `.env` variables at the bottom. -## Build/release-flow -Bestand: `build-and-push.sh` -- Bump opties: +## Build/Release Flow +File: `build-and-push.sh` +- Bump options: - `1` patch, `2` minor, `3` major, `t` test. -- Release-build: +- Release build: - update `version.txt` - commit + tag + push - - docker push van `:`, `:dev`, `:latest` -- Test-build: - - alleen `:dev` - - geen commit/tag. -- Services worden ontdekt onder `containers/*` met Dockerfile-per-service. + - docker push of `:`, `:dev`, `:latest` +- Test build: + - only `:dev` + - no commit/tag. +- Services are discovered under `containers/*` with Dockerfile-per-service. -## Technische observaties / aandachtspunten -- `README.md` is momenteel leeg; snelle instapinformatie ontbreekt. -- `LICENSE` is momenteel leeg. -- `docs/architecture.md` is momenteel leeg. -- `deploy/backupchecks-stack.yml` bevat hardcoded voorbeeldwaarden (`Changeme`), risico op foutief gebruik zonder secrets-management. -- App voert DB-init + migraties uit tijdens startup; bij grotere schemawijzigingen kan dit opstarttijd/robustness beïnvloeden. -- Er is veel functionaliteit rondom parsering en ticketing; regressierisico bij routewijzigingen is hoog zonder gerichte tests. -- Voor Autotask update-calls moet `description` expliciet behouden blijven bij updates (voorkomt ongewenste NULL-overschrijving). -- Security-hygiëne blijft belangrijk: - - geen klantnamen in parservoorbeelden/broncode, - - geen hardcoded credentials. +## Technical Observations / Attention Points +- `README.md` is currently empty; quick-start entry context is missing. +- `LICENSE` is currently empty. +- `docs/architecture.md` is currently empty. +- `deploy/backupchecks-stack.yml` contains hardcoded example values (`Changeme`), with risk if used without proper secrets management. +- The app performs DB initialization + migrations at startup; for larger schema changes this can impact startup time/robustness. +- There is significant parser and ticketing complexity; route changes carry regression risk without targeted testing. +- For Autotask update calls, the `description` field must be explicitly preserved to prevent unintended NULL overwrite. +- Security hygiene remains important: + - no customer names in parser examples/source, + - no hardcoded credentials. -## Snelle referenties +## Quick References - App entrypoint: `containers/backupchecks/src/backend/app/main.py` - App factory: `containers/backupchecks/src/backend/app/__init__.py` - Config: `containers/backupchecks/src/backend/app/config.py` -- Modellen: `containers/backupchecks/src/backend/app/models.py` +- Models: `containers/backupchecks/src/backend/app/models.py` - Parsers: `containers/backupchecks/src/backend/app/parsers/registry.py` -- Ticketing utils: `containers/backupchecks/src/backend/app/ticketing_utils.py` -- Run checks routes: `containers/backupchecks/src/backend/app/main/routes_run_checks.py` +- Ticketing utilities: `containers/backupchecks/src/backend/app/ticketing_utils.py` +- Run Checks routes: `containers/backupchecks/src/backend/app/main/routes_run_checks.py` - Compose stack: `deploy/backupchecks-stack.yml` -- Buildscript: `build-and-push.sh` +- Build script: `build-and-push.sh`