Update technical documentation with detailed system knowledge
Enhanced technical-notes-codex.md with comprehensive details from Claude's system knowledge document, including: Ticketing & Autotask: - Detailed two-ticket system explanation (internal vs Autotask) - Complete ticket propagation strategies (Strategy 1 & 2) - Where ticket linking is called (email-based, missed runs) - Display logic with two-source approach - Resolved vs Deleted distinction - All critical rules and anti-patterns Database Models: - Complete model listing - Foreign key relationships and critical deletion order - Key model fields documentation UI & UX: - Detailed navbar behavior - Status badge color coding - Complete ticket copy functionality with three-tier fallback - Checkbox autocomplete behavior Parser Architecture: - Parser types (Informational vs Regular) - Synology Updates parser example - Schedule learning behavior Recent Changes: - Documented 2026-02-13 fixes (missed run ticket linking, checkbox autoselect) - Documented 2026-02-12 fixes (Run Checks modal, Edge copy button) - Documented 2026-02-10 changes (screenshot support, link-based system) Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
This commit is contained in:
parent
084c91945a
commit
ecdb331c9b
@ -74,8 +74,35 @@ File: `containers/backupchecks/src/backend/app/models.py`
|
|||||||
- Logging:
|
- Logging:
|
||||||
- `AuditLog` (legacy alias `AdminLog`).
|
- `AuditLog` (legacy alias `AdminLog`).
|
||||||
- Domain:
|
- Domain:
|
||||||
- `Customer`, `Job`, `Override`, plus extensive run/ticket logic in other modules/routes.
|
- `Customer`, `Job`, `JobRun`, `Override`
|
||||||
- Core entities from system knowledge include `JobRun`, `MailMessage`, ticket/remark link tables, and feedback tables.
|
- `MailMessage`, `MailObject`
|
||||||
|
- `Ticket`, `TicketScope`, `TicketJobRun`
|
||||||
|
- `Remark`, `RemarkScope`, `RemarkJobRun`
|
||||||
|
- `FeedbackItem`, `FeedbackVote`, `FeedbackReply`, `FeedbackAttachment`
|
||||||
|
|
||||||
|
### Foreign Key Relationships & Deletion Order
|
||||||
|
Critical deletion order to avoid constraint violations:
|
||||||
|
1. Clean auxiliary tables (ticket_job_runs, remark_job_runs, scopes, overrides)
|
||||||
|
2. Unlink mails from jobs (UPDATE mail_messages SET job_id = NULL)
|
||||||
|
3. Delete mail_objects
|
||||||
|
4. Delete jobs (cascades to job_runs)
|
||||||
|
5. Delete mails
|
||||||
|
|
||||||
|
### Key Model Fields
|
||||||
|
**MailMessage model:**
|
||||||
|
- `from_address` (NOT `sender`!) - sender email
|
||||||
|
- `subject` - email subject
|
||||||
|
- `text_body` - plain text content
|
||||||
|
- `html_body` - HTML content
|
||||||
|
- `received_at` - timestamp
|
||||||
|
- `location` - inbox/processed/deleted
|
||||||
|
- `job_id` - link to Job (nullable)
|
||||||
|
|
||||||
|
**Job model:**
|
||||||
|
- `customer_id` - FK to Customer
|
||||||
|
- `job_name` - parsed from email
|
||||||
|
- `backup_software` - e.g., "Veeam", "Synology"
|
||||||
|
- `backup_type` - e.g., "Backup Job", "Active Backup"
|
||||||
|
|
||||||
## Parser Architecture
|
## Parser Architecture
|
||||||
- Folder: `containers/backupchecks/src/backend/app/parsers/`
|
- Folder: `containers/backupchecks/src/backend/app/parsers/`
|
||||||
@ -89,28 +116,143 @@ File: `containers/backupchecks/src/backend/app/models.py`
|
|||||||
- Practical rule:
|
- Practical rule:
|
||||||
- extend patterns by adding, not replacing (backward compatibility).
|
- extend patterns by adding, not replacing (backward compatibility).
|
||||||
|
|
||||||
|
### Parser Types
|
||||||
|
**Informational Parsers:**
|
||||||
|
- DSM Updates, Account Protection, Firmware Updates
|
||||||
|
- Set appropriate backup_type (e.g., "Updates", "Firmware Update")
|
||||||
|
- Do NOT participate in schedule learning
|
||||||
|
- Still visible in Run Checks for awareness
|
||||||
|
|
||||||
|
**Regular Parsers:**
|
||||||
|
- Backup jobs (Veeam, Synology Active Backup, NAKIVO, etc.)
|
||||||
|
- Participate in schedule learning (daily/weekly/monthly detection)
|
||||||
|
- Generate missed runs when expected runs don't occur
|
||||||
|
|
||||||
|
**Example: Synology Updates Parser (synology.py)**
|
||||||
|
- Handles multiple update notification types under same job:
|
||||||
|
- DSM automatic update cancelled
|
||||||
|
- Packages out-of-date
|
||||||
|
- Combined notifications (DSM + packages)
|
||||||
|
- Detection patterns:
|
||||||
|
- DSM: "Automatische DSM-update", "DSM-update op", "automatic DSM update"
|
||||||
|
- Packages: "Packages on", "out-of-date", "Package Center"
|
||||||
|
- Hostname extraction from multiple patterns
|
||||||
|
- Returns: backup_type "Updates", job_name "Synology Automatic Update"
|
||||||
|
|
||||||
## Ticketing and Autotask (Critical Rules)
|
## Ticketing and Autotask (Critical Rules)
|
||||||
- Two ticket tracks:
|
|
||||||
- internal tickets (`tickets`, `ticket_scopes`, `ticket_job_runs`).
|
### Two Ticket Types
|
||||||
- Autotask ticket fields on `job_runs` + synchronization with internal ticket records.
|
1. **Internal Tickets** (tickets table)
|
||||||
- Propagation to new runs is handled by `link_open_internal_tickets_to_run`.
|
- Created manually or via Autotask integration
|
||||||
- This propagation must be called for:
|
- Stored in `tickets` table with `ticket_code` (e.g., "T20250123.0001")
|
||||||
- email-based run creation (import flow),
|
- Linked to runs via `ticket_job_runs` many-to-many table
|
||||||
- missed-run generation in `routes_run_checks.py`.
|
- Scoped to jobs via `ticket_scopes` table
|
||||||
- Display logic:
|
- Have `resolved_at` field for resolution tracking
|
||||||
- link-based via explicit JOIN queries.
|
- **Auto-propagation**: Automatically linked to new runs via `link_open_internal_tickets_to_run`
|
||||||
- resolved tickets (`resolved_at` set) must no longer be linked to new runs.
|
|
||||||
- historical links remain visible for audit trail.
|
2. **Autotask Tickets** (job_runs columns)
|
||||||
- Anti-patterns (do not use):
|
- Created via Run Checks modal → "Create Autotask Ticket"
|
||||||
- date-based resolved logic like `resolved_at >= run_date`.
|
- Stored directly in JobRun columns: `autotask_ticket_id`, `autotask_ticket_number`, etc.
|
||||||
|
- When created, also creates matching internal ticket for legacy UI compatibility
|
||||||
|
- Have `autotask_ticket_deleted_at` field for deletion tracking
|
||||||
|
- Resolution tracked via matching internal ticket's `resolved_at` field
|
||||||
|
- **Auto-propagation**: Linked to new runs via two-strategy approach
|
||||||
|
|
||||||
|
### Ticket Propagation to New Runs
|
||||||
|
When a new JobRun is created (via email import OR missed run generation), `link_open_internal_tickets_to_run` ensures:
|
||||||
|
|
||||||
|
**Strategy 1: Internal ticket linking**
|
||||||
|
- Query finds tickets where: `COALESCE(ts.resolved_at, t.resolved_at) IS NULL`
|
||||||
|
- Creates `ticket_job_runs` links automatically
|
||||||
|
- Tickets remain visible until explicitly resolved
|
||||||
|
- **NO date-based logic** - resolved = immediately hidden from new runs
|
||||||
|
|
||||||
|
**Strategy 2: Autotask ticket propagation (independent)**
|
||||||
|
1. Check if internal ticket code exists → find matching Autotask run → copy ticket info
|
||||||
|
2. If no match, directly search for most recent Autotask ticket on job where:
|
||||||
|
- `autotask_ticket_deleted_at IS NULL` (not deleted in PSA)
|
||||||
|
- Internal ticket `resolved_at IS NULL` (not resolved in PSA)
|
||||||
|
3. Copy `autotask_ticket_id`, `autotask_ticket_number`, `created_at`, `created_by_user_id` to new run
|
||||||
|
|
||||||
|
### Where Ticket Linking is Called
|
||||||
|
`link_open_internal_tickets_to_run` is invoked in three locations:
|
||||||
|
1. **Email-based runs**: `routes_inbox.py` and `mail_importer.py` - after creating JobRun from parsed email
|
||||||
|
2. **Missed runs**: `routes_run_checks.py` in `_ensure_missed_runs_for_job` - after creating missed JobRun records
|
||||||
|
- Weekly schedule: After creating weekly missed run (with flush to get run.id)
|
||||||
|
- Monthly schedule: After creating monthly missed run (with flush to get run.id)
|
||||||
|
- **Critical**: Without this call, missed runs don't get ticket propagation!
|
||||||
|
|
||||||
|
### Display Logic - Link-Based System
|
||||||
|
All pages use **explicit link-based queries** (no date-based logic):
|
||||||
|
|
||||||
|
**Job Details Page:**
|
||||||
|
- **Two sources** for ticket display:
|
||||||
|
1. Direct links (`ticket_job_runs WHERE job_run_id = X`) → always show (audit trail)
|
||||||
|
2. Active window (`ticket_scopes WHERE job_id = Y AND resolved_at IS NULL`) → only unresolved
|
||||||
|
- Result: Old runs keep their ticket references, new runs don't get resolved tickets
|
||||||
|
|
||||||
|
**Run Checks Main Page (Indicators 🎫):**
|
||||||
|
- Query: `ticket_scopes JOIN tickets WHERE job_id = X AND resolved_at IS NULL`
|
||||||
|
- Only shows indicator if unresolved tickets exist for the job
|
||||||
|
|
||||||
|
**Run Checks Popup Modal:**
|
||||||
|
- API: `/api/job-runs/<run_id>/alerts`
|
||||||
|
- **Two-source ticket display**:
|
||||||
|
1. Direct links: `tickets JOIN ticket_job_runs WHERE job_run_id = X`
|
||||||
|
2. Job-level scope: `tickets JOIN ticket_scopes WHERE job_id = Y AND resolved_at IS NULL AND active_from_date <= run_date`
|
||||||
|
- Prevents duplicates by tracking seen ticket IDs
|
||||||
|
- Shows newly created tickets immediately (via scope) without waiting for resolve action
|
||||||
|
- Same for remarks: `remarks JOIN remark_job_runs WHERE job_run_id = X`
|
||||||
|
|
||||||
|
### Resolved vs Deleted
|
||||||
|
- **Resolved**: Ticket completed in Autotask (tracked in internal `tickets.resolved_at`)
|
||||||
|
- Stops propagating to new runs
|
||||||
|
- Ticket still exists in PSA
|
||||||
|
- Synced via PSA polling
|
||||||
|
- **Deleted**: Ticket removed from Autotask (tracked in `job_runs.autotask_ticket_deleted_at`)
|
||||||
|
- Also stops propagating
|
||||||
|
- Ticket no longer exists in PSA
|
||||||
|
- Rare operation
|
||||||
|
|
||||||
|
### Critical Rules
|
||||||
|
- ❌ **NEVER** use date-based resolved logic: `resolved_at >= run_date` OR `active_from_date <= run_date`
|
||||||
|
- ✅ Only show tickets that are ACTUALLY LINKED via `ticket_job_runs` table
|
||||||
|
- ✅ Resolved tickets stop linking immediately when resolved
|
||||||
|
- ✅ Old links preserved for audit trail (visible on old runs)
|
||||||
|
- ✅ All queries must use explicit JOIN to link tables
|
||||||
|
- ✅ Consistency: All pages use same "resolved = NULL" logic
|
||||||
|
- ✅ **CRITICAL**: Preserve description field during Autotask updates - must include "description" in optional_fields list
|
||||||
|
|
||||||
## UI and UX Notes
|
## UI and UX Notes
|
||||||
- Navbar is fixed-top with dynamic main container padding correction.
|
|
||||||
- Status badges use semantic color coding (success/warning/error/override/reviewed).
|
### Navbar
|
||||||
- Ticket copy button uses a three-level fallback:
|
- Fixed-top positioning
|
||||||
- Clipboard API,
|
- Collapses on mobile (hamburger menu)
|
||||||
- `execCommand('copy')`,
|
- Dynamic padding adjustment via JavaScript (measures navbar height, adjusts main content padding-top)
|
||||||
- `prompt()` fallback.
|
- Role-based menu items (Admin sees more than Operator/Viewer)
|
||||||
|
|
||||||
|
### Status Badges
|
||||||
|
- Success: Green
|
||||||
|
- Warning: Yellow/Orange
|
||||||
|
- Failed/Error: Red
|
||||||
|
- Override applied: Blue badge
|
||||||
|
- Reviewed: Checkmark indicator
|
||||||
|
|
||||||
|
### Ticket Copy Functionality
|
||||||
|
- Copy button (⧉) available on both Run Checks and Job Details pages
|
||||||
|
- Allows quick copying of ticket numbers to clipboard
|
||||||
|
- Cross-browser compatible with three-tier fallback mechanism:
|
||||||
|
1. **Modern Clipboard API**: `navigator.clipboard.writeText()` - works in modern browsers with HTTPS
|
||||||
|
2. **Legacy execCommand**: `document.execCommand('copy')` - fallback for older browsers and Edge
|
||||||
|
3. **Prompt fallback**: `window.prompt()` - last resort if clipboard access fails
|
||||||
|
- Visual feedback: button changes to ✓ checkmark for 800ms after successful copy
|
||||||
|
- Implementation uses hidden textarea for execCommand method to ensure compatibility
|
||||||
|
- No user interaction required in modern browsers (direct copy)
|
||||||
|
|
||||||
|
### Checkbox Behavior
|
||||||
|
- All checkboxes on Inbox and Run Checks pages use `autocomplete="off"`
|
||||||
|
- Prevents browser from auto-selecting checkboxes after page reload
|
||||||
|
- Fixes issue where deleting items would cause same number of new items to be selected
|
||||||
|
|
||||||
## Feedback Module with Screenshots
|
## Feedback Module with Screenshots
|
||||||
- Models: `FeedbackItem`, `FeedbackVote`, `FeedbackReply`, `FeedbackAttachment`.
|
- Models: `FeedbackItem`, `FeedbackVote`, `FeedbackReply`, `FeedbackAttachment`.
|
||||||
@ -163,3 +305,17 @@ File: `build-and-push.sh`
|
|||||||
- Run Checks routes: `containers/backupchecks/src/backend/app/main/routes_run_checks.py`
|
- Run Checks routes: `containers/backupchecks/src/backend/app/main/routes_run_checks.py`
|
||||||
- Compose stack: `deploy/backupchecks-stack.yml`
|
- Compose stack: `deploy/backupchecks-stack.yml`
|
||||||
- Build script: `build-and-push.sh`
|
- Build script: `build-and-push.sh`
|
||||||
|
|
||||||
|
## Recent Changes
|
||||||
|
|
||||||
|
### 2026-02-13
|
||||||
|
- **Fixed missed runs ticket propagation**: Added `link_open_internal_tickets_to_run` calls in `_ensure_missed_runs_for_job` (routes_run_checks.py) after creating both weekly and monthly missed JobRun records. Previously only email-based runs got ticket linking, causing missed runs to not show internal tickets or Autotask tickets. Required `db.session.flush()` before linking to ensure run.id is available.
|
||||||
|
- **Fixed checkbox auto-selection**: Added `autocomplete="off"` to all checkboxes on Inbox and Run Checks pages. Prevents browser from automatically re-selecting checkboxes after page reload following delete actions.
|
||||||
|
|
||||||
|
### 2026-02-12
|
||||||
|
- **Fixed Run Checks modal ticket display**: Implemented two-source display logic (ticket_job_runs + ticket_scopes). Previously only showed tickets after they were resolved (when ticket_job_runs entry was created). Now shows tickets immediately upon creation via scope query.
|
||||||
|
- **Fixed copy button in Edge**: Moved clipboard functions inside IIFE scope for proper closure access (Edge is stricter than Firefox about scope resolution).
|
||||||
|
|
||||||
|
### 2026-02-10
|
||||||
|
- **Added screenshot support to Feedback system**: Multiple file upload, inline display, two-stage delete (soft delete for audit trail, permanent delete for cleanup).
|
||||||
|
- **Completed transition to link-based ticket system**: All pages now use JOIN queries, no date-based logic. Added cross-browser copy ticket functionality with three-tier fallback mechanism to both Run Checks and Job Details pages.
|
||||||
|
|||||||
Loading…
Reference in New Issue
Block a user