Commit Graph

575 Commits

Author SHA1 Message Date
caff435f96 Fix Autotask propagation to also check resolved status
The previous fix only checked if tickets were deleted, but Autotask
tickets can also be resolved (which is tracked via the internal Ticket
table, not the JobRun table).

Updated Strategy 2 to:
1. Find most recent non-deleted Autotask ticket
2. Check if its internal ticket is resolved
3. Only propagate if ticket is not deleted AND not resolved

This ensures tickets stop propagating when they are resolved in Autotask
(synced via PSA polling), matching the expected behavior.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-10 09:40:27 +01:00
f3b1b56b6a Fix Autotask ticket propagation to new runs
When a new run is created, Autotask tickets were not being propagated
if the associated internal ticket was resolved. This caused users to
have to manually re-link tickets on each new run.

The previous implementation relied on finding an open internal ticket
first, then using its ticket code to find a matching Autotask-linked run.
If the internal ticket was resolved, the Autotask propagation would fail.

This commit implements a two-strategy approach:
1. Strategy 1: Use internal ticket code (existing logic, improved error handling)
2. Strategy 2: Direct Autotask propagation - find most recent non-deleted
   Autotask ticket for the job, independent of internal ticket status

Now Autotask tickets remain linked across runs regardless of internal
ticket resolution status, matching the behavior of internal tickets.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-10 09:32:23 +01:00
596fc94e69 Merge branch 'v20260209-08-veeam-vbo365-not-started' into main 2026-02-09 17:26:15 +01:00
49f24595c3 Merge branch 'v20260209-07-synology-drive-health-parser' into main 2026-02-09 17:26:04 +01:00
fd3f3765c3 Merge branch 'v20260209-06-synology-firmware-update-parser' into main 2026-02-09 17:25:30 +01:00
2a03ff0764 Merge branch 'v20260209-05-responsive-navbar-fix' into main 2026-02-09 17:25:19 +01:00
d7f6de7c23 Release v0.1.25 on branch v20260209-08-veeam-vbo365-not-started (bump type 1) 2026-02-09 17:21:39 +01:00
57196948a7 Add v0.1.25 to website changelog
Update changelog.md and changelog.py with comprehensive v0.1.25 release notes
consolidating all changes from 2026-02-09:

Sections:
- Parser Enhancements: Synology (Drive Health, DSM Updates, ABB Skipped) and
  Veeam (Job Not Started)
- Maintenance Improvements: Orphaned Jobs Cleanup, Test Email Generation
- Data Privacy: Parser Registry Cleanup, Autotask Title Simplification
- Bug Fixes: Responsive Navbar Overlap Fix

This release focuses on parser coverage expansion and system maintenance
capabilities while improving data privacy practices.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-09 17:17:47 +01:00
acb9fbb0ea Auto-commit local changes before build (2026-02-09 17:13:17) 2026-02-09 17:13:17 +01:00
3b48cd401a Add Veeam parser support for "Job did not start on schedule" error notifications
Extend Veeam parser to recognize and handle error notifications when a backup
job fails to start on its scheduled time. This commonly occurs when proxy
servers are offline or other infrastructure issues prevent job execution.

Features:
- Detects "Job did not start on schedule" pattern in subject line
- Extracts backup type from subject (e.g., "Veeam Backup for Microsoft 365")
- Extracts job name from subject after colon (e.g., "Backup MDS at Work")
- Reads error message from plain text body (handles base64 UTF-16 encoding)
- Sets overall_status to "Error" for failed-to-start jobs
- Example message: "Proxy server was offline at the time the job was scheduled to run."

This handles VBO365 and other Veeam backup types that send plain text error
notifications instead of the usual HTML formatted reports.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-09 17:12:53 +01:00
21f5c01148 Auto-commit local changes before build (2026-02-09 17:02:31) 2026-02-09 17:02:31 +01:00
f539d62daf Add Synology monthly drive health report parser
Add parser for Synology monthly drive health reports with support for both
Dutch and English notifications. Reports are classified as informational
and excluded from schedule learning and reporting logic.

Features:
- Recognizes Dutch ("Maandelijks schijfintegriteitsrapport", "Gezond") and
  English ("Monthly Drive Health Report", "Healthy") variants
- Extracts hostname from subject or body ("Van/From NAS-HOSTNAME")
- Automatic status detection: Healthy/Gezond/No problem detected → Success,
  otherwise → Warning
- Backup type: "Health Report", Job name: "Monthly Drive Health"
- Added registry entry (order 237) for /parsers page visibility

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-09 17:02:08 +01:00
b6a85d1c8e Extend Synology DSM update parser with automatic installation announcement patterns
Add detection patterns for DSM update notifications that announce automatic
installation ("belangrijke DSM-update", "kritieke oplossingen", "wordt
automatisch geïnstalleerd", "is beschikbaar op"). This is the fourth variant
of DSM update notifications now handled by the same Updates parser job.

All changes maintain backward compatibility by extending existing pattern lists.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-09 16:45:23 +01:00
5549323ff2 Extend Synology Active Backup for Business parser for skipped tasks
Extended the parser to recognize backup tasks that were skipped/ignored
because a previous backup was still running. These are treated as Warning
status for monitoring purposes.

Changes:
- Extended _ABB_COMPLETED_RE regex to match "genegeerd" (NL) and "skipped"/"ignored" (EN)
- Added "van deze taak" pattern for Dutch phrasing variations
- Added status detection for skipped tasks (Warning with "Skipped" message)
- All existing patterns remain functional (backward compatible)
- Updated changelog

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-09 16:25:27 +01:00
b4aa7ef2f6 Extend Synology Updates parser for new DSM update available notifications
Extended the parser to recognize DSM update available notifications in addition
to update cancelled and package out-of-date notifications. All variants fall
under same Updates job type for unified monitoring.

Changes:
- Added "new DSM update", "Auto Update has detected", "new version of DSM", "Update & Restore" to detection patterns
- Extended hostname extraction regex to match "detected on HOSTNAME"
- Now recognizes three notification types: update cancelled, packages out-of-date, update available
- All existing patterns remain functional (backward compatible)
- Updated changelog

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-09 16:03:11 +01:00
9576d1047e Extend Synology Updates parser to recognize out-of-date package notifications
Extended the parser to recognize both DSM update cancelled notifications AND
out-of-date package notifications under the same "Updates" job type, as they
can appear together in combined notifications.

Changes:
- Added "Packages on", "out-of-date", "Package Center" to detection patterns
- Extended hostname extraction regex to match "Packages on HOSTNAME" and "running on HOSTNAME"
- Both notification types now fall under same job (backup_type: Updates)
- All existing patterns remain functional (backward compatible)
- Updated changelog

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-09 15:58:08 +01:00
1e3a64a78a Auto-commit local changes before build (2026-02-09 15:46:49) 2026-02-09 15:46:49 +01:00
a05dbab574 Extend Synology DSM update parser with additional detection patterns
Extended the parser to recognize more email variants for Synology DSM
automatic update cancelled notifications while maintaining backward
compatibility with existing patterns.

Changes:
- Added "Automatische DSM-update" and "DSM-update op" to detection patterns
- Extended hostname extraction regex to match "DSM-update op HOSTNAME" and "DSM update on HOSTNAME"
- All existing patterns remain functional (backward compatible)
- Updated changelog

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-09 15:41:20 +01:00
0827fddaa5 Add Synology DSM update parser and remove customer names from registry
Added parser registry entry for Synology DSM automatic update cancelled
notifications. These are informational messages that don't participate in
schedule learning or reporting logic.

Also removed real customer names from parser registry examples to prevent
customer information from being stored in the codebase. Replaced with
generic placeholders like NAS-HOSTNAME, SERVER-HOSTNAME, VM-HOSTNAME.

Changes:
- Added synology_dsm_update parser entry in registry.py (order 236)
- Parser matches on DSM-update/DSM update in subject and automatic/automatische in body
- Returns backup_software: Synology, backup_type: Updates, informational status
- Replaced customer names in NTFS Auditing example (bouter.nl → example.local)
- Replaced customer names in QNAP example (BETSIES-NAS01 → NAS-HOSTNAME)
- Replaced customer names in NAKIVO example (kuiperbv.nl → VM-HOSTNAME)
- Updated changelog

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-09 15:36:40 +01:00
61b8e97e34 Auto-commit local changes before build (2026-02-09 15:14:18) 2026-02-09 15:14:18 +01:00
9197c311f2 Fix responsive navbar overlapping content on smaller screens
Added dynamic padding adjustment that measures the actual navbar height and
applies it to the main content padding-top. This prevents the navbar from
overlapping page content when it becomes taller on narrow screens.

Changes:
- Removed fixed padding-top: 80px from main content
- Added id="main-content" to main element for JavaScript targeting
- Added JavaScript function that measures navbar.offsetHeight
- Function applies dynamic padding-top with 20px buffer for spacing
- Triggers on: page load, window load, window resize (debounced), navbar collapse toggle
- Includes fallback to 80px if measurement fails
- Updated changelog

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-09 15:11:47 +01:00
26848998e1 Fix test email generator to use correct Veeam format and consistent job name
Changed test emails to use proper Veeam Backup Job format that matches parser
expectations. All test emails now use the same job name "Test-Backup-Job" so
they appear as different runs of the same job, enabling proper status testing.

Changes:
- Switched from multiple backup software to Veeam only for simplicity
- Fixed subject format to: Veeam Backup Job "Test-Backup-Job" finished with Success/WARNING/Failed
- Fixed body format to include: Backup job: Test-Backup-Job
- All 3 emails per set use same job name but different dates
- Added realistic VM objects (VM-APP01, VM-DB01, VM-WEB01) with status details
- Each set shows different failure scenarios for testing
- Updated changelog description

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-09 14:54:42 +01:00
19ef9dc32a Update test email generator with fixed sets and separate buttons
Changed from configurable count input to three separate buttons for
success, warning, and error test emails. Each button generates exactly
3 emails with consistent data for reproducible testing.

Changes:
- Updated routes_settings.py to use fixed email sets instead of random data
- Changed route from /settings/test-emails/generate to /settings/test-emails/generate/<status_type>
- Created three predefined email sets (success, warning, error) with fixed content
- Updated settings.html UI to show three separate buttons instead of count input
- Each set contains 3 emails simulating Veeam, Synology, and NAKIVO backups
- Updated changelog with detailed description

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-09 14:44:19 +01:00
e187bc3fa5 Auto-commit local changes before build (2026-02-09 14:37:36) 2026-02-09 14:37:36 +01:00
96092517b4 Add test email generator for testing and development
Added feature to generate test emails in inbox for testing purposes:
- Simulates backup notifications from Veeam, Synology, and NAKIVO
- Configurable count (1-50 emails)
- Random job names, statuses, and timestamps
- Emails are parser-compatible for testing inbox approval workflow
- Useful for testing orphaned jobs cleanup and other maintenance ops
- Admin-only feature in Settings → Maintenance

Templates include:
- Veeam: Various job statuses with detailed backup info
- Synology: Backup task notifications
- NAKIVO: Job completion reports

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-09 14:37:02 +01:00
08437aff7f Fix audit logging call for orphaned jobs deletion
Added missing 'message' parameter to _log_admin_event call and
converted details dict to JSON string to match the function signature.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-09 14:31:34 +01:00
710aba97e4 Fix foreign key constraint: delete mail_objects before mails
Added deletion of mail_objects before deleting mail_messages to
avoid foreign key constraint violation. The mail_objects table
has a foreign key to mail_messages.

Complete deletion order:
1. Clean up auxiliary tables
2. Unlink mails from jobs
3. Delete mail_objects
4. Delete jobs (cascades to runs)
5. Delete mails

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-09 14:26:53 +01:00
ff4942272f Fix foreign key constraint: unlink mails from jobs before deletion
Added UPDATE to set mail_messages.job_id = NULL before deleting jobs
to avoid foreign key constraint violation. The mail_messages table
has a foreign key to jobs, so we must unlink them first.

Complete correct order:
1. Clean up auxiliary tables
2. Unlink mails from jobs (SET job_id = NULL)
3. Delete jobs (cascades to runs)
4. Delete mails

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-09 14:25:13 +01:00
f332e61288 Fix foreign key constraint error when deleting orphaned jobs
Moved mail deletion to after job deletion to avoid foreign key
constraint violations. The job_runs have a foreign key to
mail_messages, so jobs (and their cascaded runs) must be deleted
first before the mails can be deleted.

Correct order:
1. Clean up auxiliary tables (ticket_job_runs, remark_job_runs, etc)
2. Delete jobs (cascades to runs via ORM)
3. Delete mails (no more foreign key references)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-09 14:14:17 +01:00
82fff08ebb Remove redundant Step 1 text from maintenance card
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-09 14:11:52 +01:00
e932fdf30a Remove direct delete button, enforce preview step
Removed 'Delete orphaned jobs' button from maintenance page to
enforce verification workflow. Users must now:
1. Click 'Preview orphaned jobs' to see the list
2. Verify which jobs will be deleted
3. Click 'Delete All' on the preview page

This prevents accidental deletion without verification.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-09 14:10:33 +01:00
7f8dffa3ae Add preview page for orphaned jobs before deletion
Added verification step before deleting orphaned jobs:
- New GET endpoint /settings/jobs/orphaned to preview the list
- Shows detailed table with job name, backup software/type, customer ID,
  run count, and email count
- "Preview orphaned jobs" button on maintenance page
- Delete button on preview page shows exact count
- Summary shows total jobs, runs, and emails to be deleted

This allows admins to verify which jobs will be deleted before
taking the destructive action.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-09 14:07:12 +01:00
fec28b2bfa Auto-commit local changes before build (2026-02-09 13:49:21) 2026-02-09 13:49:21 +01:00
91062bdb0d Update .last-branch 2026-02-09 13:49:02 +01:00
ff316d653a Auto-commit local changes before build (2026-02-09 13:46:55) 2026-02-09 13:46:55 +01:00
60c7e89dc2 Add cleanup orphaned jobs maintenance option
Added new maintenance option in Settings → Maintenance to delete
jobs that are no longer linked to an existing customer (customer_id
is NULL or customer doesn't exist).

Features:
- Finds all jobs without valid customer link
- Deletes jobs, runs, and related emails permanently
- Cleans up auxiliary tables (ticket_job_runs, remark_job_runs,
  scopes, overrides)
- Provides feedback on deleted items count
- Logs action to audit log

Use case: When customers are removed, their jobs and emails should
be completely removed from the database.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-09 13:46:07 +01:00
b0efa7f21d Remove customer name from Autotask ticket titles
Changed ticket title format from:
  [Backupchecks] Customer Name - Job Name - Status
To:
  [Backupchecks] Job Name - Status

Customer information is already available in the ticket's company
field, making it redundant in the title and causing unnecessarily
long ticket titles.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-09 13:34:21 +01:00
b7875dbf55 Merge v20260209-01-fix-ticket-description into main (v0.1.24) 2026-02-09 13:16:04 +01:00
d400534069 Merge v20260207-02-wiki-documentation into main (v0.1.23) 2026-02-09 13:15:58 +01:00
bb701d5f00 Release v0.1.24 on branch v20260209-01-fix-ticket-description (bump type 1) 2026-02-09 13:03:06 +01:00
82b96c3448 Add v0.1.24 to changelog for ticket description fix
Updated both docs/changelog.md and changelog.py with v0.1.24
release notes documenting the Autotask ticket description fix.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-09 13:02:02 +01:00
dee99a920d Auto-commit local changes before build (2026-02-09 12:21:34) 2026-02-09 12:21:34 +01:00
58dd2ce831 Fix Autotask ticket description being cleared on resolve
When resolving Autotask tickets via the update_ticket_resolution_safe
function, the description field was not included in the PUT payload,
causing it to be set to NULL by Autotask.

Fixed by adding "description" to the optional_fields list so it is
preserved from the GET response and included in the PUT request.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-09 11:39:48 +01:00
f021f0a2ec Release v0.1.23 on branch v20260207-02-wiki-documentation (bump type 1) 2026-02-09 09:37:52 +01:00
7a3c8c9c51 Add v0.1.23 to website changelog
Added v0.1.23 entry to changelog.py with structured Python format:
- Documentation System (Core Infrastructure, 13 completed pages, accuracy improvements)
- Audit Logging (Renaming to AuditLog, expanded event coverage)
- Timezone-Aware Display (local_datetime filter, all pages updated)
- Autotask Improvements (usability, settings organization, data portability)
- Environment Identification (sandbox banner feature)
- Job Visibility Filtering (archived/inactive customer filtering)
- Repository Security (.gitignore for .claude directory)
- Bug Fixes (imports, datetime handling, documentation corrections)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-09 09:35:39 +01:00
cdbb45fcfb Update changelog with v0.1.23 release notes
Consolidated all changes from 2026-02-06 through 2026-02-08:
- Documentation System (33 pages, 13 with full content)
- Audit Logging enhancements (settings, export, import)
- Timezone-aware datetime display across all pages
- Autotask improvements (direct links, auto-load, collapsible text)
- Environment identification (sandbox banner)
- Job visibility filtering (archived/inactive)
- Multiple bug fixes and documentation corrections

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-09 09:28:13 +01:00
0a16abb4ee Add .gitignore to protect confidential .claude directory
Added .claude/ to gitignore to prevent committing confidential files
like TODO-planka-integration.md which contains Planka API credentials
and environment-specific IDs.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-08 20:46:34 +01:00
53445ab2a4 Correct Autotask integration description in Remarks & Tickets
Fixed to clarify that Autotask integration supports MANUAL ticket creation
(not automatic creation on failures). The integration exists and handles:
- Manual ticket creation in Autotask from Run Checks modal
- Synchronization between BackupChecks and Autotask tickets
- Automatic resolution when Autotask tickets are resolved/deleted

Removed incorrect claim about automatic ticket creation on failures.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-08 20:36:06 +01:00
89f4c65e3e Remove non-existent automatic ticket creation from Remarks & Tickets
Removed entire "Integration with Autotask" section that incorrectly
described automatic ticket creation for backup failures. This feature
does not exist and is not planned.

Changes:
- Removed section explaining automatic ticket creation in Autotask
- Removed link to Autotask Integration from Next Steps
- Updated changelog with removal

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-08 20:34:46 +01:00
36f2f73ff5 Update TODO-documentation with per-job review corrections status
Added notes about latest corrections made to Backup Review documentation:
- Per-job review mechanism fixes
- Removed non-existent reviewed indicator
- Corrected bulk review descriptions
- Updated unmarking section

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-08 20:08:43 +01:00