Compare commits

..

35 Commits

Author SHA1 Message Date
596fc94e69 Merge branch 'v20260209-08-veeam-vbo365-not-started' into main 2026-02-09 17:26:15 +01:00
49f24595c3 Merge branch 'v20260209-07-synology-drive-health-parser' into main 2026-02-09 17:26:04 +01:00
fd3f3765c3 Merge branch 'v20260209-06-synology-firmware-update-parser' into main 2026-02-09 17:25:30 +01:00
2a03ff0764 Merge branch 'v20260209-05-responsive-navbar-fix' into main 2026-02-09 17:25:19 +01:00
d7f6de7c23 Release v0.1.25 on branch v20260209-08-veeam-vbo365-not-started (bump type 1) 2026-02-09 17:21:39 +01:00
57196948a7 Add v0.1.25 to website changelog
Update changelog.md and changelog.py with comprehensive v0.1.25 release notes
consolidating all changes from 2026-02-09:

Sections:
- Parser Enhancements: Synology (Drive Health, DSM Updates, ABB Skipped) and
  Veeam (Job Not Started)
- Maintenance Improvements: Orphaned Jobs Cleanup, Test Email Generation
- Data Privacy: Parser Registry Cleanup, Autotask Title Simplification
- Bug Fixes: Responsive Navbar Overlap Fix

This release focuses on parser coverage expansion and system maintenance
capabilities while improving data privacy practices.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-09 17:17:47 +01:00
acb9fbb0ea Auto-commit local changes before build (2026-02-09 17:13:17) 2026-02-09 17:13:17 +01:00
3b48cd401a Add Veeam parser support for "Job did not start on schedule" error notifications
Extend Veeam parser to recognize and handle error notifications when a backup
job fails to start on its scheduled time. This commonly occurs when proxy
servers are offline or other infrastructure issues prevent job execution.

Features:
- Detects "Job did not start on schedule" pattern in subject line
- Extracts backup type from subject (e.g., "Veeam Backup for Microsoft 365")
- Extracts job name from subject after colon (e.g., "Backup MDS at Work")
- Reads error message from plain text body (handles base64 UTF-16 encoding)
- Sets overall_status to "Error" for failed-to-start jobs
- Example message: "Proxy server was offline at the time the job was scheduled to run."

This handles VBO365 and other Veeam backup types that send plain text error
notifications instead of the usual HTML formatted reports.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-09 17:12:53 +01:00
21f5c01148 Auto-commit local changes before build (2026-02-09 17:02:31) 2026-02-09 17:02:31 +01:00
f539d62daf Add Synology monthly drive health report parser
Add parser for Synology monthly drive health reports with support for both
Dutch and English notifications. Reports are classified as informational
and excluded from schedule learning and reporting logic.

Features:
- Recognizes Dutch ("Maandelijks schijfintegriteitsrapport", "Gezond") and
  English ("Monthly Drive Health Report", "Healthy") variants
- Extracts hostname from subject or body ("Van/From NAS-HOSTNAME")
- Automatic status detection: Healthy/Gezond/No problem detected → Success,
  otherwise → Warning
- Backup type: "Health Report", Job name: "Monthly Drive Health"
- Added registry entry (order 237) for /parsers page visibility

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-09 17:02:08 +01:00
b6a85d1c8e Extend Synology DSM update parser with automatic installation announcement patterns
Add detection patterns for DSM update notifications that announce automatic
installation ("belangrijke DSM-update", "kritieke oplossingen", "wordt
automatisch geïnstalleerd", "is beschikbaar op"). This is the fourth variant
of DSM update notifications now handled by the same Updates parser job.

All changes maintain backward compatibility by extending existing pattern lists.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-09 16:45:23 +01:00
5549323ff2 Extend Synology Active Backup for Business parser for skipped tasks
Extended the parser to recognize backup tasks that were skipped/ignored
because a previous backup was still running. These are treated as Warning
status for monitoring purposes.

Changes:
- Extended _ABB_COMPLETED_RE regex to match "genegeerd" (NL) and "skipped"/"ignored" (EN)
- Added "van deze taak" pattern for Dutch phrasing variations
- Added status detection for skipped tasks (Warning with "Skipped" message)
- All existing patterns remain functional (backward compatible)
- Updated changelog

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-09 16:25:27 +01:00
b4aa7ef2f6 Extend Synology Updates parser for new DSM update available notifications
Extended the parser to recognize DSM update available notifications in addition
to update cancelled and package out-of-date notifications. All variants fall
under same Updates job type for unified monitoring.

Changes:
- Added "new DSM update", "Auto Update has detected", "new version of DSM", "Update & Restore" to detection patterns
- Extended hostname extraction regex to match "detected on HOSTNAME"
- Now recognizes three notification types: update cancelled, packages out-of-date, update available
- All existing patterns remain functional (backward compatible)
- Updated changelog

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-09 16:03:11 +01:00
9576d1047e Extend Synology Updates parser to recognize out-of-date package notifications
Extended the parser to recognize both DSM update cancelled notifications AND
out-of-date package notifications under the same "Updates" job type, as they
can appear together in combined notifications.

Changes:
- Added "Packages on", "out-of-date", "Package Center" to detection patterns
- Extended hostname extraction regex to match "Packages on HOSTNAME" and "running on HOSTNAME"
- Both notification types now fall under same job (backup_type: Updates)
- All existing patterns remain functional (backward compatible)
- Updated changelog

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-09 15:58:08 +01:00
1e3a64a78a Auto-commit local changes before build (2026-02-09 15:46:49) 2026-02-09 15:46:49 +01:00
a05dbab574 Extend Synology DSM update parser with additional detection patterns
Extended the parser to recognize more email variants for Synology DSM
automatic update cancelled notifications while maintaining backward
compatibility with existing patterns.

Changes:
- Added "Automatische DSM-update" and "DSM-update op" to detection patterns
- Extended hostname extraction regex to match "DSM-update op HOSTNAME" and "DSM update on HOSTNAME"
- All existing patterns remain functional (backward compatible)
- Updated changelog

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-09 15:41:20 +01:00
0827fddaa5 Add Synology DSM update parser and remove customer names from registry
Added parser registry entry for Synology DSM automatic update cancelled
notifications. These are informational messages that don't participate in
schedule learning or reporting logic.

Also removed real customer names from parser registry examples to prevent
customer information from being stored in the codebase. Replaced with
generic placeholders like NAS-HOSTNAME, SERVER-HOSTNAME, VM-HOSTNAME.

Changes:
- Added synology_dsm_update parser entry in registry.py (order 236)
- Parser matches on DSM-update/DSM update in subject and automatic/automatische in body
- Returns backup_software: Synology, backup_type: Updates, informational status
- Replaced customer names in NTFS Auditing example (bouter.nl → example.local)
- Replaced customer names in QNAP example (BETSIES-NAS01 → NAS-HOSTNAME)
- Replaced customer names in NAKIVO example (kuiperbv.nl → VM-HOSTNAME)
- Updated changelog

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-09 15:36:40 +01:00
61b8e97e34 Auto-commit local changes before build (2026-02-09 15:14:18) 2026-02-09 15:14:18 +01:00
9197c311f2 Fix responsive navbar overlapping content on smaller screens
Added dynamic padding adjustment that measures the actual navbar height and
applies it to the main content padding-top. This prevents the navbar from
overlapping page content when it becomes taller on narrow screens.

Changes:
- Removed fixed padding-top: 80px from main content
- Added id="main-content" to main element for JavaScript targeting
- Added JavaScript function that measures navbar.offsetHeight
- Function applies dynamic padding-top with 20px buffer for spacing
- Triggers on: page load, window load, window resize (debounced), navbar collapse toggle
- Includes fallback to 80px if measurement fails
- Updated changelog

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-09 15:11:47 +01:00
26848998e1 Fix test email generator to use correct Veeam format and consistent job name
Changed test emails to use proper Veeam Backup Job format that matches parser
expectations. All test emails now use the same job name "Test-Backup-Job" so
they appear as different runs of the same job, enabling proper status testing.

Changes:
- Switched from multiple backup software to Veeam only for simplicity
- Fixed subject format to: Veeam Backup Job "Test-Backup-Job" finished with Success/WARNING/Failed
- Fixed body format to include: Backup job: Test-Backup-Job
- All 3 emails per set use same job name but different dates
- Added realistic VM objects (VM-APP01, VM-DB01, VM-WEB01) with status details
- Each set shows different failure scenarios for testing
- Updated changelog description

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-09 14:54:42 +01:00
19ef9dc32a Update test email generator with fixed sets and separate buttons
Changed from configurable count input to three separate buttons for
success, warning, and error test emails. Each button generates exactly
3 emails with consistent data for reproducible testing.

Changes:
- Updated routes_settings.py to use fixed email sets instead of random data
- Changed route from /settings/test-emails/generate to /settings/test-emails/generate/<status_type>
- Created three predefined email sets (success, warning, error) with fixed content
- Updated settings.html UI to show three separate buttons instead of count input
- Each set contains 3 emails simulating Veeam, Synology, and NAKIVO backups
- Updated changelog with detailed description

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-09 14:44:19 +01:00
e187bc3fa5 Auto-commit local changes before build (2026-02-09 14:37:36) 2026-02-09 14:37:36 +01:00
96092517b4 Add test email generator for testing and development
Added feature to generate test emails in inbox for testing purposes:
- Simulates backup notifications from Veeam, Synology, and NAKIVO
- Configurable count (1-50 emails)
- Random job names, statuses, and timestamps
- Emails are parser-compatible for testing inbox approval workflow
- Useful for testing orphaned jobs cleanup and other maintenance ops
- Admin-only feature in Settings → Maintenance

Templates include:
- Veeam: Various job statuses with detailed backup info
- Synology: Backup task notifications
- NAKIVO: Job completion reports

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-09 14:37:02 +01:00
08437aff7f Fix audit logging call for orphaned jobs deletion
Added missing 'message' parameter to _log_admin_event call and
converted details dict to JSON string to match the function signature.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-09 14:31:34 +01:00
710aba97e4 Fix foreign key constraint: delete mail_objects before mails
Added deletion of mail_objects before deleting mail_messages to
avoid foreign key constraint violation. The mail_objects table
has a foreign key to mail_messages.

Complete deletion order:
1. Clean up auxiliary tables
2. Unlink mails from jobs
3. Delete mail_objects
4. Delete jobs (cascades to runs)
5. Delete mails

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-09 14:26:53 +01:00
ff4942272f Fix foreign key constraint: unlink mails from jobs before deletion
Added UPDATE to set mail_messages.job_id = NULL before deleting jobs
to avoid foreign key constraint violation. The mail_messages table
has a foreign key to jobs, so we must unlink them first.

Complete correct order:
1. Clean up auxiliary tables
2. Unlink mails from jobs (SET job_id = NULL)
3. Delete jobs (cascades to runs)
4. Delete mails

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-09 14:25:13 +01:00
f332e61288 Fix foreign key constraint error when deleting orphaned jobs
Moved mail deletion to after job deletion to avoid foreign key
constraint violations. The job_runs have a foreign key to
mail_messages, so jobs (and their cascaded runs) must be deleted
first before the mails can be deleted.

Correct order:
1. Clean up auxiliary tables (ticket_job_runs, remark_job_runs, etc)
2. Delete jobs (cascades to runs via ORM)
3. Delete mails (no more foreign key references)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-09 14:14:17 +01:00
82fff08ebb Remove redundant Step 1 text from maintenance card
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-09 14:11:52 +01:00
e932fdf30a Remove direct delete button, enforce preview step
Removed 'Delete orphaned jobs' button from maintenance page to
enforce verification workflow. Users must now:
1. Click 'Preview orphaned jobs' to see the list
2. Verify which jobs will be deleted
3. Click 'Delete All' on the preview page

This prevents accidental deletion without verification.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-09 14:10:33 +01:00
7f8dffa3ae Add preview page for orphaned jobs before deletion
Added verification step before deleting orphaned jobs:
- New GET endpoint /settings/jobs/orphaned to preview the list
- Shows detailed table with job name, backup software/type, customer ID,
  run count, and email count
- "Preview orphaned jobs" button on maintenance page
- Delete button on preview page shows exact count
- Summary shows total jobs, runs, and emails to be deleted

This allows admins to verify which jobs will be deleted before
taking the destructive action.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-09 14:07:12 +01:00
fec28b2bfa Auto-commit local changes before build (2026-02-09 13:49:21) 2026-02-09 13:49:21 +01:00
91062bdb0d Update .last-branch 2026-02-09 13:49:02 +01:00
ff316d653a Auto-commit local changes before build (2026-02-09 13:46:55) 2026-02-09 13:46:55 +01:00
60c7e89dc2 Add cleanup orphaned jobs maintenance option
Added new maintenance option in Settings → Maintenance to delete
jobs that are no longer linked to an existing customer (customer_id
is NULL or customer doesn't exist).

Features:
- Finds all jobs without valid customer link
- Deletes jobs, runs, and related emails permanently
- Cleans up auxiliary tables (ticket_job_runs, remark_job_runs,
  scopes, overrides)
- Provides feedback on deleted items count
- Logs action to audit log

Use case: When customers are removed, their jobs and emails should
be completely removed from the database.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-09 13:46:07 +01:00
b0efa7f21d Remove customer name from Autotask ticket titles
Changed ticket title format from:
  [Backupchecks] Customer Name - Job Name - Status
To:
  [Backupchecks] Job Name - Status

Customer information is already available in the ticket's company
field, making it redundant in the title and causing unnecessarily
long ticket titles.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-09 13:34:21 +01:00
13 changed files with 1010 additions and 16 deletions

View File

@ -1 +1 @@
v20260209-01-fix-ticket-description
v20260209-08-veeam-vbo365-not-started

View File

@ -3,6 +3,113 @@ Changelog data structure for Backupchecks
"""
CHANGELOG = [
{
"version": "v0.1.25",
"date": "2026-02-09",
"summary": "This release focuses on parser improvements and maintenance enhancements, adding support for new notification types across Synology and Veeam backup systems while improving system usability with orphaned job cleanup and test email generation features.",
"sections": [
{
"title": "Parser Enhancements",
"type": "feature",
"subsections": [
{
"subtitle": "Synology Parsers",
"changes": [
"Monthly Drive Health Reports: New parser for Synology NAS drive health notifications with Dutch and English support",
"Supports 'Maandelijks schijfintegriteitsrapport' (Dutch) and 'Monthly Drive Health Report' (English) variants",
"Automatic status detection: Healthy/Gezond/No problem detected → Success, otherwise → Warning",
"Extracts hostname from subject or body pattern (Van/From NAS-HOSTNAME)",
"Backup type: 'Health Report', Job name: 'Monthly Drive Health' (informational only, excluded from schedule learning)",
"DSM Update Notifications - Extended Coverage: Added 4 new detection patterns for automatic installation announcements",
"New patterns: 'belangrijke DSM-update', 'kritieke oplossingen', 'wordt automatisch geïnstalleerd', 'is beschikbaar op'",
"Now recognizes 4 notification types: update cancelled, packages out-of-date, new update available, automatic installation scheduled",
"All patterns added to existing lists maintaining full backward compatibility",
"Active Backup for Business - Skipped Tasks: Extended parser to recognize skipped/ignored backup tasks",
"Detects Dutch ('genegeerd') and English ('skipped', 'ignored') status indicators as Warning status",
"Common scenario: Backup skipped because previous backup still running"
]
},
{
"subtitle": "Veeam Parsers",
"changes": [
"Job Not Started Errors: New detection for 'Job did not start on schedule' error notifications",
"Recognizes VBO365 and other Veeam backup types that send plain text error notifications",
"Extracts backup type from subject (e.g., 'Veeam Backup for Microsoft 365')",
"Extracts job name from subject after colon (e.g., 'Backup MDS at Work')",
"Reads error message from plain text body (handles base64 UTF-16 encoding)",
"Sets overall_status to 'Error' for failed-to-start jobs",
"Example messages: 'Proxy server was offline at the time the job was scheduled to run.'"
]
}
]
},
{
"title": "Maintenance Improvements",
"type": "feature",
"subsections": [
{
"subtitle": "Orphaned Jobs Cleanup",
"changes": [
"Added 'Cleanup orphaned jobs' option in Settings → Maintenance",
"Removes jobs without valid customer links (useful when customers are deleted)",
"Permanently deletes job records along with all associated emails and job runs",
"'Preview orphaned jobs' button shows detailed list before deletion with email and run counts",
"Safety verification step to prevent accidental deletion"
]
},
{
"subtitle": "Test Email Generation",
"changes": [
"Added 'Generate test emails' feature in Settings → Maintenance",
"Three separate buttons to create fixed test email sets: Success, Warning, Error",
"Each set contains exactly 3 Veeam Backup Job emails with same job name 'Test-Backup-Job'",
"Different dates, objects, and statuses for reproducible testing scenarios",
"Proper status flow testing (success → warning → error progression)"
]
}
]
},
{
"title": "Data Privacy",
"type": "improvement",
"subsections": [
{
"subtitle": "Parser Registry Cleanup",
"changes": [
"Replaced real customer names in parser registry examples with generic placeholders",
"Affected parsers: NTFS Auditing, QNAP Firmware Update, NAKIVO",
"Example format now uses: NAS-HOSTNAME, SERVER-HOSTNAME, VM-HOSTNAME, example.local",
"Ensures no customer information in codebase or version control"
]
},
{
"subtitle": "Autotask Integration",
"changes": [
"Removed customer name from Autotask ticket title for concise display",
"Format changed from '[Backupchecks] Customer - Job Name - Status' to '[Backupchecks] Job Name - Status'",
"Reduces redundancy (customer already visible in ticket company field)"
]
}
]
},
{
"title": "Bug Fixes",
"type": "bugfix",
"subsections": [
{
"subtitle": "User Interface",
"changes": [
"Fixed responsive navbar overlapping page content on smaller screens",
"Implemented dynamic padding adjustment using JavaScript",
"Measures actual navbar height on page load, window resize, and navbar collapse toggle",
"Automatically adjusts main content padding-top to prevent overlap",
"Debounced resize handler for performance"
]
}
]
}
]
},
{
"version": "v0.1.24",
"date": "2026-02-09",

View File

@ -1464,7 +1464,7 @@ def api_run_checks_create_autotask_ticket():
}
)
subject = f"[Backupchecks] {customer.name} - {job.job_name or ''} - {status_display}"
subject = f"[Backupchecks] {job.job_name or ''} - {status_display}"
description = _compose_autotask_ticket_description(
settings=settings,
job=job,

View File

@ -124,6 +124,446 @@ def settings_jobs_delete_all():
return redirect(url_for("main.settings"))
@main_bp.route("/settings/jobs/orphaned", methods=["GET"])
@login_required
@roles_required("admin")
def settings_jobs_orphaned():
"""Show list of orphaned jobs for verification before deletion."""
# Find jobs without valid customer
orphaned_jobs = Job.query.outerjoin(Customer, Job.customer_id == Customer.id).filter(
db.or_(
Job.customer_id.is_(None),
Customer.id.is_(None)
)
).order_by(Job.job_name.asc()).all()
# Build list with details
jobs_list = []
for job in orphaned_jobs:
run_count = JobRun.query.filter_by(job_id=job.id).count()
mail_count = JobRun.query.filter_by(job_id=job.id).filter(JobRun.mail_message_id.isnot(None)).count()
jobs_list.append({
"id": job.id,
"job_name": job.job_name or "Unnamed",
"backup_software": job.backup_software or "-",
"backup_type": job.backup_type or "-",
"customer_id": job.customer_id,
"run_count": run_count,
"mail_count": mail_count,
})
return render_template(
"main/settings_orphaned_jobs.html",
orphaned_jobs=jobs_list,
)
@main_bp.route("/settings/jobs/delete-orphaned", methods=["POST"])
@login_required
@roles_required("admin")
def settings_jobs_delete_orphaned():
"""Delete jobs that have no customer (customer_id is NULL or customer does not exist).
Also deletes all related emails from the database since the customer is gone.
"""
try:
# Find jobs without valid customer
orphaned_jobs = Job.query.outerjoin(Customer, Job.customer_id == Customer.id).filter(
db.or_(
Job.customer_id.is_(None),
Customer.id.is_(None)
)
).all()
if not orphaned_jobs:
flash("No orphaned jobs found.", "info")
return redirect(url_for("main.settings", section="maintenance"))
job_count = len(orphaned_jobs)
mail_count = 0
run_count = 0
# Collect mail message ids and run ids for cleanup
mail_message_ids = []
run_ids = []
job_ids = [job.id for job in orphaned_jobs]
for job in orphaned_jobs:
for run in job.runs:
if run.id is not None:
run_ids.append(run.id)
run_count += 1
if run.mail_message_id:
mail_message_ids.append(run.mail_message_id)
# Helper function for safe SQL execution
def _safe_execute(stmt, params):
try:
db.session.execute(stmt, params)
except Exception:
pass
# Clean up auxiliary tables that may not have ON DELETE CASCADE
if run_ids:
from sqlalchemy import text, bindparam
_safe_execute(
text("DELETE FROM ticket_job_runs WHERE job_run_id IN :run_ids").bindparams(
bindparam("run_ids", expanding=True)
),
{"run_ids": run_ids},
)
_safe_execute(
text("DELETE FROM remark_job_runs WHERE job_run_id IN :run_ids").bindparams(
bindparam("run_ids", expanding=True)
),
{"run_ids": run_ids},
)
if job_ids:
from sqlalchemy import text, bindparam
# Clean up scopes
_safe_execute(
text("DELETE FROM ticket_scopes WHERE job_id IN :job_ids").bindparams(
bindparam("job_ids", expanding=True)
),
{"job_ids": job_ids},
)
_safe_execute(
text("DELETE FROM remark_scopes WHERE job_id IN :job_ids").bindparams(
bindparam("job_ids", expanding=True)
),
{"job_ids": job_ids},
)
# Clean up overrides
_safe_execute(
text("DELETE FROM overrides WHERE job_id IN :job_ids").bindparams(
bindparam("job_ids", expanding=True)
),
{"job_ids": job_ids},
)
# Unlink mails from jobs before deleting jobs
# mail_messages.job_id references jobs.id
_safe_execute(
text("UPDATE mail_messages SET job_id = NULL WHERE job_id IN :job_ids").bindparams(
bindparam("job_ids", expanding=True)
),
{"job_ids": job_ids},
)
# Delete mail_objects before deleting mails
# mail_objects.mail_message_id references mail_messages.id
if mail_message_ids:
from sqlalchemy import text, bindparam
_safe_execute(
text("DELETE FROM mail_objects WHERE mail_message_id IN :mail_ids").bindparams(
bindparam("mail_ids", expanding=True)
),
{"mail_ids": mail_message_ids},
)
# Delete all orphaned jobs (runs/objects are cascaded via ORM relationships)
for job in orphaned_jobs:
db.session.delete(job)
# Now delete related mails permanently (customer is gone)
# This must happen AFTER deleting jobs/runs to avoid foreign key constraint violations
if mail_message_ids:
mail_count = len(mail_message_ids)
MailMessage.query.filter(MailMessage.id.in_(mail_message_ids)).delete(synchronize_session=False)
db.session.commit()
flash(
f"Deleted {job_count} orphaned job(s), {run_count} run(s), and {mail_count} email(s).",
"success"
)
_log_admin_event(
event_type="maintenance_delete_orphaned_jobs",
message=f"Deleted {job_count} orphaned jobs, {run_count} runs, and {mail_count} emails",
details=json.dumps({
"jobs_deleted": job_count,
"runs_deleted": run_count,
"mails_deleted": mail_count,
}),
)
except Exception as exc:
db.session.rollback()
print(f"[settings-jobs] Failed to delete orphaned jobs: {exc}")
flash("Failed to delete orphaned jobs.", "danger")
return redirect(url_for("main.settings", section="maintenance"))
@main_bp.route("/settings/test-emails/generate/<status_type>", methods=["POST"])
@login_required
@roles_required("admin")
def settings_generate_test_emails(status_type):
"""Generate test emails in inbox for testing parsers and orphaned jobs cleanup.
Fixed sets for consistent testing and reproducibility.
"""
try:
from datetime import datetime, timedelta
# Fixed test email sets per status type (Veeam only for consistent testing)
# All emails use the same job name "Test-Backup-Job" so they appear as different
# runs of the same job for proper status testing
email_sets = {
"success": [
{
"from_address": "veeam@test.local",
"subject": 'Veeam Backup Job "Test-Backup-Job" finished with Success',
"body": """Backup job: Test-Backup-Job
Session details:
Start time: 2026-02-09 01:00:00
End time: 2026-02-09 02:15:00
Total size: 150 GB
Duration: 01:15:00
Processing VM-APP01
Success
Processing VM-DB01
Success
Processing VM-WEB01
Success
All backup operations completed without issues.""",
},
{
"from_address": "veeam@test.local",
"subject": 'Veeam Backup Job "Test-Backup-Job" finished with Success',
"body": """Backup job: Test-Backup-Job
Session details:
Start time: 2026-02-08 01:00:00
End time: 2026-02-08 02:10:00
Total size: 145 GB
Duration: 01:10:00
Processing VM-APP01
Success
Processing VM-DB01
Success
Processing VM-WEB01
Success
All backup operations completed without issues.""",
},
{
"from_address": "veeam@test.local",
"subject": 'Veeam Backup Job "Test-Backup-Job" finished with Success',
"body": """Backup job: Test-Backup-Job
Session details:
Start time: 2026-02-07 01:00:00
End time: 2026-02-07 02:20:00
Total size: 152 GB
Duration: 01:20:00
Processing VM-APP01
Success
Processing VM-DB01
Success
Processing VM-WEB01
Success
All backup operations completed without issues.""",
},
],
"warning": [
{
"from_address": "veeam@test.local",
"subject": 'Veeam Backup Job "Test-Backup-Job" finished with WARNING',
"body": """Backup job: Test-Backup-Job
Session details:
Start time: 2026-02-09 01:00:00
End time: 2026-02-09 02:30:00
Total size: 148 GB
Duration: 01:30:00
Processing VM-APP01
Success
Processing VM-DB01
Warning
Warning: Low free space on target datastore
Processing VM-WEB01
Success
Backup completed but some files were skipped.""",
},
{
"from_address": "veeam@test.local",
"subject": 'Veeam Backup Job "Test-Backup-Job" finished with WARNING',
"body": """Backup job: Test-Backup-Job
Session details:
Start time: 2026-02-08 01:00:00
End time: 2026-02-08 02:25:00
Total size: 142 GB
Duration: 01:25:00
Processing VM-APP01
Warning
Warning: Retry was successful after initial failure
Processing VM-DB01
Success
Processing VM-WEB01
Success
Backup completed with warnings.""",
},
{
"from_address": "veeam@test.local",
"subject": 'Veeam Backup Job "Test-Backup-Job" finished with WARNING',
"body": """Backup job: Test-Backup-Job
Session details:
Start time: 2026-02-07 01:00:00
End time: 2026-02-07 02:35:00
Total size: 140 GB
Duration: 01:35:00
Processing VM-APP01
Success
Processing VM-DB01
Success
Processing VM-WEB01
Warning
Warning: Some files were locked and skipped
Backup completed with warnings.""",
},
],
"error": [
{
"from_address": "veeam@test.local",
"subject": 'Veeam Backup Job "Test-Backup-Job" finished with Failed',
"body": """Backup job: Test-Backup-Job
Session details:
Start time: 2026-02-09 01:00:00
End time: 2026-02-09 01:15:00
Total size: 0 GB
Duration: 00:15:00
Processing VM-APP01
Failed
Error: Cannot create snapshot: VSS error 0x800423f4
Processing VM-DB01
Success
Processing VM-WEB01
Success
Backup failed. Please check the logs for details.""",
},
{
"from_address": "veeam@test.local",
"subject": 'Veeam Backup Job "Test-Backup-Job" finished with Failed',
"body": """Backup job: Test-Backup-Job
Session details:
Start time: 2026-02-08 01:00:00
End time: 2026-02-08 01:10:00
Total size: 0 GB
Duration: 00:10:00
Processing VM-APP01
Success
Processing VM-DB01
Failed
Error: Disk space exhausted on backup repository
Processing VM-WEB01
Success
Backup failed due to storage issue.""",
},
{
"from_address": "veeam@test.local",
"subject": 'Veeam Backup Job "Test-Backup-Job" finished with Failed',
"body": """Backup job: Test-Backup-Job
Session details:
Start time: 2026-02-07 01:00:00
End time: 2026-02-07 01:05:00
Total size: 0 GB
Duration: 00:05:00
Processing VM-APP01
Success
Processing VM-DB01
Success
Processing VM-WEB01
Failed
Error: Network connection lost to ESXi host
Backup failed. Network issue detected.""",
},
],
}
if status_type not in email_sets:
flash("Invalid status type.", "danger")
return redirect(url_for("main.settings", section="maintenance"))
emails = email_sets[status_type]
created_count = 0
now = datetime.utcnow()
for email_data in emails:
mail = MailMessage(
from_address=email_data["from_address"],
subject=email_data["subject"],
text_body=email_data["body"],
html_body=f"<pre>{email_data['body']}</pre>",
received_at=now - timedelta(hours=created_count),
location="inbox",
job_id=None,
)
db.session.add(mail)
created_count += 1
db.session.commit()
flash(f"Generated {created_count} {status_type} test email(s) in inbox.", "success")
_log_admin_event(
event_type="maintenance_generate_test_emails",
message=f"Generated {created_count} {status_type} test emails",
details=json.dumps({"status_type": status_type, "count": created_count}),
)
except Exception as exc:
db.session.rollback()
print(f"[settings-test] Failed to generate test emails: {exc}")
flash("Failed to generate test emails.", "danger")
return redirect(url_for("main.settings", section="maintenance"))
@main_bp.route("/settings/objects/backfill", methods=["POST"])
@login_required
@roles_required("admin")

View File

@ -50,13 +50,13 @@ PARSER_DEFINITIONS = [
},
"description": "Parses NTFS Auditing file audit report mails (attachment-based HTML reports).",
"example": {
"subject": "Bouter btr-dc001.bouter.nl file audits → 6 ↑ 12",
"from_address": "auditing@bouter.nl",
"subject": "SERVER-HOSTNAME file audits → 6 ↑ 12",
"from_address": "auditing@example.local",
"body_snippet": "(empty body, HTML report in attachment)",
"parsed_result": {
"backup_software": "NTFS Auditing",
"backup_type": "Audit",
"job_name": "btr-dc001.bouter.nl file audits",
"job_name": "SERVER-HOSTNAME file audits",
"objects": [],
},
},
@ -73,16 +73,68 @@ PARSER_DEFINITIONS = [
},
"description": "Parses QNAP Notification Center firmware update notifications (informational; excluded from reporting and missing logic).",
"example": {
"subject": "[Info][Firmware Update] Notification from your device: BETSIES-NAS01",
"subject": "[Info][Firmware Update] Notification from your device: NAS-HOSTNAME",
"from_address": "notifications@customer.tld",
"body_snippet": "NAS Name: BETSIES-NAS01\n...\nMessage: ...",
"body_snippet": "NAS Name: NAS-HOSTNAME\n...\nMessage: ...",
"parsed_result": {
"backup_software": "QNAP",
"backup_type": "Firmware Update",
"job_name": "Firmware Update",
"overall_status": "Warning",
"objects": [
{"name": "BETSIES-NAS01", "status": "Warning", "error_message": None}
{"name": "NAS-HOSTNAME", "status": "Warning", "error_message": None}
],
},
},
},
{
"name": "synology_dsm_update",
"backup_software": "Synology",
"backup_types": ["Updates"],
"order": 236,
"enabled": True,
"match": {
"subject_contains_any": ["DSM-update", "DSM update"],
"body_contains_any": ["automatische DSM-update", "automatic DSM update", "Automatic update of DSM"],
},
"description": "Parses Synology DSM automatic update cancelled notifications (informational; excluded from reporting and missing logic).",
"example": {
"subject": "Synology NAS-HOSTNAME - Automatische DSM-update op NAS-HOSTNAME is geannuleerd door het systeem",
"from_address": "backup@example.local",
"body_snippet": "Het systeem heeft de automatische DSM-update op NAS-HOSTNAME geannuleerd...",
"parsed_result": {
"backup_software": "Synology",
"backup_type": "Updates",
"job_name": "Synology Automatic Update",
"overall_status": "Warning",
"objects": [
{"name": "NAS-HOSTNAME", "status": "Warning"}
],
},
},
},
{
"name": "synology_drive_health",
"backup_software": "Synology",
"backup_types": ["Health Report"],
"order": 237,
"enabled": True,
"match": {
"subject_contains_any": ["schijfintegriteitsrapport", "Drive Health Report"],
"body_contains_any": ["health of the drives", "integriteitsrapport van de schijven"],
},
"description": "Parses Synology monthly drive health reports (informational; excluded from reporting and missing logic).",
"example": {
"subject": "[NAS-HOSTNAME] Monthly Drive Health Report on NAS-HOSTNAME - Healthy",
"from_address": "nas@example.local",
"body_snippet": "The following is your monthly report regarding the health of the drives on NAS-HOSTNAME. No problem detected with the drives in DSM.",
"parsed_result": {
"backup_software": "Synology",
"backup_type": "Health Report",
"job_name": "Monthly Drive Health",
"overall_status": "Success",
"objects": [
{"name": "NAS-HOSTNAME", "status": "Success"}
],
},
},
@ -383,16 +435,16 @@ PARSER_DEFINITIONS = [
},
"description": "Parses NAKIVO Backup & Replication reports for VMware backup jobs.",
"example": {
"subject": '"exchange01.kuiperbv.nl" job: Successful',
"subject": '"VM-HOSTNAME" job: Successful',
"from_address": "NAKIVO Backup & Replication <administrator@customer.local>",
"body_snippet": "Job Run Report... Backup job for VMware ... Successful",
"parsed_result": {
"backup_software": "NAKIVO",
"backup_type": "Backup job for VMware",
"job_name": "exchange01.kuiperbv.nl",
"job_name": "VM-HOSTNAME",
"objects": [
{
"name": "exchange01.kuiperbv.nl",
"name": "VM-HOSTNAME",
"status": "Success",
"error_message": "",
}

View File

@ -18,10 +18,23 @@ DSM_UPDATE_CANCELLED_PATTERNS = [
"Automatische update van DSM is geannuleerd",
"Automatic DSM update was cancelled",
"Automatic update of DSM was cancelled",
"Automatische DSM-update",
"DSM-update op",
"Packages on",
"out-of-date",
"Package Center",
"new DSM update",
"Auto Update has detected",
"new version of DSM",
"Update & Restore",
"belangrijke DSM-update",
"kritieke oplossingen",
"wordt automatisch geïnstalleerd",
"is beschikbaar op",
]
_DSM_UPDATE_CANCELLED_HOST_RE = re.compile(
r"\b(?:geannuleerd\s+op|cancelled\s+on)\s+(?P<host>[A-Za-z0-9._-]+)\b",
r"\b(?:geannuleerd\s+op|cancelled\s+on|DSM-update\s+op|DSM\s+update\s+on|Packages\s+on|running\s+on|detected\s+on)\s+(?P<host>[A-Za-z0-9._-]+)\b",
re.I,
)
@ -60,6 +73,75 @@ def _parse_synology_dsm_update_cancelled(subject: str, text: str) -> Tuple[bool,
return True, result, objects
# --- Synology Drive Health Report (informational, excluded from reporting) ---
DRIVE_HEALTH_PATTERNS = [
"schijfintegriteitsrapport",
"Drive Health Report",
"Monthly Drive Health",
"health of the drives",
"integriteitsrapport van de schijven",
]
_DRIVE_HEALTH_SUBJECT_RE = re.compile(
r"\b(?:schijfintegriteitsrapport\s+over|Drive\s+Health\s+Report\s+on)\s+(?P<host>[A-Za-z0-9._-]+)",
re.I,
)
_DRIVE_HEALTH_FROM_RE = re.compile(r"\b(?:Van|From)\s+(?P<host>[A-Za-z0-9._-]+)\b", re.I)
_DRIVE_HEALTH_STATUS_HEALTHY_RE = re.compile(
r"\b(?:Gezond|Healthy|geen\s+problemen\s+gedetecteerd|No\s+problem\s+detected)\b",
re.I,
)
def _is_synology_drive_health(subject: str, text: str) -> bool:
haystack = f"{subject}\n{text}".lower()
return any(p.lower() in haystack for p in DRIVE_HEALTH_PATTERNS)
def _parse_synology_drive_health(subject: str, text: str) -> Tuple[bool, Dict, List[Dict]]:
haystack = f"{subject}\n{text}"
host = ""
# Try to extract hostname from subject first
m = _DRIVE_HEALTH_SUBJECT_RE.search(subject or "")
if m:
host = (m.group("host") or "").strip()
# Fallback: extract from body "Van/From NAS-NAME"
if not host:
m = _DRIVE_HEALTH_FROM_RE.search(text or "")
if m:
host = (m.group("host") or "").strip()
# Determine status based on health indicators
overall_status = "Success"
overall_message = "Healthy"
if not _DRIVE_HEALTH_STATUS_HEALTHY_RE.search(haystack):
# If we don't find healthy indicators, mark as Warning
overall_status = "Warning"
overall_message = "Drive health issue detected"
# Informational job: show in Run Checks, but do not participate in schedules / reporting.
result: Dict = {
"backup_software": "Synology",
"backup_type": "Health Report",
"job_name": "Monthly Drive Health",
"overall_status": overall_status,
"overall_message": overall_message + (f" ({host})" if host else ""),
}
objects: List[Dict] = []
if host:
objects.append({"name": host, "status": overall_status})
return True, result, objects
_BR_RE = re.compile(r"<\s*br\s*/?\s*>", re.I)
_TAG_RE = re.compile(r"<[^>]+>")
_WS_RE = re.compile(r"[\t\r\f\v ]+")
@ -176,12 +258,14 @@ _ABB_SUBJECT_RE = re.compile(r"\bactive\s+backup\s+for\s+business\b", re.I)
# Examples (NL):
# "De back-uptaak vSphere-Task-1 op KANTOOR-NEW is voltooid."
# "Virtuele machine back-uptaak vSphere-Task-1 op KANTOOR-NEW is gedeeltelijk voltooid."
# "back-uptaak vSphere-Task-1 op KANTOOR-NEW is genegeerd"
# Examples (EN):
# "The backup task vSphere-Task-1 on KANTOOR-NEW has completed."
# "Virtual machine backup task vSphere-Task-1 on KANTOOR-NEW partially completed."
# "backup task vSphere-Task-1 on KANTOOR-NEW was skipped"
_ABB_COMPLETED_RE = re.compile(
r"\b(?:virtuele\s+machine\s+)?(?:de\s+)?back-?up\s*taak\s+(?P<job>.+?)\s+op\s+(?P<host>[A-Za-z0-9._-]+)\s+is\s+(?P<status>voltooid|gedeeltelijk\s+voltooid)\b"
r"|\b(?:virtual\s+machine\s+)?(?:the\s+)?back-?up\s+task\s+(?P<job_en>.+?)\s+on\s+(?P<host_en>[A-Za-z0-9._-]+)\s+(?:is\s+)?(?P<status_en>completed|finished|has\s+completed|partially\s+completed)\b",
r"\b(?:virtuele\s+machine\s+)?(?:de\s+)?back-?up\s*(?:taak|job)\s+(?:van\s+deze\s+taak\s+)?(?P<job>.+?)\s+op\s+(?P<host>[A-Za-z0-9._-]+)\s+is\s+(?P<status>voltooid|gedeeltelijk\s+voltooid|genegeerd)\b"
r"|\b(?:virtual\s+machine\s+)?(?:the\s+)?back-?up\s+(?:task|job)\s+(?P<job_en>.+?)\s+on\s+(?P<host_en>[A-Za-z0-9._-]+)\s+(?:is\s+|was\s+)?(?P<status_en>completed|finished|has\s+completed|partially\s+completed|skipped|ignored)\b",
re.I,
)
@ -233,6 +317,11 @@ def _parse_active_backup_for_business(subject: str, text: str) -> Tuple[bool, Di
overall_status = "Warning"
overall_message = "Partially completed"
# "genegeerd" / "skipped" / "ignored" should be treated as Warning
if "genegeerd" in status_raw or "skipped" in status_raw or "ignored" in status_raw:
overall_status = "Warning"
overall_message = "Skipped"
# Explicit failure wording overrides everything
if _ABB_FAILED_RE.search(haystack):
overall_status = "Error"
@ -489,6 +578,12 @@ def try_parse_synology(msg: MailMessage) -> Tuple[bool, Dict, List[Dict]]:
if ok:
return True, result, objects
# Drive Health Report (informational; no schedule; excluded from reporting)
if _is_synology_drive_health(subject, text):
ok, result, objects = _parse_synology_drive_health(subject, text)
if ok:
return True, result, objects
# DSM Account Protection (informational; no schedule)
if _is_synology_account_protection(subject, text):
ok, result, objects = _parse_account_protection(subject, text)

View File

@ -1177,6 +1177,38 @@ def try_parse_veeam(msg: MailMessage) -> Tuple[bool, Dict, List[Dict]]:
}
return True, result, []
# Job did not start on schedule: special error notification (no objects, plain text body).
# Example subject: "[Veeam Backup for Microsoft 365] Job did not start on schedule: Backup MDS at Work"
subject_lower = subject.lower()
if 'job did not start on schedule' in subject_lower:
# Extract backup type from subject (e.g., "Veeam Backup for Microsoft 365")
backup_type = None
for candidate in VEEAM_BACKUP_TYPES:
if candidate.lower() in subject_lower:
backup_type = candidate
break
if not backup_type:
backup_type = "Backup Job"
# Extract job name after the colon (e.g., "Backup MDS at Work")
job_name = None
m_job = re.search(r'job did not start on schedule:\s*(.+)$', subject, re.IGNORECASE)
if m_job:
job_name = (m_job.group(1) or '').strip()
# Get overall message from text_body (can be base64 encoded)
text_body = (getattr(msg, 'text_body', None) or '').strip()
overall_message = text_body if text_body else 'Job did not start on schedule'
result = {
'backup_software': 'Veeam',
'backup_type': backup_type,
'job_name': job_name or 'Unknown Job',
'overall_status': 'Error',
'overall_message': overall_message,
}
return True, result, []
# Configuration Job detection (may not have object details)
subj_lower = subject.lower()
is_config_job = ('backup configuration job' in subj_lower) or ('configuration backup for' in html_lower)

View File

@ -197,7 +197,7 @@
</div>
</nav>
<main class="{% block main_class %}container content-container{% endblock %}" style="padding-top: 80px;">
<main class="{% block main_class %}container content-container{% endblock %}" id="main-content">
{% with messages = get_flashed_messages(with_categories=true) %}
{% if messages %}
<div class="mb-3">
@ -216,6 +216,58 @@
<script src="https://cdn.jsdelivr.net/npm/bootstrap@5.3.3/dist/js/bootstrap.bundle.min.js"></script>
<script>
// Dynamic navbar height adjustment
(function () {
function adjustContentPadding() {
try {
var navbar = document.querySelector('.navbar.fixed-top');
var mainContent = document.getElementById('main-content');
if (!navbar || !mainContent) return;
// Get actual navbar height
var navbarHeight = navbar.offsetHeight;
// Add small buffer (20px) for visual spacing
var paddingTop = navbarHeight + 20;
// Apply padding to main content
mainContent.style.paddingTop = paddingTop + 'px';
} catch (e) {
// Fallback to 80px if something goes wrong
var mainContent = document.getElementById('main-content');
if (mainContent) {
mainContent.style.paddingTop = '80px';
}
}
}
// Run on page load
if (document.readyState === 'loading') {
document.addEventListener('DOMContentLoaded', adjustContentPadding);
} else {
adjustContentPadding();
}
// Run after navbar is fully rendered
window.addEventListener('load', adjustContentPadding);
// Run on window resize
var resizeTimeout;
window.addEventListener('resize', function () {
clearTimeout(resizeTimeout);
resizeTimeout = setTimeout(adjustContentPadding, 100);
});
// Run when navbar collapse is toggled
var navbarCollapse = document.getElementById('navbarNav');
if (navbarCollapse) {
navbarCollapse.addEventListener('shown.bs.collapse', adjustContentPadding);
navbarCollapse.addEventListener('hidden.bs.collapse', adjustContentPadding);
}
})();
</script>
<script>
(function () {
function isOverflowing(el) {

View File

@ -549,6 +549,36 @@
</div>
</div>
<div class="col-12 col-lg-6">
<div class="card h-100 border-warning">
<div class="card-header bg-warning">Cleanup orphaned jobs</div>
<div class="card-body">
<p class="mb-3">Delete jobs that are no longer linked to an existing customer. Related emails and runs will be <strong>permanently deleted</strong> from the database.</p>
<a href="{{ url_for('main.settings_jobs_orphaned') }}" class="btn btn-warning">Preview orphaned jobs</a>
</div>
</div>
</div>
<div class="col-12 col-lg-6">
<div class="card h-100 border-info">
<div class="card-header bg-info text-white">Generate test emails</div>
<div class="card-body">
<p class="mb-3">Generate fixed test email sets in the inbox for testing parsers and maintenance operations. Each set contains 3 emails simulating Veeam, Synology, and NAKIVO backups.</p>
<div class="d-flex flex-column gap-2">
<form method="post" action="{{ url_for('main.settings_generate_test_emails', status_type='success') }}">
<button type="submit" class="btn btn-success w-100">Generate success emails (3)</button>
</form>
<form method="post" action="{{ url_for('main.settings_generate_test_emails', status_type='warning') }}">
<button type="submit" class="btn btn-warning w-100">Generate warning emails (3)</button>
</form>
<form method="post" action="{{ url_for('main.settings_generate_test_emails', status_type='error') }}">
<button type="submit" class="btn btn-danger w-100">Generate error emails (3)</button>
</form>
</div>
</div>
</div>
</div>
<div class="col-12 col-lg-6">
<div class="card h-100 border-danger">
<div class="card-header bg-danger text-white">Jobs maintenance</div>

View File

@ -0,0 +1,87 @@
{% extends "layout/base.html" %}
{% block title %}Orphaned Jobs Preview{% endblock %}
{% block content %}
<div class="container-fluid py-4">
<div class="d-flex justify-content-between align-items-center mb-4">
<div>
<h2>Orphaned Jobs Preview</h2>
<p class="text-muted mb-0">Jobs without a valid customer link</p>
</div>
<a href="{{ url_for('main.settings', section='maintenance') }}" class="btn btn-outline-secondary">Back to Settings</a>
</div>
{% if orphaned_jobs %}
<div class="alert alert-warning">
<strong>⚠️ Warning:</strong> Found {{ orphaned_jobs|length }} orphaned job(s). Review the list below before deleting.
</div>
<div class="card mb-4">
<div class="card-header d-flex justify-content-between align-items-center">
<span>Orphaned Jobs List</span>
<form method="post" action="{{ url_for('main.settings_jobs_delete_orphaned') }}" onsubmit="return confirm('Delete all {{ orphaned_jobs|length }} orphaned jobs and their emails? This cannot be undone.');">
<button type="submit" class="btn btn-sm btn-danger">Delete All ({{ orphaned_jobs|length }} jobs)</button>
</form>
</div>
<div class="card-body p-0">
<div class="table-responsive">
<table class="table table-hover mb-0">
<thead>
<tr>
<th>Job Name</th>
<th>Backup Software</th>
<th>Backup Type</th>
<th>Customer ID</th>
<th class="text-end">Runs</th>
<th class="text-end">Emails</th>
</tr>
</thead>
<tbody>
{% for job in orphaned_jobs %}
<tr>
<td>{{ job.job_name }}</td>
<td>{{ job.backup_software }}</td>
<td>{{ job.backup_type }}</td>
<td>
{% if job.customer_id %}
<span class="badge bg-danger">{{ job.customer_id }} (deleted)</span>
{% else %}
<span class="badge bg-secondary">NULL</span>
{% endif %}
</td>
<td class="text-end">{{ job.run_count }}</td>
<td class="text-end">{{ job.mail_count }}</td>
</tr>
{% endfor %}
</tbody>
<tfoot>
<tr class="table-light">
<td colspan="4"><strong>Total</strong></td>
<td class="text-end"><strong>{{ orphaned_jobs|sum(attribute='run_count') }}</strong></td>
<td class="text-end"><strong>{{ orphaned_jobs|sum(attribute='mail_count') }}</strong></td>
</tr>
</tfoot>
</table>
</div>
</div>
</div>
<div class="alert alert-info">
<strong> What will be deleted:</strong>
<ul class="mb-0">
<li>{{ orphaned_jobs|length }} job(s)</li>
<li>{{ orphaned_jobs|sum(attribute='run_count') }} job run(s)</li>
<li>{{ orphaned_jobs|sum(attribute='mail_count') }} email(s)</li>
<li>All related data (backup objects, ticket/remark links, scopes, overrides)</li>
</ul>
</div>
{% else %}
<div class="alert alert-success">
<strong>✅ No orphaned jobs found.</strong>
<p class="mb-0">All jobs are properly linked to existing customers.</p>
</div>
{% endif %}
</div>
{% endblock %}

View File

@ -4,8 +4,25 @@ This file documents all changes made to this project via Claude Code.
## [2026-02-09]
### Added
- Extended Veeam parser to recognize "Job did not start on schedule" error notifications for Veeam Backup for Microsoft 365 (and other Veeam backup types) with job name extraction from subject and error message from plain text body (proxy server offline, scheduled run failed)
- Added parser for Synology monthly drive health reports (backup software: Synology, backup type: Health Report, job name: Monthly Drive Health, informational only, no schedule learning) with support for both Dutch and English notifications ("schijfintegriteitsrapport"/"Drive Health Report") and automatic status detection (Healthy/Gezond → Success, problems → Warning)
- Added "Cleanup orphaned jobs" maintenance option in Settings → Maintenance to delete jobs without valid customer links and their associated emails/runs permanently from database (useful when customers are removed)
- Added "Preview orphaned jobs" button to show detailed list of jobs to be deleted with run/email counts before confirming deletion (verification step for safety)
- Added "Generate test emails" feature in Settings → Maintenance with three separate buttons to create fixed test email sets (success/warning/error) in inbox for testing parsers and maintenance operations (each set contains exactly 3 Veeam Backup Job emails with the same job name "Test-Backup-Job" and different dates/objects/statuses for reproducible testing and proper status flow testing)
- Added parser registry entry for Synology DSM automatic update cancelled notifications (backup software: Synology, backup type: Updates, informational only, no schedule learning)
- Extended Synology DSM update parser with additional detection patterns ("Automatische DSM-update", "DSM-update op", "Packages on", "out-of-date", "Package Center", "new DSM update", "Auto Update has detected", "Update & Restore", "belangrijke DSM-update", "kritieke oplossingen", "wordt automatisch geïnstalleerd", "is beschikbaar op") and hostname extraction regex to recognize DSM update cancelled, out-of-date packages, new update available, and automatic installation announcements under same Updates job type while maintaining backward compatibility with existing patterns
- Extended Synology Active Backup for Business parser to recognize skipped/ignored backup tasks ("genegeerd", "skipped", "ignored") as Warning status when backup was skipped due to previous backup still running
### Changed
- Updated `docs/changelog.md` with comprehensive v0.1.25 release notes consolidating all changes from 2026-02-09 (Parser Enhancements for Synology and Veeam, Maintenance Improvements, Data Privacy, Bug Fixes)
- Updated `containers/backupchecks/src/backend/app/changelog.py` with v0.1.25 entry in Python structure for website display (4 sections with subsections matching changelog.md content)
- Removed customer name from Autotask ticket title to keep titles concise (format changed from "[Backupchecks] Customer - Job Name - Status" to "[Backupchecks] Job Name - Status")
- Replaced real customer names in parser registry examples with generic placeholders (NTFS Auditing, QNAP Firmware Update, NAKIVO) to prevent customer information in codebase
### Fixed
- Fixed Autotask ticket description being set to NULL when resolving tickets via `update_ticket_resolution_safe` by adding "description" to the optional_fields list, ensuring the original description is preserved during PUT operations
- Fixed responsive navbar overlapping page content on smaller screens by implementing dynamic padding adjustment (JavaScript measures actual navbar height and adjusts main content padding-top automatically on page load, window resize, and navbar collapse toggle events)
### Changed
- Updated `docs/changelog.md` with comprehensive v0.1.23 release notes consolidating all changes from 2026-02-06 through 2026-02-08 (Documentation System, Audit Logging, Timezone-Aware Display, Autotask Improvements, Environment Identification, Bug Fixes)

View File

@ -1,3 +1,85 @@
## v0.1.25
This release focuses on parser improvements and maintenance enhancements, adding support for new notification types across Synology and Veeam backup systems while improving system usability with orphaned job cleanup and test email generation features.
### Parser Enhancements
**Synology Parsers:**
- **Monthly Drive Health Reports**: New parser for Synology NAS drive health notifications
- Supports both Dutch ("Maandelijks schijfintegriteitsrapport", "Gezond") and English ("Monthly Drive Health Report", "Healthy") variants
- Automatic status detection: Healthy/Gezond/No problem detected → Success, otherwise → Warning
- Extracts hostname from subject or body pattern (Van/From NAS-HOSTNAME)
- Backup type: "Health Report", Job name: "Monthly Drive Health"
- Informational only (excluded from schedule learning and reporting logic)
- Registry entry added (order 237) for /parsers page visibility
- **DSM Update Notifications - Extended Coverage**: Added support for additional DSM update notification variants
- New patterns: "belangrijke DSM-update", "kritieke oplossingen", "wordt automatisch geïnstalleerd", "is beschikbaar op"
- Now recognizes 4 different notification types under same job:
1. Automatic update cancelled
2. Packages out-of-date warnings
3. New update available announcements
4. Automatic installation scheduled notifications
- All patterns added to existing lists maintaining full backward compatibility
- **Active Backup for Business - Skipped Tasks**: Extended parser to recognize skipped/ignored backup tasks
- Detects Dutch ("genegeerd") and English ("skipped", "ignored") status indicators
- Status mapping: Skipped/Ignored → Warning with "Skipped" message
- Common scenario: Backup skipped because previous backup still running
**Veeam Parsers:**
- **Job Not Started Errors**: New detection for "Job did not start on schedule" error notifications
- Recognizes VBO365 and other Veeam backup types that send plain text error notifications
- Extracts backup type from subject (e.g., "Veeam Backup for Microsoft 365")
- Extracts job name from subject after colon (e.g., "Backup MDS at Work")
- Reads error message from plain text body (handles base64 UTF-16 encoding)
- Sets overall_status to "Error" for failed-to-start jobs
- Example messages: "Proxy server was offline at the time the job was scheduled to run."
### Maintenance Improvements
**Orphaned Jobs Cleanup:**
- Added "Cleanup orphaned jobs" option in Settings → Maintenance
- Removes jobs without valid customer links (useful when customers are deleted)
- Permanently deletes job records along with all associated emails and job runs
- "Preview orphaned jobs" button shows detailed list before deletion
- Displays job information with email and run counts
- Safety verification step to prevent accidental deletion
**Test Email Generation:**
- Added "Generate test emails" feature in Settings → Maintenance
- Three separate buttons to create fixed test email sets for parser testing:
- Success emails (3 emails with success status)
- Warning emails (3 emails with warning status)
- Error emails (3 emails with error status)
- Each set contains exactly 3 Veeam Backup Job emails with:
- Same job name "Test-Backup-Job" for consistency
- Different dates, objects, and statuses
- Reproducible testing scenarios
- Proper status flow testing (success → warning → error progression)
### Data Privacy
**Parser Registry Cleanup:**
- Replaced real customer names in parser registry examples with generic placeholders
- Affected parsers: NTFS Auditing, QNAP Firmware Update, NAKIVO
- Example format now uses: NAS-HOSTNAME, SERVER-HOSTNAME, VM-HOSTNAME, example.local
- Ensures no customer information in codebase or version control
**Autotask Integration:**
- Removed customer name from Autotask ticket title for concise display
- Format changed from "[Backupchecks] Customer - Job Name - Status" to "[Backupchecks] Job Name - Status"
- Reduces redundancy (customer already visible in ticket company field)
### Bug Fixes
**User Interface:**
- Fixed responsive navbar overlapping page content on smaller screens
- Implemented dynamic padding adjustment using JavaScript
- Measures actual navbar height on page load, window resize, and navbar collapse toggle
- Automatically adjusts main content padding-top to prevent overlap
- Debounced resize handler for performance
## v0.1.24
### Bug Fixes

View File

@ -1 +1 @@
v0.1.24
v0.1.25