Compare commits
35 Commits
b7875dbf55
...
596fc94e69
| Author | SHA1 | Date | |
|---|---|---|---|
| 596fc94e69 | |||
| 49f24595c3 | |||
| fd3f3765c3 | |||
| 2a03ff0764 | |||
| d7f6de7c23 | |||
| 57196948a7 | |||
| acb9fbb0ea | |||
| 3b48cd401a | |||
| 21f5c01148 | |||
| f539d62daf | |||
| b6a85d1c8e | |||
| 5549323ff2 | |||
| b4aa7ef2f6 | |||
| 9576d1047e | |||
| 1e3a64a78a | |||
| a05dbab574 | |||
| 0827fddaa5 | |||
| 61b8e97e34 | |||
| 9197c311f2 | |||
| 26848998e1 | |||
| 19ef9dc32a | |||
| e187bc3fa5 | |||
| 96092517b4 | |||
| 08437aff7f | |||
| 710aba97e4 | |||
| ff4942272f | |||
| f332e61288 | |||
| 82fff08ebb | |||
| e932fdf30a | |||
| 7f8dffa3ae | |||
| fec28b2bfa | |||
| 91062bdb0d | |||
| ff316d653a | |||
| 60c7e89dc2 | |||
| b0efa7f21d |
@ -1 +1 @@
|
|||||||
v20260209-01-fix-ticket-description
|
v20260209-08-veeam-vbo365-not-started
|
||||||
|
|||||||
@ -3,6 +3,113 @@ Changelog data structure for Backupchecks
|
|||||||
"""
|
"""
|
||||||
|
|
||||||
CHANGELOG = [
|
CHANGELOG = [
|
||||||
|
{
|
||||||
|
"version": "v0.1.25",
|
||||||
|
"date": "2026-02-09",
|
||||||
|
"summary": "This release focuses on parser improvements and maintenance enhancements, adding support for new notification types across Synology and Veeam backup systems while improving system usability with orphaned job cleanup and test email generation features.",
|
||||||
|
"sections": [
|
||||||
|
{
|
||||||
|
"title": "Parser Enhancements",
|
||||||
|
"type": "feature",
|
||||||
|
"subsections": [
|
||||||
|
{
|
||||||
|
"subtitle": "Synology Parsers",
|
||||||
|
"changes": [
|
||||||
|
"Monthly Drive Health Reports: New parser for Synology NAS drive health notifications with Dutch and English support",
|
||||||
|
"Supports 'Maandelijks schijfintegriteitsrapport' (Dutch) and 'Monthly Drive Health Report' (English) variants",
|
||||||
|
"Automatic status detection: Healthy/Gezond/No problem detected → Success, otherwise → Warning",
|
||||||
|
"Extracts hostname from subject or body pattern (Van/From NAS-HOSTNAME)",
|
||||||
|
"Backup type: 'Health Report', Job name: 'Monthly Drive Health' (informational only, excluded from schedule learning)",
|
||||||
|
"DSM Update Notifications - Extended Coverage: Added 4 new detection patterns for automatic installation announcements",
|
||||||
|
"New patterns: 'belangrijke DSM-update', 'kritieke oplossingen', 'wordt automatisch geïnstalleerd', 'is beschikbaar op'",
|
||||||
|
"Now recognizes 4 notification types: update cancelled, packages out-of-date, new update available, automatic installation scheduled",
|
||||||
|
"All patterns added to existing lists maintaining full backward compatibility",
|
||||||
|
"Active Backup for Business - Skipped Tasks: Extended parser to recognize skipped/ignored backup tasks",
|
||||||
|
"Detects Dutch ('genegeerd') and English ('skipped', 'ignored') status indicators as Warning status",
|
||||||
|
"Common scenario: Backup skipped because previous backup still running"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"subtitle": "Veeam Parsers",
|
||||||
|
"changes": [
|
||||||
|
"Job Not Started Errors: New detection for 'Job did not start on schedule' error notifications",
|
||||||
|
"Recognizes VBO365 and other Veeam backup types that send plain text error notifications",
|
||||||
|
"Extracts backup type from subject (e.g., 'Veeam Backup for Microsoft 365')",
|
||||||
|
"Extracts job name from subject after colon (e.g., 'Backup MDS at Work')",
|
||||||
|
"Reads error message from plain text body (handles base64 UTF-16 encoding)",
|
||||||
|
"Sets overall_status to 'Error' for failed-to-start jobs",
|
||||||
|
"Example messages: 'Proxy server was offline at the time the job was scheduled to run.'"
|
||||||
|
]
|
||||||
|
}
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"title": "Maintenance Improvements",
|
||||||
|
"type": "feature",
|
||||||
|
"subsections": [
|
||||||
|
{
|
||||||
|
"subtitle": "Orphaned Jobs Cleanup",
|
||||||
|
"changes": [
|
||||||
|
"Added 'Cleanup orphaned jobs' option in Settings → Maintenance",
|
||||||
|
"Removes jobs without valid customer links (useful when customers are deleted)",
|
||||||
|
"Permanently deletes job records along with all associated emails and job runs",
|
||||||
|
"'Preview orphaned jobs' button shows detailed list before deletion with email and run counts",
|
||||||
|
"Safety verification step to prevent accidental deletion"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"subtitle": "Test Email Generation",
|
||||||
|
"changes": [
|
||||||
|
"Added 'Generate test emails' feature in Settings → Maintenance",
|
||||||
|
"Three separate buttons to create fixed test email sets: Success, Warning, Error",
|
||||||
|
"Each set contains exactly 3 Veeam Backup Job emails with same job name 'Test-Backup-Job'",
|
||||||
|
"Different dates, objects, and statuses for reproducible testing scenarios",
|
||||||
|
"Proper status flow testing (success → warning → error progression)"
|
||||||
|
]
|
||||||
|
}
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"title": "Data Privacy",
|
||||||
|
"type": "improvement",
|
||||||
|
"subsections": [
|
||||||
|
{
|
||||||
|
"subtitle": "Parser Registry Cleanup",
|
||||||
|
"changes": [
|
||||||
|
"Replaced real customer names in parser registry examples with generic placeholders",
|
||||||
|
"Affected parsers: NTFS Auditing, QNAP Firmware Update, NAKIVO",
|
||||||
|
"Example format now uses: NAS-HOSTNAME, SERVER-HOSTNAME, VM-HOSTNAME, example.local",
|
||||||
|
"Ensures no customer information in codebase or version control"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"subtitle": "Autotask Integration",
|
||||||
|
"changes": [
|
||||||
|
"Removed customer name from Autotask ticket title for concise display",
|
||||||
|
"Format changed from '[Backupchecks] Customer - Job Name - Status' to '[Backupchecks] Job Name - Status'",
|
||||||
|
"Reduces redundancy (customer already visible in ticket company field)"
|
||||||
|
]
|
||||||
|
}
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"title": "Bug Fixes",
|
||||||
|
"type": "bugfix",
|
||||||
|
"subsections": [
|
||||||
|
{
|
||||||
|
"subtitle": "User Interface",
|
||||||
|
"changes": [
|
||||||
|
"Fixed responsive navbar overlapping page content on smaller screens",
|
||||||
|
"Implemented dynamic padding adjustment using JavaScript",
|
||||||
|
"Measures actual navbar height on page load, window resize, and navbar collapse toggle",
|
||||||
|
"Automatically adjusts main content padding-top to prevent overlap",
|
||||||
|
"Debounced resize handler for performance"
|
||||||
|
]
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
]
|
||||||
|
},
|
||||||
{
|
{
|
||||||
"version": "v0.1.24",
|
"version": "v0.1.24",
|
||||||
"date": "2026-02-09",
|
"date": "2026-02-09",
|
||||||
|
|||||||
@ -1464,7 +1464,7 @@ def api_run_checks_create_autotask_ticket():
|
|||||||
}
|
}
|
||||||
)
|
)
|
||||||
|
|
||||||
subject = f"[Backupchecks] {customer.name} - {job.job_name or ''} - {status_display}"
|
subject = f"[Backupchecks] {job.job_name or ''} - {status_display}"
|
||||||
description = _compose_autotask_ticket_description(
|
description = _compose_autotask_ticket_description(
|
||||||
settings=settings,
|
settings=settings,
|
||||||
job=job,
|
job=job,
|
||||||
|
|||||||
@ -124,6 +124,446 @@ def settings_jobs_delete_all():
|
|||||||
return redirect(url_for("main.settings"))
|
return redirect(url_for("main.settings"))
|
||||||
|
|
||||||
|
|
||||||
|
@main_bp.route("/settings/jobs/orphaned", methods=["GET"])
|
||||||
|
@login_required
|
||||||
|
@roles_required("admin")
|
||||||
|
def settings_jobs_orphaned():
|
||||||
|
"""Show list of orphaned jobs for verification before deletion."""
|
||||||
|
# Find jobs without valid customer
|
||||||
|
orphaned_jobs = Job.query.outerjoin(Customer, Job.customer_id == Customer.id).filter(
|
||||||
|
db.or_(
|
||||||
|
Job.customer_id.is_(None),
|
||||||
|
Customer.id.is_(None)
|
||||||
|
)
|
||||||
|
).order_by(Job.job_name.asc()).all()
|
||||||
|
|
||||||
|
# Build list with details
|
||||||
|
jobs_list = []
|
||||||
|
for job in orphaned_jobs:
|
||||||
|
run_count = JobRun.query.filter_by(job_id=job.id).count()
|
||||||
|
mail_count = JobRun.query.filter_by(job_id=job.id).filter(JobRun.mail_message_id.isnot(None)).count()
|
||||||
|
|
||||||
|
jobs_list.append({
|
||||||
|
"id": job.id,
|
||||||
|
"job_name": job.job_name or "Unnamed",
|
||||||
|
"backup_software": job.backup_software or "-",
|
||||||
|
"backup_type": job.backup_type or "-",
|
||||||
|
"customer_id": job.customer_id,
|
||||||
|
"run_count": run_count,
|
||||||
|
"mail_count": mail_count,
|
||||||
|
})
|
||||||
|
|
||||||
|
return render_template(
|
||||||
|
"main/settings_orphaned_jobs.html",
|
||||||
|
orphaned_jobs=jobs_list,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@main_bp.route("/settings/jobs/delete-orphaned", methods=["POST"])
|
||||||
|
@login_required
|
||||||
|
@roles_required("admin")
|
||||||
|
def settings_jobs_delete_orphaned():
|
||||||
|
"""Delete jobs that have no customer (customer_id is NULL or customer does not exist).
|
||||||
|
|
||||||
|
Also deletes all related emails from the database since the customer is gone.
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
# Find jobs without valid customer
|
||||||
|
orphaned_jobs = Job.query.outerjoin(Customer, Job.customer_id == Customer.id).filter(
|
||||||
|
db.or_(
|
||||||
|
Job.customer_id.is_(None),
|
||||||
|
Customer.id.is_(None)
|
||||||
|
)
|
||||||
|
).all()
|
||||||
|
|
||||||
|
if not orphaned_jobs:
|
||||||
|
flash("No orphaned jobs found.", "info")
|
||||||
|
return redirect(url_for("main.settings", section="maintenance"))
|
||||||
|
|
||||||
|
job_count = len(orphaned_jobs)
|
||||||
|
mail_count = 0
|
||||||
|
run_count = 0
|
||||||
|
|
||||||
|
# Collect mail message ids and run ids for cleanup
|
||||||
|
mail_message_ids = []
|
||||||
|
run_ids = []
|
||||||
|
job_ids = [job.id for job in orphaned_jobs]
|
||||||
|
|
||||||
|
for job in orphaned_jobs:
|
||||||
|
for run in job.runs:
|
||||||
|
if run.id is not None:
|
||||||
|
run_ids.append(run.id)
|
||||||
|
run_count += 1
|
||||||
|
if run.mail_message_id:
|
||||||
|
mail_message_ids.append(run.mail_message_id)
|
||||||
|
|
||||||
|
# Helper function for safe SQL execution
|
||||||
|
def _safe_execute(stmt, params):
|
||||||
|
try:
|
||||||
|
db.session.execute(stmt, params)
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
|
||||||
|
# Clean up auxiliary tables that may not have ON DELETE CASCADE
|
||||||
|
if run_ids:
|
||||||
|
from sqlalchemy import text, bindparam
|
||||||
|
_safe_execute(
|
||||||
|
text("DELETE FROM ticket_job_runs WHERE job_run_id IN :run_ids").bindparams(
|
||||||
|
bindparam("run_ids", expanding=True)
|
||||||
|
),
|
||||||
|
{"run_ids": run_ids},
|
||||||
|
)
|
||||||
|
_safe_execute(
|
||||||
|
text("DELETE FROM remark_job_runs WHERE job_run_id IN :run_ids").bindparams(
|
||||||
|
bindparam("run_ids", expanding=True)
|
||||||
|
),
|
||||||
|
{"run_ids": run_ids},
|
||||||
|
)
|
||||||
|
|
||||||
|
if job_ids:
|
||||||
|
from sqlalchemy import text, bindparam
|
||||||
|
# Clean up scopes
|
||||||
|
_safe_execute(
|
||||||
|
text("DELETE FROM ticket_scopes WHERE job_id IN :job_ids").bindparams(
|
||||||
|
bindparam("job_ids", expanding=True)
|
||||||
|
),
|
||||||
|
{"job_ids": job_ids},
|
||||||
|
)
|
||||||
|
_safe_execute(
|
||||||
|
text("DELETE FROM remark_scopes WHERE job_id IN :job_ids").bindparams(
|
||||||
|
bindparam("job_ids", expanding=True)
|
||||||
|
),
|
||||||
|
{"job_ids": job_ids},
|
||||||
|
)
|
||||||
|
# Clean up overrides
|
||||||
|
_safe_execute(
|
||||||
|
text("DELETE FROM overrides WHERE job_id IN :job_ids").bindparams(
|
||||||
|
bindparam("job_ids", expanding=True)
|
||||||
|
),
|
||||||
|
{"job_ids": job_ids},
|
||||||
|
)
|
||||||
|
|
||||||
|
# Unlink mails from jobs before deleting jobs
|
||||||
|
# mail_messages.job_id references jobs.id
|
||||||
|
_safe_execute(
|
||||||
|
text("UPDATE mail_messages SET job_id = NULL WHERE job_id IN :job_ids").bindparams(
|
||||||
|
bindparam("job_ids", expanding=True)
|
||||||
|
),
|
||||||
|
{"job_ids": job_ids},
|
||||||
|
)
|
||||||
|
|
||||||
|
# Delete mail_objects before deleting mails
|
||||||
|
# mail_objects.mail_message_id references mail_messages.id
|
||||||
|
if mail_message_ids:
|
||||||
|
from sqlalchemy import text, bindparam
|
||||||
|
_safe_execute(
|
||||||
|
text("DELETE FROM mail_objects WHERE mail_message_id IN :mail_ids").bindparams(
|
||||||
|
bindparam("mail_ids", expanding=True)
|
||||||
|
),
|
||||||
|
{"mail_ids": mail_message_ids},
|
||||||
|
)
|
||||||
|
|
||||||
|
# Delete all orphaned jobs (runs/objects are cascaded via ORM relationships)
|
||||||
|
for job in orphaned_jobs:
|
||||||
|
db.session.delete(job)
|
||||||
|
|
||||||
|
# Now delete related mails permanently (customer is gone)
|
||||||
|
# This must happen AFTER deleting jobs/runs to avoid foreign key constraint violations
|
||||||
|
if mail_message_ids:
|
||||||
|
mail_count = len(mail_message_ids)
|
||||||
|
MailMessage.query.filter(MailMessage.id.in_(mail_message_ids)).delete(synchronize_session=False)
|
||||||
|
|
||||||
|
db.session.commit()
|
||||||
|
|
||||||
|
flash(
|
||||||
|
f"Deleted {job_count} orphaned job(s), {run_count} run(s), and {mail_count} email(s).",
|
||||||
|
"success"
|
||||||
|
)
|
||||||
|
|
||||||
|
_log_admin_event(
|
||||||
|
event_type="maintenance_delete_orphaned_jobs",
|
||||||
|
message=f"Deleted {job_count} orphaned jobs, {run_count} runs, and {mail_count} emails",
|
||||||
|
details=json.dumps({
|
||||||
|
"jobs_deleted": job_count,
|
||||||
|
"runs_deleted": run_count,
|
||||||
|
"mails_deleted": mail_count,
|
||||||
|
}),
|
||||||
|
)
|
||||||
|
|
||||||
|
except Exception as exc:
|
||||||
|
db.session.rollback()
|
||||||
|
print(f"[settings-jobs] Failed to delete orphaned jobs: {exc}")
|
||||||
|
flash("Failed to delete orphaned jobs.", "danger")
|
||||||
|
|
||||||
|
return redirect(url_for("main.settings", section="maintenance"))
|
||||||
|
|
||||||
|
|
||||||
|
@main_bp.route("/settings/test-emails/generate/<status_type>", methods=["POST"])
|
||||||
|
@login_required
|
||||||
|
@roles_required("admin")
|
||||||
|
def settings_generate_test_emails(status_type):
|
||||||
|
"""Generate test emails in inbox for testing parsers and orphaned jobs cleanup.
|
||||||
|
|
||||||
|
Fixed sets for consistent testing and reproducibility.
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
from datetime import datetime, timedelta
|
||||||
|
|
||||||
|
# Fixed test email sets per status type (Veeam only for consistent testing)
|
||||||
|
# All emails use the same job name "Test-Backup-Job" so they appear as different
|
||||||
|
# runs of the same job for proper status testing
|
||||||
|
email_sets = {
|
||||||
|
"success": [
|
||||||
|
{
|
||||||
|
"from_address": "veeam@test.local",
|
||||||
|
"subject": 'Veeam Backup Job "Test-Backup-Job" finished with Success',
|
||||||
|
"body": """Backup job: Test-Backup-Job
|
||||||
|
|
||||||
|
Session details:
|
||||||
|
Start time: 2026-02-09 01:00:00
|
||||||
|
End time: 2026-02-09 02:15:00
|
||||||
|
Total size: 150 GB
|
||||||
|
Duration: 01:15:00
|
||||||
|
|
||||||
|
Processing VM-APP01
|
||||||
|
Success
|
||||||
|
|
||||||
|
Processing VM-DB01
|
||||||
|
Success
|
||||||
|
|
||||||
|
Processing VM-WEB01
|
||||||
|
Success
|
||||||
|
|
||||||
|
All backup operations completed without issues.""",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"from_address": "veeam@test.local",
|
||||||
|
"subject": 'Veeam Backup Job "Test-Backup-Job" finished with Success',
|
||||||
|
"body": """Backup job: Test-Backup-Job
|
||||||
|
|
||||||
|
Session details:
|
||||||
|
Start time: 2026-02-08 01:00:00
|
||||||
|
End time: 2026-02-08 02:10:00
|
||||||
|
Total size: 145 GB
|
||||||
|
Duration: 01:10:00
|
||||||
|
|
||||||
|
Processing VM-APP01
|
||||||
|
Success
|
||||||
|
|
||||||
|
Processing VM-DB01
|
||||||
|
Success
|
||||||
|
|
||||||
|
Processing VM-WEB01
|
||||||
|
Success
|
||||||
|
|
||||||
|
All backup operations completed without issues.""",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"from_address": "veeam@test.local",
|
||||||
|
"subject": 'Veeam Backup Job "Test-Backup-Job" finished with Success',
|
||||||
|
"body": """Backup job: Test-Backup-Job
|
||||||
|
|
||||||
|
Session details:
|
||||||
|
Start time: 2026-02-07 01:00:00
|
||||||
|
End time: 2026-02-07 02:20:00
|
||||||
|
Total size: 152 GB
|
||||||
|
Duration: 01:20:00
|
||||||
|
|
||||||
|
Processing VM-APP01
|
||||||
|
Success
|
||||||
|
|
||||||
|
Processing VM-DB01
|
||||||
|
Success
|
||||||
|
|
||||||
|
Processing VM-WEB01
|
||||||
|
Success
|
||||||
|
|
||||||
|
All backup operations completed without issues.""",
|
||||||
|
},
|
||||||
|
],
|
||||||
|
"warning": [
|
||||||
|
{
|
||||||
|
"from_address": "veeam@test.local",
|
||||||
|
"subject": 'Veeam Backup Job "Test-Backup-Job" finished with WARNING',
|
||||||
|
"body": """Backup job: Test-Backup-Job
|
||||||
|
|
||||||
|
Session details:
|
||||||
|
Start time: 2026-02-09 01:00:00
|
||||||
|
End time: 2026-02-09 02:30:00
|
||||||
|
Total size: 148 GB
|
||||||
|
Duration: 01:30:00
|
||||||
|
|
||||||
|
Processing VM-APP01
|
||||||
|
Success
|
||||||
|
|
||||||
|
Processing VM-DB01
|
||||||
|
Warning
|
||||||
|
Warning: Low free space on target datastore
|
||||||
|
|
||||||
|
Processing VM-WEB01
|
||||||
|
Success
|
||||||
|
|
||||||
|
Backup completed but some files were skipped.""",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"from_address": "veeam@test.local",
|
||||||
|
"subject": 'Veeam Backup Job "Test-Backup-Job" finished with WARNING',
|
||||||
|
"body": """Backup job: Test-Backup-Job
|
||||||
|
|
||||||
|
Session details:
|
||||||
|
Start time: 2026-02-08 01:00:00
|
||||||
|
End time: 2026-02-08 02:25:00
|
||||||
|
Total size: 142 GB
|
||||||
|
Duration: 01:25:00
|
||||||
|
|
||||||
|
Processing VM-APP01
|
||||||
|
Warning
|
||||||
|
Warning: Retry was successful after initial failure
|
||||||
|
|
||||||
|
Processing VM-DB01
|
||||||
|
Success
|
||||||
|
|
||||||
|
Processing VM-WEB01
|
||||||
|
Success
|
||||||
|
|
||||||
|
Backup completed with warnings.""",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"from_address": "veeam@test.local",
|
||||||
|
"subject": 'Veeam Backup Job "Test-Backup-Job" finished with WARNING',
|
||||||
|
"body": """Backup job: Test-Backup-Job
|
||||||
|
|
||||||
|
Session details:
|
||||||
|
Start time: 2026-02-07 01:00:00
|
||||||
|
End time: 2026-02-07 02:35:00
|
||||||
|
Total size: 140 GB
|
||||||
|
Duration: 01:35:00
|
||||||
|
|
||||||
|
Processing VM-APP01
|
||||||
|
Success
|
||||||
|
|
||||||
|
Processing VM-DB01
|
||||||
|
Success
|
||||||
|
|
||||||
|
Processing VM-WEB01
|
||||||
|
Warning
|
||||||
|
Warning: Some files were locked and skipped
|
||||||
|
|
||||||
|
Backup completed with warnings.""",
|
||||||
|
},
|
||||||
|
],
|
||||||
|
"error": [
|
||||||
|
{
|
||||||
|
"from_address": "veeam@test.local",
|
||||||
|
"subject": 'Veeam Backup Job "Test-Backup-Job" finished with Failed',
|
||||||
|
"body": """Backup job: Test-Backup-Job
|
||||||
|
|
||||||
|
Session details:
|
||||||
|
Start time: 2026-02-09 01:00:00
|
||||||
|
End time: 2026-02-09 01:15:00
|
||||||
|
Total size: 0 GB
|
||||||
|
Duration: 00:15:00
|
||||||
|
|
||||||
|
Processing VM-APP01
|
||||||
|
Failed
|
||||||
|
Error: Cannot create snapshot: VSS error 0x800423f4
|
||||||
|
|
||||||
|
Processing VM-DB01
|
||||||
|
Success
|
||||||
|
|
||||||
|
Processing VM-WEB01
|
||||||
|
Success
|
||||||
|
|
||||||
|
Backup failed. Please check the logs for details.""",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"from_address": "veeam@test.local",
|
||||||
|
"subject": 'Veeam Backup Job "Test-Backup-Job" finished with Failed',
|
||||||
|
"body": """Backup job: Test-Backup-Job
|
||||||
|
|
||||||
|
Session details:
|
||||||
|
Start time: 2026-02-08 01:00:00
|
||||||
|
End time: 2026-02-08 01:10:00
|
||||||
|
Total size: 0 GB
|
||||||
|
Duration: 00:10:00
|
||||||
|
|
||||||
|
Processing VM-APP01
|
||||||
|
Success
|
||||||
|
|
||||||
|
Processing VM-DB01
|
||||||
|
Failed
|
||||||
|
Error: Disk space exhausted on backup repository
|
||||||
|
|
||||||
|
Processing VM-WEB01
|
||||||
|
Success
|
||||||
|
|
||||||
|
Backup failed due to storage issue.""",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"from_address": "veeam@test.local",
|
||||||
|
"subject": 'Veeam Backup Job "Test-Backup-Job" finished with Failed',
|
||||||
|
"body": """Backup job: Test-Backup-Job
|
||||||
|
|
||||||
|
Session details:
|
||||||
|
Start time: 2026-02-07 01:00:00
|
||||||
|
End time: 2026-02-07 01:05:00
|
||||||
|
Total size: 0 GB
|
||||||
|
Duration: 00:05:00
|
||||||
|
|
||||||
|
Processing VM-APP01
|
||||||
|
Success
|
||||||
|
|
||||||
|
Processing VM-DB01
|
||||||
|
Success
|
||||||
|
|
||||||
|
Processing VM-WEB01
|
||||||
|
Failed
|
||||||
|
Error: Network connection lost to ESXi host
|
||||||
|
|
||||||
|
Backup failed. Network issue detected.""",
|
||||||
|
},
|
||||||
|
],
|
||||||
|
}
|
||||||
|
|
||||||
|
if status_type not in email_sets:
|
||||||
|
flash("Invalid status type.", "danger")
|
||||||
|
return redirect(url_for("main.settings", section="maintenance"))
|
||||||
|
|
||||||
|
emails = email_sets[status_type]
|
||||||
|
created_count = 0
|
||||||
|
now = datetime.utcnow()
|
||||||
|
|
||||||
|
for email_data in emails:
|
||||||
|
mail = MailMessage(
|
||||||
|
from_address=email_data["from_address"],
|
||||||
|
subject=email_data["subject"],
|
||||||
|
text_body=email_data["body"],
|
||||||
|
html_body=f"<pre>{email_data['body']}</pre>",
|
||||||
|
received_at=now - timedelta(hours=created_count),
|
||||||
|
location="inbox",
|
||||||
|
job_id=None,
|
||||||
|
)
|
||||||
|
db.session.add(mail)
|
||||||
|
created_count += 1
|
||||||
|
|
||||||
|
db.session.commit()
|
||||||
|
|
||||||
|
flash(f"Generated {created_count} {status_type} test email(s) in inbox.", "success")
|
||||||
|
|
||||||
|
_log_admin_event(
|
||||||
|
event_type="maintenance_generate_test_emails",
|
||||||
|
message=f"Generated {created_count} {status_type} test emails",
|
||||||
|
details=json.dumps({"status_type": status_type, "count": created_count}),
|
||||||
|
)
|
||||||
|
|
||||||
|
except Exception as exc:
|
||||||
|
db.session.rollback()
|
||||||
|
print(f"[settings-test] Failed to generate test emails: {exc}")
|
||||||
|
flash("Failed to generate test emails.", "danger")
|
||||||
|
|
||||||
|
return redirect(url_for("main.settings", section="maintenance"))
|
||||||
|
|
||||||
|
|
||||||
@main_bp.route("/settings/objects/backfill", methods=["POST"])
|
@main_bp.route("/settings/objects/backfill", methods=["POST"])
|
||||||
@login_required
|
@login_required
|
||||||
@roles_required("admin")
|
@roles_required("admin")
|
||||||
|
|||||||
@ -50,13 +50,13 @@ PARSER_DEFINITIONS = [
|
|||||||
},
|
},
|
||||||
"description": "Parses NTFS Auditing file audit report mails (attachment-based HTML reports).",
|
"description": "Parses NTFS Auditing file audit report mails (attachment-based HTML reports).",
|
||||||
"example": {
|
"example": {
|
||||||
"subject": "Bouter btr-dc001.bouter.nl file audits → 6 ↑ 12",
|
"subject": "SERVER-HOSTNAME file audits → 6 ↑ 12",
|
||||||
"from_address": "auditing@bouter.nl",
|
"from_address": "auditing@example.local",
|
||||||
"body_snippet": "(empty body, HTML report in attachment)",
|
"body_snippet": "(empty body, HTML report in attachment)",
|
||||||
"parsed_result": {
|
"parsed_result": {
|
||||||
"backup_software": "NTFS Auditing",
|
"backup_software": "NTFS Auditing",
|
||||||
"backup_type": "Audit",
|
"backup_type": "Audit",
|
||||||
"job_name": "btr-dc001.bouter.nl file audits",
|
"job_name": "SERVER-HOSTNAME file audits",
|
||||||
"objects": [],
|
"objects": [],
|
||||||
},
|
},
|
||||||
},
|
},
|
||||||
@ -73,16 +73,68 @@ PARSER_DEFINITIONS = [
|
|||||||
},
|
},
|
||||||
"description": "Parses QNAP Notification Center firmware update notifications (informational; excluded from reporting and missing logic).",
|
"description": "Parses QNAP Notification Center firmware update notifications (informational; excluded from reporting and missing logic).",
|
||||||
"example": {
|
"example": {
|
||||||
"subject": "[Info][Firmware Update] Notification from your device: BETSIES-NAS01",
|
"subject": "[Info][Firmware Update] Notification from your device: NAS-HOSTNAME",
|
||||||
"from_address": "notifications@customer.tld",
|
"from_address": "notifications@customer.tld",
|
||||||
"body_snippet": "NAS Name: BETSIES-NAS01\n...\nMessage: ...",
|
"body_snippet": "NAS Name: NAS-HOSTNAME\n...\nMessage: ...",
|
||||||
"parsed_result": {
|
"parsed_result": {
|
||||||
"backup_software": "QNAP",
|
"backup_software": "QNAP",
|
||||||
"backup_type": "Firmware Update",
|
"backup_type": "Firmware Update",
|
||||||
"job_name": "Firmware Update",
|
"job_name": "Firmware Update",
|
||||||
"overall_status": "Warning",
|
"overall_status": "Warning",
|
||||||
"objects": [
|
"objects": [
|
||||||
{"name": "BETSIES-NAS01", "status": "Warning", "error_message": None}
|
{"name": "NAS-HOSTNAME", "status": "Warning", "error_message": None}
|
||||||
|
],
|
||||||
|
},
|
||||||
|
},
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "synology_dsm_update",
|
||||||
|
"backup_software": "Synology",
|
||||||
|
"backup_types": ["Updates"],
|
||||||
|
"order": 236,
|
||||||
|
"enabled": True,
|
||||||
|
"match": {
|
||||||
|
"subject_contains_any": ["DSM-update", "DSM update"],
|
||||||
|
"body_contains_any": ["automatische DSM-update", "automatic DSM update", "Automatic update of DSM"],
|
||||||
|
},
|
||||||
|
"description": "Parses Synology DSM automatic update cancelled notifications (informational; excluded from reporting and missing logic).",
|
||||||
|
"example": {
|
||||||
|
"subject": "Synology NAS-HOSTNAME - Automatische DSM-update op NAS-HOSTNAME is geannuleerd door het systeem",
|
||||||
|
"from_address": "backup@example.local",
|
||||||
|
"body_snippet": "Het systeem heeft de automatische DSM-update op NAS-HOSTNAME geannuleerd...",
|
||||||
|
"parsed_result": {
|
||||||
|
"backup_software": "Synology",
|
||||||
|
"backup_type": "Updates",
|
||||||
|
"job_name": "Synology Automatic Update",
|
||||||
|
"overall_status": "Warning",
|
||||||
|
"objects": [
|
||||||
|
{"name": "NAS-HOSTNAME", "status": "Warning"}
|
||||||
|
],
|
||||||
|
},
|
||||||
|
},
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "synology_drive_health",
|
||||||
|
"backup_software": "Synology",
|
||||||
|
"backup_types": ["Health Report"],
|
||||||
|
"order": 237,
|
||||||
|
"enabled": True,
|
||||||
|
"match": {
|
||||||
|
"subject_contains_any": ["schijfintegriteitsrapport", "Drive Health Report"],
|
||||||
|
"body_contains_any": ["health of the drives", "integriteitsrapport van de schijven"],
|
||||||
|
},
|
||||||
|
"description": "Parses Synology monthly drive health reports (informational; excluded from reporting and missing logic).",
|
||||||
|
"example": {
|
||||||
|
"subject": "[NAS-HOSTNAME] Monthly Drive Health Report on NAS-HOSTNAME - Healthy",
|
||||||
|
"from_address": "nas@example.local",
|
||||||
|
"body_snippet": "The following is your monthly report regarding the health of the drives on NAS-HOSTNAME. No problem detected with the drives in DSM.",
|
||||||
|
"parsed_result": {
|
||||||
|
"backup_software": "Synology",
|
||||||
|
"backup_type": "Health Report",
|
||||||
|
"job_name": "Monthly Drive Health",
|
||||||
|
"overall_status": "Success",
|
||||||
|
"objects": [
|
||||||
|
{"name": "NAS-HOSTNAME", "status": "Success"}
|
||||||
],
|
],
|
||||||
},
|
},
|
||||||
},
|
},
|
||||||
@ -383,16 +435,16 @@ PARSER_DEFINITIONS = [
|
|||||||
},
|
},
|
||||||
"description": "Parses NAKIVO Backup & Replication reports for VMware backup jobs.",
|
"description": "Parses NAKIVO Backup & Replication reports for VMware backup jobs.",
|
||||||
"example": {
|
"example": {
|
||||||
"subject": '"exchange01.kuiperbv.nl" job: Successful',
|
"subject": '"VM-HOSTNAME" job: Successful',
|
||||||
"from_address": "NAKIVO Backup & Replication <administrator@customer.local>",
|
"from_address": "NAKIVO Backup & Replication <administrator@customer.local>",
|
||||||
"body_snippet": "Job Run Report... Backup job for VMware ... Successful",
|
"body_snippet": "Job Run Report... Backup job for VMware ... Successful",
|
||||||
"parsed_result": {
|
"parsed_result": {
|
||||||
"backup_software": "NAKIVO",
|
"backup_software": "NAKIVO",
|
||||||
"backup_type": "Backup job for VMware",
|
"backup_type": "Backup job for VMware",
|
||||||
"job_name": "exchange01.kuiperbv.nl",
|
"job_name": "VM-HOSTNAME",
|
||||||
"objects": [
|
"objects": [
|
||||||
{
|
{
|
||||||
"name": "exchange01.kuiperbv.nl",
|
"name": "VM-HOSTNAME",
|
||||||
"status": "Success",
|
"status": "Success",
|
||||||
"error_message": "",
|
"error_message": "",
|
||||||
}
|
}
|
||||||
|
|||||||
@ -18,10 +18,23 @@ DSM_UPDATE_CANCELLED_PATTERNS = [
|
|||||||
"Automatische update van DSM is geannuleerd",
|
"Automatische update van DSM is geannuleerd",
|
||||||
"Automatic DSM update was cancelled",
|
"Automatic DSM update was cancelled",
|
||||||
"Automatic update of DSM was cancelled",
|
"Automatic update of DSM was cancelled",
|
||||||
|
"Automatische DSM-update",
|
||||||
|
"DSM-update op",
|
||||||
|
"Packages on",
|
||||||
|
"out-of-date",
|
||||||
|
"Package Center",
|
||||||
|
"new DSM update",
|
||||||
|
"Auto Update has detected",
|
||||||
|
"new version of DSM",
|
||||||
|
"Update & Restore",
|
||||||
|
"belangrijke DSM-update",
|
||||||
|
"kritieke oplossingen",
|
||||||
|
"wordt automatisch geïnstalleerd",
|
||||||
|
"is beschikbaar op",
|
||||||
]
|
]
|
||||||
|
|
||||||
_DSM_UPDATE_CANCELLED_HOST_RE = re.compile(
|
_DSM_UPDATE_CANCELLED_HOST_RE = re.compile(
|
||||||
r"\b(?:geannuleerd\s+op|cancelled\s+on)\s+(?P<host>[A-Za-z0-9._-]+)\b",
|
r"\b(?:geannuleerd\s+op|cancelled\s+on|DSM-update\s+op|DSM\s+update\s+on|Packages\s+on|running\s+on|detected\s+on)\s+(?P<host>[A-Za-z0-9._-]+)\b",
|
||||||
re.I,
|
re.I,
|
||||||
)
|
)
|
||||||
|
|
||||||
@ -60,6 +73,75 @@ def _parse_synology_dsm_update_cancelled(subject: str, text: str) -> Tuple[bool,
|
|||||||
|
|
||||||
return True, result, objects
|
return True, result, objects
|
||||||
|
|
||||||
|
|
||||||
|
# --- Synology Drive Health Report (informational, excluded from reporting) ---
|
||||||
|
|
||||||
|
DRIVE_HEALTH_PATTERNS = [
|
||||||
|
"schijfintegriteitsrapport",
|
||||||
|
"Drive Health Report",
|
||||||
|
"Monthly Drive Health",
|
||||||
|
"health of the drives",
|
||||||
|
"integriteitsrapport van de schijven",
|
||||||
|
]
|
||||||
|
|
||||||
|
_DRIVE_HEALTH_SUBJECT_RE = re.compile(
|
||||||
|
r"\b(?:schijfintegriteitsrapport\s+over|Drive\s+Health\s+Report\s+on)\s+(?P<host>[A-Za-z0-9._-]+)",
|
||||||
|
re.I,
|
||||||
|
)
|
||||||
|
|
||||||
|
_DRIVE_HEALTH_FROM_RE = re.compile(r"\b(?:Van|From)\s+(?P<host>[A-Za-z0-9._-]+)\b", re.I)
|
||||||
|
|
||||||
|
_DRIVE_HEALTH_STATUS_HEALTHY_RE = re.compile(
|
||||||
|
r"\b(?:Gezond|Healthy|geen\s+problemen\s+gedetecteerd|No\s+problem\s+detected)\b",
|
||||||
|
re.I,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _is_synology_drive_health(subject: str, text: str) -> bool:
|
||||||
|
haystack = f"{subject}\n{text}".lower()
|
||||||
|
return any(p.lower() in haystack for p in DRIVE_HEALTH_PATTERNS)
|
||||||
|
|
||||||
|
|
||||||
|
def _parse_synology_drive_health(subject: str, text: str) -> Tuple[bool, Dict, List[Dict]]:
|
||||||
|
haystack = f"{subject}\n{text}"
|
||||||
|
host = ""
|
||||||
|
|
||||||
|
# Try to extract hostname from subject first
|
||||||
|
m = _DRIVE_HEALTH_SUBJECT_RE.search(subject or "")
|
||||||
|
if m:
|
||||||
|
host = (m.group("host") or "").strip()
|
||||||
|
|
||||||
|
# Fallback: extract from body "Van/From NAS-NAME"
|
||||||
|
if not host:
|
||||||
|
m = _DRIVE_HEALTH_FROM_RE.search(text or "")
|
||||||
|
if m:
|
||||||
|
host = (m.group("host") or "").strip()
|
||||||
|
|
||||||
|
# Determine status based on health indicators
|
||||||
|
overall_status = "Success"
|
||||||
|
overall_message = "Healthy"
|
||||||
|
|
||||||
|
if not _DRIVE_HEALTH_STATUS_HEALTHY_RE.search(haystack):
|
||||||
|
# If we don't find healthy indicators, mark as Warning
|
||||||
|
overall_status = "Warning"
|
||||||
|
overall_message = "Drive health issue detected"
|
||||||
|
|
||||||
|
# Informational job: show in Run Checks, but do not participate in schedules / reporting.
|
||||||
|
result: Dict = {
|
||||||
|
"backup_software": "Synology",
|
||||||
|
"backup_type": "Health Report",
|
||||||
|
"job_name": "Monthly Drive Health",
|
||||||
|
"overall_status": overall_status,
|
||||||
|
"overall_message": overall_message + (f" ({host})" if host else ""),
|
||||||
|
}
|
||||||
|
|
||||||
|
objects: List[Dict] = []
|
||||||
|
if host:
|
||||||
|
objects.append({"name": host, "status": overall_status})
|
||||||
|
|
||||||
|
return True, result, objects
|
||||||
|
|
||||||
|
|
||||||
_BR_RE = re.compile(r"<\s*br\s*/?\s*>", re.I)
|
_BR_RE = re.compile(r"<\s*br\s*/?\s*>", re.I)
|
||||||
_TAG_RE = re.compile(r"<[^>]+>")
|
_TAG_RE = re.compile(r"<[^>]+>")
|
||||||
_WS_RE = re.compile(r"[\t\r\f\v ]+")
|
_WS_RE = re.compile(r"[\t\r\f\v ]+")
|
||||||
@ -176,12 +258,14 @@ _ABB_SUBJECT_RE = re.compile(r"\bactive\s+backup\s+for\s+business\b", re.I)
|
|||||||
# Examples (NL):
|
# Examples (NL):
|
||||||
# "De back-uptaak vSphere-Task-1 op KANTOOR-NEW is voltooid."
|
# "De back-uptaak vSphere-Task-1 op KANTOOR-NEW is voltooid."
|
||||||
# "Virtuele machine back-uptaak vSphere-Task-1 op KANTOOR-NEW is gedeeltelijk voltooid."
|
# "Virtuele machine back-uptaak vSphere-Task-1 op KANTOOR-NEW is gedeeltelijk voltooid."
|
||||||
|
# "back-uptaak vSphere-Task-1 op KANTOOR-NEW is genegeerd"
|
||||||
# Examples (EN):
|
# Examples (EN):
|
||||||
# "The backup task vSphere-Task-1 on KANTOOR-NEW has completed."
|
# "The backup task vSphere-Task-1 on KANTOOR-NEW has completed."
|
||||||
# "Virtual machine backup task vSphere-Task-1 on KANTOOR-NEW partially completed."
|
# "Virtual machine backup task vSphere-Task-1 on KANTOOR-NEW partially completed."
|
||||||
|
# "backup task vSphere-Task-1 on KANTOOR-NEW was skipped"
|
||||||
_ABB_COMPLETED_RE = re.compile(
|
_ABB_COMPLETED_RE = re.compile(
|
||||||
r"\b(?:virtuele\s+machine\s+)?(?:de\s+)?back-?up\s*taak\s+(?P<job>.+?)\s+op\s+(?P<host>[A-Za-z0-9._-]+)\s+is\s+(?P<status>voltooid|gedeeltelijk\s+voltooid)\b"
|
r"\b(?:virtuele\s+machine\s+)?(?:de\s+)?back-?up\s*(?:taak|job)\s+(?:van\s+deze\s+taak\s+)?(?P<job>.+?)\s+op\s+(?P<host>[A-Za-z0-9._-]+)\s+is\s+(?P<status>voltooid|gedeeltelijk\s+voltooid|genegeerd)\b"
|
||||||
r"|\b(?:virtual\s+machine\s+)?(?:the\s+)?back-?up\s+task\s+(?P<job_en>.+?)\s+on\s+(?P<host_en>[A-Za-z0-9._-]+)\s+(?:is\s+)?(?P<status_en>completed|finished|has\s+completed|partially\s+completed)\b",
|
r"|\b(?:virtual\s+machine\s+)?(?:the\s+)?back-?up\s+(?:task|job)\s+(?P<job_en>.+?)\s+on\s+(?P<host_en>[A-Za-z0-9._-]+)\s+(?:is\s+|was\s+)?(?P<status_en>completed|finished|has\s+completed|partially\s+completed|skipped|ignored)\b",
|
||||||
re.I,
|
re.I,
|
||||||
)
|
)
|
||||||
|
|
||||||
@ -233,6 +317,11 @@ def _parse_active_backup_for_business(subject: str, text: str) -> Tuple[bool, Di
|
|||||||
overall_status = "Warning"
|
overall_status = "Warning"
|
||||||
overall_message = "Partially completed"
|
overall_message = "Partially completed"
|
||||||
|
|
||||||
|
# "genegeerd" / "skipped" / "ignored" should be treated as Warning
|
||||||
|
if "genegeerd" in status_raw or "skipped" in status_raw or "ignored" in status_raw:
|
||||||
|
overall_status = "Warning"
|
||||||
|
overall_message = "Skipped"
|
||||||
|
|
||||||
# Explicit failure wording overrides everything
|
# Explicit failure wording overrides everything
|
||||||
if _ABB_FAILED_RE.search(haystack):
|
if _ABB_FAILED_RE.search(haystack):
|
||||||
overall_status = "Error"
|
overall_status = "Error"
|
||||||
@ -489,6 +578,12 @@ def try_parse_synology(msg: MailMessage) -> Tuple[bool, Dict, List[Dict]]:
|
|||||||
if ok:
|
if ok:
|
||||||
return True, result, objects
|
return True, result, objects
|
||||||
|
|
||||||
|
# Drive Health Report (informational; no schedule; excluded from reporting)
|
||||||
|
if _is_synology_drive_health(subject, text):
|
||||||
|
ok, result, objects = _parse_synology_drive_health(subject, text)
|
||||||
|
if ok:
|
||||||
|
return True, result, objects
|
||||||
|
|
||||||
# DSM Account Protection (informational; no schedule)
|
# DSM Account Protection (informational; no schedule)
|
||||||
if _is_synology_account_protection(subject, text):
|
if _is_synology_account_protection(subject, text):
|
||||||
ok, result, objects = _parse_account_protection(subject, text)
|
ok, result, objects = _parse_account_protection(subject, text)
|
||||||
|
|||||||
@ -1177,6 +1177,38 @@ def try_parse_veeam(msg: MailMessage) -> Tuple[bool, Dict, List[Dict]]:
|
|||||||
}
|
}
|
||||||
return True, result, []
|
return True, result, []
|
||||||
|
|
||||||
|
# Job did not start on schedule: special error notification (no objects, plain text body).
|
||||||
|
# Example subject: "[Veeam Backup for Microsoft 365] Job did not start on schedule: Backup MDS at Work"
|
||||||
|
subject_lower = subject.lower()
|
||||||
|
if 'job did not start on schedule' in subject_lower:
|
||||||
|
# Extract backup type from subject (e.g., "Veeam Backup for Microsoft 365")
|
||||||
|
backup_type = None
|
||||||
|
for candidate in VEEAM_BACKUP_TYPES:
|
||||||
|
if candidate.lower() in subject_lower:
|
||||||
|
backup_type = candidate
|
||||||
|
break
|
||||||
|
if not backup_type:
|
||||||
|
backup_type = "Backup Job"
|
||||||
|
|
||||||
|
# Extract job name after the colon (e.g., "Backup MDS at Work")
|
||||||
|
job_name = None
|
||||||
|
m_job = re.search(r'job did not start on schedule:\s*(.+)$', subject, re.IGNORECASE)
|
||||||
|
if m_job:
|
||||||
|
job_name = (m_job.group(1) or '').strip()
|
||||||
|
|
||||||
|
# Get overall message from text_body (can be base64 encoded)
|
||||||
|
text_body = (getattr(msg, 'text_body', None) or '').strip()
|
||||||
|
overall_message = text_body if text_body else 'Job did not start on schedule'
|
||||||
|
|
||||||
|
result = {
|
||||||
|
'backup_software': 'Veeam',
|
||||||
|
'backup_type': backup_type,
|
||||||
|
'job_name': job_name or 'Unknown Job',
|
||||||
|
'overall_status': 'Error',
|
||||||
|
'overall_message': overall_message,
|
||||||
|
}
|
||||||
|
return True, result, []
|
||||||
|
|
||||||
# Configuration Job detection (may not have object details)
|
# Configuration Job detection (may not have object details)
|
||||||
subj_lower = subject.lower()
|
subj_lower = subject.lower()
|
||||||
is_config_job = ('backup configuration job' in subj_lower) or ('configuration backup for' in html_lower)
|
is_config_job = ('backup configuration job' in subj_lower) or ('configuration backup for' in html_lower)
|
||||||
|
|||||||
@ -197,7 +197,7 @@
|
|||||||
</div>
|
</div>
|
||||||
</nav>
|
</nav>
|
||||||
|
|
||||||
<main class="{% block main_class %}container content-container{% endblock %}" style="padding-top: 80px;">
|
<main class="{% block main_class %}container content-container{% endblock %}" id="main-content">
|
||||||
{% with messages = get_flashed_messages(with_categories=true) %}
|
{% with messages = get_flashed_messages(with_categories=true) %}
|
||||||
{% if messages %}
|
{% if messages %}
|
||||||
<div class="mb-3">
|
<div class="mb-3">
|
||||||
@ -216,6 +216,58 @@
|
|||||||
|
|
||||||
<script src="https://cdn.jsdelivr.net/npm/bootstrap@5.3.3/dist/js/bootstrap.bundle.min.js"></script>
|
<script src="https://cdn.jsdelivr.net/npm/bootstrap@5.3.3/dist/js/bootstrap.bundle.min.js"></script>
|
||||||
|
|
||||||
|
<script>
|
||||||
|
// Dynamic navbar height adjustment
|
||||||
|
(function () {
|
||||||
|
function adjustContentPadding() {
|
||||||
|
try {
|
||||||
|
var navbar = document.querySelector('.navbar.fixed-top');
|
||||||
|
var mainContent = document.getElementById('main-content');
|
||||||
|
if (!navbar || !mainContent) return;
|
||||||
|
|
||||||
|
// Get actual navbar height
|
||||||
|
var navbarHeight = navbar.offsetHeight;
|
||||||
|
|
||||||
|
// Add small buffer (20px) for visual spacing
|
||||||
|
var paddingTop = navbarHeight + 20;
|
||||||
|
|
||||||
|
// Apply padding to main content
|
||||||
|
mainContent.style.paddingTop = paddingTop + 'px';
|
||||||
|
} catch (e) {
|
||||||
|
// Fallback to 80px if something goes wrong
|
||||||
|
var mainContent = document.getElementById('main-content');
|
||||||
|
if (mainContent) {
|
||||||
|
mainContent.style.paddingTop = '80px';
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Run on page load
|
||||||
|
if (document.readyState === 'loading') {
|
||||||
|
document.addEventListener('DOMContentLoaded', adjustContentPadding);
|
||||||
|
} else {
|
||||||
|
adjustContentPadding();
|
||||||
|
}
|
||||||
|
|
||||||
|
// Run after navbar is fully rendered
|
||||||
|
window.addEventListener('load', adjustContentPadding);
|
||||||
|
|
||||||
|
// Run on window resize
|
||||||
|
var resizeTimeout;
|
||||||
|
window.addEventListener('resize', function () {
|
||||||
|
clearTimeout(resizeTimeout);
|
||||||
|
resizeTimeout = setTimeout(adjustContentPadding, 100);
|
||||||
|
});
|
||||||
|
|
||||||
|
// Run when navbar collapse is toggled
|
||||||
|
var navbarCollapse = document.getElementById('navbarNav');
|
||||||
|
if (navbarCollapse) {
|
||||||
|
navbarCollapse.addEventListener('shown.bs.collapse', adjustContentPadding);
|
||||||
|
navbarCollapse.addEventListener('hidden.bs.collapse', adjustContentPadding);
|
||||||
|
}
|
||||||
|
})();
|
||||||
|
</script>
|
||||||
|
|
||||||
<script>
|
<script>
|
||||||
(function () {
|
(function () {
|
||||||
function isOverflowing(el) {
|
function isOverflowing(el) {
|
||||||
|
|||||||
@ -549,6 +549,36 @@
|
|||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
|
|
||||||
|
<div class="col-12 col-lg-6">
|
||||||
|
<div class="card h-100 border-warning">
|
||||||
|
<div class="card-header bg-warning">Cleanup orphaned jobs</div>
|
||||||
|
<div class="card-body">
|
||||||
|
<p class="mb-3">Delete jobs that are no longer linked to an existing customer. Related emails and runs will be <strong>permanently deleted</strong> from the database.</p>
|
||||||
|
<a href="{{ url_for('main.settings_jobs_orphaned') }}" class="btn btn-warning">Preview orphaned jobs</a>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div class="col-12 col-lg-6">
|
||||||
|
<div class="card h-100 border-info">
|
||||||
|
<div class="card-header bg-info text-white">Generate test emails</div>
|
||||||
|
<div class="card-body">
|
||||||
|
<p class="mb-3">Generate fixed test email sets in the inbox for testing parsers and maintenance operations. Each set contains 3 emails simulating Veeam, Synology, and NAKIVO backups.</p>
|
||||||
|
<div class="d-flex flex-column gap-2">
|
||||||
|
<form method="post" action="{{ url_for('main.settings_generate_test_emails', status_type='success') }}">
|
||||||
|
<button type="submit" class="btn btn-success w-100">Generate success emails (3)</button>
|
||||||
|
</form>
|
||||||
|
<form method="post" action="{{ url_for('main.settings_generate_test_emails', status_type='warning') }}">
|
||||||
|
<button type="submit" class="btn btn-warning w-100">Generate warning emails (3)</button>
|
||||||
|
</form>
|
||||||
|
<form method="post" action="{{ url_for('main.settings_generate_test_emails', status_type='error') }}">
|
||||||
|
<button type="submit" class="btn btn-danger w-100">Generate error emails (3)</button>
|
||||||
|
</form>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
<div class="col-12 col-lg-6">
|
<div class="col-12 col-lg-6">
|
||||||
<div class="card h-100 border-danger">
|
<div class="card h-100 border-danger">
|
||||||
<div class="card-header bg-danger text-white">Jobs maintenance</div>
|
<div class="card-header bg-danger text-white">Jobs maintenance</div>
|
||||||
|
|||||||
@ -0,0 +1,87 @@
|
|||||||
|
{% extends "layout/base.html" %}
|
||||||
|
|
||||||
|
{% block title %}Orphaned Jobs Preview{% endblock %}
|
||||||
|
|
||||||
|
{% block content %}
|
||||||
|
<div class="container-fluid py-4">
|
||||||
|
<div class="d-flex justify-content-between align-items-center mb-4">
|
||||||
|
<div>
|
||||||
|
<h2>Orphaned Jobs Preview</h2>
|
||||||
|
<p class="text-muted mb-0">Jobs without a valid customer link</p>
|
||||||
|
</div>
|
||||||
|
<a href="{{ url_for('main.settings', section='maintenance') }}" class="btn btn-outline-secondary">Back to Settings</a>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
{% if orphaned_jobs %}
|
||||||
|
<div class="alert alert-warning">
|
||||||
|
<strong>⚠️ Warning:</strong> Found {{ orphaned_jobs|length }} orphaned job(s). Review the list below before deleting.
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div class="card mb-4">
|
||||||
|
<div class="card-header d-flex justify-content-between align-items-center">
|
||||||
|
<span>Orphaned Jobs List</span>
|
||||||
|
<form method="post" action="{{ url_for('main.settings_jobs_delete_orphaned') }}" onsubmit="return confirm('Delete all {{ orphaned_jobs|length }} orphaned jobs and their emails? This cannot be undone.');">
|
||||||
|
<button type="submit" class="btn btn-sm btn-danger">Delete All ({{ orphaned_jobs|length }} jobs)</button>
|
||||||
|
</form>
|
||||||
|
</div>
|
||||||
|
<div class="card-body p-0">
|
||||||
|
<div class="table-responsive">
|
||||||
|
<table class="table table-hover mb-0">
|
||||||
|
<thead>
|
||||||
|
<tr>
|
||||||
|
<th>Job Name</th>
|
||||||
|
<th>Backup Software</th>
|
||||||
|
<th>Backup Type</th>
|
||||||
|
<th>Customer ID</th>
|
||||||
|
<th class="text-end">Runs</th>
|
||||||
|
<th class="text-end">Emails</th>
|
||||||
|
</tr>
|
||||||
|
</thead>
|
||||||
|
<tbody>
|
||||||
|
{% for job in orphaned_jobs %}
|
||||||
|
<tr>
|
||||||
|
<td>{{ job.job_name }}</td>
|
||||||
|
<td>{{ job.backup_software }}</td>
|
||||||
|
<td>{{ job.backup_type }}</td>
|
||||||
|
<td>
|
||||||
|
{% if job.customer_id %}
|
||||||
|
<span class="badge bg-danger">{{ job.customer_id }} (deleted)</span>
|
||||||
|
{% else %}
|
||||||
|
<span class="badge bg-secondary">NULL</span>
|
||||||
|
{% endif %}
|
||||||
|
</td>
|
||||||
|
<td class="text-end">{{ job.run_count }}</td>
|
||||||
|
<td class="text-end">{{ job.mail_count }}</td>
|
||||||
|
</tr>
|
||||||
|
{% endfor %}
|
||||||
|
</tbody>
|
||||||
|
<tfoot>
|
||||||
|
<tr class="table-light">
|
||||||
|
<td colspan="4"><strong>Total</strong></td>
|
||||||
|
<td class="text-end"><strong>{{ orphaned_jobs|sum(attribute='run_count') }}</strong></td>
|
||||||
|
<td class="text-end"><strong>{{ orphaned_jobs|sum(attribute='mail_count') }}</strong></td>
|
||||||
|
</tr>
|
||||||
|
</tfoot>
|
||||||
|
</table>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div class="alert alert-info">
|
||||||
|
<strong>ℹ️ What will be deleted:</strong>
|
||||||
|
<ul class="mb-0">
|
||||||
|
<li>{{ orphaned_jobs|length }} job(s)</li>
|
||||||
|
<li>{{ orphaned_jobs|sum(attribute='run_count') }} job run(s)</li>
|
||||||
|
<li>{{ orphaned_jobs|sum(attribute='mail_count') }} email(s)</li>
|
||||||
|
<li>All related data (backup objects, ticket/remark links, scopes, overrides)</li>
|
||||||
|
</ul>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
{% else %}
|
||||||
|
<div class="alert alert-success">
|
||||||
|
<strong>✅ No orphaned jobs found.</strong>
|
||||||
|
<p class="mb-0">All jobs are properly linked to existing customers.</p>
|
||||||
|
</div>
|
||||||
|
{% endif %}
|
||||||
|
</div>
|
||||||
|
{% endblock %}
|
||||||
@ -4,8 +4,25 @@ This file documents all changes made to this project via Claude Code.
|
|||||||
|
|
||||||
## [2026-02-09]
|
## [2026-02-09]
|
||||||
|
|
||||||
|
### Added
|
||||||
|
- Extended Veeam parser to recognize "Job did not start on schedule" error notifications for Veeam Backup for Microsoft 365 (and other Veeam backup types) with job name extraction from subject and error message from plain text body (proxy server offline, scheduled run failed)
|
||||||
|
- Added parser for Synology monthly drive health reports (backup software: Synology, backup type: Health Report, job name: Monthly Drive Health, informational only, no schedule learning) with support for both Dutch and English notifications ("schijfintegriteitsrapport"/"Drive Health Report") and automatic status detection (Healthy/Gezond → Success, problems → Warning)
|
||||||
|
- Added "Cleanup orphaned jobs" maintenance option in Settings → Maintenance to delete jobs without valid customer links and their associated emails/runs permanently from database (useful when customers are removed)
|
||||||
|
- Added "Preview orphaned jobs" button to show detailed list of jobs to be deleted with run/email counts before confirming deletion (verification step for safety)
|
||||||
|
- Added "Generate test emails" feature in Settings → Maintenance with three separate buttons to create fixed test email sets (success/warning/error) in inbox for testing parsers and maintenance operations (each set contains exactly 3 Veeam Backup Job emails with the same job name "Test-Backup-Job" and different dates/objects/statuses for reproducible testing and proper status flow testing)
|
||||||
|
- Added parser registry entry for Synology DSM automatic update cancelled notifications (backup software: Synology, backup type: Updates, informational only, no schedule learning)
|
||||||
|
- Extended Synology DSM update parser with additional detection patterns ("Automatische DSM-update", "DSM-update op", "Packages on", "out-of-date", "Package Center", "new DSM update", "Auto Update has detected", "Update & Restore", "belangrijke DSM-update", "kritieke oplossingen", "wordt automatisch geïnstalleerd", "is beschikbaar op") and hostname extraction regex to recognize DSM update cancelled, out-of-date packages, new update available, and automatic installation announcements under same Updates job type while maintaining backward compatibility with existing patterns
|
||||||
|
- Extended Synology Active Backup for Business parser to recognize skipped/ignored backup tasks ("genegeerd", "skipped", "ignored") as Warning status when backup was skipped due to previous backup still running
|
||||||
|
|
||||||
|
### Changed
|
||||||
|
- Updated `docs/changelog.md` with comprehensive v0.1.25 release notes consolidating all changes from 2026-02-09 (Parser Enhancements for Synology and Veeam, Maintenance Improvements, Data Privacy, Bug Fixes)
|
||||||
|
- Updated `containers/backupchecks/src/backend/app/changelog.py` with v0.1.25 entry in Python structure for website display (4 sections with subsections matching changelog.md content)
|
||||||
|
- Removed customer name from Autotask ticket title to keep titles concise (format changed from "[Backupchecks] Customer - Job Name - Status" to "[Backupchecks] Job Name - Status")
|
||||||
|
- Replaced real customer names in parser registry examples with generic placeholders (NTFS Auditing, QNAP Firmware Update, NAKIVO) to prevent customer information in codebase
|
||||||
|
|
||||||
### Fixed
|
### Fixed
|
||||||
- Fixed Autotask ticket description being set to NULL when resolving tickets via `update_ticket_resolution_safe` by adding "description" to the optional_fields list, ensuring the original description is preserved during PUT operations
|
- Fixed Autotask ticket description being set to NULL when resolving tickets via `update_ticket_resolution_safe` by adding "description" to the optional_fields list, ensuring the original description is preserved during PUT operations
|
||||||
|
- Fixed responsive navbar overlapping page content on smaller screens by implementing dynamic padding adjustment (JavaScript measures actual navbar height and adjusts main content padding-top automatically on page load, window resize, and navbar collapse toggle events)
|
||||||
|
|
||||||
### Changed
|
### Changed
|
||||||
- Updated `docs/changelog.md` with comprehensive v0.1.23 release notes consolidating all changes from 2026-02-06 through 2026-02-08 (Documentation System, Audit Logging, Timezone-Aware Display, Autotask Improvements, Environment Identification, Bug Fixes)
|
- Updated `docs/changelog.md` with comprehensive v0.1.23 release notes consolidating all changes from 2026-02-06 through 2026-02-08 (Documentation System, Audit Logging, Timezone-Aware Display, Autotask Improvements, Environment Identification, Bug Fixes)
|
||||||
|
|||||||
@ -1,3 +1,85 @@
|
|||||||
|
## v0.1.25
|
||||||
|
|
||||||
|
This release focuses on parser improvements and maintenance enhancements, adding support for new notification types across Synology and Veeam backup systems while improving system usability with orphaned job cleanup and test email generation features.
|
||||||
|
|
||||||
|
### Parser Enhancements
|
||||||
|
|
||||||
|
**Synology Parsers:**
|
||||||
|
- **Monthly Drive Health Reports**: New parser for Synology NAS drive health notifications
|
||||||
|
- Supports both Dutch ("Maandelijks schijfintegriteitsrapport", "Gezond") and English ("Monthly Drive Health Report", "Healthy") variants
|
||||||
|
- Automatic status detection: Healthy/Gezond/No problem detected → Success, otherwise → Warning
|
||||||
|
- Extracts hostname from subject or body pattern (Van/From NAS-HOSTNAME)
|
||||||
|
- Backup type: "Health Report", Job name: "Monthly Drive Health"
|
||||||
|
- Informational only (excluded from schedule learning and reporting logic)
|
||||||
|
- Registry entry added (order 237) for /parsers page visibility
|
||||||
|
|
||||||
|
- **DSM Update Notifications - Extended Coverage**: Added support for additional DSM update notification variants
|
||||||
|
- New patterns: "belangrijke DSM-update", "kritieke oplossingen", "wordt automatisch geïnstalleerd", "is beschikbaar op"
|
||||||
|
- Now recognizes 4 different notification types under same job:
|
||||||
|
1. Automatic update cancelled
|
||||||
|
2. Packages out-of-date warnings
|
||||||
|
3. New update available announcements
|
||||||
|
4. Automatic installation scheduled notifications
|
||||||
|
- All patterns added to existing lists maintaining full backward compatibility
|
||||||
|
|
||||||
|
- **Active Backup for Business - Skipped Tasks**: Extended parser to recognize skipped/ignored backup tasks
|
||||||
|
- Detects Dutch ("genegeerd") and English ("skipped", "ignored") status indicators
|
||||||
|
- Status mapping: Skipped/Ignored → Warning with "Skipped" message
|
||||||
|
- Common scenario: Backup skipped because previous backup still running
|
||||||
|
|
||||||
|
**Veeam Parsers:**
|
||||||
|
- **Job Not Started Errors**: New detection for "Job did not start on schedule" error notifications
|
||||||
|
- Recognizes VBO365 and other Veeam backup types that send plain text error notifications
|
||||||
|
- Extracts backup type from subject (e.g., "Veeam Backup for Microsoft 365")
|
||||||
|
- Extracts job name from subject after colon (e.g., "Backup MDS at Work")
|
||||||
|
- Reads error message from plain text body (handles base64 UTF-16 encoding)
|
||||||
|
- Sets overall_status to "Error" for failed-to-start jobs
|
||||||
|
- Example messages: "Proxy server was offline at the time the job was scheduled to run."
|
||||||
|
|
||||||
|
### Maintenance Improvements
|
||||||
|
|
||||||
|
**Orphaned Jobs Cleanup:**
|
||||||
|
- Added "Cleanup orphaned jobs" option in Settings → Maintenance
|
||||||
|
- Removes jobs without valid customer links (useful when customers are deleted)
|
||||||
|
- Permanently deletes job records along with all associated emails and job runs
|
||||||
|
- "Preview orphaned jobs" button shows detailed list before deletion
|
||||||
|
- Displays job information with email and run counts
|
||||||
|
- Safety verification step to prevent accidental deletion
|
||||||
|
|
||||||
|
**Test Email Generation:**
|
||||||
|
- Added "Generate test emails" feature in Settings → Maintenance
|
||||||
|
- Three separate buttons to create fixed test email sets for parser testing:
|
||||||
|
- Success emails (3 emails with success status)
|
||||||
|
- Warning emails (3 emails with warning status)
|
||||||
|
- Error emails (3 emails with error status)
|
||||||
|
- Each set contains exactly 3 Veeam Backup Job emails with:
|
||||||
|
- Same job name "Test-Backup-Job" for consistency
|
||||||
|
- Different dates, objects, and statuses
|
||||||
|
- Reproducible testing scenarios
|
||||||
|
- Proper status flow testing (success → warning → error progression)
|
||||||
|
|
||||||
|
### Data Privacy
|
||||||
|
|
||||||
|
**Parser Registry Cleanup:**
|
||||||
|
- Replaced real customer names in parser registry examples with generic placeholders
|
||||||
|
- Affected parsers: NTFS Auditing, QNAP Firmware Update, NAKIVO
|
||||||
|
- Example format now uses: NAS-HOSTNAME, SERVER-HOSTNAME, VM-HOSTNAME, example.local
|
||||||
|
- Ensures no customer information in codebase or version control
|
||||||
|
|
||||||
|
**Autotask Integration:**
|
||||||
|
- Removed customer name from Autotask ticket title for concise display
|
||||||
|
- Format changed from "[Backupchecks] Customer - Job Name - Status" to "[Backupchecks] Job Name - Status"
|
||||||
|
- Reduces redundancy (customer already visible in ticket company field)
|
||||||
|
|
||||||
|
### Bug Fixes
|
||||||
|
|
||||||
|
**User Interface:**
|
||||||
|
- Fixed responsive navbar overlapping page content on smaller screens
|
||||||
|
- Implemented dynamic padding adjustment using JavaScript
|
||||||
|
- Measures actual navbar height on page load, window resize, and navbar collapse toggle
|
||||||
|
- Automatically adjusts main content padding-top to prevent overlap
|
||||||
|
- Debounced resize handler for performance
|
||||||
|
|
||||||
## v0.1.24
|
## v0.1.24
|
||||||
|
|
||||||
### Bug Fixes
|
### Bug Fixes
|
||||||
|
|||||||
@ -1 +1 @@
|
|||||||
v0.1.24
|
v0.1.25
|
||||||
|
|||||||
Loading…
Reference in New Issue
Block a user