backupchecks/docs/changelog.md

## v20260109-01-veeam-m365-overall-message

- Fixed Veeam Backup for Microsoft 365 mail parsing where the overall summary message was not stored.
- Improved extraction of the overall details message from the mail content, ensuring permission and role warnings are correctly captured.
- Ensured the extracted overall message is consistently available in job details, run checks, and reporting views.

---

## v20260109-02-object-list-sorting

- Updated object list sorting logic to improve readability and prioritization.
- Objects are now grouped by severity in the following order:
  1. Errors
  2. Warnings
  3. All other statuses
- Within each severity group, objects are sorted alphabetically (A–Z).
- Applied consistent sorting across all relevant views, including Inbox, Job Details, Run Checks, Daily Jobs, and Admin All Mail.

---

## v20260109-03-preserve-ampersand-errors
- Decode HTML entities (e.g. &) in parsed object fields (name/type/status/error_message) before storing them.
- Ensures ampersands and other entities are displayed correctly in the UI error messages.

---

## v20260109-04-parsers-page-all-parsers

- Updated the Parsers overview page to use the central parser registry instead of a static list.
- All currently available parsers are now automatically displayed on the page.
- Ensures the Parsers page stays in sync with newly added or removed parsers without manual template updates.
- Improved maintainability by removing hardcoded parser definitions from the template.

---

## v20260109-05-fix-parsers-route-import
- Fixed parsers page route startup crash by replacing an invalid absolute import with a package-relative import for the parser registry.
- Prevented Gunicorn worker boot failure (Bad Gateway) caused by "No module named 'app'" during application initialization.

---

## v20260109-06-user-management-edit-roles

- Added support to edit user roles directly in the User Management interface.
- Implemented backend logic to update existing user role assignments without requiring user deletion.
- Updated validation to ensure role changes are applied immediately and consistently.
- Ensured role updates are reflected correctly in permissions and access control.

---

## v20260109-07-feedback-open-default-resolved-sort

- Updated feedback listing to show only Open items by default.
- Adjusted sorting logic so Resolved items are always placed at the bottom when viewing all items.
- Preserved existing filtering, searching, and user-controlled sorting behavior.

---

## v20260109-08-ui-ellipsis-and-remove-objects-header

- Added reusable ellipsis styling for detail fields to prevent long values (e.g., email addresses) from overlapping other UI elements.
- Implemented click-to-expand for ellipsized fields so the full text becomes visible and selectable; double-click expands and selects all text.
- Added automatic tooltip (title) showing the full value when the field is truncated.
- Removed the redundant "Objects" heading above the objects table (table column header remains as the single label).
- Applied the detail-field truncation behavior consistently across Inbox, Deleted Mail, Run Checks, Daily Jobs, Job Detail, and Admin All Mail popups/pages where long values can occur.

---

## v20260109-09-ellipsis-reset-on-popup-close

- Reset expanded ellipsis fields when a Bootstrap modal is shown or hidden, so expanded state does not persist between openings.
- Added the same reset behavior for Bootstrap offcanvas components to keep behavior consistent across popups.

---

## v20260109-10-veeam-cloud-connect-report-parser

- Updated the Veeam Cloud Connect Report parser to correctly handle object naming by combining User and Repository Name with a "|" separator.
- Excluded the row containing "TOTAL" from being processed as an object.
- Adjusted status detection so red rows are interpreted as Errors and yellow/orange rows as Warnings.
- Ensured the overall status is set to Error when one or more objects are detected as errors.
- Improved parsing logic to correctly classify mixed-status reports within a single mail.

---

## v20260109-11-ntfs-auditing-audit-parser

- Added parser support for “NTFS Auditing” with type “Audit”.
- Implemented job name extraction from the mail subject (e.g. “<hostname> file audits”).
- Handled reports without an overall status by defaulting the result to Success.
- Ensured existing missing-run logic is applied when no audit report mail is received.
- Defined that these audit mails contain no objects; object list remains empty.

---

## v20260109-12-ntfs-audit-fqdn-jobname

- Updated NTFS Auditing (Audit) subject parsing to extract the hostname/FQDN directly from the start of the subject before "file audits".
- Removed the dependency on a fixed "Bouter <host>" subject prefix, so hosts like dc01.totall-it.local and fs01.totall-it.local are recognized.

---

## v20260109-13-ntfs-audit-jobname-prefix-flex

- Updated NTFS Auditing (Audit) subject parsing to support both formats:
  - Subjects starting with "Bouter <host> file audits"
  - Subjects starting directly with "<host> file audits"
- Ensures both btr-dc001/btr-dc002 and dc01/fs01 style hostnames are recognized consistently.

---

## v20260112-01-synology-abb-partial-warning

- Updated the Synology Active Backup for Business parser to detect “partially completed” jobs as Warning instead of ignoring them.
- Added support for localized variants of the completion message (including cases without an explicit host in the first line).
- Improved host detection by extracting the system name from the “From” header when not present in the subject/body.
- Extended object parsing to recognize per-device backup results and correctly classify them as Success or Error.
- Fixed the ABB-specific regular expression to prevent parser failures on valid warning mails.

---

## v20260112-02-synology-abb-subject-partial-warning

- Updated the Synology Active Backup for Business subject matching to also recognize “Virtual machine backup task …” variants.
- Added subject-based detection for “partially completed / gedeeltelijk voltooid” and map this to Overall status: Warning.
- Fixed the ABB completion regex (it previously failed to match valid subjects).
- Improved per-device object parsing by detecting “backup succeeded/failed” device-list variants and mapping them to Success/Error accordingly.

---

## v20260112-03-ntfs-audit-recognize-bouter-hosts
- Improved NTFS Auditing (Audit) subject parsing to extract the hostname reliably, including subjects with a leading company prefix (e.g. "Bouter ...") and underscores-as-spaces variants.
- Set overall status to Warning when the subject indicates any detected changes (↑/↓ counts > 0); otherwise Success.
- Ensured job name is consistently generated as "<hostname> file audits" for all supported senders.

---

## v20260112-04-remove-runchecks-success-override-button

- Removed the “Mark success (override)” button from the Run Checks popup.
- Prevented creation of actual overrides when marking individual runs as success.
- Simplified override overview by ensuring Run Checks actions no longer affect override administration.

---

## v20260112-05-qnap-firmware-update-info-parser

- Added parser support for QNAP Firmware Update notifications.
- Classified firmware update messages as informational warnings.
- Excluded QNAP firmware update messages from missing-run detection logic.
- Excluded QNAP firmware update messages from reports and scheduling.
- Ensured affected NAS devices are only shown in Run Checks when the message occurs.

---

## v20260112-06-veeam-spc-alarm-summary-parser
- Added support for Veeam Service Provider Console (VSPC) summary daily alarm notification emails.
- Implemented parsing that aggregates alarms per Company and stores one object per company, with the alarm details in the object error message.
- Derived overall status from the worst status found across all companies (Failed > Warning > Success).
- Registered "Service Provider Console" as a supported Veeam backup type for consistent reporting.

--- 

## v20260112-07-veeam-vspc-active-alarms-parser
- Improved detection for VSPC “Active Alarms” summary emails based on HTML content to prevent fallback to other Veeam parsers.
- Implemented parsing of alarms per Company and per alarm row, creating objects named “<Company> | <Object>”.
- Derived object status from “Current State” and attached alarm details where available (fallback to Alarm Name).

---

## v20260112-08-fix-veeam-vspc-parser-syntaxerror

- Fixed a SyntaxError in the Veeam VSPC Active Alarms parser caused by an incomplete regular expression.
- Restored valid parser loading to prevent Gunicorn startup failure and Bad Gateway errors.
- No functional logic changes; fix is limited to syntax correction only.

---

## v20260112-09-veeam-vspc-company-mapping-popup

- Added a dedicated company mapping popup for Veeam Service Provider Console "Active alarms summary" reports.
- Enabled manual linking of companies found in the mail to existing customers, as automatic customer matching is not applicable for this report type.
- Introduced per-company job and run creation (job name format: "Active alarms summary | <Company>") to ensure correct customer association.
- Ensured only alarms and objects belonging to the selected company are attached to the corresponding run.
- Disabled the standard job approval flow for this report type and replaced it with the company mapping workflow.

---

## v20260112-10-fix-vspc-approve-endpoint-duplicate

- Fixed a Flask startup crash caused by duplicate endpoint registration for the VSPC company approve route.
- Corrected misplaced route decorators that caused the approve handler to be mapped to the delete endpoint.
- Restored proper route separation between inbox message delete and VSPC company approve actions.
- Added missing authentication requirement to the VSPC approve endpoint to align with other inbox actions.

---

## v20260112-11-show-vspc-company-mapping-popup
- Fixed VSPC company-mapping popup visibility by removing strict job name matching in the Inbox modal logic.
- Made VSPC detection case-insensitive for backup software/type to improve robustness across mail variants and legacy data.

---

## v20260112-12-vspc-company-mapping-popup-visible
- Enabled VSPC company-mapping popup visibility for “Active alarms summary” messages even when no objects are parsed.
- Exposed parsed VSPC company list via the inbox message detail endpoint.
- Updated inbox popup logic to trigger the dedicated VSPC company-mapping workflow based on detected companies instead of parsed objects.

---

## v20260112-13-vspc-company-mapping-popup-ui
- Added VSPC company extraction to the inbox message detail response so multi-company summaries expose a company list even when no objects are stored.
- Updated the Inbox message modal to show the “Map companies” button based on the returned VSPC company list (with fallback to legacy object-name parsing).
- Disabled the standard Customer selector when VSPC company mapping is available to avoid using the wrong approval flow.

---

## v20260112-14-vspc-company-mapping-require-all
- Filtered VSPC company extraction to include only companies with alarms > 0 to avoid showing unrelated companies in the mapping popup.
- Added default customer prefill for each VSPC company based on existing per-company jobs (previous mappings).
- Changed VSPC approval flow to require all companies in the email to be mapped before approval; when mappings are incomplete the message stays in the Inbox while newly provided mappings are still saved.

---

## v20260112-15-vspc-scroll-partial-approve-objects
- Added a vertical scrollbar to the VSPC company mapping popup so “Approve mapped companies” stays reachable without browser zoom.
- Changed VSPC approval to approve only mapped companies; if some companies are still unmapped, the email stays in the Inbox while runs for mapped companies are created.
- Added de-duplication for VSPC approvals to prevent creating duplicate runs on repeated approvals.
- Improved VSPC error detection by treating both “Error” and “Failed” states as errors for the run status.
- Enhanced VSPC parsing to use the “Alarm Details” column so object rows (e.g., HV01, USB Disk) persist with their full alarm messages and become visible in the customer job view.

================================================================================================================================================
## v0.1.19
This release delivers a broad set of improvements focused on reliability, transparency, and operational control across mail processing, administrative auditing, and Run Checks workflows. The changes aim to make message handling more robust, provide better insight for administrators, and give operators clearer and more flexible control when reviewing backup runs.

Mail Import Reliability and Data Integrity
- Updated the mail import flow so messages are only moved to the processed folder after a successful database store and commit.
- Prevented Graph emails from being moved when parsing, storing, or committing data fails, ensuring no messages are lost due to partial failures.
- Added explicit commit and rollback handling to guarantee database consistency before any mail state changes occur.
- Improved logging around import, commit, and rollback failures to make skipped or retried mails easier to trace and troubleshoot.

Administrative Mail Auditing and Visibility
- Introduced an admin-only “All Mail” audit page that provides a complete overview of all received mail messages.
- Implemented pagination with a fixed page size of 50 items to ensure consistent performance and predictable navigation.
- Added always-visible search filters that can be combined using AND logic, including From, Subject, Backup, Type, Job name, and a received date/time range.
- Added an “Only unlinked” filter to quickly identify messages that are not associated with any job.
- Reused the existing Inbox message detail modal to allow consistent inspection of messages from the All Mail page.
- Added a dedicated navigation entry so administrators can easily access the All Mail audit view.
- Fixed modal opening behavior in the All Mail page to fully align click handling and popups with the Inbox implementation.

Inbox and Mail Body Rendering Improvements
- Treated whitespace-only email bodies as empty during import so HTML report attachments can be extracted and displayed correctly.
- Added legacy fallback logic in the Inbox message detail API to extract the first HTML attachment from stored EML files when bodies are empty or invalid.
- Improved iframe rendering in the Inbox so full HTML documents (commonly used for report attachments) are rendered directly instead of being wrapped.
- Added detection for “effectively empty” HTML bodies, such as empty Graph-generated HTML skeletons.
- Ensured that both newly imported and already-stored messages can dynamically fall back to EML attachment extraction without requiring a reset.

Run Checks Usability and Control
- Added a copy-to-clipboard icon next to ticket numbers in the Run Checks popup to quickly copy only the ticket code.
- Prevented accidental selection of appended status text when copying ticket numbers.
- Introduced a manual “Success (override)” action that allows Operators and Admins to mark a run as successful even if it originally failed or produced warnings.
- Implemented backend support to persist the override state without modifying the original run data.
- Updated UI indicators so overridden runs are clearly shown with a blue success status.
- Ensured overrides apply only to the selected run and do not affect historical or related runs.
- Improved Run Checks mail rendering by falling back to text bodies when HTML bodies are missing, matching Inbox and All Mail behavior.
- Added support for extracting HTML content from stored EML files when both HTML and text bodies are unavailable.
- Ensured plain-text emails are safely rendered using preformatted HTML to preserve readability.

Customer, Ticket, and Scope Cleanup
- Updated customer deletion logic to allow removal of customers even when tickets or remarks are linked.
- Added explicit cleanup of related TicketScope and RemarkScope records prior to customer deletion.
- Ensured jobs linked to a deleted customer are safely unassigned to prevent foreign key constraint errors.
- Eliminated deletion failures caused by lingering ticket or remark relationships.

Parser Enhancements and Informational Messages
- Added parser support for 3CX SSL Certificate notification emails.
- Classified these messages as Backup: 3CX with Type: SSL Certificate.
- Parsed and displayed certificate information in the Run Checks popup.
- Stored these messages as successful runs so certificate status can be tracked over time.
- Added detection for Synology DSM automatic update cancellation messages in both Dutch and English.
- Classified Synology Updates messages as informational and set their overall status to Warning.
- Excluded Synology Updates informational messages from scheduling logic and reporting output.

UI Layout and Status Accuracy Improvements
- Moved the Details section above the email body in both Inbox and Job Details popups to improve readability.
- Avoided long detail texts being constrained to narrow side columns.
- Adjusted missed run detection to include a ±1 hour grace window around scheduled run times.
- Prevented runs that arrive shortly after the scheduled time from being temporarily marked as Missed.
- Ensured the Missed status is only applied after the full grace window has elapsed, reducing false alerts in Run Checks and Daily Jobs views.

---

## v0.1.18

This release focuses on improving ticket reuse, scoping, and visibility across jobs, runs, and history views to ensure consistent and flexible ticket handling.

### Ticket Linking and Reuse
- Updated ticket linking logic to allow the same existing ticket number to be associated with multiple jobs and job runs.
- Prevented duplicate ticket creation errors when reusing an existing ticket code.
- Ensured existing tickets are consistently reused and linked instead of being rejected when already present in the system.

### Ticket Scope and Resolution
- Fixed missing ticket number display in job and run popups by always creating or reusing a ticket scope when linking an existing ticket to a job.
- Updated ticket resolution logic to support per-job resolution when resolving tickets from a job or run context.
- Ensured resolving a ticket from the central Tickets view resolves the ticket globally and closes all associated job scopes.
- Updated ticket active status determination to be based on open job scopes, allowing the same ticket number to remain active for other jobs when applicable.

### Job History – Tickets and Remarks
- Added a Tickets and Remarks section to the Job History mail popup.
- Aligned ticket handling in Job History with the existing Run Checks popup behavior.
- Enabled viewing of active and resolved tickets and remarks per job run.
- Added support for creating new tickets and remarks directly from the Job History popup.
- Enabled resolving tickets and remarks directly from the Job History popup.
- Ensured tickets and remarks are correctly scoped to the selected run (run_id).

---

## v0.1.17

### Release Summary

This release focuses on improving job normalization, ticket and remark handling, UI usability, and the robustness of run and object detection across the platform.

**Job normalization and aggregation**
- Veeam job names are now normalized to prevent duplicates:
  - Jobs with “(Combined)” and “(Full)” suffixes are merged with their base job names.
- Ensures accurate aggregation, reporting, and statistics for Veeam Backup and Microsoft 365 jobs.
- Added support for archiving inactive jobs while keeping all historical runs fully included in reports.

**Inbox and bulk operations**
- Introduced multi-select inbox functionality for Operator and Admin roles.
- Added a bulk “Delete selected” action with validation, counters, and admin audit logging.

**Jobs UI and navigation**
- Restored row-click navigation on the Jobs page.
- Moved the Archive action from the Jobs table to the Job Details page for consistency.
- Improved layout and behavior of job run popups, ensuring objects are visible, correctly rendered, and consistently sorted.

**Tickets and remarks**
- Ticket creation now always uses a user-provided ticket code with strict format and uniqueness validation.
- Editing of tickets and remarks has been fully disabled; items must be resolved and recreated instead.
- Removed ticket description fields from creation and detail views to prevent inconsistent data.
- Fixed backend indentation errors that previously caused startup and Bad Gateway failures.

**Customer deletion stability**
- Fixed foreign key constraint issues when deleting customers.
- Customer deletion now safely unlinks jobs while preserving historical job and run data.
- Enforced cascading deletes where appropriate to prevent integrity errors.

**Feedback handling**
- Users can now reply directly to Feedback items while they are in the “Open” state.
- Replies are stored for audit and history tracking.

**Veeam parser improvements**
- Configuration Backup parser now correctly captures multi-line overall warning messages.
- Improved date parsing to support formats without leading zeros.
- Microsoft 365 parser now always persists overall warning/info messages, even on successful runs.

**Run checks and missed run detection**
- Improved weekly and monthly schedule inference to reduce false positives.
- Monthly jobs are now detected and marked as missed on the correct expected date.
- Added fallback mechanisms for loading run objects in Run Checks to support legacy and transitional data.

Overall, this release significantly improves data consistency, usability, and reliability across jobs, tickets, inbox management, and run analysis.

---

## v0.1.16

This release significantly expands and stabilizes the reporting functionality, focusing on configurability, correctness, and richer output formats.

### Key Highlights
- Reports are now **job-aggregated** instead of object-level, making them suitable for high-level status reviews.
- Full **report lifecycle management** was added, including secure deletion and reliable refresh behavior.
- Introduced **advanced reporting foundations** with configurable report definitions (columns, filters, layout, charts).
- Added support for **multiple export formats**: CSV, HTML, and PDF, including graphical HTML previews and basic PDF charts.
- Implemented extensive **column selection** across Summary, Snapshot, and Jobs views, with drag-and-drop ordering and persistence.
- Added **job-level aggregated metrics**, including per-job success rates and charts over the selected period.
- Improved **filtering** to exclude informational (non-backup) jobs and allow selection by backup software and type.
- Ensured **success rate and total runs calculations** are correct and consistently based only on selected run statuses.
- Added **Customer** as a selectable column and improved multi-customer report clarity.
- Introduced configurable **HTML/PDF content scope** (Customers, Jobs, or both).
- Fixed numerous **stability issues** (loading states, NameErrors, missing imports, endpoint collisions, JS errors).
- Improved HTML report layout, table rendering, column labeling, sorting logic, and visual consistency.
- Cleaned up column metadata, removed deprecated/duplicate options, and added migration logic to fix existing configurations.

---

## v0.1.15

This release focused on improving operational clarity and usability by strengthening dashboard guidance, introducing a solid reporting foundation, and significantly refining the Run Checks review workflow for daily accountability.

### Dashboard & User Guidance
- Added a comprehensive explanatory section to the Dashboard describing how Backupchecks collects, normalizes, and evaluates backup results.
- Clarified dashboard counters, job statuses, overrides, and operational workflows.
- Introduced clear guidance emphasizing the goal of keeping the Run Checks page empty or near-empty as proof of daily review and accountability.
- Positioned Backupchecks explicitly as a monitoring and validation layer that complements existing backup software.

### Navigation & Usability
- Implemented automatic redirection to the Dashboard on a user’s first visit of each day, improving daily visibility of system status.
- Refactored the Settings area into clearly separated sections (General, Imports, Users, etc.) for better usability and structure.

### Dashboard News
- Added a per-user Dashboard News section with read/unread tracking.
- Enabled full admin management of news items (create, edit, delete, pin, severity, publish window).
- Provided admin insight into read statistics and the ability to reset read status.

### Reporting – Foundation
- Introduced the initial reporting framework focused on raw, immutable data for auditability.
- Added object-based reporting across multiple jobs with aggregated success, warning, failed, missed, and override metrics.
- Prepared the data model for CSV export, future PDF generation, scheduling, and email delivery.
- Enabled reporting access for admin, operator, and reporter roles in phase 1.

### Reports UI & Generation
- Added an initial Reports UI to define, generate, and manage reports.
- Enabled raw data preview directly in the UI with pagination support.
- Implemented backend endpoints to generate and retrieve report data on demand.
- Replaced the “New report” modal with a dedicated Create Report page.
- Added month-based date selection, quick-select options, and multi-customer reporting support.
- Enabled actual report generation while keeping scheduling as a non-functional placeholder.

### Run Checks – Multi-Select Improvements
- Added Shift-click multi-selection for efficient bulk review.
- Iteratively fixed edge cases including:
  - Off-by-one range selection issues
  - Checkbox and row highlight desynchronization
  - Last-row selection bugs
  - Browser text selection interference
- Improved “Select all” checkbox behavior:
  - Clearing selection when indeterminate
  - Correctly resetting from the indeterminate state
  - Ensuring consistent mouse and keyboard interaction

### Overall Result
- Improved clarity, accountability, and daily workflow guidance across the application.
- Established a solid, extensible reporting foundation.
- Significantly enhanced usability and reliability of the Run Checks review process.

---

## v0.1.14

### Daily Jobs
- Introduced a consistent, case-insensitive multi-level sort order for the Daily Jobs overview: Customer → Backup Software → Backup Type → Job Name.
- Fixed backend ordering issues to ensure server-side data no longer overrides the intended sort logic.
- Ensured sorting is applied before serialization so the UI always reflects the correct order.
- Improved predictability and readability of job listings across environments.

### Veeam Backup for Microsoft 365
- Improved parsing of overall warning messages to correctly extract and display permission- and role-related issues.
- Added support for combined permission and role warnings in M365 reports.
- Ensured detailed permission warnings take precedence over generic “X of X objects processed” messages.
- Fixed incorrect overall message selection and filtered out misleading banner fragments.
- Resolved an indentation error in the parser that caused backend startup failures, restoring stability.

### Overrides – Configuration and Matching
- Replaced free-text inputs with dropdowns for Backup Software and Backup Type in Overrides, including:
  - Alphabetical sorting
  - Preselection of existing values
  - A global option at the top of each dropdown
- Fixed PostgreSQL compatibility issues by replacing DISTINCT queries with GROUP BY while preserving case-insensitive sorting.
- Ensured Overrides endpoints no longer crash due to invalid query constructions.

### Overrides – Application, Editing, and Deletion
- Made newly created overrides apply immediately and retroactively to all unreviewed runs by default.
- Added full support for editing existing overrides and reapplying changes to unreviewed runs.
- Restricted override deletion to Admin users and ensured proper reprocessing after removal.
- Fixed datetime handling in override edit flows so unchanged values are preserved and NULL constraint violations are avoided.
- Ensured Admin users always see delete actions by consistently passing permission flags to the UI.

### Overrides – Matching Logic Improvements
- Extended override matching to use persisted run_object_links joined with customer_objects instead of legacy or non-existent relationships.
- Improved global override matching by resolving backup software and type from MailMessage data when missing on jobs.
- Added support for matching against object-level error messages as well as run-level remarks.
- Ensured all override matching remains case-insensitive and consistent across run-level and object-level evaluations.

### Overrides – UI Indicators and Reporting
- Introduced a blue status indicator for runs and jobs where overrides are applied.
- Updated status labels to display “Success (override)” for clearer distinction without changing canonical stored statuses.
- Added persistent override reporting metadata to job runs, including applied override ID, level, and reason.
- Ensured dashboards, Daily Jobs, Run Checks, and popups correctly propagate and display override-based success states.
- Fixed multiple UI rendering issues so overridden runs are no longer misclassified as warnings or missed jobs.

### Daily Jobs Popups
- Fixed popup loading failures and backend unpacking errors related to override handling.
- Ensured popup details consistently load correct run data.
- Aligned popup override detection and status coloring with Run Checks and Daily Jobs overviews.

---

## v0.1.13

This release focuses on improving visibility and consistency of Tickets and Remarks across Run Checks and Job Details, alongside several UI fixes and backend stability improvements.

### Highlights
- Added clear visual indicators for active Tickets and Remarks in the Run Checks overview.
- Enhanced Job Details and Job History to display actual ticket numbers and related remark messages, both in tables and popups.
- Improved navigation consistency by adding direct “Job page” links for Tickets and Remarks.

### Improvements
- Job History popups now reliably show associated ticket numbers and remark content.
- Backend job history data is enriched to support consistent UI rendering.
- Missed-run detection now includes a ±1 hour tolerance window and respects the configured UI timezone.
- Run Checks UI is simplified by hiding last-reviewed columns (data is still retained in the backend).

### Fixes
- Resolved a backend indentation issue that caused Gunicorn startup failures.
- Made frontend parsing of ticket/remark data more robust against malformed or unexpected payloads.
- Fixed JSON encoding issues in HTML data attributes to prevent popup rendering errors.

### Changelog Update
- Simplified the changelog by removing “Current Version” and “In testing” sections.
- The changelog now only shows completed changes.

---

## v0.1.12

### Dashboard & UI
- Corrected dashboard counters so **Expected**, **Missed**, and **Success (override)** statuses are shown accurately.
- Added dedicated counters for Expected and Success (override).
- Fixed layout issues on the Inbox dashboard tiles and improved label wrapping.
- Added safe timezone fallbacks to prevent incorrect status aggregation.
- Restored missing status icons and symbols across Dashboard and Daily Jobs views.
- Cleaned up Job Details UI by removing redundant columns and clarifying schedule display.
- Extended Job History with weekday labels and review metadata (Admin-only visibility).

### Stability & Reliability
- Fixed a Gunicorn startup crash caused by incorrect Python indentation.
- Improved migration robustness for soft-delete columns to prevent startup 502 errors on busy databases.
- Prevented duplicate or unintended regeneration of reviewed “Missed” runs.

### Inbox & Mail Handling
- Introduced soft-delete for Inbox messages with full Admin restore capability.
- Added an Admin-only “Deleted mails” page with audit details (who/when).
- Added popup previews for deleted mails without requiring restore.
- Improved HTML mail handling by extracting content from HTML attachments when the body is empty.
- Added an Admin maintenance action to backfill HTML bodies from existing attachments.

### Feedback & Settings
- Changed Feedback behavior so resolved items remain visible until explicitly deleted.
- Restricted feedback deletion to Admin users only.
- Added a User Settings page allowing users to change their own password securely.

### Backup Parser Enhancements
- Improved Veeam parsing:
  - Fixed Health Check Summary parsing.
  - Normalized job names by stripping “(Retry)”.
  - Added and refined License Key parsing with correct status detection and exclusions from scheduling logic.
- Added and expanded Synology support:
  - Active Backup for Business (NL/EN).
  - R-Sync (NL/EN).
  - Account Protection notifications (informational, no scheduling).
- Added new parsers:
  - R-Drive Image.
  - Syncovery.
- Ensured correct handling of objects, statuses, and scheduling exclusions where applicable.

### Changelog
- Removed the “Planned” section from the Changelog.
- Future planning is now handled exclusively via the Feedback page.

---

## v0.1.11

### Stability & Bug Fixes
- Fixed multiple page crashes caused by missing imports after refactoring (Jobs, Feedback, Run Checks, Inbox, Daily Jobs).
- Resolved Jinja2 template errors and SQL/runtime issues related to timezone handling.
- Improved robustness by explicitly importing shared helpers to prevent NameError exceptions.

### Run Checks & Review Workflow
- Introduced a new **Run Checks** page to review job runs independently from Daily Jobs.
- Displays all unreviewed runs with no time-based filtering.
- Supports bulk review actions and per-job review via popups.
- Added admin-only features: show reviewed runs, unmark reviewed runs, reviewer metadata, and full audit logging.
- Enhanced popups to group runs per job, include missed runs, and show ticket/remark indicators.
- Added per-job and per-popup status summaries using visual indicators only.

### UI & Visual Consistency
- Unified all job and run status indicators to a single shape differentiated by color.
- Added a clear status legend to the Dashboard, including the new **Expected** state.
- Removed textual status labels across Daily Jobs and Run Checks for a cleaner UI.
- Improved table layouts and widened content areas for better use of 1080p screens.
- Ensured consistent indicator rendering across all pages.

### Timezone & Display Improvements
- Added a configurable timezone setting in Settings.
- Updated all frontend date/time rendering to use the configured timezone instead of UTC.
- Fixed offset issues and restored missing timestamps across multiple pages.

### Missed Runs Logic
- Refined missed run detection to rely only on historically received mail reports.
- Prevented synthetic or never-run schedules from generating false missed runs.

### Settings & Maintenance
- Stabilized **Delete all jobs** by adding schema-tolerant cleanup of all related foreign key references.
- Refactored the Settings page layout using accordions and cards for improved clarity.
- Improved alignment and usability of import/export and user management sections.

### Roles & Access Control
- Added support for multiple roles per user with an active role switcher.
- Fixed role-based menu rendering and ensured permissions are evaluated against the active role.
- Ensured role switching consistently redirects to the Dashboard.

### Theme & UX Fixes
- Fixed manual theme switching (Light/Dark/Auto) and ensured user preferences persist.
- Corrected Inbox EML download functionality by restoring the missing shared import.

Overall, this release significantly improves stability, review workflows, visual consistency, timezone correctness, and administrative reliability, while refining the operator experience and access control model.

---

## v0.1.10

### Performance & Stability
- Reworked **Re-parse all** to process inbox messages in controlled batches, preventing gateway and Gunicorn timeouts on large inboxes.
- Added execution time guards to stop processing before proxy limits are reached.
- Optimized job-matching queries and disabled session autoflush during batch operations to reduce database load.
- Ensured auto-approval and persistence logic only finalize after a full, successful re-parse cycle.
- Restored stable backend startup by fixing decorator ordering issues that caused 502 errors.

### Job Matching & Parsing
- Fixed approved job imports to persist `from_address`, ensuring correct matching during re-parse.
- Improved Veeam Backup Job parsing:
  - Extracted and stored multi-line warnings/errors and object-level details with preserved line breaks.
  - Ignored VM summary lines (e.g., “X of X VMs processed”) for overall status detection.
  - Prevented incorrect overall warnings when issues are object-level only.
  - Fixed regressions to ensure backup objects are consistently detected, stored, and displayed across all views.

### UI & UX Improvements
- Added EML download support for Job Details and Daily Jobs, with automatic availability handling and proper 404s when missing.
- Improved rendering to preserve line breaks (pre-wrap) in remarks, overall messages, and object details.
- Reduced visual clutter by moving overall status/messages out of tables and into context-specific popups.
- Standardized changelog version display by removing date suffixes.
- Reordered main navigation for better consistency.

### Daily Jobs & Status Accuracy
- Clarified Daily Jobs status logic:
  - Introduced **Expected** for backups not yet due.
  - Reserved **Missed** only for jobs past their final expected run time.
- Added last remark excerpts and ensured object details are visible in Daily Jobs popups.

### Tickets, Remarks & Overrides
- Introduced run-date–scoped ticket activity with `active_from_date`, ensuring accurate historical and current visibility.
- Implemented identical scoping for remarks, preserving visibility across runs even after resolution.
- Fixed resolve actions to redirect properly in the UI while keeping JSON responses for API/AJAX.
- Improved override handling so changes apply immediately to existing job runs with correct priority resolution.

### New Features
- Added a **Feedback** board with per-user upvoting, admin moderation (resolve/reopen, soft delete), database migrations, and navigation entry.

### Navigation
- Updated menu order to: Inbox, Customers, Jobs, Daily Jobs, Tickets, Overrides, Reports, Settings, Logging, Changelog, Feedback.

---

## v0.1.9

### Changelog System Improvements
- Added and maintained multiple **Completed** changelog entries (v0.1.2 through v0.1.8) with correct release dates.
- Ensured all existing Completed, Testing, and Planned changelog entries were preserved without loss.
- Migrated the Completed changelog from markdown-based content to a **structured, non-markdown format** aligned with the Planned section.
- Simplified changelog rendering logic to use explicit section titles and bullet handling instead of full markdown parsing.
- Standardized formatting across all versions for long-term maintainability and consistent UI rendering.

### Bug Fixes & Stability
- Fixed multiple backend **Python syntax and runtime errors** related to changelog definitions (missing commas, indentation issues, invalid list entries).
- Resolved rendering issues where markdown content was displayed as plain text or collapsed incorrectly.
- Restored application startup stability by fixing missing imports (`re`, `html`) and indentation errors in changelog-related routes.

### Refactoring & Maintainability
- Refactored a large `routes.py` file into multiple smaller route modules.
- Introduced a shared routes module for common imports, helpers, and access control.
- Fixed NameError issues after refactoring by explicitly importing underscored helper functions that are not included via wildcard imports.
- Ensured all split route modules retained full functional parity with the original implementation.

### Release Management Updates
- Moved versions through **Testing → Completed** states correctly:
  - v0.1.7 marked as Completed (2025-12-23).
  - v0.1.8 added as Completed (2025-12-24) and restored as Current Version.
  - Testing advanced to v0.1.9.
- Updated v0.1.8 release notes to document consistent job-matching and auto-approval behavior across all mail processing flows.
- Verified no regressions in changelog structure or rendering after updates.

Overall, v20251225 focused on **hardening the changelog system**, improving backend stability, cleaning up technical debt in routing, and ensuring consistent, reliable release tracking across the application. :contentReference[oaicite:0]{index=0}

---

## v0.1.8

### Overview
This release focuses on making job matching and auto-approval behavior fully consistent across manual inbox actions, automatic mail imports, and the “Re-parse all” process. It also fixes a critical backend startup issue introduced in the re-parse logic.

### Key Changes
- Introduced a single, shared job-matching helper based on a full unique key:
  - From address  
  - Backup software  
  - Backup type  
  - Job name
- Updated manual inbox approval to reuse existing jobs when the unique key matches, instead of relying on customer-only matching.
- Aligned inbox “Re-parse all” auto-approve logic with the same shared matching behavior.
- Fixed automatic mail import auto-approve so it correctly:
  - Creates a JobRun
  - Marks the mail as approved
  - Moves the mail to history when a matching job exists

### Re-parse All Improvements
- Auto-approve is now executed during “Re-parse all”, not only on initial mail import.
- After re-parsing, all successfully parsed mails without a linked job are re-evaluated against existing jobs using the full unique key.
- When a matching active job with auto-approve enabled is found:
  - The mail is automatically approved
  - The mail is linked to the job
  - The mail is moved to history
  - A corresponding job run is created and shown in Job History

### Fixes
- Resolved an issue where “Re-parse all” previously only updated parse metadata and skipped auto-approve logic, causing historical mails not to appear in job history.
- Fixed a SyntaxError in the re-parse auto-approve logic that caused backend startup failures (Bad Gateway).
- Corrected try/except structure and indentation to ensure re-parse auto-approve runs safely per mail without breaking the overall process.

### Result
- Job matching and auto-approval behavior is now consistent across all mail processing flows.
- Historical mails are correctly linked to jobs and visible in job history.
- Backend stability during startup and re-parse operations is restored.

---

## v0.1.7

### Key Features
- Introduced **export and import functionality for approved jobs**, allowing approved job definitions to be migrated between clean installations via JSON.
- Import process automatically creates missing customers and updates existing jobs based on a unique job identity to prevent duplicates.

### Versioning & Changelog
- Promoted version **v0.1.7** from *Testing* to *Completed*.
- Introduced **v0.1.8** as the new *Testing* release.
- Updated the changelog structure and testing notes to reflect active export/import functionality.

### Parser Enhancements
- **Boxafe**
  - Improved parsing for Shared Drives, Domain Accounts (Email, Contact, Drive, Calendar), and handling of *Warning* statuses.
  - Corrected object detection logic, ensuring no false objects are created when jobs contain no object data.
  - Removed object parsing for Shared Drives backups entirely.
- **Synology Hyper Backup**
  - Added full support for Dutch notification emails.
  - Improved status detection for Dutch phrasing.
  - Confirmed that no objects are parsed for Hyper Backup jobs.
- **Veeam**
  - Added support for **Scale-out Backup Repository (SOBR)** notifications, including storage capacity metrics suitable for logging and graphing.
  - Added support for **Veeam Health Check** reports with correct object filtering to exclude summary-only entries.

### Job Approval & Auto-Approval Logic
- Significantly refined approved job matching logic to prevent cross-customer approvals.
- Improved auto-approve behavior during **Re-parse all**, Inbox reprocessing, and Graph imports.
- Enhanced resilience against case differences, whitespace, unicode characters, and hidden formatting issues.
- Iteratively refined normalization strategy:
  - Ultimately simplified matching to rely primarily on a normalized **From address**, while ensuring parser-consistent values for Backup, Type, and Job Name.
- Ensured deterministic behavior during reprocessing by preventing mutation of message data.

### Stability Fixes
- Fixed multiple crashes and approval errors caused by undefined or incorrect job name handling.
- Resolved issues with duplicate job records (including `NULL` customer IDs) that previously blocked auto-approval.
- Ensured consistent JobRun creation and mail linking during automatic imports and re-parsing.

### Notes
- Due to changes in approval matching logic, **previously approved jobs are expected to be recreated** after these updates.

---

## v0.1.6

### Fixed
- Corrected auto-approve logic to ensure it is properly applied during automatic mail imports.
- Prevented the **Re-parse all** action from re-processing emails that were already approved.
- Ensured approved status is always respected and never overwritten during re-parsing or automatic imports.
- Fixed multiple Jinja2 `TemplateSyntaxError` issues in the base layout that caused 500 Internal Server Errors.
- Restored correct rendering of all pages affected by template errors, including Dashboard, Parsers, and Changelog.
- Resolved Changelog page rendering issues by fixing dictionary access in templates and avoiding conflicts with built-in methods.

### Added
- Introduced a centralized **Changelog** page containing:
  - Active production version
  - Testing version
  - Planned / Todo items
  - Completed changes
- Added the Changelog entry point to the main navigation.
- Applied a clear versioning convention, e.g. `v0.1.7 (v20251222-03)` for testing releases.
- Marked version `0.1.6` as the active production release.

### Planned
- Export and import of jobs to allow restoring approved jobs after a clean installation.
- Always register “New license key is not available” as an error.
- Support for a scale-out backup repository Cloud Connect Immutable parser.
- Ability to attach EML files to Daily Jobs and Job Details.
- Fix for Light/Dark theme switching so users can properly change themes.
- Restrict ticket creation and editing to Operator and Admin roles only.

### Known Bugs
- Emails that were previously approved remain in the Inbox instead of being removed, even though they appear auto-approved and linked to Jobs.

---

## v0.1.5

### Overview
This release focuses on restoring Microsoft Graph functionality, improving application reliability, and introducing a robust reset mechanism to allow a clean restart of the application state.

### Key Fixes
- Restored Microsoft Graph folder retrieval by fixing an incorrect import that caused a `ModuleNotFoundError`.
- Resolved failures in the automatic mail importer caused by `signal`-based timeout handling by replacing it with a thread-safe mechanism.
- Fixed backend startup crashes and Bad Gateway errors related to the automatic mail importer.
- Implemented missing backend logic required for automatic imports to function correctly.

### New Features
- Added an **Application Reset** option in the Settings page.
- Introduced a confirmation step to prevent accidental resets.

### Improvements & Changes
- Implemented full backend support for a complete application reset.
- Reset now clears all application data, including:
  - Approved and pending jobs
  - Imported and processed emails
  - Daily job runs
  - Logs
  - User-defined settings (system defaults are preserved)
- Ensured database cleanup runs in the correct order to respect foreign key constraints.
- Aligned automatic mail import logic with the existing manual import flow for consistent behavior.
- Applied the automatic import cutoff date directly via a Microsoft Graph `$filter`, leaving older emails untouched in the inbox.

### Result
- Graph API functionality is fully restored.
- Automatic mail import runs reliably on its configured schedule.
- The application can now be safely reset to a clean, fresh-install state when needed.

---

## v0.1.4

This release focuses on improving backend stability, database reliability, and consistency in object parsing and mail handling.

### Key Changes
- Database migrations for tickets and remarks were stabilized by running each migration in its own transaction scope, preventing closed-connection errors during startup.
- Backend startup issues causing Gunicorn failures and 502 Bad Gateway errors were resolved.
- The title field was fully removed from tickets and remarks, simplifying both backend validation and UI forms to use only descriptive content.
- Manual mail imports were aligned with the **Re-parse all** behavior, ensuring immediate and consistent object detection.
- Object visibility on the Daily Jobs page was corrected for previously approved jobs.
- Manual imports were hardened against Microsoft Graph timeouts by adding DNS preflight checks and safely skipping message moves when Graph is unreachable.

### Improvements
- Eliminated `ResourceClosedError` exceptions during backend boot.
- Increased reliability of migrations and overall application startup.
- Ensured object parsing is consistently re-evaluated on every job run, with correct detection of added or removed objects.
- Prevented internal server errors and Gunicorn worker timeouts caused by long-running external Graph operations.

Overall, v0.1.4 significantly improves robustness, consistency, and fault tolerance across database migrations, job parsing, and manual mail imports.

---

## v0.1.3

### Logging & Stability
- Fixed logging persistence so log entries are consistently stored in the database.
- Resolved cases where certain log events were not stored due to object lifecycle handling.
- Improved reliability of log creation during background/asynchronous processes.
- Corrected log retrieval so stored logs are properly fetched and shown in the web UI.
- Added pagination to the logging overview (20 entries per page).
- Extended the logging view to show all available log fields and fixed missing columns in the UI.

### UI & Table Layout Improvements
- Improved the logging page usability by placing pagination controls at the top and keeping them available at the bottom.
- Increased logging table width to better fit a 1080p layout.
- Fixed column layout so all columns remain in consistent positions regardless of content length.
- Updated status styling to use colored text only within the status column (Success, Warning, Error/Failed, Missed), including clear differentiation for overrides.
- Fixed JavaScript errors in the Daily Jobs popup that prevented rendering.

### Jobs & Daily Jobs Enhancements
- Standardized default sorting for both Jobs and Daily Jobs tables (Customer → Backup → Type → Job name).
- Persisted the Daily Jobs start date setting in the database and ensured it reloads correctly in the Settings UI.
- Corrected missed-status calculation to start from the configured Daily Jobs start date.
- Improved Daily Jobs table readability:
  - Moved the number of runs into a dedicated Runs column.
  - Prevented layout shifting caused by variable text in the Last result column.
  - Restored the original Runs visual representation and adjusted placement for better readability.
  - Reduced the Last result column width so only status text is shown and the Runs column remains visible.

### Parsing & Data Normalization
- Stripped retry suffixes like “(Retry 1)”, “(Retry 2)”, etc. from job names so retries don’t create separate job identities.
- Extended the NAKIVO parser to support VMware Replication job emails:
  - Detects job type (Backup vs Replication) based on email content.
  - Improves replication job name parsing.
  - Extracts VM names from the Objects/Virtual Machines section.
  - Maps overall job status correctly for replication reports.

### Tickets & Remarks (New Feature Area)
- Added database schema for globally unique, persistent tickets linked to job runs for long-term reporting:
  - New tables: tickets, ticket_scopes, ticket_job_runs.
  - Ticket codes are globally unique using format TYYYYMMDD.NNNN.
  - Tickets require at least one customer scope.
- Added database schema for remarks with scoped attachment and persistent linkage to job runs:
  - New tables: remarks, remark_scopes, remark_job_runs.
- Implemented a new Tickets page with tabbed navigation (Tickets / Remarks):
  - Overviews with filtering.
  - Detail views showing scopes, linked job runs, and actions.
  - Added indicators in Daily Jobs to show active tickets and/or remarks.
  - Added ticket/remark management in the job run popup (create, edit, resolve).
  - Introduced consistent icons for tickets, remarks, actions, and status indicators.
- Added backend API endpoints for listing/creating/updating/resolving/linking tickets and remarks, plus an endpoint to retrieve all alerts for a specific job run.

---

## v0.1.2

This release focuses on improved parser support, more robust data cleanup, and a fully reworked in-app logging and object persistence system.

### Parser & support
- Extended the Synology Hyper Backup mail parser with proper recognition of Strato HiDrive backups.
- Added support for parsing job names from the “Backup Task:” field for Strato HiDrive.
- Correct handling of successful runs without listed objects.
- Added a Strato HiDrive example to the parser templates for validation and reference.

### Administration & cleanup
- Introduced an admin-only action to delete all jobs in a single operation.
- Ensured related run mails are moved back to the Inbox when jobs are deleted.
- Fixed foreign key constraint issues by enforcing the correct deletion order:
  - run_object_links first
  - job_object_links next
  - then job runs and jobs
- Stabilized the “Delete all jobs” action to fully clean up all related data.

### Logging (reworked)
- Moved logging away from container/stdout logging to in-app logging.
- Introduced AdminLog-based logging for:
  - Mail import
  - Auto-approval
  - Manual job approval
  - Job deletion
- Added detailed logging per imported and auto-approved email.
- Added summary logging at the end of each mail import run.
- Ensured all relevant events are logged exclusively via the AdminLog table and visible on the Logging page.

### Object persistence
- Restored persistence of parsed objects after manual approval of inbox mails.
- Restored persistence of parsed objects during auto-approval (reparse-all).
- Ensured objects from approved mails are:
  - Upserted into `customer_objects`
  - Linked to jobs via `job_object_links` (with first/last seen tracking)
  - Linked to runs via `run_object_links` (with status and error details)
- Added centralized helper logic to ensure consistent object persistence.
- Added an admin-only maintenance action to backfill missing object links for already approved runs.
- Object persistence failures no longer block mail approval.
- Daily Jobs and Run detail views correctly display objects again for both new and historical runs after backfilling.