backupchecks/docs/changelog-claude.md

Changelog - Claude Code

This file documents all changes made to this project via Claude Code.

[2026-02-08]

Added

• Completed Mail & Import documentation section (4 pages):
  • Mail Import Setup: Microsoft Graph API configuration with enhanced security (Azure AD app registration, start with Mail.Read only, Application Access Policy to restrict access to one mailbox via Exchange PowerShell, add Mail.ReadWrite after testing), two-step save workflow clarification (same "Save settings" button clicked twice - once for credentials to enable folder browsing, then again for complete configuration), automatic folder browser popup (no manual path entry), test configuration, troubleshooting (authentication errors, folder not found, no emails imported), advanced configuration (email retention, multiple mailboxes), comprehensive security best practices including principle of least privilege, note about planned UI improvement to make two-step save workflow more obvious
  • Inbox Management: Inbox overview and workflow (inbox shows ONLY unapproved emails - approved emails immediately disappear), table columns explanation (EML column shows download link, not checkmark), viewing email details with screenshot (approve-job.png showing metadata, body iframe, and approval interface), email detail modal sections (metadata with action buttons, body iframe, parsed objects - removed incorrect "Customer" and "Approved" fields), approving emails (customer selection, job name), what happens after approval (email disappears from inbox, future emails auto-approved and never appear in inbox), re-parsing emails (single and bulk), deleting emails (single and bulk delete, soft delete behavior), downloading .eml files, common workflows (new customer setup with clear approval flow, troubleshooting unparsed emails, cleaning spam), best practices
  • Mail Parsing: Parsing overview and workflow (retrieval, preprocessing, parser selection, matching, parsing, storage), parsers (viewing available parsers, parser information), match criteria (sender, subject, body match), supported backup software (Veeam, Synology, NAKIVO, Boxafe, Panel3, QNAP, Syncovery, 3CX, RDrive, NTFS Auditing), parser execution order, enabling/disabling parsers, re-parsing emails, parsing workflow example, troubleshooting parsing issues, parser limitations, best practices
  • Auto-Import Configuration: Auto-import overview and features, enabling auto-import, import interval configuration (5/15/30/60 minutes), batch size explanation, how auto-import works (timer, check settings, authenticate, fetch, parse, auto-approve, move, log, persist, wait), auto-approval logic and conditions, when auto-approval fails, manual import (triggering, batch size 1-50), import settings summary table, monitoring import activity (audit log, inbox page, jobs/daily jobs), troubleshooting (auto-import not running, emails not auto-approved, import errors), best practices, advanced configuration (disabling temporarily, EML retention, high-volume environments)

Added

• Added comprehensive documentation system for user onboarding and reference:
  • Documentation Blueprint: New /documentation route with dedicated blueprint (doc_bp)
  • Navigation Structure: Hierarchical documentation with 9 sections and 33 pages covering all features
    • Getting Started (3 pages): What is BackupChecks, First Login, Quick Start
    • User Management (3 pages): Users & Roles, Authentication, Profile Settings
    • Customers & Jobs (4 pages): Managing Customers, Configuring Jobs, Approved Jobs, Job Schedules
    • Mail & Import (4 pages): Setup, Inbox Management, Mail Parsing, Auto-Import
    • Backup Review (5 pages): Approving Backups, Daily Jobs, Run Checks Modal, Overrides, Remarks & Tickets
    • Reports (4 pages): Creating Reports, Relative Periods, Scheduling, Exporting Data
    • Autotask Integration (4 pages): Setup, Company Mapping, Creating Tickets, Ticket Management
    • Settings (6 pages): General, Mail, Autotask, Reporting, User Management, Maintenance
    • Troubleshooting (3 pages): Common Issues, FAQ, Support Contact
  • UI Components:
    • Sidebar navigation with collapsible sections and active page highlighting
    • Breadcrumb navigation for current location context
    • Previous/Next pagination buttons for sequential reading
    • Documentation menu item in main navbar (📖 icon) visible to all authenticated users
  • Styling:
    • Custom CSS with support for dark mode
    • Callout boxes (info, warning, tip, danger) for highlighting important content
    • Code blocks, tables, and image support
    • Responsive design for mobile and desktop
  • Access Control: Login required (@login_required) - accessible to all user roles
  • Current Status: Core infrastructure complete, getting-started (3 pages), users (3 pages), and customers-jobs (4 pages) sections completed with comprehensive content
  • Placeholder Pages: Remaining 23 pages created with basic structure for future content
  • Full content for remaining sections will be added incrementally in future updates

Changed

• Enhanced documentation accuracy in Users section:
  • Login & Authentication page: Added captcha requirement explanation (simple math problem), noted future plan to make it optional via Settings
  • Browser Security section: Clarified HTTPS context - only relevant for external access via domain name, not for internal IP-based deployments (typical use case)
  • Browser Compatibility: Marked Mozilla Firefox as "Recommended - Most tested" browser for optimal experience
  • Profile Settings page: Streamlined to reflect actual functionality - only password change available in User Settings page, theme and role selectors located in navigation bar

Removed

• Removed non-existent features from Profile Settings documentation:
  • Notification Preferences section (no plans for implementation)
  • Session Information section (not displayed in User Settings)
  • Redundant theme/role configuration sections (these are in navbar, not settings page)

Added

• Completed Customers & Jobs documentation section (4 pages):
  • Managing Customers: Customer creation, editing, activation/deactivation, Autotask company mapping, CSV export/import, delete operations
  • Configuring Jobs: Inbox-based job approval workflow, Mail Parser automatic configuration, Reparse All functionality, job archiving and deletion
  • Approved Jobs: Jobs list overview, job details, status indicators, archive/unarchive workflow, JSON export/import for migration
  • Job Schedules: Automatic schedule learning, schedule types (daily/weekly/monthly), Daily Jobs integration, schedule accuracy and troubleshooting
• Added user-settings.png screenshot showing password change form
• Enhanced documentation CSS: centered all images horizontally (display: block, margin: 20px auto)

[2026-02-07]

Changed

• Renamed AdminLog to AuditLog for better semantic clarity:
  • Model: AdminLog → AuditLog (backwards compatible alias maintained)
  • Table: admin_logs → audit_logs (automatic migration)
  • Function: log_admin_event() → log_audit_event() (alias provided)
  • Better reflects purpose as comprehensive audit trail for both user and system events
• Updated UI labels to reflect audit log semantics:
  • Changed "Admin activity" to "System Audit Log" in logging page header

Added

• Expanded audit logging for critical operations:
  • Settings Changes: Now logs all changes to General, Mail, and Autotask settings
    • Tracks which settings changed with old value → new value comparison
    • Event types: settings_general, settings_mail, settings_autotask
    • Excludes sensitive data (passwords are never logged)
    • Example logged fields: ui_timezone, require_daily_dashboard_visit, is_sandbox_environment, graph_mailbox, autotask_enabled
  • Export Operations: Logs when users export data
    • Customers export (event type: export_customers): CSV format, record count
    • Jobs export (event type: export_jobs): JSON schema version, customer/job counts
  • Import Operations: Logs when users import data
    • Customers import (event type: import_customers): CSV format, created/updated/skipped counts
    • Jobs import (event type: import_jobs): JSON schema version, all operation counts (customers and jobs)
  • All logging uses structured event_type for filtering and includes detailed JSON in details field
  • Maintains 7-day retention policy
  • No performance impact (async logging)
  • Helps with compliance, troubleshooting, and security audits

[2026-02-06]

Added

• Added Sandbox/Development environment indicator feature:
  • Database Model: Added is_sandbox_environment boolean column to SystemSettings model (defaults to False for production safety)
  • Database Migration: Created migrate_system_settings_sandbox_environment() function for automatic schema update on deployment (idempotent, safe to run multiple times)
  • Backend Routes:
    • Extended routes_settings.py to process the new checkbox setting in the General tab
    • Updated routes_shared.py context processor to inject system_settings into all template contexts
  • Settings UI:
    • Added new "Environment" card in Settings > General (placed after Navigation section)
    • Toggle switch with clear English description: "Mark this as a Sandbox/Development environment"
    • Help text explains visual banner behavior
  • CSS Styling:
    • Created sandbox.css with diagonal red banner styling
    • Position: top-left corner, rotated 45 degrees
    • Color: Bootstrap danger red (#dc3545) with white text
    • Z-index: 9999 (always on top)
    • pointer-events: none - banner itself is non-interactive, elements behind it remain clickable
    • Box-shadow for depth effect
  • Base Template:
    • Banner conditionally displayed only when setting is enabled
    • CSS include added to <head> section
    • Banner placed directly after <body> tag, before navbar
  • The banner displays "SANDBOX" in uppercase with letter-spacing for clear visibility across all pages
• Auto-load Autotask reference data on Settings page load:
  • Automatically fetches queues, ticket sources, statuses, and priorities when opening Settings
  • Only triggers when Autotask is enabled, credentials are configured, and cache is empty
  • Eliminates need to manually click "Refresh reference data" before selecting defaults
  • Displays info message with loaded data counts
• Extended export/import functionality to include Autotask company mappings:
  • Customer Export/Import (CSV at /customers/export):
    • Export now includes autotask_company_id and autotask_company_name columns
    • Import reads Autotask mapping fields and applies them to existing or new customers
    • Backwards compatible - old CSV files without Autotask columns still work
    • Import resets autotask_mapping_status and autotask_last_sync_at to allow resynchronization
  • Jobs Export/Import (JSON at Settings > Maintenance):
    • Export now includes Autotask fields in customers array (autotask_company_id, autotask_company_name)
    • Import processes customers array first, applying Autotask mappings before creating jobs
    • Schema remains approved_jobs_export_v1 for backwards compatibility
    • Import message now shows both created and updated customer counts
  • Enables preservation of Autotask company mappings during system reset/migration workflows
• Implemented timezone-aware datetime display across all pages:
  • Template Filter: Added local_datetime Jinja2 filter to convert UTC datetimes to UI timezone
  • Automatic Conversion: All datetime fields now automatically display in configured timezone (Settings > General > UI Timezone)
  • Database: All datetime values remain stored in UTC for consistency
  • Affected Pages: Customers (Autotask sync time), Settings (reference data sync), Feedback, Logging, Overrides, Archived Jobs, Tickets, Remarks, Inbox, Job Detail, Admin Mail
  • Custom Format Support: Filter accepts strftime format parameter (e.g., |local_datetime('%d-%m-%Y %H:%M'))
  • Graceful Fallback: Falls back to UTC display if timezone conversion fails
  • Default Timezone: Europe/Amsterdam (configurable via SystemSettings.ui_timezone)

Changed

• Renamed "Refresh" button to "Search" in Link existing Autotask ticket modal for better clarity (the button performs a search operation)
• Added ellipsis-field functionality to "Overall remark" and "Remark" fields in Run Checks modal to prevent long text from hiding action buttons (click to expand/collapse)
• Enhanced Autotask ticket display in Run Checks modal:
  • Added "Open in Autotask" button next to ticket number for direct navigation to ticket in Autotask PSA
  • Button only appears when ticket is linked (autotask_ticket_id exists) and Autotask integration is enabled
  • Opens in new tab with proper URL format including workspace and ticket ID parameters
  • Styled as small outline button to maintain compact layout
• Merged all feature branches from v20260203-01 through v20260205-13 into main branch
  • Consolidated 29 feature branches spanning three development cycles
  • Used final state from v20260205-13-changelog-python-structure to preserve all functionality
  • Successfully integrated Autotask PSA integration, changelog restructuring, and UI improvements
  • All features from individual branches now available in main
• Reorganized Autotask settings into two separate forms with dedicated save buttons:
  • Autotask Settings form with "Save Autotask Settings" button inside the card
  • Ticket Defaults form with "Save Ticket Defaults" button inside the card
  • All fields marked as required with red asterisks (*)
  • HTML5 validation prevents saving incomplete configurations
  • Clear visual separation improves UX and prevents accidental saves

Fixed

• Jobs from archived jobs and inactive customers no longer appear on Daily Jobs, Run Checks, and Jobs list pages
  • Added customer active status filtering to all job list queries
  • Jobs now filtered where customer is NULL or active=True alongside existing archived=False filter
  • Prevents showing jobs with "-" status from deleted customers or archived jobs
• Default Ticket Status dropdown no longer appears empty on first visit to Settings page
• Default Ticket Status value is now protected against accidental clearing:
  • Only updates when a valid status is selected
  • Only allows clearing if reference data is loaded (dropdown has options)
  • Preserves existing value if dropdown is empty (prevents data loss)
  • Fixes issue where "Create Autotask ticket" failed due to missing default status after saving settings with empty dropdown

Removed

• Cleaned up 66 merged feature branches from repository (both local and remote):
  • Removed all v20260113-* branches (8 branches)
  • Removed all v20260115-* branches (17 branches)
  • Removed all v20260116-* branches (12 branches)
  • Removed all v20260119-* branches (18 branches)
  • Removed all v20260120-* branches (11 branches)
  • Repository now contains only main branch and current working branches (v20260206-*)

[2026-02-05]

Added

• Redesigned changelog system to use Python-based structure instead of Markdown:
  • Created app/changelog.py with structured changelog data (21 versions from v0.1.22 to v0.1.2)
  • Each version contains: version number, date, summary, and structured sections
  • Sections include: title, type (feature/improvement/fixed/documentation), and list of changes
  • Removed Gitea dependency for changelog rendering - now fully self-contained
  • No external dependencies, faster loading, always available
• New changelog.html template with modern design:
  • Sidebar navigation with all versions for quick jumping between releases
  • Sticky sidebar that remains visible during scrolling
  • Bootstrap cards for each version with gradient blue headers
  • Color-coded type badges for sections:
    • Green gradient: Feature
    • Blue gradient: Improvement
    • Red gradient: Fixed
    • Purple gradient: Documentation
  • Responsive design (sidebar hidden on mobile devices)
  • Summary section with blue left border highlight
  • Click-to-expand sections with smooth animations
• Created static/css/changelog.css with comprehensive styling:
  • Modern gradients for badges and headers
  • Dark mode support via CSS variables
  • Hover effects on navigation links and list items
  • Smooth scrolling to version anchors
  • Compact spacing optimizations (reduced padding, margins, font sizes)
  • CSS specificity enhancements with !important flags to override Bootstrap defaults
• Added {% block head %} to base.html template to allow pages to inject custom CSS

Changed

• Updated routes_changelog.py to load data from changelog.py instead of fetching from Gitea
• Simplified changelog route - removed markdown parsing and external HTTP requests
• Removed dependency on markdown library for changelog rendering
• Template now receives structured Python data instead of HTML string

Fixed

• Fixed module import path in routes_changelog.py (changed from from app.changelog to from ..changelog)
• Fixed dictionary key conflict - renamed items to changes to avoid collision with dict.items() method
• Added missing {% block head %} in base.html that prevented custom CSS from loading

Technical Details

• Changelog data structure uses dictionaries with keys: version, date, summary, sections
• Sections contain: title, type, subsections (optional), changes
• Subsections contain: subtitle, changes
• All list items use "changes" key instead of "items" to avoid Python reserved method conflicts
• CSS uses !important flags and increased specificity (.changelog-nav .changelog-nav-link) to override Bootstrap
• Compact spacing achieved with: 0.15rem padding, 0.15rem margins, 0.85rem/0.7rem font sizes, 1.1/1.0 line heights

Added

• Autotask customer mapping now auto-searches for similar company names when opening unmapped customers:
  • Automatically populates search box with customer name
  • Displays matching Autotask companies as suggestions
  • Speeds up mapping process by eliminating manual search for most customers
• Autotask "Link existing ticket" now supports cross-company ticket search:
  • Added query_tickets_by_number() to search tickets by number across all companies
  • When searching with a ticket number (e.g., "T20260205.0001"), results include:
    • Tickets from the customer's company (primary results)
    • Matching tickets from other companies (for overarching issues)
  • Enables linking tickets for multi-company infrastructure issues

Changed

• Autotask resolve confirmation and note messages now correctly indicate ticket closure status:
  • Frontend confirmation dialog explains conditional closure based on time entries
  • Backend route checks time entries before creating note and generates dynamic message:
    • "ticket will be closed in Autotask" when no time entries exist
    • "ticket remains open in Autotask due to existing time entries" when time entries exist
  • Route docstring updated to reflect conditional status update behaviour

Added

• Autotask conditional ticket status update based on time entries (API contract section 9):
  • query_time_entries_by_ticket_id() - Query time entries for a ticket via POST /TimeEntries/query
  • update_ticket_resolution_safe() now checks for time entries and conditionally sets status:
    • If NO time entries exist: sets status to 5 (Complete) with completedDate and resolvedDateTime
    • If time entries exist: keeps current status unchanged (ticket remains open)

Fixed

• Automatic mail import can now be disabled in Settings after being enabled (fixed unchecked checkbox not being processed)
• Autotask "Link existing" search box now clears when opening the modal instead of retaining previous search text
• Autotask customer mapping search box now clears when opening the edit modal instead of retaining previous search text
• Autotask ticket resolution update now correctly preserves exact field values from GET response in PUT payload. The issueType, subIssueType, and source fields are copied with their exact values (including null) from the GET response, as required by Autotask API. Previously these fields were being skipped or modified.

Added

• Restored Autotask PSA integration from branch v20260203-13-autotask-resolution-item-wrapper:
  • integrations/autotask/client.py - Autotask REST API client with full support for:
    • Zone information discovery
    • Ticket CRUD operations (create, get, update)
    • Ticket notes via /Tickets/{id}/Notes endpoint
    • Safe resolution updates preserving stabilizing fields
    • Query support for companies, tickets, time entries, deleted ticket logs
    • Reference data retrieval (queues, ticket sources, priorities, statuses)
  • ticketing_utils.py - Utilities for internal ticket management and Autotask linkage
  • Database migrations for Autotask fields:
    • SystemSettings: Autotask connection settings, defaults, and cached reference data
    • Customer: Autotask company mapping fields
    • JobRun: Autotask ticket linkage and deletion tracking fields
    • Ticket: Resolution origin tracking
  • Settings UI for Autotask configuration (connection test, reference data sync)
  • Run Checks integration for Autotask ticket creation
  • Customers page with Autotask company mapping
  • Documentation files for Autotask integration design and implementation
• Added docs/autotask_rest_api.md - Validated Autotask REST API contract based on Postman testing

[2026-02-04]

Added

• docs/changelog-claude.md - Changelog file for tracking changes made via Claude Code
• Setting to enable/disable daily dashboard redirect requirement (Settings > General > Navigation)
  • New require_daily_dashboard_visit column in SystemSettings model
  • Migration in migrations.py to add the column
  • Toggle in Settings General page under new "Navigation" card
  • Default value is OFF (disabled) - users can navigate directly to any page

Changed

• Converted changelog to English (all project documentation must be in English)
• Documented branch naming convention and build workflow in Claude memory
• Filled README.md with comprehensive project documentation based on source code analysis

Performance

• Added database indexes migration (migrations.py) for frequently queried foreign key columns:
  • JobRun: indexes on job_id, job_id+run_at, job_id+reviewed_at, mail_message_id
  • MailMessage: indexes on job_id, location, job_id+location
  • MailObject: index on mail_message_id
  • TicketScope: indexes on ticket_id, job_id
  • RemarkScope: indexes on remark_id, job_id
• Fixed N+1 query in _recompute_override_flags_for_runs() - batch loads all jobs instead of per-run queries
• Optimized Daily Jobs page with batch queries:
  • Batch load all today's runs for all jobs in single query
  • Batch infer weekly schedules for all jobs (was per-job query)
  • Batch infer monthly schedules for jobs without weekly schedule
  • Batch load ticket/remark indicators for all jobs