ivooskamp/backupchecks
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
5700f795e4
backupchecks/TODO-reports-improvements.md
Ivo Oskamp 8ebfb9e90f Add comprehensive TODO for reports improvements 
Complete planning document for reporting feature expansion including:

Phase 1 - Settings > Reporting:
- Email settings (from_email, from_name, reply_to)
- Branding (logo upload, company_name, brand_color)
- Footer customization (contact info, footer text)
- Default email template with HTML support
- Test email function with Graph API permission check

Phase 2 - Relative Periods:
- 15 relative period options (yesterday to previous year)
- Timezone-aware period calculation (follows ui_timezone)
- "To date" periods end at yesterday 23:59 (complete days only)
- Email body template per report (HTML with placeholders)
- Template validation and XSS protection

Phase 3 - Scheduling (Future):
- Weekly, Monthly, Quarterly, Yearly frequencies
- Per-report scheduling configuration
- Graph API email sending (no SMTP/SPF issues)
- Retry logic (max 3 attempts)
- Audit logging for deliveries

Includes:
- Complete implementation plan with code snippets
- Database model changes (25+ fields)
- UI mockups for Settings and report creation
- 60+ test cases covering all features
- Success criteria per phase
- All design decisions documented

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-07 23:36:23 +01:00

1159 lines
46 KiB
Markdown
Raw Blame History

# TODO: Reports Improvements
**Branch:** `v20260207-01-reports-improvements`
**Datum:** 2026-02-07
**Status:** Planning fase
---
## ✅ Wat is al gedaan
- ✅ Scheduling placeholder verwijderd van reports overview pagina (reports.html)
- ✅ Report definitions tabel gebruikt nu volle breedte
---
## 🔄 Scope Uitbreiding: Report Settings & Branding
**Nieuwe requirement:** Settings sectie voor reporting met email/branding configuratie.
---
## 🔄 Wat moet nog: Relative Period Selector
### Doel
Gebruikers moeten relatieve periodes kunnen kiezen (zoals "Previous month", "Last 7 days") in plaats van alleen vaste datums. Dit is essentieel voor scheduled reports: als je een maandelijks rapport hebt met "Previous month", wordt elke keer automatisch de juiste maand berekend.
### UI Design
**Period selector in reports_new.html:**
```
┌─ Reporting period ────────────────────────────────────┐
│ │
│ Period type: │
│ ○ Relative period (recommended for scheduled reports)│
│ └─ Select period: │
│ [Dropdown ▼] │
│ --- Recent periods --- │
│ - Yesterday │
│ - Last 7 days │
│ - Last 14 days │
│ - Last 30 days │
│ - Last 90 days │
│ - Last 6 months │
│ --- Weeks --- │
│ - Last week (full) │
│ - Current week (to date) │
│ --- Months --- │
│ - Previous month (full) │
│ - Current month (to date) │
│ --- Quarters --- │
│ - Last quarter (full) │
│ - Current quarter (to date) │
│ --- Years --- │
│ - Previous year (full) │
│ - Current year (to date) │
│ │
│ ○ Custom date range (for one-time reports) │
│ └─ Start: [date] [time] │
│ End: [date] [time] │
│ │
│ [Preset: Current month] [Last month] [Last month full]│
└───────────────────────────────────────────────────────┘
```
### Voordelen
- ✅ Scheduled reports werken automatisch zonder datum aanpassingen
- ✅ Duidelijk zichtbaar welke periode gebruikt wordt ("Previous month" ipv "2026-01-01 to 2026-01-31")
- ✅ Eenvoudiger te configureren
- ✅ Minder foutgevoelig
- ✅ Betere UX voor recurring reports
---
## 📋 Implementatie Plan (Prioriteit)
### Phase 1: Settings > Reporting Section
#### 1A. Database Model - SystemSettings
**Bestand:** `containers/backupchecks/src/backend/app/models.py`
**Add reporting_* fields** (see "Database Model - SystemSettings" section above)
#### 1B. Database Migration
**Bestand:** `containers/backupchecks/src/backend/app/migrations.py`
```python
def migrate_add_reporting_settings():
"""Add reporting configuration fields to system_settings table."""
pass
```
#### 1C. Settings UI
**Bestand:** `containers/backupchecks/src/templates/main/settings.html`
- Add new "Reporting" section/tab
- Email settings form (from_email, from_name, reply_to)
- Branding form (company_name, logo upload, brand_color)
- Footer form (contact info, footer text)
- Save button + validation
#### 1D. Settings Routes
**Bestand:** `containers/backupchecks/src/backend/app/main/routes_settings.py`
- Extend POST /settings to handle reporting_form_touched
- Add POST /settings/reporting/test-email route
- Logo upload handling (validate, store blob)
- Email validation
### Phase 2: Relative Periods
#### 2A. Database Model - Report
**Bestand:** `containers/backupchecks/src/backend/app/models.py`
**Huidige Report model velden:**
- `period_start` (String) - ISO datetime
- `period_end` (String) - ISO datetime
**Nieuwe velden toevoegen:**
- `period_type` (String) - "relative" of "custom"
- `relative_period` (String) - "previous_month", "last_7_days", etc.
- `email_body_template` (Text) - Custom HTML email body with placeholders
- `schedule_enabled` (Boolean) - Is scheduling active for this report?
- `schedule_frequency` (String) - "weekly", "monthly", "quarterly", "yearly"
- `schedule_time` (String) - "HH:MM" in ui_timezone
- `schedule_day_of_week` (Integer) - 0-6 (Monday-Sunday) for weekly
- `schedule_day_of_month` (Integer) - 1-31 for monthly
- `schedule_month` (Integer) - 1-12 for yearly
- `schedule_recipients` (String) - Comma-separated email addresses
- `schedule_last_run_at` (DateTime) - Last successful run
- `schedule_next_run_at` (DateTime) - Calculated next run time
- `schedule_last_error` (String) - Last error message
- `schedule_retry_count` (Integer) - Current retry count (max 3)
**Migratie:**
- `migrate_add_report_relative_periods()` functie
- Bestaande reports krijgen `period_type="custom"` (backwards compatible)
- Idempotent, veilig te draaien
#### 2B. Backend Routes Update
**Bestand:** `containers/backupchecks/src/backend/app/main/routes_reports.py`
**POST/PUT /api/reports:**
- Accept new fields: `period_type`, `relative_period`
- Validate relative_period values
- Store in database
**POST /api/reports/{id}/generate:**
- Als `period_type == "relative"`:
- Calculate concrete dates from `relative_period`
- Use calculated dates for report generation
- Als `period_type == "custom"`:
- Use existing `period_start` and `period_end`
#### 2C. Relative Period Calculator
**Nieuw bestand:** `containers/backupchecks/src/backend/app/report_periods.py`
```python
def calculate_period(relative_period: str, ui_timezone: str = "Europe/Amsterdam") -> tuple[datetime, datetime]:
"""
Calculate concrete start/end dates for a relative period.
Returns (start_datetime, end_datetime) in the specified timezone, then converted to UTC.
IMPORTANT: All calculations use ui_timezone (not UTC).
"To date" periods go up to END OF YESTERDAY (23:59:59) for complete days.
Example for ui_timezone="Europe/Amsterdam" on 2026-02-07:
- yesterday: 2026-02-06 00:00:00 → 2026-02-06 23:59:59 (Amsterdam time)
- last_7_days: 2026-01-31 00:00:00 → 2026-02-06 23:59:59 (Amsterdam time)
- current_month: 2026-02-01 00:00:00 → 2026-02-06 23:59:59 (Amsterdam time)
Supported periods:
- yesterday: Yesterday full day (00:00 to 23:59)
- last_7_days: 7 days ago 00:00 to yesterday 23:59
- last_14_days: 14 days ago 00:00 to yesterday 23:59
- last_30_days: 30 days ago 00:00 to yesterday 23:59
- last_90_days: 90 days ago 00:00 to yesterday 23:59
- last_6_months: 6 months ago 00:00 to yesterday 23:59
- last_week: Previous week Monday 00:00 to Sunday 23:59
- current_week: This week Monday 00:00 to yesterday 23:59
- previous_month: Full previous month (1st 00:00 to last day 23:59)
- current_month: Current month 1st 00:00 to yesterday 23:59
- last_quarter: Previous quarter first day 00:00 to last day 23:59
- current_quarter: Current quarter first day 00:00 to yesterday 23:59
- previous_year: Full previous year (Jan 1 00:00 to Dec 31 23:59)
- current_year: Current year Jan 1 00:00 to yesterday 23:59
"""
pass
```
#### 2D. UI Update - reports_new.html
**Bestand:** `containers/backupchecks/src/templates/main/reports_new.html`
**Changes:**
- Add radio buttons: "Relative period" vs "Custom date range"
- Add dropdown for relative periods (only visible when "Relative" selected)
- Show/hide date pickers based on selection
- Update JavaScript to handle new fields
- Update validation logic
- Preserve presets for custom date range mode
**JavaScript changes:**
- `selectedPeriodType()` function
- `updatePeriodUi()` function to show/hide sections
- Update `createReport()` payload building
- Update `applyInitialReport()` for edit mode
- Add email body template field with placeholders help text
#### 2E. UI Update - reports.html
**Bestand:** `containers/backupchecks/src/templates/main/reports.html`
**Changes:**
- Display relative period label instead of dates when `period_type == "relative"`
- Format: Badge with label - Example: `[Previous month ●]` where ● is a small badge/indicator
- Show tooltip on hover: "Automatically calculated on each generation"
- For custom dates: Keep current format "2026-01-01 → 2026-01-31"
### Phase 3: Email Sending (Future - Scheduling)
#### 3A. Graph API Email Sender
**Nieuw bestand:** `containers/backupchecks/src/backend/app/report_email.py`
See "Graph API Email Sending" section above for implementation details.
#### 3B. Template Processor
**Functie om placeholders te vervangen:**
```python
import re
import html
from typing import Dict, List
ALLOWED_PLACEHOLDERS = {
'customer_name', 'period', 'total_jobs', 'success_count',
'failed_count', 'success_rate', 'company_name', 'logo_base64'
}
def validate_email_template(template: str) -> tuple[bool, List[str]]:
"""
Validate email template for invalid placeholders.
Returns: (is_valid, list_of_invalid_placeholders)
"""
if not template:
return True, []
# Find all placeholders
placeholders = re.findall(r'\{([^}]+)\}', template)
invalid = [p for p in placeholders if p not in ALLOWED_PLACEHOLDERS]
return len(invalid) == 0, invalid
def process_email_template(template: str, context: dict, settings) -> str:
"""
Replace placeholders in email template with actual values.
Placeholders: {customer_name}, {period}, {total_jobs}, etc.
Sanitizes HTML to prevent XSS.
"""
if not template:
# Use default template from settings
template = settings.reporting_default_email_template or get_hardcoded_default_template()
# Add logo to context if not present
if 'logo_base64' not in context and settings.reporting_logo_blob:
mime_type = settings.reporting_logo_mime_type or 'image/png'
logo_b64 = base64.b64encode(settings.reporting_logo_blob).decode('utf-8')
context['logo_base64'] = f"data:{mime_type};base64,{logo_b64}"
else:
context['logo_base64'] = "" # Empty if no logo
# Add company name
context['company_name'] = settings.reporting_company_name or "Backup Reports"
# Escape HTML in context values (prevent XSS from data)
safe_context = {k: html.escape(str(v)) for k, v in context.items() if k != 'logo_base64'}
safe_context['logo_base64'] = context.get('logo_base64', '') # Don't escape base64
# Replace placeholders
result = template
for key, value in safe_context.items():
result = result.replace(f'{{{key}}}', str(value))
# Sanitize HTML (allow safe tags only)
result = sanitize_html(result)
return result
def sanitize_html(html_content: str) -> str:
"""
Sanitize HTML to prevent XSS while allowing basic formatting.
Allowed tags: <strong>, <em>, <u>, <br>, <p>, <ul>, <ol>, <li>, <a>, <img>
"""
# Use bleach library or similar
import bleach
allowed_tags = ['strong', 'em', 'u', 'br', 'p', 'ul', 'ol', 'li', 'a', 'img', 'div', 'span']
allowed_attrs = {
'a': ['href', 'title'],
'img': ['src', 'alt', 'style'],
'div': ['style'],
'span': ['style'],
'p': ['style']
}
return bleach.clean(
html_content,
tags=allowed_tags,
attributes=allowed_attrs,
strip=True
)
def get_hardcoded_default_template() -> str:
"""Fallback template if nothing configured."""
return """
<div style="font-family: Arial, sans-serif; max-width: 600px;">
<img src="{logo_base64}" alt="{company_name}" style="max-width: 200px; margin-bottom: 20px;" />
<p>Dear <strong>{customer_name}</strong>,</p>
<p>Attached is your backup report for <strong>{period}</strong>.</p>
<ul>
<li>Total jobs: {total_jobs}</li>
<li>Successful: {success_count}</li>
<li>Failed: {failed_count}</li>
<li>Success rate: {success_rate}%</li>
</ul>
<p>Best regards,<br/>{company_name}</p>
</div>
"""
```
**Dependencies:**
```bash
pip install bleach # For HTML sanitization
```
---
## ✅ Testing Checklist
### Settings > Reporting (Phase 1)
**Basic Settings:**
- [ ] Navigate to Settings > Reporting section
- [ ] Save from_email - validate email format
- [ ] Save from_name - verify stored correctly
- [ ] Save reply_to - validate email format (optional field)
- [ ] Leave reply_to empty - verify falls back to from_email
**Logo Upload:**
- [ ] Upload logo (PNG, 100 KB) - verify preview shows
- [ ] Upload logo (JPG, 200 KB) - verify preview shows
- [ ] Upload logo (SVG, 50 KB) - verify preview shows
- [ ] Upload logo too large (600 KB) - verify error message
- [ ] Upload invalid file type (TXT) - verify error rejected
- [ ] Remove logo - verify cleared and preview empty
**Branding:**
- [ ] Save company_name - verify stored
- [ ] Save brand_color - validate hex format (#1a73e8)
- [ ] Invalid brand_color (not hex) - verify error
- [ ] Brand color without # prefix - auto-add or error?
**Footer:**
- [ ] Save contact info (phone, email, website) - verify stored
- [ ] Save footer_text - verify stored
**Default Email Template:**
- [ ] Save default_email_template with placeholders - verify stored
- [ ] Invalid placeholder in template - verify validation error
- [ ] Live preview updates as typing - verify works
- [ ] Leave template empty - verify uses hardcoded default
**Graph API & Test Email:**
- [ ] Mail.Send permission NOT granted - verify error message when testing
- [ ] Mail.Send permission granted - verify test email works
- [ ] Click "Send test email" - verify email received
- [ ] Check test email has logo inline (base64)
- [ ] Check test email has company name
- [ ] Check test email has brand color (if applied to template)
- [ ] Check test email has footer text
- [ ] Update settings - verify changes reflected in new test email
- [ ] Test email uses default template - verify placeholders replaced
- [ ] Reporting_from_email different from graph_mailbox - verify works
### Relative Periods (Phase 2)
**Basic Functionality:**
- [ ] Create report with relative period "Previous month"
- [ ] Create report with relative period "Last 7 days"
- [ ] Create report with custom date range (backwards compatibility)
- [ ] Edit report: change from custom to relative
- [ ] Edit report: change from relative to custom
- [ ] Edit existing custom report - should still work (backwards compatibility)
**Period Calculation (verify end date = yesterday 23:59):**
- [ ] Yesterday: verify full day
- [ ] Last 7 days: verify 7 days ago to yesterday
- [ ] Last 14 days: verify 14 days ago to yesterday
- [ ] Last 30 days: verify 30 days ago to yesterday
- [ ] Last 90 days: verify 90 days ago to yesterday
- [ ] Last 6 months: verify 6 months ago to yesterday
- [ ] Last week (full): verify previous Monday-Sunday
- [ ] Current week (to date): verify this Monday to yesterday
- [ ] Previous month (full): verify full previous month
- [ ] Current month (to date): verify 1st to yesterday
- [ ] Last quarter (full): verify previous quarter
- [ ] Current quarter (to date): verify quarter start to yesterday
- [ ] Previous year (full): verify full previous year
- [ ] Current year (to date): verify Jan 1 to yesterday
**Timezone Handling:**
- [ ] Set ui_timezone to "Europe/Amsterdam" - verify periods calculated in Amsterdam time
- [ ] Set ui_timezone to "America/New_York" - verify periods calculated in NY time
- [ ] Set ui_timezone to "UTC" - verify periods calculated in UTC
- [ ] Verify generated report shows correct timezone in period description
**UI/UX:**
- [ ] Generate report with relative period - verify correct dates in output
- [ ] Generate same relative period report on different days - verify dates change automatically
- [ ] Check reports overview displays relative period with badge correctly
- [ ] Check tooltip shows "Automatically calculated on each generation"
- [ ] Check raw data modal works with relative periods
- [ ] Check CSV export filename includes relative period or calculated dates
- [ ] Check PDF/HTML export shows relative period in header
**Edge Cases:**
- [ ] Generate "current_month" on 1st day of month - verify works
- [ ] Generate "last_quarter" on first day of new quarter - verify correct previous quarter
- [ ] Generate report in different timezone than when created - verify still correct
**Email Body Template (Per Report):**
- [ ] Create report with custom email_body_template (HTML)
- [ ] Leave email_body_template empty - verify uses default from Settings
- [ ] Use all placeholders in template - verify replaced correctly
- [ ] Invalid placeholder in template - verify validation error on save
- [ ] HTML tags in template - verify sanitized (XSS-safe)
- [ ] Dangerous HTML (script tag) - verify stripped/rejected
- [ ] Live preview in reports_new.html - verify shows sample data
- [ ] Edit report - change email_body_template - verify saved
- [ ] Logo placeholder {logo_base64} - verify renders inline image
- [ ] No logo uploaded in Settings - verify {logo_base64} is empty string
---
## 🎯 Relative Period Options
**IMPORTANT:** All periods use ui_timezone (from SystemSettings). "To date" periods end at **yesterday 23:59:59** (complete days only).
### Recent Periods
| Key | Label | Description |
|-----|-------|-------------|
| `yesterday` | Yesterday | Yesterday 00:00 to yesterday 23:59 |
| `last_7_days` | Last 7 days | 7 days ago 00:00 to yesterday 23:59 |
| `last_14_days` | Last 14 days | 14 days ago 00:00 to yesterday 23:59 |
| `last_30_days` | Last 30 days | 30 days ago 00:00 to yesterday 23:59 |
| `last_90_days` | Last 90 days | 90 days ago 00:00 to yesterday 23:59 |
| `last_6_months` | Last 6 months | 6 months ago 00:00 to yesterday 23:59 |
### Week Periods
| Key | Label | Description |
|-----|-------|-------------|
| `last_week` | Last week (full) | Previous week Monday 00:00 to Sunday 23:59 |
| `current_week` | Current week (to date) | This week Monday 00:00 to yesterday 23:59 |
### Month Periods
| Key | Label | Description |
|-----|-------|-------------|
| `previous_month` | Previous month (full) | 1st day 00:00 to last day 23:59 of previous month |
| `current_month` | Current month (to date) | 1st day 00:00 to yesterday 23:59 |
### Quarter Periods
| Key | Label | Description |
|-----|-------|-------------|
| `last_quarter` | Last quarter (full) | Previous quarter Q1/Q2/Q3/Q4 first day 00:00 to last day 23:59 |
| `current_quarter` | Current quarter (to date) | Current quarter first day 00:00 to yesterday 23:59 |
### Year Periods
| Key | Label | Description |
|-----|-------|-------------|
| `previous_year` | Previous year (full) | Jan 1 00:00 to Dec 31 23:59 of previous year |
| `current_year` | Current year (to date) | Jan 1 00:00 to yesterday 23:59 |
**Total: 15 relative period options**
---
## 📝 Beslissingen Genomen
### ✅ Tijdstippen
- **Besluit**: Optie B - Tot einde van gisteren (volledige dagen)
- **Reden**: Complete days only, geen partial data van vandaag
- **Voorbeeld**: "Last 7 days" op 7 feb = 31 jan 00:00 tot 6 feb 23:59
### ✅ Extra Periods
- **Besluit**: Alle voorgestelde periods toevoegen (15 totaal)
- **Toegevoegd**: Yesterday, Last 14 days, Last 6 months, Week periods, Quarter periods
### ✅ Weergave in Reports Lijst
- **Besluit**: Optie A - Badge met label
- **Format**: `[Previous month ●]` met tooltip
- **Tooltip**: "Automatically calculated on each generation"
### ✅ Scheduling
- **Besluit**: Wordt later besproken
- **Nu**: Alleen relative periods voor handmatig genereren
- **Later**: Scheduling implementatie met relative periods
### ✅ Timezone
- **Besluit**: ui_timezone setting volgen (niet UTC)
- **Implementatie**: Alle berekeningen in configured timezone, dan converteren naar UTC
- **Voorbeeld**: "Previous month" in Amsterdam timezone, niet UTC month
### ✅ Email Body Template
- **Besluit**: Per rapport configureren (niet algemene setting)
- **Locatie**: In report definition (reports_new.html), niet in Settings
- **Reden**: Elk rapport kan andere tekst/tone hebben afhankelijk van klant
### ✅ Email Sending (Technical)
- **Besluit**: Graph API hergebruiken (geen SMTP)
- **Redenen**:
- Microsoft 365 Basic Authentication/SMTP wordt uitgefaseerd
- Geen SPF record aanpassingen nodig (DNS lookup limit van 10 al bereikt)
- Consistent met bestaande mail import functionaliteit
- **Implementatie**: Hergebruik bestaande Graph API client/token flow
---
## 📧 Settings > Reporting Section (NEW)
### Doel
Centrale configuratie voor report branding en email settings. Deze settings gelden voor **alle** scheduled reports (tenzij per rapport overschreven).
### UI Design - settings.html
Nieuwe tab/sectie: **"Reporting"** (naast General, Mail, Integrations, etc.)
```
┌─ Reporting Settings ─────────────────────────────────┐
│ │
│ ┌─ Email Settings ───────────────────────────────┐ │
│ │ │ │
│ │ From email address * │ │
│ │ [email@example.com________________________] │ │
│ │ The email address used to send reports. │ │
│ │ Must be a mailbox in your Microsoft 365 tenant. │ │
│ │ │ │
│ │ From name * │ │
│ │ [Acme Backup Services_________________] │ │
│ │ Display name shown in recipient's inbox. │ │
│ │ │ │
│ │ Reply-to address (optional) │ │
│ │ [support@example.com__________________] │ │
│ │ Where replies should go (leave empty to use │ │
│ │ from address). │ │
│ │ │ │
│ │ [Send test email] │ │
│ │ │ │
│ └──────────────────────────────────────────────────┘ │
│ │
│ ┌─ Report Branding ──────────────────────────────┐ │
│ │ │ │
│ │ Company name * │ │
│ │ [Acme Backup Services_________________] │ │
│ │ Displayed in report headers and footers. │ │
│ │ │ │
│ │ Company logo │ │
│ │ [Current: logo.png (45 KB)] │ │
│ │ [Choose file...] [Remove logo] │ │
│ │ Max 500 KB. PNG, JPG, or SVG. Recommended: │ │
│ │ 300x100px or similar landscape aspect ratio. │ │
│ │ │ │
│ │ Brand color (hex) │ │
│ │ [#] [1a73e8___] [Color picker] │ │
│ │ Primary color for HTML/PDF reports. │ │
│ │ │ │
│ │ ┌─ Preview ─────────────────────────────────┐ │ │
│ │ │ [LOGO] Acme Backup Services │ │ │
│ │ │ Backup Report - January 2026 │ │ │
│ │ └────────────────────────────────────────────┘ │ │
│ │ │ │
│ └──────────────────────────────────────────────────┘ │
│ │
│ ┌─ Report Footer ────────────────────────────────┐ │
│ │ │ │
│ │ Contact information (optional) │ │
│ │ Phone: [+31 20 123 4567_______________] │ │
│ │ Email: [info@example.com______________] │ │
│ │ Website: [https://example.com__________] │ │
│ │ │ │
│ │ Footer text (optional) │ │
│ │ ┌────────────────────────────────────────────┐ │ │
│ │ │ This is an automated report. For questions │ │ │
│ │ │ please contact our support team. │ │ │
│ │ │ │ │ │
│ │ └────────────────────────────────────────────┘ │ │
│ │ Shown at bottom of all reports. │ │
│ │ │ │
│ └──────────────────────────────────────────────────┘ │
│ │
│ ┌─ Default Email Template ───────────────────────┐ │
│ │ │ │
│ │ Used when report has no custom template. │ │
│ │ Supports HTML and placeholders. │ │
│ │ │ │
│ │ Template (HTML): │ │
│ │ ┌────────────────────────────────────────────┐ │ │
│ │ │ <div style="font-family: Arial;"> │ │ │
│ │ │ <img src="{logo_base64}" /> │ │ │
│ │ │ <p>Dear <strong>{customer_name}</strong> │ │ │
│ │ │ ... │ │ │
│ │ │ │ │ │
│ │ └────────────────────────────────────────────┘ │ │
│ │ │ │
│ │ Available placeholders: │ │
│ │ {customer_name}, {period}, {total_jobs}, │ │
│ │ {success_count}, {failed_count}, {success_rate},│ │
│ │ {company_name}, {logo_base64} │ │
│ │ │ │
│ │ ┌─ Live Preview ─────────────────────────────┐ │ │
│ │ │ [LOGO] Acme Backup Services │ │ │
│ │ │ │ │ │
│ │ │ Dear Customer Name, │ │ │
│ │ │ Attached is your report for January 2026. │ │ │
│ │ │ • Total jobs: 25 │ │ │
│ │ │ • Success rate: 96% │ │ │
│ │ └─────────────────────────────────────────────┘ │ │
│ │ │ │
│ └──────────────────────────────────────────────────┘ │
│ │
│ [Save Reporting Settings] │
│ │
└───────────────────────────────────────────────────────┘
```
### Database Model - SystemSettings
**Nieuwe velden toevoegen:**
```python
# Email settings
reporting_from_email = Column(String, nullable=True) # "reports@example.com"
reporting_from_name = Column(String, nullable=True) # "Acme Backup Services"
reporting_reply_to = Column(String, nullable=True) # "support@example.com"
# Branding
reporting_company_name = Column(String, nullable=True) # "Acme Backup Services"
reporting_logo_blob = Column(LargeBinary, nullable=True) # Logo image bytes
reporting_logo_filename = Column(String, nullable=True) # "logo.png"
reporting_logo_mime_type = Column(String, nullable=True) # "image/png"
reporting_brand_color = Column(String, nullable=True) # "#1a73e8"
# Footer
reporting_contact_phone = Column(String, nullable=True) # "+31 20 123 4567"
reporting_contact_email = Column(String, nullable=True) # "info@example.com"
reporting_contact_website = Column(String, nullable=True) # "https://example.com"
reporting_footer_text = Column(Text, nullable=True) # Custom footer text
# Email Template
reporting_default_email_template = Column(Text, nullable=True) # Default HTML email body
```
**Total: 11 nieuwe velden**
### Database Migration
**Bestand:** `migrations.py`
```python
def migrate_add_reporting_settings():
"""Add reporting configuration fields to system_settings table."""
# Check if columns already exist (idempotent)
# Add columns with ALTER TABLE
# Safe to run multiple times
```
### Backend Routes - routes_settings.py
**POST /settings (extend existing route):**
Detect reporting form submission:
```python
reporting_form_touched = any(k.startswith("reporting_") for k in request.form.keys())
if reporting_form_touched:
# Update reporting_* fields
# Handle logo upload (validate size, type)
# Validate email address (must be valid + in tenant?)
# Validate hex color format
db.session.commit()
```
**POST /settings/reporting/test-email:**
New route to send test email:
```python
@main_bp.route("/settings/reporting/test-email", methods=["POST"])
@login_required
@roles_required("admin")
def settings_reporting_test_email():
"""Send test report email to validate settings and permissions."""
settings = _get_or_create_settings()
# Validate required fields
if not settings.reporting_from_email:
return jsonify({"error": "From email not configured"}), 400
# Check Graph API Mail.Send permission
try:
has_permission = check_graph_mail_send_permission(settings)
if not has_permission:
return jsonify({
"error": "Mail.Send permission not granted. Please add it in Azure Portal."
}), 403
except Exception as e:
return jsonify({"error": f"Permission check failed: {str(e)}"}), 500
# Send test email to admin
test_email_body = render_test_email_template(settings)
try:
send_report_email(
to_address=current_user.email,
subject=f"Test Email - {settings.reporting_company_name or 'Backup Reports'}",
body_html=test_email_body,
attachment_bytes=b"", # No attachment for test
attachment_name=""
)
return jsonify({"success": True, "message": f"Test email sent to {current_user.email}"}), 200
except Exception as e:
return jsonify({"error": f"Failed to send test email: {str(e)}"}), 500
```
**Helper function:**
```python
def check_graph_mail_send_permission(settings) -> bool:
"""Check if Graph API has Mail.Send permission."""
access_token = _get_access_token(settings)
headers = _build_auth_headers(access_token)
# Try to get mailbox info - if this works, Mail.Send is likely granted
# Or parse token claims to check permissions
url = f"{GRAPH_BASE_URL}/users/{settings.reporting_from_email}"
response = requests.get(url, headers=headers)
if response.status_code == 403:
return False # No permission
elif response.status_code == 200:
return True # Has permission (simplified check)
else:
raise Exception(f"Permission check failed: HTTP {response.status_code}")
```
### Logo Upload Handling
**Validation:**
- Max file size: 500 KB
- Allowed formats: PNG, JPG, JPEG, SVG
- Store as blob in database (not filesystem - easier backup/restore)
**Preview:**
- Show thumbnail in settings UI
- Option to remove/replace logo
### Graph API Email Sending
**Reuse existing Graph API setup:**
```python
from ..mail_importer import _get_access_token, _build_auth_headers, GRAPH_BASE_URL
def send_report_email(to_address, subject, body_html, attachment_bytes, attachment_name):
"""
Send report email via Microsoft Graph API.
Uses SendMail endpoint with attachment.
Requires Mail.Send permission (already have for mail import).
"""
settings = _get_or_create_settings()
access_token = _get_access_token(settings)
# Build email payload
message = {
"message": {
"subject": subject,
"body": {
"contentType": "HTML",
"content": body_html
},
"toRecipients": [{"emailAddress": {"address": to_address}}],
"from": {
"emailAddress": {
"address": settings.reporting_from_email,
"name": settings.reporting_from_name
}
},
"attachments": [{
"@odata.type": "#microsoft.graph.fileAttachment",
"name": attachment_name,
"contentType": "application/pdf",
"contentBytes": base64.b64encode(attachment_bytes).decode()
}]
}
}
if settings.reporting_reply_to:
message["message"]["replyTo"] = [
{"emailAddress": {"address": settings.reporting_reply_to}}
]
# POST to /users/{from_email}/sendMail
url = f"{GRAPH_BASE_URL}/users/{settings.reporting_from_email}/sendMail"
response = requests.post(url, headers=headers, json=message)
# Handle response
```
### Report Model Extension
**Email body template per report:**
```python
# In Report model, add field:
email_body_template = Column(Text, nullable=True) # Custom email body with placeholders
```
**Placeholders supported:**
- `{customer_name}` - Customer name
- `{period}` - Report period (e.g. "January 2026" or "Last 7 days")
- `{total_jobs}` - Number of jobs in report
- `{success_count}` - Number of successful jobs
- `{failed_count}` - Number of failed jobs
- `{success_rate}` - Success percentage
- `{company_name}` - From reporting settings
**Default template if empty:**
```
Dear {customer_name},
Attached is your backup report for {period}.
Summary:
- Total jobs: {total_jobs}
- Successful: {success_count}
- Failed: {failed_count}
- Success rate: {success_rate}%
Best regards,
{company_name}
```
### UI Update - reports_new.html
**Add email body template field:**
After the "Report content" section, add new section:
```html
<hr class="my-4" />
<div class="fw-semibold mb-1">Email settings (for scheduled delivery)</div>
<div class="text-muted small mb-3">
Configure the email body for this report. Leave empty to use default template.
</div>
<div class="mb-3">
<label class="form-label">Email body template (optional)</label>
<textarea class="form-control font-monospace" id="rep_email_body" rows="10"
placeholder="Dear {customer_name},&#10;&#10;Attached is your backup report for {period}..."></textarea>
<div class="form-text">
Available placeholders:
<code>{customer_name}</code>,
<code>{period}</code>,
<code>{total_jobs}</code>,
<code>{success_count}</code>,
<code>{failed_count}</code>,
<code>{success_rate}</code>,
<code>{company_name}</code>
</div>
</div>
<div class="alert alert-info">
<strong>Note:</strong> Email settings like "From address" and "Logo" are configured in
<a href="{{ url_for('main.settings', section='reporting') }}">Settings > Reporting</a>.
</div>
```
---
## 📂 Belangrijke Bestanden
```
containers/backupchecks/src/backend/app/
├── models.py # Report model (add period_type, relative_period, email_body)
│ # SystemSettings model (add reporting_* fields)
├── migrations.py # Add migration functions (reports + settings)
├── report_periods.py # NEW: Period calculator
└── main/
├── routes_reports.py # API routes (accept/process new fields)
└── routes_settings.py # Settings routes (add reporting section handling)
containers/backupchecks/src/templates/main/
├── reports.html # Overview (display relative periods)
├── reports_new.html # Create/Edit (new period selector + email body template)
└── settings.html # Settings (add Reporting section)
docs/
└── changelog-claude.md # Changelog entry
```
---
## 🎯 Implementatie Fases Samenvatting
### Phase 1: Settings > Reporting (Start hier) ⭐
**Focus:** Basis configuratie voor report branding en email
- SystemSettings model uitbreiden (10 velden)
- Settings UI met Reporting sectie
- Logo upload functionaliteit
- Test email functie
- **Schatting:** Medium complexity
- **Dependencies:** Geen
### Phase 2: Relative Periods
**Focus:** Dynamische periode selectie
- Report model uitbreiden (3 velden)
- Period calculator met timezone support (15 periods)
- UI updates (reports_new.html + reports.html)
- Email body template per rapport
- **Schatting:** Medium-High complexity
- **Dependencies:** Phase 1 settings (voor email body preview)
### Phase 3: Email Sending (Toekomstig - Scheduling)
**Focus:** Automatisch versturen van reports
- Graph API email sender
- Template processor (placeholder replacement)
- Scheduling implementatie (wordt later besproken)
- **Schatting:** High complexity
- **Dependencies:** Phase 1 + Phase 2
---
## ✅ Alle Beslissingen - Scheduling & Email (Phase 3)
### Scheduling Configuration
**Frequentie opties:**
- ✅ Weekly (elke week op bepaalde dag)
- ✅ Monthly (elke maand op bepaalde dag, bijv. 1e)
- ✅ Quarterly (elk kwartaal)
- ✅ Yearly (elk jaar op bepaalde datum)
- ❌ GEEN Daily (niet nodig)
- ❌ GEEN Custom cron (te complex)
**Tijdstip:**
- ✅ Per rapport configureerbaar (niet centraal)
- Format: HH:MM in ui_timezone
- Voorbeeld: "08:00" in Amsterdam timezone
**Recipients (ontvangers):**
- ✅ Per rapport: comma-separated email adressen
- Format: `user1@example.com, user2@example.com`
- GEEN per-customer emails (te complex voor MVP)
**Delivery Failures:**
- ✅ Max 3 retry attempts (met exponential backoff?)
- ✅ Log in audit log (event_type: `report_delivery_failed`)
- ✅ Error indicator bij scheduled report in UI
- ✅ Admin kan failed reports zien en manually retry
**Generated Reports Opslag:**
- ✅ Instelling in Settings: bewaren ja/nee + aantal dagen
- ✅ Rapport kan opnieuw gegenereerd worden (on-demand)
- ❌ Oude versies worden NIET bewaard (altijd verse data bij regenerate)
### Email Body Template
**Live Preview:**
- ✅ Ja - toon live preview met sample data in UI
- Shows: sample customer name, period, statistics
- Updates in real-time as user types
**Invalid Placeholders:**
- ✅ Validation error bij opslaan (niet toegestaan)
- Toegestane placeholders: `{customer_name}`, `{period}`, `{total_jobs}`, `{success_count}`, `{failed_count}`, `{success_rate}`, `{company_name}`
- Foutmelding: "Invalid placeholder: {wrong_name}. Allowed: ..."
**HTML Support:**
- ✅ HTML toegestaan in email body template
- **Reden:** Professionele weergave met logo/opmaak → klanten vertrouwen legitimiteit
- **Veiligheid:** XSS-safe (sanitize user input)
- Toegestane tags: `<strong>`, `<em>`, `<u>`, `<br>`, `<p>`, `<ul>`, `<ol>`, `<li>`, `<a>`, `<img>` (voor logo)
- Voorbeeld default template:
```html
<div style="font-family: Arial, sans-serif;">
<img src="{logo_base64}" alt="{company_name}" style="max-width: 200px; margin-bottom: 20px;" />
<p>Dear <strong>{customer_name}</strong>,</p>
<p>Attached is your backup report for <strong>{period}</strong>.</p>
<ul>
<li>Total jobs: {total_jobs}</li>
<li>Successful: {success_count}</li>
<li>Failed: {failed_count}</li>
<li>Success rate: {success_rate}%</li>
</ul>
<p>Best regards,<br/>{company_name}</p>
</div>
```
**Default Template:**
- ✅ Ook configureerbaar in Settings > Reporting
- Field: `reporting_default_email_template` (Text)
- Used when report's `email_body_template` is NULL/empty
- Includes placeholders + HTML
### Logo & Branding
**Logo in Email:**
- ✅ Inline/embedded (base64 in HTML)
- **Reden:** Backupreports app is beveiligd, alleen toegankelijk vanaf bepaalde locaties
- **Method:** `<img src="data:image/png;base64,{base64_data}" />`
- Placeholder in template: `{logo_base64}` (automatically filled)
**Multiple Logos:**
- ❌ Nee - 1 logo voor alle reports (simpel, voldoende)
- Configured in Settings > Reporting
**Dark Mode:**
- ❌ Nee - standaard styling, geen dark mode support
- **Reden:** Reports als PDF attachment (PDF heeft geen dark mode)
### Graph API Permissions
**Mail.Send Permission:**
- ⚠️ **Nog niet toegekend** - moet toegevoegd worden in Azure Portal
- **Validatie bij opslaan:**
- Optie 1: Check Graph API permissions via API call
- Optie 2: Test email field in Settings (admin vult eigen email in, send test)
- **Keuze:** Beide - check permissions + test email button
**Mailbox Verschillend:**
- ✅ Ja - `reporting_from_email` KAN anders zijn dan `graph_mailbox`
- **Reden:** Klanten mogen NIET naar import mailbox mailen (wordt niet meer uitgelezen)
- **Voorbeeld:**
- Import mailbox: `backups@company.com` (voor import)
- Reporting mailbox: `reports@company.com` (voor sending)
**Impersonation:**
- ❌ Nee - altijd vanaf 1 centraal reporting mailadres
- ✅ Reply-to adres kan ingevuld worden
- **Voorbeeld:**
- From: `reports@company.com`
- Reply-To: `helpdesk@company.com`
- Klanten replyen naar helpdesk, niet naar reports mailbox
---
## 🚀 Volgende Stappen
### ✅ Planning Compleet!
Alle vragen zijn beantwoord. Het TODO bevat nu:
- ✅ 15 relative period options met timezone support
- ✅ Settings > Reporting sectie (11 velden)
- ✅ Email template systeem met HTML + placeholders
- ✅ Logo upload en branding
- ✅ Graph API email sending (geen SMTP)
- ✅ Scheduling specificaties (voor Phase 3)
- ✅ 60+ test cases
### Implementatie Start
**Aanbevolen volgorde:**
1. **Start Phase 1: Settings > Reporting**
- SystemSettings model + 11 velden
- Migratie functie
- Settings UI met nieuwe Reporting tab
- Logo upload handler
- Test email functie + Graph permission check
- Default email template editor
2. **Dan Phase 2: Relative Periods**
- Report model + 14 velden (periods + scheduling)
- Period calculator met timezone
- UI updates (period selector + email template)
- Template validation + sanitization
3. **Later Phase 3: Scheduling** (toekomstig)
- Scheduler daemon/cron
- Email sender via Graph API
- Retry logic + error handling
- Reports opslag
**Klaar om te beginnen?** We kunnen starten met Phase 1!
---
## 📝 Belangrijke Ontwerpbeslissingen
### Technisch
- **SPF/DNS:** Bewust geen SMTP - Graph API voorkomt SPF lookup limiet (max 10 al bereikt)
- **Backwards compatibility:** Bestaande reports blijven werken (period_type="custom")
- **Logo storage:** Database blob (niet filesystem) voor eenvoudige backup/restore
- **HTML sanitization:** Gebruik bleach library voor XSS protection
- **Logo embedding:** Base64 inline in email (app is beveiligd, geen public URLs nodig)
### User Experience
- **Timezone:** Volgt ui_timezone setting overal (consistent met rest van app)
- **"To date" periodes:** Eindigen altijd op gisteren 23:59 (volledige dagen, geen partial data)
- **Email legitimiteit:** HTML templates met logo → professioneel → klanten vertrouwen
- **Reply-to scheiding:** Reports vanaf reports@, replies naar helpdesk@ (mailbox scheiding)
- **Validation:** Strict placeholder validation (voorkomen errors bij sending)
### Scheduling
- **Frequenties:** Weekly/Monthly/Quarterly/Yearly (geen daily - overkill)
- **Retry logic:** Max 3 attempts voor failed deliveries
- **Error visibility:** Admin ziet failed reports in UI + audit log
- **Recipients:** Per rapport (comma-separated) - geen customer-level emails in MVP
### Scope Limitaties (MVP)
- ❌ Geen multiple logos per klant (1 logo voor alle reports)
- ❌ Geen dark mode email support (PDF attachment toch)
- ❌ Geen custom cron expressions (te complex)
- ❌ Geen per-customer email addresses (te complex)
- ❌ Geen report history storage (kan wel opnieuw genereren)
## 🎯 Success Criteria
### Phase 1 Success
- [ ] Admin kan Settings > Reporting configureren
- [ ] Logo upload werkt (PNG/JPG/SVG, max 500KB)
- [ ] Test email verstuurd met alle settings toegepast
- [ ] Graph API Mail.Send permission check werkt
- [ ] Default email template configureerbaar met live preview
### Phase 2 Success
- [ ] Admin kan relative period kiezen bij report maken
- [ ] 15 relative period options beschikbaar
- [ ] Period calculator werkt correct (timezone-aware, to yesterday)
- [ ] Reports lijst toont relative period badges
- [ ] Email body template per rapport met HTML + placeholders
- [ ] Template validation werkt (invalid placeholders rejected)
### Phase 3 Success (Toekomstig)
- [ ] Reports kunnen gescheduled worden (weekly/monthly/quarterly/yearly)
- [ ] Scheduled reports versturen automatisch
- [ ] Failed deliveries hebben retry (max 3x)
- [ ] Admin ziet schedule status in UI
- [ ] Audit log bevat delivery events
```
Reference in New Issue View Git Blame Copy Permalink