v20260113-08-vspc-object-linking-normalize #114

Merged
ivooskamp merged 3 commits from v20260113-08-vspc-object-linking-normalize into main 2026-01-13 16:46:00 +01:00
5 changed files with 121 additions and 104 deletions

View File

@ -1 +1 @@
v20260113-08-vspc-object-linking v20260113-08-vspc-object-linking-normalize

View File

@ -68,4 +68,53 @@ def find_matching_job(msg: MailMessage) -> Optional[Job]:
if len(matches) == 1: if len(matches) == 1:
return matches[0] return matches[0]
# Backwards-compatible matching for Veeam VSPC Active Alarms summary per-company jobs.
# Earlier versions could store company names with slightly different whitespace / HTML entities,
# while parsers store objects using a normalized company prefix. When the exact match fails,
# try a normalized company comparison so existing jobs continue to match.
try:
bsw = (backup or "").strip().lower()
bt = (btype or "").strip().lower()
jn = (job_name or "").strip()
if bsw == "veeam" and bt == "service provider console" and "|" in jn:
left, right = [p.strip() for p in jn.split("|", 1)]
if left.lower() == "active alarms summary" and right:
from .parsers.veeam import normalize_vspc_company_name # lazy import
target_company = normalize_vspc_company_name(right)
if not target_company:
return None
q2 = Job.query
if norm_from is None:
q2 = q2.filter(Job.from_address.is_(None))
else:
q2 = q2.filter(Job.from_address == norm_from)
q2 = q2.filter(Job.backup_software == backup)
q2 = q2.filter(Job.backup_type == btype)
q2 = q2.filter(Job.job_name.ilike("Active alarms summary | %"))
# Load a small set of candidates and compare the company portion.
candidates = q2.order_by(Job.updated_at.desc(), Job.id.desc()).limit(25).all()
normalized_matches: list[Job] = []
for cand in candidates:
cand_name = (cand.job_name or "").strip()
if "|" not in cand_name:
continue
c_left, c_right = [p.strip() for p in cand_name.split("|", 1)]
if c_left.lower() != "active alarms summary" or not c_right:
continue
if normalize_vspc_company_name(c_right) == target_company:
normalized_matches.append(cand)
if len(normalized_matches) > 1:
customer_ids = {m.customer_id for m in normalized_matches}
if len(customer_ids) == 1:
return normalized_matches[0]
return None
if len(normalized_matches) == 1:
return normalized_matches[0]
except Exception:
pass
return None return None

View File

@ -228,45 +228,6 @@ def _parse_vspc_active_alarms_from_html(html: str) -> Tuple[List[Dict], str, Opt
return objects, overall_status, overall_message return objects, overall_status, overall_message
def extract_vspc_active_alarms_companies(raw: str) -> List[str]:
"""Extract company names with alarms > 0 from a VSPC Active Alarms summary body."""
if not raw:
return []
txt = raw
if "<" in txt and ">" in txt:
txt = re.sub(r"<[^>]+>", " ", txt)
txt = _html.unescape(txt)
txt = txt.replace("\xa0", " ")
txt = re.sub(r"\s+", " ", txt).strip()
seen: set[str] = set()
out: List[str] = []
for m in re.finditer(
r"\bCompany:\s*([^\(\r\n]+?)\s*\(\s*alarms?\s*:\s*(\d+)\s*\)",
txt,
flags=re.IGNORECASE,
):
cname = (m.group(1) or "").strip()
cname = cname.replace("\xa0", " ")
cname = re.sub(r"\s+", " ", cname).strip()
try:
alarms = int(m.group(2))
except Exception:
alarms = 0
if not cname or alarms <= 0:
continue
if cname in seen:
continue
seen.add(cname)
out.append(cname)
return out
def _parse_cloud_connect_report_from_html(html: str) -> Tuple[List[Dict], str]: def _parse_cloud_connect_report_from_html(html: str) -> Tuple[List[Dict], str]:
"""Parse Veeam Cloud Connect daily report (provider) HTML. """Parse Veeam Cloud Connect daily report (provider) HTML.

View File

@ -1,69 +1,76 @@
## v20260113-01-changelog-markdown-render
- Updated the changelog route to render remote Markdown content with proper formatting.
- Enabled Markdown parsing so headings, lists, links, and code blocks are displayed as intended.
- Ensured the changelog always fetches the latest version from the source repository at request time.
- Removed plain-text rendering to prevent loss of Markdown structure.
---
## v20260113-02-edge-mail-lightmode
- Forced light color-scheme for embedded mail content to prevent Edge auto dark mode rendering.
- Added explicit color-scheme and forced-color-adjust rules so original mail CSS is respected.
- Ensured consistent mail readability across Edge and Firefox in Inbox, Deleted Inbox, Job Detail, Run Checks, Daily Jobs, and Admin All Mail views.
---
## v20260113-03-runchecks-overall-remark
- Added support to display the overall remark (overall_message) on the Run Checks page.
- Ensured consistency between the Run Checks view and the Job Details page where the overall remark was already available.
---
## v20260113-04-edge-initial-setup-users-exist
- Fixed incorrect redirect to the “Initial admin setup” page when users already exist.
- Changed the setup check from “admin user exists” to “any user exists”, so existing environments always show the login page instead of allowing a new initial admin to be created.
- Prevented direct access to the initial setup route when at least one user is present (redirects to login).
---
## v20260113-05-reporter-menu-restrict
- Restricted the Reporter role to only access Dashboard, Reports, Changelog, and Feedback.
- Updated menu rendering to hide all unauthorized menu items for Reporter users.
- Adjusted route access to ensure Feedback pages are accessible for the Reporter role.
---
## v20260113-06-overrides-error-match-modes
- Added configurable error text matching modes for overrides: contains, exact, starts with, ends with
- Updated override evaluation logic to apply the selected match mode across run remarks and object error messages
- Extended overrides UI with a match type selector and improved edit support for existing overrides
- Added database migration to create and backfill overrides.match_error_mode for existing records
---
## v20260113-07-job-delete-fix
- Fixed an error that occurred when deleting a job.
- Corrected backend deletion logic to prevent exceptions during job removal.
- Ensured related records are handled safely to avoid constraint or reference errors.
---
## v20260113-08-vspc-object-linking
- Fixed VSPC company name normalization so company detection and object prefixing match consistently.
- Fixed filtered object persistence to respect UNIQUE(customer_id, object_name) and to update last_seen timestamps correctly.
- Added auto object persistence routing for VSPC per-company runs so objects are linked to the correct customer/job (prefix stripped).
- Improved auto-approval for VSPC Active Alarms summary:
- Creates per-company runs automatically when company jobs are mapped (new imports and inbox re-parse).
- Uses case-insensitive matching for "<company> | <object>" mail objects.
- Added best-effort retroactive processing after approving VSPC company mappings to automatically link older inbox messages that are now fully mapped.
*** ***
## v0.1.21
This release focuses on improving correctness, consistency, and access control across core application workflows, with particular attention to changelog rendering, browser-specific mail readability, Run Checks visibility, role-based access restrictions, override flexibility, and VSPC object linking reliability. The goal is to ensure predictable behavior, clearer diagnostics, and safer administration across both day-to-day operations and complex multi-entity reports.
### Changelog Rendering and Documentation Accuracy
- Updated the Changelog route to render remote Markdown content instead of plain text.
- Enabled full Markdown parsing so headings, lists, links, and code blocks are displayed correctly.
- Ensured the changelog always fetches the latest version directly from the source repository at request time.
- Removed legacy plain-text rendering to prevent loss of structure and formatting.
### Mail Rendering and Browser Compatibility
- Forced a light color scheme for embedded mail content to prevent Microsoft Edge from applying automatic dark mode styling.
- Added explicit `color-scheme` and `forced-color-adjust` rules so original mail CSS is respected.
- Ensured consistent mail readability across Edge and Firefox.
- Applied these fixes consistently across Inbox, Deleted Inbox, Job Details, Run Checks, Daily Jobs, and Admin All Mail views.
### Run Checks Visibility and Consistency
- Added support for displaying the overall remark (overall_message) directly on the Run Checks page.
- Ensured consistency between Run Checks and Job Details, where the overall remark was already available.
- Improved operator visibility of high-level run context without requiring navigation to job details.
### Initial Setup and User Existence Safeguards
- Fixed an incorrect redirect to the “Initial admin setup” page when users already exist.
- Changed setup detection logic from “admin user exists” to “any user exists”.
- Ensured existing environments always show the login page instead of allowing a new initial admin to be created.
- Prevented direct access to the initial setup route when at least one user is present.
### Role-Based Access Control and Menu Restrictions
- Restricted the Reporter role to only access Dashboard, Reports, Changelog, and Feedback.
- Updated menu rendering to fully hide unauthorized menu items for Reporter users.
- Adjusted route access to ensure Feedback pages remain accessible for the Reporter role.
- Improved overall consistency between visible navigation and backend access rules.
### Override Matching Flexibility and Maintainability
- Added configurable error text matching modes for overrides: contains, exact, starts with, and ends with.
- Updated override evaluation logic to apply the selected match mode across run remarks and object error messages.
- Extended the overrides UI with a match type selector and improved edit support for existing overrides.
- Added a database migration to create and backfill the `overrides.match_error_mode` field for existing records.
### Job Deletion Stability
- Fixed an error that occurred during job deletion.
- Corrected backend deletion logic to prevent runtime exceptions.
- Ensured related records are handled safely to avoid constraint or reference errors during removal.
### VSPC Object Linking and Normalization
- Fixed VSPC company name normalization so detection and object prefixing behave consistently.
- Ensured filtered object persistence respects the UNIQUE(customer_id, object_name) constraint.
- Correctly update `last_seen` timestamps for existing objects.
- Added automatic object persistence routing for VSPC per-company runs, ensuring objects are linked to the correct customer and job.
- Improved auto-approval for VSPC Active Alarms summaries with per-company run creation and case-insensitive object matching.
- Added best-effort retroactive processing to automatically link older inbox messages once company mappings are approved.
### VSPC Normalization Bug Fixes and Backward Compatibility
- Removed duplicate definitions of VSPC Active Alarms company extraction logic that caused inconsistent normalization.
- Ensured a single, consistent normalization path is used when creating jobs and linking objects.
- Improved object linking so real objects (e.g. HV01, USB Disk) are reliably associated with their jobs.
- Restored automatic re-linking for both new and historical VSPC mails.
- Added backward-compatible matching to prevent existing VSPC jobs from breaking due to earlier inconsistent company naming.
---
## v0.1.20 ## v0.1.20
This release delivers a comprehensive set of improvements focused on parser correctness, data consistency, and clearer operator workflows across Inbox handling, Run Checks, and administrative tooling. The main goal of these changes is to ensure that backup notifications are parsed reliably, presented consistently, and handled through predictable and auditable workflows, even for complex or multi-entity reports. This release delivers a comprehensive set of improvements focused on parser correctness, data consistency, and clearer operator workflows across Inbox handling, Run Checks, and administrative tooling. The main goal of these changes is to ensure that backup notifications are parsed reliably, presented consistently, and handled through predictable and auditable workflows, even for complex or multi-entity reports.

View File

@ -1 +1 @@
v0.1.19 v0.1.20