clearview/docs/changelog-develop.md

Changelog - Develop

This file documents changes on the develop branch of this project.

[2026-04-13]

Added

• Site filter in Job Details — dropdown in the Selected Job Details panel to filter Targets and Deviations tables by site URL (client-side, no extra API call).
• Excel export — GET /api/scan-jobs/{id}/export endpoint (optional ?site_url= filter) returns a .xlsx file with two sheets:
  • Targets: URL, status, attempts, error, timestamps.
  • Deviations: Site URL, relative Object URL, Object Type, Principal, Link Risk (colour-coded), Resolved Members, Role, Delta — sorted by Site URL → Object URL → Principal.
• Hierarchical deduplication — after scanning a target, deviations are post-processed to suppress child-level entries already covered by a parent (library/folder). Prevents result explosion on large sites with deeply inherited permissions. No additional API calls.
• SharingLinks classification and colour coding — SharePoint sharing-link principals are parsed and displayed with a risk badge in the Deviations table:
  • Anonymous* → Critical (red)
  • Flexible → High (orange)
  • Organization* → Low (blue)
  • Direct* → Low (green)
• Resolve Sharing Links — post-scan action in the Job Details panel. Fetches the actual member list of sharing-link groups via /_api/web/sitegroups/getbyname/users. Stored in new permission_deviations.resolved_members column. Anonymous links produce an empty member list (shown as (public link)). New endpoint: POST /api/scan-jobs/{id}/resolve-sharing-links.
• Role name normalisation — common Dutch SharePoint role names (e.g. "Volledig beheer", "Bijdragen") are translated to their English equivalents at scan time before being stored.
• openpyxl dependency added to requirements.txt.
• Favicon replaced with a dedicated icon (blue rounded square with eye/keyhole symbol) instead of the concept design SVG.

Changed

• SCAN_TARGET_TIMEOUT_SEC default raised from 180 s to 3600 s (1 hour) to accommodate large sites with tens of thousands of files.
• permission_deviations table extended with resolved_members TEXT column (auto-migrated on startup).
• Object URL in the Deviations table and Excel export is now shown relative to the site URL (site URL prefix stripped).
• Principal display in the Deviations table strips the SharePoint claim prefix (e.g. i:0#.f|membership|) and shows only the email/name; full value visible on hover.
• Site URL in the Deviations table is abbreviated to the last path segment with full URL on hover.
• Deviations table uses table-layout: fixed with column widths sized to fit on a 1080p display.
• docs/TECHNICAL.md and README.md updated to reflect all new functionality.

[2026-04-13]

Added

• Certificate-based authentication for SharePoint app-only access:
  • Clearview generates a self-signed RSA-2048 certificate per tenant (no external CA required).
  • New endpoint POST /api/tenants/{id}/generate-certificate stores the private key and returns the public cert.
  • Public certificate downloadable as a .cer file from the UI, named after the tenant.
  • Scanner uses MSAL with certificate when available; client secret remains as fallback.
  • Resolves SharePoint error "Unsupported app only token" when using client secret authentication.
  • TenantProfile extended with cert_private_key, cert_thumbprint, and cert_expires_at.
  • Tenant table shows auth method (cert with expiry date or secret).
• Client secret is now optional when creating a tenant profile (can be omitted when a certificate will be used).
• Job deletion: DELETE /api/scan-jobs/{id} endpoint added (not allowed for queued or running jobs).
  • Delete button per job in the UI; cascades to targets and deviations.

Fixed

• SharePoint REST API error when fetching list items: removed $filter=HasUniqueRoleAssignments eq true as SharePoint does not support this field as an OData filter. The check is now performed client-side.

[2026-04-13]

Added

• Multi-tenant support: Clearview now manages multiple customer tenants from a single instance.
  • New TenantProfile data model (tenant_profiles table) for storing customer credentials.
  • ScanJob linked to a tenant profile via tenant_profile_id FK.
  • API endpoints for tenant profile management: GET/POST /api/tenants, DELETE /api/tenants/{id}.
  • GET /api/scan-jobs supports filtering by tenant_profile_id.
• UI fully redesigned for multi-tenant use:
  • New Tenants panel with a table of configured customers, Add/Delete actions, and a Scan shortcut per tenant.
  • Onboarding flow (Connect Microsoft / manual instructions) moved into the Add Tenant form.
  • Scan form uses a tenant profile dropdown; manual credentials only shown as a fallback option.
  • Jobs table extended with a Tenant column and a tenant filter dropdown.
  • Hero stats now show: Tenants / Jobs / Active Jobs.
• XSS escaping added for all user-supplied data rendered in the jobs and deviations tables.

Changed

• TECHNICAL.md updated with multi-tenant model documentation, tenant profile API, and redesigned onboarding flow.

[2026-04-13]

Added

• Initial repository structure.
• containers/ directory added with the clearview starter service.
• build-and-push.sh added for container build and push.
• docs/TECHNICAL.md added.
• docs/changelog-develop.md added.
• version.txt added with initial version.
• .last-branch added for branch tracking in the build script.

[2026-04-13]

Added

• FastAPI backend integrated into the clearview container (single-container app runtime).
• PostgreSQL-backed scan job model (scan_jobs, scan_targets, permission_deviations).
• Background scan worker with queue processing, retries, and per-target timeout controls.
• API endpoints for manual URL scan creation, CSV import, job listing, and job detail retrieval.
• CSV parsing support for Microsoft Sites export format with URL normalization and de-duplication.
• Default-site skip rules for tenant root and app catalog paths.
• Frontend replaced with production-oriented scan UI:
  • Manual URL submission
  • CSV upload
  • Job status overview
  • Target-level result view
  • Deviation table view
• Stack configuration extended with scan worker runtime environment settings.

Changed

• containers/clearview/Dockerfile switched from static nginx hosting to Python FastAPI runtime.

Added

• Real SharePoint scan implementation for app-only authentication mode (SHAREPOINT_SCAN_MODE=sharepoint_app_only):
  • OAuth2 client credentials token acquisition via Microsoft Entra ID.
  • Site root permission baseline loading through SharePoint REST roleassignments.
  • Document library, folder, and file traversal with unique-permission detection (HasUniqueRoleAssignments).
  • Deviation persistence only for rights not present on site root (delta_type=added).
  • HTTP retry/backoff and throttle handling (429/503), plus list-level scan caps.
• Scanner HTTP retry/backoff and list-limit controls added in backend configuration.

Changed

• Authentication flow updated to universal multi-tenant style:
  • Azure credentials are now supplied per scan job from the web UI/API payload.
  • No Azure tenant/client/secret dependency in stack .env.
• Added UI and technical documentation guidance for one-time Entra app setup and required SharePoint permission (Sites.FullControl.All + admin consent).

Added

• Automated onboarding endpoint POST /api/onboarding/create-scan-app:
  • Creates a dedicated scan app in Entra for the connected tenant.
  • Configures SharePoint app permission Sites.FullControl.All.
  • Creates service principal and assigns app role consent.
  • Generates and returns a new client secret.
• Microsoft connect/admin-consent flow endpoints:
  • GET /api/onboarding/microsoft/connect-url
  • GET /api/onboarding/microsoft/callback
• UI onboarding flow updated:
  • Connect Microsoft button for admin consent redirect
  • Callback handling to capture tenant id
  • Automatic scan-app creation without manual bootstrap app input