diff --git a/.last-branch b/.last-branch index 3209668..d784745 100644 --- a/.last-branch +++ b/.last-branch @@ -1 +1 @@ -v20260219-03-fix-remark-visibility +v20260223-05-cove-integration diff --git a/TODO-cove-data-protection.md b/TODO-cove-data-protection.md index df0aa9d..55b217e 100644 --- a/TODO-cove-data-protection.md +++ b/TODO-cove-data-protection.md @@ -1,772 +1,249 @@ # TODO: Cove Data Protection Integration -**Date:** 2026-02-10 -**Status:** Research phase +**Date:** 2026-02-23 +**Status:** Research COMPLETED β€” Ready for implementation **Priority:** Medium --- ## 🎯 Goal -Integrate Cove Data Protection (formerly N-able Backup / SolarWinds Backup) into Backupchecks for backup status monitoring. +Integrate Cove Data Protection (formerly N-able Backup / SolarWinds Backup) into Backupchecks for backup status monitoring via scheduled API polling. The integration runs server-side within the Backupchecks web application. -**Challenge:** Cove does NOT work with email notifications like other backup systems (Veeam, Synology, NAKIVO). We need to find an alternative method to import backup status information. +**Challenge:** Cove does NOT work with email notifications like other backup systems (Veeam, Synology, NAKIVO). We use the JSON-RPC API instead. --- -## πŸ” Research Questions +## βœ… Research Phase β€” COMPLETED (2026-02-23) -### 1. API Availability -- [x] Does Cove Data Protection have a public API? **YES - Confirmed in documentation** -- [ ] **CRITICAL:** How to enable/activate API access? (settings location, admin portal?) -- [ ] What authentication method does the API use? (API key, OAuth, basic auth?) -- [ ] Which endpoints are available for backup status? -- [ ] Is there rate limiting on the API? -- [ ] Documentation URL: ? -- [ ] Is API access available in all Cove subscription tiers or only specific plans? +### Confirmed findings -### 2. Data Structure -- [ ] What information can we retrieve per backup job? - - Job name - - Status (success/warning/failed) - - Start/end time - - Backup type - - Client/device name - - Error messages - - Objects/files backed up -- [ ] Is there a webhook system available? -- [ ] How often should the API be polled? +- **API endpoint:** `https://api.backup.management/jsonapi` +- **Protocol:** JSON-RPC 2.0, POST requests, `Content-Type: application/json` +- **Authentication:** Login method returns a `visa` token β€” include in all subsequent calls +- **PartnerId:** `139124` (MCC Automatisering) β€” required for all queries, partnernaam is NIET nodig +- **Alle benodigde data is beschikbaar** β€” eerdere blockers (D02/D03 errors) waren door gebruik van legacy column codes. Vervangen door D10/D11. +- **Geen MSP-level beperking** β€” elke API user heeft dezelfde toegang. Toegang tot alle sub-customers via top-level account. +- **Geen EnumerateAccounts nodig** β€” `EnumerateAccountStatistics` met juiste columns geeft alles wat we nodig hebben. -### 3. Multi-Tenancy -- [ ] Does Cove support multi-tenant setups? (MSP use case) -- [ ] Can we monitor multiple customers/partners from 1 account? -- [ ] How are permissions/access managed? - -### 4. Integration Strategy -- [ ] **Option A: Scheduled Polling** - - Cronjob that periodically calls API - - Parse results to JobRun records - - Pro: Simple, consistent with current flow - - Con: Delay between backup and registration in system - -- [ ] **Option B: Webhook/Push** - - Cove sends notifications to our endpoint - - Pro: Real-time updates - - Con: Requires external endpoint, security considerations - -- [ ] **Option C: Email Forwarding** - - If Cove has email support after all (hidden setting?) - - Pro: Reuses existing email import flow - - Con: Possibly not available +### OfficiΓ«le documentatie (van N-able support, Andrew Robinson) +- **Getting Started:** https://developer.n-able.com/n-able-cove/docs/getting-started +- **Column Codes:** https://developer.n-able.com/n-able-cove/docs/column-codes +- **Construct a Call:** https://developer.n-able.com/n-able-cove/docs/construct-a-json-rpc-api-call +- **Authorization:** https://developer.n-able.com/n-able-cove/docs/authorization --- -## πŸ“‹ Technical Considerations +## πŸ“‘ API β€” Vastgestelde werking -### Database Model -Current JobRun model expects: -- `mail_message_id` (FK) - how do we adapt this for API-sourced runs? -- Possible new field: `source_type` ("email" vs "api") -- Possible new field: `external_id` (Cove job ID) +### Stap 1: Login -### Parser System -Current parser system works with email content. For API: -- New "parser" concept for API responses? -- Or direct JobRun creation without parser layer? +```json +POST https://api.backup.management/jsonapi +Content-Type: application/json -### Architecture Options +{ + "jsonrpc": "2.0", + "id": "jsonrpc", + "method": "Login", + "params": { + "username": "{{cove_api_username}}", + "password": "{{cove_api_password}}" + } +} +``` -**Option 1: Extend Email Import System** -``` -API Poller β†’ Pseudo-MailMessage β†’ Existing Parser β†’ JobRun -``` -- Pro: Reuse existing flow -- Con: Hacky, email fields have no meaning +**Response bevat:** +- `visa` β€” sessie token (meegeven in alle vervolg calls) +- `result.PartnerId` β€” het partner ID (139124 voor MCC Automatisering) -**Option 2: Parallel Import System** -``` -API Poller β†’ API Parser β†’ JobRun (direct) -``` -- Pro: Clean separation, no email dependency -- Con: Logic duplication +### Stap 2: EnumerateAccountStatistics -**Option 3: Unified Import Layer** +```json +{ + "jsonrpc": "2.0", + "visa": "{{visa}}", + "id": "jsonrpc", + "method": "EnumerateAccountStatistics", + "params": { + "query": { + "PartnerId": 139124, + "StartRecordNumber": 0, + "RecordsCount": 250, + "Columns": [ + "I1", "I18", "I8", "I78", + "D09F00", "D09F09", "D09F15", "D09F08", + "D1F00", "D1F15", + "D10F00", "D10F15", + "D11F00", "D11F15", + "D19F00", "D19F15", + "D20F00", "D20F15", + "D5F00", "D5F15", + "D23F00", "D23F15" + ] + } + } +} ``` - β†’ Email Import β†’ -Unified β†’ β†’ Common Processor β†’ JobRun - β†’ API Import β†’ -``` -- Pro: Future-proof, scalable -- Con: Larger refactor --- -## πŸ”§ Implementation Steps (After Research) +## πŸ“‹ Column codes β€” wat ze betekenen -### Phase 0: API Access Activation (FIRST!) -**Critical step before any development can begin:** +### Device info +| Column | Betekenis | Type | +|--------|-----------|------| +| `I1` | Device naam (intern, uniek) | String | +| `I18` | Computer naam (leesbaar) β€” leeg bij M365 | String | +| `I8` | Klant naam | String | +| `I78` | Actieve datasources, bijv. `D01D02D10` | String | -1. [ ] **Find API activation location** - - Check Cove admin portal/dashboard - - Look in: Settings β†’ API / Integrations / Developer section - - Check: Account settings, Company settings, Partner settings - - Search documentation for: "API activation", "API access", "enable API" +### Datasource status (per datasource herhaalbaar) +| Suffix | Betekenis | Type | +|--------|-----------|------| +| `F00` | Status van laatste sessie | Int (zie tabel) | +| `F09` | Tijdstip laatste **succesvolle** sessie | Unix timestamp | +| `F15` | Tijdstip laatste sessie (ongeacht status) | Unix timestamp | +| `F08` | Color bar laatste 28 dagen (28 cijfers) | String | -2. [ ] **Generate API credentials** - - API key generation - - Client ID / Client Secret (if OAuth) - - Note: which user/role can generate API keys? +### Status waarden (F00) +| Waarde | Betekenis | +|--------|-----------| +| `1` | In process | +| `2` | Failed ❌ | +| `3` | Aborted | +| `5` | Completed βœ… | +| `6` | Interrupted | +| `8` | CompletedWithErrors ⚠️ | +| `9` | InProgressWithFaults | +| `10` | OverQuota | +| `11` | NoSelection (geconfigureerd maar niets geselecteerd) | +| `12` | Restarted | -3. [ ] **Document API base URL** - - Production API endpoint - - Sandbox/test environment (if available) - - Regional endpoints (EU vs US?) +### Datasources +| Code | Naam | Gebruik | +|-------|------|---------| +| `D09` | Total (alle datasources gecombineerd) | Altijd aanwezig, beste voor overall status | +| `D1` | Files & Folders | Servers/workstations | +| `D2` | System State | Servers/workstations | +| `D10` | VssMsSql (SQL Server) | Servers met SQL | +| `D11` | VssSharePoint | Servers met SharePoint | +| `D19` | Microsoft 365 Exchange | M365 tenants | +| `D20` | Microsoft 365 OneDrive | M365 tenants | +| `D5` | Microsoft 365 SharePoint | M365 tenants | +| `D23` | Microsoft 365 Teams | M365 tenants | -4. [ ] **Document API authentication flow** - - Header format (Bearer token, API key in header, query param?) - - Token expiration and refresh - - Rate limit headers to watch +**Let op:** D02 en D03 zijn legacy codes β€” gebruik D10 en D11. -5. [ ] **Find API documentation portal** - - Developer documentation URL - - Interactive API explorer (Swagger/OpenAPI?) - - Code examples/SDKs - - Support channels for API questions +### Device types herkennen via I78 +- `I78` bevat waarden zoals `D01D02`, `D01D02D10`, `D19D20D05D23` +- Leeg `I18` veld = Microsoft 365 tenant +- Gevuld `I18` veld = server of workstation -**Resources to check:** -- Cove admin portal: https://backup.management (or similar) -- N-able partner portal -- Cove knowledge base / support docs -- Contact Cove support for API access instructions - -### Phase 1: API Research & POC - -**Step 1: Read Authentication Documentation** βœ… DOCUMENTATION FOUND! -- [x] API documentation located -- [ ] **Read:** https://documentation.n-able.com/covedataprotection/USERGUIDE/documentation/Content/unused/service-management/json-api/login.htm -- [ ] **Read:** https://documentation.n-able.com/covedataprotection/USERGUIDE/documentation/Content/unused/service-management/json-api/construct-a-call.htm -- [ ] Document API base URL from docs -- [ ] Document authentication flow (likely JSON-RPC style based on "construct-a-call") -- [ ] Note any required request format (headers, body structure) - -**Step 2: Test Authentication** -- [ ] Determine token format (Bearer token? API key header? Query param?) -- [ ] Common authentication patterns to test: - ```bash - # Option 1: Bearer token - curl -H "Authorization: Bearer YOUR_TOKEN" https://api.example.com/endpoint - - # Option 2: API Key header - curl -H "X-API-Key: YOUR_TOKEN" https://api.example.com/endpoint - - # Option 3: Custom header - curl -H "X-Auth-Token: YOUR_TOKEN" https://api.example.com/endpoint - ``` -- [ ] Test with simple endpoint (e.g., `/api/v1/status`, `/api/accounts`, `/api/devices`) - -**Step 3: Discover Available Endpoints** -- [ ] Find API documentation/reference -- [ ] Look for OpenAPI/Swagger spec -- [ ] Key endpoints we need: - - List customers/accounts - - List backup devices/jobs - - Get backup job history - - Get backup job status/details - - Get backup run results (success/failed/warnings) - -**Step 4: Test Data Retrieval** -- [ ] Test listing customers (verify top-level access works) -- [ ] Test listing backup jobs for one customer -- [ ] Test retrieving details for one backup job -- [ ] Document response format (JSON structure) -- [ ] Save example API responses for reference - -**Step 5: Proof of Concept Script** -1. [ ] Create standalone Python script (outside Backupchecks) -2. [ ] Test authentication and data retrieval -3. [ ] Parse API response to extract key fields -4. [ ] Mapping of Cove data β†’ Backupchecks JobRun model -5. [ ] Document findings in this TODO - -### Phase 2: Database Changes -1. [ ] Decide: extend MailMessage model or new source type? -2. [ ] Migration: add `source_type` field to JobRun -3. [ ] Migration: add `external_id` field to JobRun -4. [ ] Update constraints/validations - -### Phase 3: Import Mechanism -1. [ ] New file: `containers/backupchecks/src/backend/app/cove_importer.py` -2. [ ] API client for Cove -3. [ ] Data transformation to JobRun format -4. [ ] Error handling & retry logic -5. [ ] Logging & audit trail - -### Phase 4: Scheduling -1. [ ] Cronjob/scheduled task for polling (every 15 min?) -2. [ ] Or: webhook endpoint if Cove supports it -3. [ ] Rate limiting & throttling -4. [ ] Duplicate detection (avoid double imports) - -### Phase 5: UI Updates -1. [ ] Job Details: indication that job is from API (not email) -2. [ ] No "Download EML" button for API-sourced runs -3. [ ] Possibly different metadata display +### D09F08 β€” Color bar decoderen +28 tekens, elk karakter = 1 dag (oudste eerst): +- `5` = Completed βœ… +- `8` = CompletedWithErrors ⚠️ +- `2` = Failed ❌ +- `1` = In progress +- `0` = Geen backup --- -## πŸ“š References +## πŸ—οΈ Architectuur beslissing -### Cove Data Protection -- **Product name:** Cove Data Protection (formerly N-able Backup, SolarWinds Backup) -- **Website:** https://www.n-able.com/products/cove-data-protection -- **API Documentation Base:** https://documentation.n-able.com/covedataprotection/USERGUIDE/documentation/Content/unused/service-management/json-api/ +**Gekozen: Option 2 β€” Parallel Import System** -### JSON API Documentation (Found!) +``` +API Poller β†’ Cove API Parser β†’ JobRun (direct, zonder MailMessage) +``` -**Core Documentation:** -- πŸ“˜ **API Home:** https://documentation.n-able.com/covedataprotection/USERGUIDE/documentation/Content/unused/service-management/json-api/home.htm -- πŸ”‘ **Login/Authentication:** https://documentation.n-able.com/covedataprotection/USERGUIDE/documentation/Content/unused/service-management/json-api/login.htm -- πŸ”§ **Construct API Calls:** https://documentation.n-able.com/covedataprotection/USERGUIDE/documentation/Content/unused/service-management/json-api/construct-a-call.htm +Rationale: +- Schone scheiding van email- en API-gebaseerde imports +- Geen misbruik van MailMessage model voor data zonder email context +- Toekomstbestendig voor andere API-gebaseerde backup systemen -**Key Endpoints for Backupchecks:** -- πŸ‘₯ **Enumerate Customers:** https://documentation.n-able.com/covedataprotection/USERGUIDE/documentation/Content/unused/service-management/json-api/enumerate-customers.htm -- πŸ’» **Enumerate Devices:** https://documentation.n-able.com/covedataprotection/USERGUIDE/documentation/Content/unused/service-management/json-api/enumerate-devices.htm -- πŸ“Š **Enumerate Device Statistics:** https://documentation.n-able.com/covedataprotection/USERGUIDE/documentation/Content/unused/service-management/json-api/enumerate-device-statistics.htm - -**Reference:** -- πŸ“‹ **API Column Codes:** https://documentation.n-able.com/covedataprotection/USERGUIDE/documentation/Content/unused/service-management/json-api/API-column-codes.htm -- πŸ“‹ **Legacy Column Codes:** https://documentation.n-able.com/covedataprotection/USERGUIDE/documentation/Content/unused/service-management/json-api/API-column-codes-legacy.htm -- πŸ“ **Schema Documentation:** https://documentation.n-able.com/covedataprotection/USERGUIDE/documentation/Content/unused/service-management/json-api/how-to-schema.htm - -**Other Resources:** -- πŸ—οΈ **Architecture Guide:** https://documentation.n-able.com/covedataprotection/USERGUIDE/documentation/Content/Architecture-and-Security/Cove-Architecture-Guide.htm -- πŸ”’ **Security Guide:** https://documentation.n-able.com/covedataprotection/USERGUIDE/documentation/Content/Architecture-and-Security/Cove-Security-Guide.htm - -**Note:** API docs are in "unused" folder - likely legacy but still functional! - -### Similar Integrations -Other backup systems that use APIs: -- Veeam: Has both email and REST API -- Acronis: REST API available -- MSP360: API for management - -### Resources -- [ ] API documentation (yet to find) -- [ ] SDK/Client libraries available? -- [ ] Community/forum for integration questions? -- [ ] Example code/integrations? +### Database wijzigingen nodig +- `JobRun.source_type` β€” nieuw veld: `"email"` of `"api"` +- `JobRun.external_id` β€” Cove `AccountId` als externe referentie +- `JobRun.mail_message` β€” moet nullable worden (of aparte tabel) --- -## ❓ Open Questions +## πŸ”§ Implementatie fases -1. **Performance:** How many Cove jobs do we need to monitor? (impact on polling frequency) -2. **Historical Data:** Can we retrieve old backup runs, or only new ones? -3. **Filtering:** Can we apply filters (only failed jobs, specific clients)? -4. **Authentication:** Where do we store Cove API credentials? (SystemSettings?) -5. **Multi-Account:** Do we support multiple Cove accounts? (MSP scenario) +### Phase 1: Database migratie +- [ ] `source_type` veld toevoegen aan JobRun (`email` / `api`) +- [ ] `external_id` veld toevoegen aan JobRun (voor Cove AccountId) +- [ ] `mail_message` FK nullable maken voor API-gebaseerde runs +- [ ] Migratie schrijven en testen + +### Phase 2: Cove API client +- [ ] Nieuw bestand: `app/services/cove_client.py` +- [ ] Login methode (visa token ophalen) +- [ ] `enumerate_account_statistics()` methode +- [ ] Paginatie afhandelen (RecordsCount / StartRecordNumber) +- [ ] Token verloop afhandelen (opnieuw inloggen) +- [ ] Error handling & retry logic + +### Phase 3: Data transformatie +- [ ] Nieuw bestand: `app/services/cove_importer.py` +- [ ] Settings lijst omzetten naar dict voor makkelijke lookup +- [ ] Unix timestamps omzetten naar datetime +- [ ] Datasource status mappen naar Backupchecks status (success/warning/failed) +- [ ] Device type bepalen (server vs M365) via `I18` en `I78` +- [ ] JobRun records aanmaken per device + +### Phase 4: Scheduled polling +- [ ] Cronjob of scheduled task (elke 15-60 minuten?) +- [ ] Duplicate detectie op basis van `external_id` + tijdstip +- [ ] Logging & audit trail +- [ ] Rate limiting respecteren + +### Phase 5: UI aanpassingen +- [ ] Job Details: geen "Download EML" knop voor API-gebaseerde runs +- [ ] Indicatie dat job afkomstig is van Cove API (niet email) +- [ ] 28-daagse color bar eventueel tonen + +### Phase 6: Configuratie +- [ ] Cove API credentials opslaan in SystemSettings +- [ ] PartnerId configureerbaar maken +- [ ] Polling interval instelbaar --- -## 🎯 Success Criteria +## πŸ”‘ API Credentials -### Minimum Viable Product (MVP) -- [ ] Backup runs from Cove are automatically imported -- [ ] Status (success/warning/failed) displayed correctly -- [ ] Job name and timestamp available -- [ ] Visible in Daily Jobs & Run Checks -- [ ] Errors and warnings are shown +- **API User:** `backupchecks-cove-01` +- **User ID:** `1665555` +- **PartnerId:** `139124` +- **Role:** SuperUser + SecurityOfficer +- **Portal:** https://backup.management/#/api-users + +**BELANGRIJK:** Token opslaan in password manager β€” kan niet opnieuw worden opgevraagd! + +--- + +## ❓ Openstaande vragen voor implementatie + +1. Hoe slaan we de Cove API credentials veilig op in Backupchecks? (SystemSettings? Environment variable?) +2. Wat is de gewenste polling frequentie? (15 min / 30 min / 1 uur?) +3. Willen we historische data importeren bij eerste run, of alleen nieuwe sessies? +4. Willen we de 28-daagse color bar (`D09F08`) tonen in de UI? +5. Ondersteunen we meerdere Cove accounts (meerdere MSPs)? + +--- + +## 🎯 Success Criteria (MVP) + +- [ ] Backup status (success/warning/failed) per device zichtbaar in Backupchecks +- [ ] Klant naam en device naam correct gekoppeld +- [ ] Tijdstip laatste backup beschikbaar +- [ ] Zichtbaar in Daily Jobs & Run Checks +- [ ] Servers Γ©n Microsoft 365 tenants worden ondersteund +- [ ] Geen duplicates bij herhaalde polling ### Nice to Have -- [ ] Real-time import (webhook instead of polling) -- [ ] Backup object details (individual files/folders) -- [ ] Retry history -- [ ] Storage usage metrics -- [ ] Multi-tenant support - ---- - -## ⚠️ Critical Limitations Discovered (2026-02-10) - -### What the API CAN provide: -- βœ… Account/device identifiers (I1) -- βœ… Storage usage metrics (I14 - bytes used) -- βœ… Computer/hostname (I18) -- βœ… Numeric metrics (D01F00-D01F07, D09F00) -- βœ… Basic partner metadata - -### What the API CANNOT provide (security restrictions): -- ❌ **Last backup timestamp** - No reliable date/time fields accessible -- ❌ **Backup status** (success/failed/warning) - No explicit status fields -- ❌ **Error messages** - All D02Fxx/D03Fxx ranges blocked -- ❌ **Backup run history** - No detailed run information -- ❌ **Cross-customer aggregation** - API users are customer-scoped -- ❌ **Device enumeration** - EnumerateAccounts method blocked (error 13501) - -### Root Cause -**Security error 13501** ("Operation failed because of security reasons") occurs when: -- Any restricted column code is requested in EnumerateAccountStatistics -- EnumerateAccounts method is called (always fails) -- This applies even with SuperUser + SecurityOfficer roles - -**Column restrictions are per-tenant and not documented.** The allow-list is extremely limited. - -### Impact on Backupchecks Integration -**Current API access is insufficient for backup monitoring** because: -1. No way to determine if a backup succeeded or failed -2. No error messages to display to users -3. No timestamps to track backup frequency -4. Cannot import backup "runs" in meaningful way - -**Possible with current API:** -- Storage usage dashboard only -- Device inventory list -- But NOT backup status monitoring (core Backupchecks function) - ---- - -## πŸ”€ Decision Point: Integration Feasibility - -### Option A: Implement Metrics-Only Integration -**Pros:** -- Can display storage usage per device -- Simple implementation -- Works with current API access - -**Cons:** -- Does NOT meet core Backupchecks requirement (backup status monitoring) -- No success/failure tracking -- No alerting on backup issues -- Limited value compared to email-based systems - -**Effort:** Low (2-3 days) -**Value:** Low (storage metrics only, no backup monitoring) - -### Option B: Request Expanded API Access from N-able ⭐ RECOMMENDED -**Contact N-able support and request:** -1. MSP-level API user capability (cross-customer access) -2. Access to restricted column codes: - - Backup timestamps (last successful backup) - - Status fields (success/warning/failed) - - Error message fields (D02Fxx/D03Fxx) - - Session/run history fields - -**Pros:** -- Could enable full backup monitoring if granted -- Proper integration matching other backup systems - -**Cons:** -- May require vendor cooperation -- No guarantee N-able will grant access -- Possible additional licensing costs? -- Timeline uncertain (support ticket process) - -**Effort:** Unknown (depends on N-able response) -**Value:** High (if successful) - ---- - -### πŸ“§ Support Ticket Template (Ready to Send) - -**To:** N-able Cove Data Protection Support -**Subject:** API Access Request - Backup Monitoring Integration - -**Email Body:** - -``` -Hello N-able Support Team, - -We are developing a backup monitoring solution for MSPs and are integrating -with Cove Data Protection via the JSON-RPC API for our customers. - -Current Situation: -- We have successfully authenticated with the API -- API endpoint: https://api.backup.management/jsonapi -- API user management: https://backup.management/#/api-users -- Method tested: EnumerateAccountStatistics -- Role: SuperUser + SecurityOfficer - -Current Limitations (Blocking Integration): -We are encountering "Operation failed because of security reasons (error 13501)" -when attempting to access essential backup monitoring data: - -1. Backup Status Fields - - Cannot determine if backups succeeded, failed, or completed with warnings - - Need access to status/result columns - -2. Timestamp Information - - Cannot access last backup date/time - - Need reliable timestamp fields to track backup frequency - -3. Error Messages - - D02Fxx and D03Fxx column ranges are blocked - - Cannot retrieve error details to show users what went wrong - -4. API User Scope - - API users are customer-scoped only - - Need MSP-level API user capability for cross-customer monitoring - -Impact: -Without access to these fields, we can only retrieve storage usage metrics, -which is insufficient for backup status monitoring - the core requirement -for our MSP customers. - -Request: -Can you please: -1. Enable MSP-level API user creation for cross-customer access -2. Grant access to restricted column codes containing: - - Backup status (success/failed/warning) - - Last backup timestamps - - Error messages and details - - Session/run history -3. Provide documentation on the semantic meaning of column codes (especially - D01F00-D01F07 and D09F00 which currently work) -4. OR suggest an alternative integration method if expanded API access is - not available (webhooks, reporting API, email notifications, etc.) - -Technical Details: -- Our test results are documented at: - docs/cove_data_protection_api_calls_known_info.md (can provide upon request) -- Safe columns identified: I1, I14, I18, D01F00-D01F07, D09F00 -- Restricted columns: Entire D02Fxx and D03Fxx ranges - -Use Case: -We need this integration to provide our MSP customers with centralized backup -monitoring across multiple backup vendors (Veeam, Synology, NAKIVO, and Cove). -Without proper API access, Cove customers cannot benefit from our monitoring -solution. - -Please advise on the best path forward for enabling comprehensive backup -monitoring via the Cove API. - -Thank you for your assistance. - -Best regards, -[Your Name] -[Company Name] -[Contact Information] -``` - -**Alternative Contact Methods:** -- N-able Partner Portal support ticket -- Cove support email (if available) -- N-able account manager (if assigned) - ---- - -### Option C: Alternative Integration Methods -Explore if Cove has: -1. **Reporting API** (separate from JSON-RPC) -2. **Webhook system** (push notifications for backup events) -3. **Email notifications** (if available, use existing email parser) -4. **Export/CSV reports** (scheduled export that can be imported) - -**Effort:** Medium (research required) -**Value:** Unknown - -### Option D: Defer Integration -**Wait until:** -- Customer requests Cove support specifically -- N-able improves API capabilities -- Alternative integration method discovered - -**Pros:** -- No wasted effort on limited implementation -- Focus on systems with better API support - -**Cons:** -- Cove customers cannot use Backupchecks -- Competitive disadvantage if other MSPs support Cove - ---- - -## 🎯 Recommended Next Steps - -### Immediate (This Week) -1. **Decision:** Choose Option A, B, C, or D above -2. **If Option B (contact N-able):** - - Open support ticket with N-able - - Reference API user creation at https://backup.management/#/api-users - - Explain need for expanded column access for monitoring solution - - Attach findings from `/docker/develop/cove_data_protection_api_calls_known_info.md` - - Ask specifically for: - - MSP-level API user creation - - Access to backup status/timestamp columns - - Documentation of column codes semantics - -3. **If Option C (alternative methods):** - - Check Cove portal for webhook/reporting settings - - Search N-able docs for "reporting API", "webhooks", "notifications" - - Test if email notifications can be enabled per customer - -### Long Term (Future) -- Monitor N-able API changelog for improvements -- Check if other MSPs have found workarounds -- Consider partnering with N-able for integration - ---- - -## πŸš€ Next Steps - -### Immediate Actions (Ready to Start!) - -**1. Read API Documentation** βœ… FOUND! -Priority reading order: -1. **Start here:** [Login/Auth](https://documentation.n-able.com/covedataprotection/USERGUIDE/documentation/Content/unused/service-management/json-api/login.htm) - How to authenticate with your token -2. **Then read:** [Construct a Call](https://documentation.n-able.com/covedataprotection/USERGUIDE/documentation/Content/unused/service-management/json-api/construct-a-call.htm) - Request format -3. **Key endpoint:** [Enumerate Device Statistics](https://documentation.n-able.com/covedataprotection/USERGUIDE/documentation/Content/unused/service-management/json-api/enumerate-device-statistics.htm) - This likely has backup job data! - -**What to extract from docs:** -- API base URL/endpoint -- Request format (JSON-RPC? REST? POST body structure?) -- How to use the token in requests -- Response format examples -- Which fields contain backup status/results - -**2. Quick API Test with Postman** (can be done now with token!) - -### Postman Setup Instructions - -**Step 1: Create New Request** -1. Open Postman -2. Click "New" β†’ "HTTP Request" -3. Name it "Cove API - Test Authentication" - -**Step 2: Configure Request** -- **Method:** GET -- **URL:** Try these in order: - 1. `https://api.backup.management/api/accounts` - 2. `https://backup.management/api/accounts` - 3. `https://api.backup.management/api/customers` - -**Step 3: Add Authentication (try both methods)** - -**Option A: Bearer Token** -- Go to "Authorization" tab -- Type: "Bearer Token" -- Token: `YOUR_TOKEN` (paste token from backup.management) - -**Option B: API Key in Header** -- Go to "Headers" tab -- Add header: - - Key: `X-API-Key` - - Value: `YOUR_TOKEN` - -**Step 4: Send Request and Analyze Response** - -**Expected Results:** -- βœ… **200 OK** β†’ Success! API works, save this configuration - - Copy the JSON response β†’ we'll analyze structure - - Note which URL and auth method worked - - Check for pagination info in response - -- ❌ **401 Unauthorized** β†’ Wrong auth method - - Try other authentication option (Bearer vs X-API-Key) - - Check if token was copied correctly - -- ❌ **404 Not Found** β†’ Wrong endpoint URL - - Try alternative base URL (api.backup.management vs backup.management) - - Try different endpoint (/api/customers, /api/devices) - -- ❌ **403 Forbidden** β†’ Token works but insufficient permissions - - Verify API user has SuperUser role - - Check customer scope selection - -**Step 5: Discover Available Endpoints** - -Once authentication works, try these endpoints: -``` -GET /api/accounts -GET /api/customers -GET /api/devices -GET /api/jobs -GET /api/statistics -GET /api/sessions -``` - -For each successful endpoint, save: -- The request in Postman collection -- Example response in TODO or separate file -- Note any query parameters (page, limit, filter, etc.) - -**Step 6: Look for API Documentation** - -Try these URLs in browser or Postman: -- `https://api.backup.management/swagger` -- `https://api.backup.management/docs` -- `https://api.backup.management/api-docs` -- `https://backup.management/api/documentation` - -**Step 7: Document Findings** - -After successful testing, document in this TODO: -- βœ… Working API base URL -- βœ… Correct authentication method (Bearer vs header) -- βœ… List of available endpoints discovered -- βœ… JSON response structure examples -- βœ… Any pagination/filtering patterns -- βœ… Rate limits (check response headers: X-RateLimit-*) - -### Postman Tips for This Project - -**Save Everything:** -- Create a "Cove API" collection in Postman -- Save all working requests -- Export collection to JSON for documentation - -**Use Variables:** -- Create Postman environment "Cove Production" -- Add variable: `cove_token` = your token -- Add variable: `cove_base_url` = working base URL -- Use `{{cove_token}}` and `{{cove_base_url}}` in requests - -**Check Response Headers:** -- Look for `X-RateLimit-Limit` (API call limits) -- Look for `X-RateLimit-Remaining` (calls left) -- Look for `Link` header (pagination) - -**Save Response Examples:** -- For each endpoint, save example response -- Use Postman's "Save Response" feature -- Or copy JSON to separate file for reference - -**3. Document Findings** - -**After successful Postman testing, update this TODO with:** - -```markdown -## βœ… API Testing Results (Add after testing) - -### Working Configuration -- **Base URL:** [fill in] -- **Authentication:** Bearer Token / X-API-Key header (circle one) -- **Token Location:** Authorization header / X-API-Key header (circle one) - -### Available Endpoints Discovered -| Endpoint | Method | Purpose | Response Fields | -|----------|--------|---------|-----------------| -| /api/accounts | GET | List accounts | [list key fields] | -| /api/customers | GET | List customers | [list key fields] | -| /api/devices | GET | List backup devices | [list key fields] | -| /api/jobs | GET | List backup jobs | [list key fields] | - -### Key Response Fields for Backupchecks Integration -From backup job/session endpoint: -- Job ID: `[field name]` -- Job Name: `[field name]` -- Status: `[field name]` (values: success/warning/failed) -- Start Time: `[field name]` -- End Time: `[field name]` -- Customer/Device: `[field name]` -- Error Messages: `[field name]` -- Backup Objects: `[field name or nested path]` - -### Pagination -- Method: [Link headers / page parameter / cursor / none] -- Page size: [default and max] -- Total count: [available in response?] - -### Rate Limiting -- Limit: [X requests per Y time] -- Headers: [X-RateLimit-* header names] - -### API Documentation URL -- [URL if found, or "Not found" if unavailable] -``` - -**Save Postman Collection:** -- Export collection as JSON -- Save to: `/docker/develop/backupchecks/docs/cove-api-postman-collection.json` -- Or share Postman workspace link in this TODO - -**4. Create POC Script** -Once API works, create standalone Python test script: -```python -import requests - -# Test script to retrieve Cove backup data -token = "YOUR_TOKEN" -base_url = "https://api.example.com" - -headers = { - "Authorization": f"Bearer {token}", - "Content-Type": "application/json" -} - -# Get list of customers -response = requests.get(f"{base_url}/api/customers", headers=headers) -print(response.json()) -``` - -**5. Plan Integration** -Based on POC results, decide architecture approach and start implementation - -**Status:** Ready for API testing - token available! - ---- - -## πŸ“ Notes - -- This TODO document should be updated after each research step -- Add API examples as soon as available -- Document edge cases and limitations -- Consider security implications (API key storage, rate limits, etc.) - -### Current Status (2026-02-23) πŸŽ‰ BREAKTHROUGH -- βœ… **Confirmed:** Cove Data Protection HAS API access (mentioned in documentation) -- βœ… **Found:** API user creation location in Cove portal -- βœ… **Created:** API user with SuperUser role and token -- βœ… **Found:** Complete JSON API documentation (N-able docs site) -- βœ… **Tested:** API authentication and multiple methods (with ChatGPT assistance) -- βœ… **BLOCKER RESOLVED:** N-able support (Andrew Robinson) confirmed D02/D03 are legacy! - - Use D10/D11 instead of D02/D03 - - No MSP-level restrictions – all users have same access - - New docs: https://developer.n-able.com/n-able-cove/docs/ -- βœ… **New column codes identified:** - - D9F00 = Last Session Status (2=Failed, 5=Completed, 8=CompletedWithErrors) - - D9F09 = Last Successful Session Timestamp - - D9F15 = Last Session Timestamp - - D9F06 = Error Count -- πŸ”„ **Next step:** Run `cove_api_test.py` to verify new column codes work -- πŸ“‹ **After test:** Implement full integration in Backupchecks website - -### Test Results Summary (see docs/cove_data_protection_api_calls_known_info.md) -- **Endpoint:** https://api.backup.management/jsonapi (JSON-RPC 2.0) -- **Authentication:** Login method β†’ visa token β†’ include in all subsequent calls -- **Working method:** EnumerateAccountStatistics (with limited columns) -- **Blocked method:** EnumerateAccounts (security error 13501) -- **Safe columns:** I1, I14, I18, D01F00-D01F07, D09F00 -- **Restricted columns:** D02Fxx, D03Fxx ranges (cause entire request to fail) -- **Scope limitation:** API users are customer-scoped, not MSP-level - -### API Credentials (Created) -- **Authentication:** Token-based -- **Role:** SuperUser (full access) -- **Scope:** Top-level customer (access to all sub-customers) -- **Token:** Generated (store securely!) -- **Portal URL:** https://backup.management -- **API User Management:** https://backup.management/#/api-users - -**IMPORTANT:** Store token in secure location (password manager) - cannot be retrieved again if lost! - -### Likely API Base URLs to Test -Based on portal URL `backup.management`: -1. `https://api.backup.management` (most common pattern) -2. `https://backup.management/api` -3. `https://api.backup.management/jsonapi` (some backup systems use this) -4. Check API user page for hints or documentation links - -### Possible Admin Portal Locations -Check these sections in Cove dashboard: -- Settings β†’ API Keys / Developer -- Settings β†’ Integrations -- Account β†’ API Access -- Partner Portal β†’ API Management -- Company Settings β†’ Advanced β†’ API - -### Support Channels -If API activation is not obvious: -- Cove support ticket: Ask "How do I enable API access for backup monitoring?" -- N-able partner support (if MSP) -- Check Cove community forums -- Review onboarding documentation for API mentions +- [ ] 28-daagse history grafiek +- [ ] Per-datasource status (SQL, Exchange, etc.) +- [ ] Polling frequentie instelbaar per klant