diff --git a/TODO-cove-data-protection.md b/TODO-cove-data-protection.md index 73eb992..6e8b598 100644 --- a/TODO-cove-data-protection.md +++ b/TODO-cove-data-protection.md @@ -1,204 +1,204 @@ # TODO: Cove Data Protection Integration -**Datum:** 2026-02-10 -**Status:** Research fase -**Prioriteit:** Medium +**Date:** 2026-02-10 +**Status:** Research phase +**Priority:** Medium --- -## 🎯 Doel +## 🎯 Goal -Cove Data Protection (formerly N-able Backup / SolarWinds Backup) integreren in Backupchecks voor monitoring van backup status. +Integrate Cove Data Protection (formerly N-able Backup / SolarWinds Backup) into Backupchecks for backup status monitoring. -**Uitdaging:** Cove werkt NIET met email notificaties zoals andere backup systemen (Veeam, Synology, NAKIVO). We moeten een alternatieve methode vinden om backup status informatie binnen te halen. +**Challenge:** Cove does NOT work with email notifications like other backup systems (Veeam, Synology, NAKIVO). We need to find an alternative method to import backup status information. --- -## πŸ” Research Vragen +## πŸ” Research Questions -### 1. API Beschikbaarheid -- [ ] Heeft Cove Data Protection een publieke API? -- [ ] Welke authenticatie methode gebruikt de API? (API key, OAuth, basic auth?) -- [ ] Welke endpoints zijn beschikbaar voor backup status? -- [ ] Is er rate limiting op de API? -- [ ] Documentatie URL: ? +### 1. API Availability +- [ ] Does Cove Data Protection have a public API? +- [ ] What authentication method does the API use? (API key, OAuth, basic auth?) +- [ ] Which endpoints are available for backup status? +- [ ] Is there rate limiting on the API? +- [ ] Documentation URL: ? -### 2. Data Structuur -- [ ] Welke informatie kunnen we ophalen per backup job? - - Job naam +### 2. Data Structure +- [ ] What information can we retrieve per backup job? + - Job name - Status (success/warning/failed) - - Start/eind tijd + - Start/end time - Backup type - - Client/device naam + - Client/device name - Error messages - Objects/files backed up -- [ ] Is er een webhook systeem beschikbaar? -- [ ] Hoe vaak moet de API gepolld worden? +- [ ] Is there a webhook system available? +- [ ] How often should the API be polled? ### 3. Multi-Tenancy -- [ ] Ondersteunt Cove multi-tenant setups? (MSP use case) -- [ ] Kunnen we meerdere customers/partners monitoren vanuit 1 account? -- [ ] Hoe worden permissions/access geregeld? +- [ ] Does Cove support multi-tenant setups? (MSP use case) +- [ ] Can we monitor multiple customers/partners from 1 account? +- [ ] How are permissions/access managed? -### 4. Integratie Strategie -- [ ] **Optie A: Scheduled Polling** - - Cronjob die periodiek API aanroept - - Parse resultaten naar JobRun records - - Pro: Eenvoudig, consistent met huidige flow - - Con: Delay tussen backup en registratie in systeem +### 4. Integration Strategy +- [ ] **Option A: Scheduled Polling** + - Cronjob that periodically calls API + - Parse results to JobRun records + - Pro: Simple, consistent with current flow + - Con: Delay between backup and registration in system -- [ ] **Optie B: Webhook/Push** - - Cove stuurt notificaties naar ons endpoint +- [ ] **Option B: Webhook/Push** + - Cove sends notifications to our endpoint - Pro: Real-time updates - Con: Requires external endpoint, security considerations -- [ ] **Optie C: Email Forwarding** - - Als Cove toch email support heeft (hidden setting?) - - Pro: Hergebruikt bestaande email import flow - - Con: Mogelijk niet beschikbaar +- [ ] **Option C: Email Forwarding** + - If Cove has email support after all (hidden setting?) + - Pro: Reuses existing email import flow + - Con: Possibly not available --- -## πŸ“‹ Technische Overwegingen +## πŸ“‹ Technical Considerations ### Database Model -Huidige JobRun model verwacht: -- `mail_message_id` (FK) - hoe gaan we dit aanpassen voor API-sourced runs? -- Mogelijk nieuw veld: `source_type` ("email" vs "api") -- Mogelijk nieuw veld: `external_id` (Cove job ID) +Current JobRun model expects: +- `mail_message_id` (FK) - how do we adapt this for API-sourced runs? +- Possible new field: `source_type` ("email" vs "api") +- Possible new field: `external_id` (Cove job ID) -### Parser Systeem -Huidige parser systeem werkt met email content. Voor API: -- Nieuw "parser" concept voor API responses? -- Of direct JobRun creatie zonder parser layer? +### Parser System +Current parser system works with email content. For API: +- New "parser" concept for API responses? +- Or direct JobRun creation without parser layer? -### Architecture Opties +### Architecture Options -**Optie 1: Extend Email Import System** +**Option 1: Extend Email Import System** ``` API Poller β†’ Pseudo-MailMessage β†’ Existing Parser β†’ JobRun ``` -- Pro: Hergebruik bestaande flow -- Con: Hacky, email velden hebben geen betekenis +- Pro: Reuse existing flow +- Con: Hacky, email fields have no meaning -**Optie 2: Parallel Import System** +**Option 2: Parallel Import System** ``` API Poller β†’ API Parser β†’ JobRun (direct) ``` -- Pro: Clean separation, geen email dependency -- Con: Duplicatie van logic +- Pro: Clean separation, no email dependency +- Con: Logic duplication -**Optie 3: Unified Import Layer** +**Option 3: Unified Import Layer** ``` β†’ Email Import β†’ Unified β†’ β†’ Common Processor β†’ JobRun β†’ API Import β†’ ``` -- Pro: Future-proof, schaalbaar -- Con: Grotere refactor +- Pro: Future-proof, scalable +- Con: Larger refactor --- -## πŸ”§ Implementatie Stappen (Na Research) +## πŸ”§ Implementation Steps (After Research) ### Phase 1: API Research & POC -1. [ ] Onderzoek Cove API documentatie -2. [ ] Test API authenticatie -3. [ ] Test data ophalen (1 backup job) -4. [ ] Mapping van Cove data β†’ Backupchecks model +1. [ ] Research Cove API documentation +2. [ ] Test API authentication +3. [ ] Test data retrieval (1 backup job) +4. [ ] Mapping of Cove data β†’ Backupchecks model 5. [ ] Proof of concept script (standalone) -### Phase 2: Database Aanpassingen -1. [ ] Beslis: extend MailMessage model of nieuwe source type? -2. [ ] Migratie: `source_type` veld toevoegen aan JobRun -3. [ ] Migratie: `external_id` veld toevoegen aan JobRun +### Phase 2: Database Changes +1. [ ] Decide: extend MailMessage model or new source type? +2. [ ] Migration: add `source_type` field to JobRun +3. [ ] Migration: add `external_id` field to JobRun 4. [ ] Update constraints/validations -### Phase 3: Import Mechanisme -1. [ ] Nieuw bestand: `containers/backupchecks/src/backend/app/cove_importer.py` -2. [ ] API client voor Cove -3. [ ] Data transformatie naar JobRun format +### Phase 3: Import Mechanism +1. [ ] New file: `containers/backupchecks/src/backend/app/cove_importer.py` +2. [ ] API client for Cove +3. [ ] Data transformation to JobRun format 4. [ ] Error handling & retry logic 5. [ ] Logging & audit trail ### Phase 4: Scheduling -1. [ ] Cronjob/scheduled task voor polling (elke 15 min?) -2. [ ] Of: webhook endpoint als Cove dat ondersteunt +1. [ ] Cronjob/scheduled task for polling (every 15 min?) +2. [ ] Or: webhook endpoint if Cove supports it 3. [ ] Rate limiting & throttling -4. [ ] Duplicate detection (niet dubbel importeren) +4. [ ] Duplicate detection (avoid double imports) ### Phase 5: UI Updates -1. [ ] Job Details: indicatie dat job van API komt (niet email) -2. [ ] No "Download EML" button voor API-sourced runs -3. [ ] Mogelijk andere metadata weergave +1. [ ] Job Details: indication that job is from API (not email) +2. [ ] No "Download EML" button for API-sourced runs +3. [ ] Possibly different metadata display --- -## πŸ“š Referenties +## πŸ“š References ### Cove Data Protection -- **Product naam:** Cove Data Protection (formerly N-able Backup, SolarWinds Backup) +- **Product name:** Cove Data Protection (formerly N-able Backup, SolarWinds Backup) - **Website:** https://www.n-able.com/products/cove-data-protection -- **API Docs:** [TODO: link toevoegen na research] +- **API Docs:** [TODO: add link after research] - **Support:** [TODO: contact info] -### Vergelijkbare Integraties -Andere backup systemen die API gebruiken: -- Veeam: Heeft zowel email als REST API -- Acronis: REST API beschikbaar -- MSP360: API voor management +### Similar Integrations +Other backup systems that use APIs: +- Veeam: Has both email and REST API +- Acronis: REST API available +- MSP360: API for management ### Resources -- [ ] API documentation (nog te vinden) -- [ ] SDK/Client libraries beschikbaar? -- [ ] Community/forum voor integratie vragen? +- [ ] API documentation (yet to find) +- [ ] SDK/Client libraries available? +- [ ] Community/forum for integration questions? - [ ] Example code/integrations? --- -## ❓ Open Vragen +## ❓ Open Questions -1. **Performance:** Hoeveel Cove jobs moeten we monitoren? (impact op polling frequency) -2. **Historical Data:** Kunnen we oude backup runs ophalen, of alleen nieuwe? -3. **Filtering:** Kunnen we filters toepassen (alleen failed jobs, specifieke clients)? -4. **Authentication:** Waar bewaren we Cove API credentials? (SystemSettings?) -5. **Multi-Account:** Ondersteunen we meerdere Cove accounts? (MSP scenario) +1. **Performance:** How many Cove jobs do we need to monitor? (impact on polling frequency) +2. **Historical Data:** Can we retrieve old backup runs, or only new ones? +3. **Filtering:** Can we apply filters (only failed jobs, specific clients)? +4. **Authentication:** Where do we store Cove API credentials? (SystemSettings?) +5. **Multi-Account:** Do we support multiple Cove accounts? (MSP scenario) --- ## 🎯 Success Criteria ### Minimum Viable Product (MVP) -- [ ] Backup runs van Cove worden automatisch geΓ―mporteerd -- [ ] Status (success/warning/failed) correct weergegeven -- [ ] Job naam en timestamp beschikbaar -- [ ] Zichtbaar in Daily Jobs & Run Checks -- [ ] Errors en warnings worden getoond +- [ ] Backup runs from Cove are automatically imported +- [ ] Status (success/warning/failed) displayed correctly +- [ ] Job name and timestamp available +- [ ] Visible in Daily Jobs & Run Checks +- [ ] Errors and warnings are shown ### Nice to Have -- [ ] Real-time import (webhook ipv polling) -- [ ] Backup object details (individuele files/folders) +- [ ] Real-time import (webhook instead of polling) +- [ ] Backup object details (individual files/folders) - [ ] Retry history - [ ] Storage usage metrics - [ ] Multi-tenant support --- -## πŸš€ Volgende Stappen +## πŸš€ Next Steps -1. **Research eerst!** - Begin met API documentatie onderzoek -2. Maak POC script (standalone, buiten Backupchecks) -3. Documenteer bevindingen in dit bestand -4. Beslis welke architecture optie (1, 2, of 3) -5. Dan pas implementatie starten +1. **Research first!** - Start with API documentation investigation +2. Create POC script (standalone, outside Backupchecks) +3. Document findings in this file +4. Decide which architecture option (1, 2, or 3) +5. Only then start implementation -**Status:** Wachten op API research completion. +**Status:** Waiting on API research completion. --- ## πŸ“ Notes -- Dit TODO document moet bijgewerkt worden na elke research stap -- Voeg API voorbeelden toe zodra beschikbaar -- Documenteer edge cases en beperkingen -- Overweeg security implications (API keys opslag, rate limits, etc.) +- This TODO document should be updated after each research step +- Add API examples as soon as available +- Document edge cases and limitations +- Consider security implications (API key storage, rate limits, etc.)