Translate docs to English and add project README
This commit is contained in:
parent
58268a4906
commit
ec64502265
49
README.md
Normal file
49
README.md
Normal file
@ -0,0 +1,49 @@
|
||||
# Novela
|
||||
|
||||
Novela is a self-hosted web application for managing and reading a personal digital library.
|
||||
It supports EPUB, PDF, and CBR/CBZ, with metadata editing, reading progress tracking, and Dropbox backups.
|
||||
|
||||
## What Novela Provides
|
||||
- Library import and indexing for EPUB/PDF/CBR/CBZ
|
||||
- Home dashboard with continue reading and unread/read sections
|
||||
- Reader support for EPUB, PDF, and comics (CBR/CBZ)
|
||||
- Metadata editing (title, author, publisher, series, volume, tags, genres)
|
||||
- `New` review workflow with list/grid view, column toggles, and bulk actions
|
||||
- Reading analytics/statistics dashboard
|
||||
- Dropbox backup with:
|
||||
- versioned snapshots
|
||||
- object deduplication
|
||||
- retention policy
|
||||
- scheduled background runs
|
||||
|
||||
## Tech Stack
|
||||
- FastAPI
|
||||
- Jinja2 templates + vanilla JavaScript
|
||||
- PostgreSQL 16
|
||||
- Docker / Docker Compose style deployment
|
||||
|
||||
## Repository Layout
|
||||
- `containers/novela/` - application code (routers, templates, static assets, migrations)
|
||||
- `stack/` - deployment stack files and environment configuration
|
||||
- `docs/` - technical status and changelog documentation
|
||||
- `build-and-push.sh` - helper script for container build/push
|
||||
|
||||
## Quick Start (Development)
|
||||
1. Configure environment values in `stack/novela.env`.
|
||||
2. Start the stack using your preferred Docker workflow (Compose/Portainer stack).
|
||||
3. Open the app and complete backup credential setup on `/backup` if needed.
|
||||
|
||||
## Key Environment Variables
|
||||
- `POSTGRES_DB`
|
||||
- `POSTGRES_USER`
|
||||
- `POSTGRES_PASSWORD`
|
||||
- `NOVELA_MASTER_KEY`
|
||||
- `CONFIG_DIR`
|
||||
- `LIBRARY_DIR` (optional override)
|
||||
|
||||
## Documentation
|
||||
- Technical status: `docs/TECHNICAL.md`
|
||||
- Develop changelog: `docs/changelog-develop.md`
|
||||
|
||||
## Status
|
||||
This repository is actively evolving. The develop documentation reflects implemented behavior in the current codebase.
|
||||
@ -1,20 +1,20 @@
|
||||
# Novela 2.0 - Technical Status (Develop)
|
||||
|
||||
## Scope
|
||||
Dit document beschrijft de actuele technische status van de `develop` codebase.
|
||||
Dit document is de primaire technische documentatie voor de huidige codebase.
|
||||
This document describes the current technical status of the `develop` codebase.
|
||||
It is the primary technical reference for the current implementation.
|
||||
|
||||
## Architecture
|
||||
- Stack: FastAPI, Jinja2 templates, plain JS, PostgreSQL 16, Docker.
|
||||
- Stack: FastAPI, Jinja2 templates, plain JavaScript, PostgreSQL 16, Docker.
|
||||
- Startup lifecycle (`main.py`):
|
||||
1. `init_pool()`
|
||||
2. `run_migrations()`
|
||||
3. `start_backup_scheduler()`
|
||||
4. routers mounten
|
||||
4. mount routers
|
||||
- Shutdown lifecycle:
|
||||
1. `stop_backup_scheduler()`
|
||||
2. `close_pool()`
|
||||
- Source-of-truth regel: bestand op schijf leidend, database als index/cache.
|
||||
- Source-of-truth rule: files on disk are authoritative, the database is an index/cache.
|
||||
|
||||
## Router Status
|
||||
|
||||
@ -36,33 +36,33 @@ Dit document is de primaire technische documentatie voor de huidige codebase.
|
||||
- `GET /api/stats`
|
||||
- `GET /library/list` (compat)
|
||||
|
||||
`GET /api/library` draait standaard in fast-path (DB-only, geen full disk rescan).
|
||||
Voor geforceerde sync: `GET /api/library?rescan=true` of `POST /library/rescan`.
|
||||
`include_file_info=true` is optioneel voor bestandsgrootte/mtime verrijking.
|
||||
`GET /api/library` runs in fast-path mode by default (DB-only, no full disk rescan).
|
||||
For a forced sync: `GET /api/library?rescan=true` or `POST /library/rescan`.
|
||||
`include_file_info=true` is optional for file size/mtime enrichment.
|
||||
|
||||
`/api/home` levert:
|
||||
`/api/home` returns:
|
||||
- `continue_reading`
|
||||
- `shorts_unread`
|
||||
- `novels_unread`
|
||||
- `shorts_read`
|
||||
- `novels_read`
|
||||
|
||||
`/api/stats` levert naast totals ook chart- en history-data voor `stats.html`:
|
||||
`/api/stats` returns totals plus chart/history data for `stats.html`:
|
||||
- `reads_by_month`, `reads_by_dow`, `reads_by_hour`
|
||||
- `genre_counts`, `publisher_counts`, `fav_genre`, `fav_publisher`
|
||||
- `top_books`, `history`
|
||||
|
||||
Home-secties filteren series uit met:
|
||||
Home sections exclude series books via:
|
||||
- `COALESCE(series, '') = ''`
|
||||
- `filename NOT LIKE '%/Series/%'`
|
||||
|
||||
Read-secties op Home zijn gesorteerd op oudste eerst:
|
||||
Home read sections are ordered oldest-first:
|
||||
- `shorts_read`: `ORDER BY MAX(read_at) ASC`
|
||||
- `novels_read`: `ORDER BY MAX(read_at) ASC`
|
||||
|
||||
### `routers/reader.py`
|
||||
- Epub serving/chapters/images
|
||||
- Reader pagina + book detail
|
||||
- EPUB serving/chapters/images
|
||||
- Reader page + book detail
|
||||
- Metadata patch (`PATCH /library/book/{filename}`)
|
||||
- Progress read/write/delete
|
||||
- Mark-as-read
|
||||
@ -71,15 +71,15 @@ Read-secties op Home zijn gesorteerd op oudste eerst:
|
||||
- Genres endpoint
|
||||
|
||||
### `routers/editor.py`
|
||||
- Editor pagina
|
||||
- Editor page
|
||||
- Chapter get/save
|
||||
- Chapter add
|
||||
- Chapter delete
|
||||
|
||||
### `routers/grabber.py`
|
||||
- Grabber pagina + convert/debug flows
|
||||
- Grabber page + convert/debug flows
|
||||
- SSE events
|
||||
- Credentials beheer voor scraper-sites
|
||||
- Credential management for scraper sites
|
||||
- Credentials manager UI (`/credentials-manager`)
|
||||
|
||||
### `routers/backup.py`
|
||||
@ -91,44 +91,44 @@ Read-secties op Home zijn gesorteerd op oudste eerst:
|
||||
- `POST /api/backup/run`
|
||||
|
||||
## Backup & Security
|
||||
- Dropbox token encrypted-at-rest in `credentials` (`site='dropbox'`).
|
||||
- Dropbox backup root encrypted opgeslagen in `credentials` (`site='dropbox_backup_root'`).
|
||||
- Retentie (`snapshots to keep`) encrypted opgeslagen in `credentials` (`site='dropbox_backup_retention'`).
|
||||
- Backup schedule (`enabled` + `interval_hours`) encrypted opgeslagen in `credentials` (`site='dropbox_backup_schedule'`).
|
||||
- Encryptie via `NOVELA_MASTER_KEY` (Fernet).
|
||||
- Dropbox token is stored encrypted-at-rest in `credentials` (`site='dropbox'`).
|
||||
- Dropbox backup root is stored encrypted in `credentials` (`site='dropbox_backup_root'`).
|
||||
- Retention (`snapshots to keep`) is stored encrypted in `credentials` (`site='dropbox_backup_retention'`).
|
||||
- Backup schedule (`enabled` + `interval_hours`) is stored encrypted in `credentials` (`site='dropbox_backup_schedule'`).
|
||||
- Encryption uses `NOVELA_MASTER_KEY` (Fernet).
|
||||
|
||||
Implementatie:
|
||||
- Versie-gebaseerde backups met deduplicatie:
|
||||
- bestandsobjecten in Dropbox: `library_objects/{sha256_prefix}/{sha256}`
|
||||
Implementation details:
|
||||
- Versioned backups with deduplication:
|
||||
- file objects in Dropbox: `library_objects/{sha256_prefix}/{sha256}`
|
||||
- snapshots in Dropbox: `library_snapshots/snapshot-YYYYMMDD-HHMMSS.json`
|
||||
- Elke run maakt een nieuwe snapshot (versie) en uploadt alleen ontbrekende objecten.
|
||||
- Retentie verwijdert oude snapshots boven de ingestelde limiet.
|
||||
- Orphan object-prune verwijdert objecten die niet meer door retained snapshots worden gerefereerd.
|
||||
- Lokale manifestcache (`config/backup_manifest.json`) versnelt changeddetectie.
|
||||
- Database backup via `pg_dump` naar Dropbox `postgres/`.
|
||||
- `POST /api/backup/run` start altijd als background task en geeft direct status terug.
|
||||
- Scheduler draait in achtergrond (`start_backup_scheduler`) en triggert op interval als backup aanstaat.
|
||||
- Concurrentierestrictie: slechts 1 backup tegelijk.
|
||||
- Bij container restart/crash worden stale `running` logs automatisch gemarkeerd als interrupted/error.
|
||||
- Each run creates a new snapshot version and uploads only missing objects.
|
||||
- Retention removes older snapshots above the configured limit.
|
||||
- Orphan object pruning removes objects no longer referenced by retained snapshots.
|
||||
- Local manifest cache (`config/backup_manifest.json`) speeds up change detection.
|
||||
- Database backup is done via `pg_dump` to Dropbox `postgres/`.
|
||||
- `POST /api/backup/run` always starts a background task and returns immediately.
|
||||
- Scheduler runs in the background (`start_backup_scheduler`) and triggers on interval when enabled.
|
||||
- Concurrency guard: only one backup can run at a time.
|
||||
- After container restart/crash, stale `running` logs are auto-marked as interrupted/error.
|
||||
|
||||
## Environment
|
||||
`stack/novela.env` bevat nu minimaal:
|
||||
`stack/novela.env` should include at least:
|
||||
- `POSTGRES_DB`
|
||||
- `POSTGRES_USER`
|
||||
- `POSTGRES_PASSWORD`
|
||||
- `NOVELA_MASTER_KEY`
|
||||
- `CONFIG_DIR`
|
||||
|
||||
Dropbox settings verlopen via webinterface op `/backup`.
|
||||
Dropbox settings are managed via the web UI on `/backup`.
|
||||
|
||||
## UI Notes
|
||||
- Library import accepteert: EPUB/PDF/CBR/CBZ.
|
||||
- Home heeft dezelfde importmogelijkheden.
|
||||
- Home heeft zoekfunctionaliteit.
|
||||
- Home header/dropzone uitlijning gelijkgetrokken met Library (zoek rechtsboven, dropzone eronder).
|
||||
- `New` view ondersteunt `Grid` en `List` mode.
|
||||
- Bulk selectie + `Remove from New` werkt alleen in `List` mode.
|
||||
- `List` mode heeft kolomfilter (aan/uit) met kolommen:
|
||||
- Library import accepts EPUB/PDF/CBR/CBZ.
|
||||
- Home supports the same import formats.
|
||||
- Home includes search.
|
||||
- Home header/dropzone alignment matches Library (search top-right, dropzone below).
|
||||
- `New` view supports `Grid` and `List` mode.
|
||||
- Bulk selection + `Remove from New` works only in `List` mode.
|
||||
- `List` mode has a column visibility filter with columns:
|
||||
- Publisher
|
||||
- Author
|
||||
- Series
|
||||
@ -140,27 +140,26 @@ Dropbox settings verlopen via webinterface op `/backup`.
|
||||
- Sub-genres
|
||||
- Tags
|
||||
- Status
|
||||
- `List` mode ondersteunt multi-select met `Shift+klik` range-select op checkboxes.
|
||||
- `Grid` mode toont geen selectie-checkboxes of bulkacties.
|
||||
- Backup pagina ondersteunt:
|
||||
- handmatige run en dry-run
|
||||
- instellingen voor Dropbox root
|
||||
- retentie-aantal snapshots
|
||||
- geplande backup (aan/uit + interval in uren)
|
||||
- status + history overzicht
|
||||
- `List` mode supports multi-select with `Shift+click` range selection on checkboxes.
|
||||
- `Grid` mode shows no selection checkboxes or bulk actions.
|
||||
- Backup page supports:
|
||||
- manual run and dry-run
|
||||
- Dropbox root settings
|
||||
- snapshot retention count
|
||||
- scheduled backup (on/off + interval in hours)
|
||||
- status + history overview
|
||||
|
||||
## Known Conventions
|
||||
- Verwijderen boek: bestand verwijderen, lege mappen prunen, daarna `DELETE FROM library` (cascade op child-tabellen).
|
||||
- Cover strategie:
|
||||
- EPUB: cover uit bestand + cache
|
||||
- Book deletion flow: delete file, prune empty directories, then `DELETE FROM library` (cascade removes child rows).
|
||||
- Cover strategy:
|
||||
- EPUB: cover from file + cache
|
||||
- PDF/CBR: thumbnail via cover cache
|
||||
|
||||
|
||||
## Performance Notes
|
||||
- Library-load geoptimaliseerd voor grote datasets:
|
||||
- `list_library_json()` gebruikt pre-aggregatie voor `reading_sessions`.
|
||||
- `has_cached_cover` komt direct uit SQL join i.p.v. losse volledige cache-fetch.
|
||||
- Nieuwe migrations-indexen:
|
||||
- Library load is optimized for large datasets:
|
||||
- `list_library_json()` uses pre-aggregation for `reading_sessions`.
|
||||
- `has_cached_cover` is provided directly via SQL join instead of full cache fetch.
|
||||
- Additional migration indexes:
|
||||
- `idx_library_sort_coalesce`
|
||||
- `idx_library_needs_review`
|
||||
- `idx_library_archived`
|
||||
|
||||
@ -1,47 +1,47 @@
|
||||
# Changelog Develop
|
||||
# Develop Changelog
|
||||
|
||||
Dit bestand houdt wijzigingen op de `develop` lijn bij.
|
||||
`changelog.md` wordt later gebruikt voor release-samenvattingen.
|
||||
This file tracks changes on the `develop` line.
|
||||
`changelog.md` can later be used for release summaries.
|
||||
|
||||
## 2026-03-22
|
||||
- Blueprint en technische documentatie toegevoegd in `docs/`.
|
||||
- Router-splitsing en bootstrapstructuur afgerond (`main.py`, routers, migrations, db pool).
|
||||
- Media support uitgebreid naar EPUB/PDF/CBR/CBZ in import- en scanflow.
|
||||
- Home UI uitgebreid met:
|
||||
- import-dropzone voor EPUB/PDF/CBR/CBZ
|
||||
- zoekfunctie
|
||||
- uitlijning gelijk aan Library (zoek rechtsboven, dropzone eronder)
|
||||
- Library UI importteksten en drag/drop filtering bijgewerkt voor multi-format.
|
||||
- Library `New` view uitgebreid:
|
||||
- Added blueprint/technical documentation structure in `docs/`.
|
||||
- Completed router split and bootstrap structure (`main.py`, routers, migrations, DB pool).
|
||||
- Expanded media support to EPUB/PDF/CBR/CBZ in import and scan flows.
|
||||
- Expanded Home UI with:
|
||||
- import dropzone for EPUB/PDF/CBR/CBZ
|
||||
- search functionality
|
||||
- alignment matching Library (search top-right, dropzone below)
|
||||
- Updated Library import texts and drag/drop filtering for multi-format support.
|
||||
- Expanded Library `New` view with:
|
||||
- `Grid`/`List` toggle
|
||||
- kolomfilter in `List`
|
||||
- column visibility filter in `List`
|
||||
- multi-select + bulk `Remove from New`
|
||||
- selectie alleen in `List` mode
|
||||
- `Shift+klik` range-select op checkboxes
|
||||
- Nieuwe route toegevoegd: `POST /library/new/mark-reviewed` (bulk `needs_review=false`).
|
||||
- Library performance verbeterd:
|
||||
- `/api/library` fast-path (geen full rescan per page-load)
|
||||
- optionele `rescan=true`/`include_file_info=true`
|
||||
- SQL-optimalisatie in `list_library_json()`
|
||||
- extra DB-indexen voor schaal
|
||||
- `/api/home` hersteld naar volledige dataset-output:
|
||||
- selection only in `List` mode
|
||||
- `Shift+click` range selection on checkboxes
|
||||
- Added route: `POST /library/new/mark-reviewed` (bulk set `needs_review=false`).
|
||||
- Improved library performance:
|
||||
- `/api/library` fast-path (no full rescan on every page load)
|
||||
- optional `rescan=true` / `include_file_info=true`
|
||||
- SQL optimizations in `list_library_json()`
|
||||
- additional DB indexes for scale
|
||||
- Restored `/api/home` full dataset output:
|
||||
- `continue_reading`
|
||||
- `shorts_unread`
|
||||
- `novels_unread`
|
||||
- `shorts_read`
|
||||
- `novels_read`
|
||||
- Home-sectiefilters expliciet zonder serieboeken gezet.
|
||||
- Home read-volgorde gecorrigeerd: in `shorts_read` en `novels_read` staat de oudste bovenaan (`ORDER BY MAX(read_at) ASC`).
|
||||
- Statistics pagina hersteld: `/api/stats` levert weer volledige payload voor charts, favorieten, topboeken en reading history.
|
||||
- Backup verbeterd:
|
||||
- Dropbox token encrypted opgeslagen in DB
|
||||
- Dropbox backup root instelbaar via webinterface en encrypted in DB
|
||||
- versie-gebaseerde snapshots + object-store deduplicatie in Dropbox (`library_snapshots` / `library_objects`)
|
||||
- instelbare snapshot-retentie (`snapshots to keep`) via backup settings
|
||||
- object prune op basis van retained snapshots
|
||||
- geplande backup (enable + interval in uren)
|
||||
- backup runs als background process zodat navigeren op site door kan lopen
|
||||
- herstel op stale running state na restart/crash (oude running logs markeren als interrupted/error)
|
||||
- dry-run ondersteuning op nieuwe flow
|
||||
- Docker image aangepast met `postgresql-client` voor `pg_dump`.
|
||||
- Meerdere test builds uitgevoerd en gepusht naar `gitea.oskamp.info/ivooskamp/novela:dev`.
|
||||
- Explicitly filtered series books out of Home sections.
|
||||
- Corrected Home read ordering: `shorts_read` and `novels_read` now show oldest first (`ORDER BY MAX(read_at) ASC`).
|
||||
- Restored Statistics page by returning the full `/api/stats` payload required for charts, favorites, top books, and reading history.
|
||||
- Improved backup implementation:
|
||||
- Dropbox token stored encrypted in DB
|
||||
- Dropbox backup root configurable via web UI and stored encrypted in DB
|
||||
- versioned snapshots + object-store deduplication in Dropbox (`library_snapshots` / `library_objects`)
|
||||
- configurable snapshot retention (`snapshots to keep`) via backup settings
|
||||
- object pruning based on retained snapshots
|
||||
- scheduled backup (enable + interval in hours)
|
||||
- backup runs as a background process, so page navigation does not block execution
|
||||
- stale running state recovery after restart/crash (old running logs marked interrupted/error)
|
||||
- dry-run support in the new flow
|
||||
- Updated Docker image with `postgresql-client` for `pg_dump`.
|
||||
- Multiple test builds pushed to `gitea.oskamp.info/ivooskamp/novela:dev`.
|
||||
|
||||
Loading…
Reference in New Issue
Block a user