novela/docs/changelog-develop.md

# Develop Changelog

This file tracks changes on the `develop` line.
`changelog.md` can later be used for release summaries.

## 2026-03-28 (2)
- Performance: library page now loads instantly for large collections (1000+ books)
  - `IntersectionObserver` defers both cover image loading and placeholder canvas drawing until cards enter the viewport — eliminates hundreds of upfront canvas ops that blocked the initial render
  - `ETag` caching on `/library/list`: server returns `304 Not Modified` when nothing changed, client skips JSON parse and re-download
  - Single DOM pass in `renderBooksGrid`, `renderDuplicatesView`, `renderSeriesDetail`: canvas and img set up via `card.querySelector` immediately after `innerHTML`, removing a second iteration with `document.getElementById` per card
  - `book_tags` joined via `json_agg` in the main `list_library_json()` query, eliminating a separate `SELECT * FROM book_tags` query and Python merge loop
  - `loadLibrary` now shows an error message instead of staying stuck on "Loading…" when the fetch or render fails

## 2026-03-28 (1)
- Added Following page (`/following`): track external author URLs outside Library and Tools
  - New `authors` table: `name` (unique), `url`, `created_at`, `updated_at`
  - New `routers/following.py`: `GET /following` page, `GET /api/following` (all authors + URL + book count + last added), `POST /api/following/{name}` (set/clear URL)
  - Sidebar: new Following section between Library and Tools; counter shows number of followed authors
  - Following page: two tabs — Following (authors with URL) and All Authors; inline URL editing with Enter/Escape keyboard support; Visit button opens external URL in a new tab; author name links to library author view
- Added Incomplete view to Library (`#incomplete`): shows all non-archived books where `publication_status ≠ Complete`; sidebar counter included; entry placed after New in the Library section

## 2026-03-27 (1)
- Convert page: duplicate warning shown after loading metadata when a book with the same title+author already exists in the library; warning includes a link to the existing book; user can still proceed with conversion
- Library: added Duplicates section to sidebar (between Rated and Statistics); counter shows total number of books that are part of a duplicate group (same title+author, case-insensitive); Duplicates view groups books by title+author with a subheading per group
- Fixed Duplicates view not loading covers: card renderer now uses the same canvas + two-pass img/`makePlaceholderCover` pattern as `renderBooksGrid`

## 2026-03-26 (2)
- Fixed Book Builder page showing white background: `library.css` added to `builder.html` to load `:root` CSS variables and dark `body` background; all CSS variable references in `builder.css` aligned with library theme names (`--text`, `--surface`, `--surface2`, `--text-dim`, `--border`, `--accent`, `--sidebar`)

## 2026-03-26 (1)
- Added Book Builder: create EPUB books manually from scratch via a WYSIWYG editor
  - New `builder_drafts` table (`id UUID`, `title`, `author`, `publisher`, `source_url`, `chapters JSONB`)
  - `build_epub()` in `epub.py`: builds a standards-compliant EPUB 2.0 ZIP from title/author/publisher/chapters; embeds inline CSS and `break.png` if present
  - `normalize_wysiwyg_html()` in `xhtml.py`: converts contenteditable HTML to EPUB-safe XHTML; handles scene-breaks, `<blockquote class="author-note">`, inline formatting, and `<hr>` → break image
  - `routers/builder.py`: draft CRUD, chapter CRUD (`GET/POST/PUT/DELETE`), normalize preview endpoint, publish endpoint (normalizes all chapters → builds EPUB → writes to `library/epub/` → upserts to DB → deletes draft → redirects to book detail)
  - `templates/builder.html` + `static/builder.js` + `static/builder.css`: index page (new draft form + draft list) and editor (chapter panel, contenteditable pane, toolbar with bold/italic/underline/blockquote/author-note/scene-break/normalize, autosave every 30 s, Ctrl+S, publish button)
  - Book Builder link added to sidebar Tools section (between Convert and Credentials)
  - `.author-note` blockquote style added to `static/epub-style.css`

## 2026-03-25 (20)
- Added Rated section to library sidebar: shows all non-archived books with `rating > 0`, sorted by rating descending then title alphabetically; badge displays total count; navigable via `#rated` URL hash

## 2026-03-25 (19)
- Fixed series index 0 not displaying in series slot view, grid cards, list volume column, and book detail:
  - Series slot labels now show `#0` for all slots when the series has at least one positively-indexed sibling (consistent label height across all cards)
  - Grid card and list volume column use the same cross-book heuristic (`indexedSeriesSet`) to show `[0]` where appropriate
  - Book detail page queries whether any sibling in the same series has `series_index > 0` (`series_is_indexed`) and shows `[0]` accordingly

## 2026-03-25 (18)
- Added autocomplete for Author, Publisher, and Series in the book edit panel: typing shows a filtered dropdown of existing values from the library; selecting a value fills the field (same dropdown styling as genres/tags)
- Status field now defaults to "Complete" when opening the edit panel for a book that has no status set yet

## 2026-03-25 (17)
- Fixed CBR reader showing only first page: `cbr_page_count` was missing from the `cbr` import in `reader.py`; `/api/cbr/info/` was returning an error, causing `page_count` to fall back to 1 and the Next button to remain disabled

## 2026-03-25 (16)
- Fixed CBR/CBZ reader stuck on loading: added `/api/cbr/info/{filename}` endpoint returning `{page_count}`, and added a CBR/CBZ branch in `reader.html` that mirrors the PDF paged reader (page images served via `/library/cbr/{filename}/{page}`)

## 2026-03-25 (15)
- Added series volume 0 and letter suffix support (e.g. "21a", "21b"):
  - New `series_suffix VARCHAR(10)` column added to `library` table via `migrate_series_suffix`
  - `series_index` lower bound changed from 1 to 0 throughout (index 0 = special/prequel edition)
  - Volume field in the book editor changed from `type="number"` to `type="text"` — accepts "0", "1", "21a", etc.
  - Server parses the combined volume string into `series_index` (INTEGER) + `series_suffix` (VARCHAR) via `parse_volume_str`
  - File naming includes suffix: `021a - Title.epub`
  - `novela:series_suffix` meta tag written to/read from EPUB OPF for persistence across rescans
  - Series detail view: books sorted by `(series_index, series_suffix)`; slot labels show "21a"; index 0 shown as slot when any sibling has index > 0
  - Grid cards, list view Volume column, author/publisher sort all include suffix

## 2026-03-25 (14)
- Added multi-select and bulk delete to `All books` List view:
  - Checkbox column added to the list table (List mode only; Grid mode unchanged)
  - Select all / Clear all / Clear selection controls in the controls bar
  - `Shift+click` range selection on checkboxes
  - `Delete selected` button (red, disabled when nothing is selected) triggers a confirmation dialog; confirmed deletions remove files from disk and database, then reload the library

## 2026-03-25 (13)
- Fixed EPUB cover replacement reverting to original after upload:
  - `GET /library/cover/{filename}` for EPUBs now checks the DB cover cache first (populated on upload) before falling back to extracting from the EPUB file; the cache is also warmed on cache-miss so subsequent requests are fast.
  - `add_cover_to_epub` fully rewritten: locates the OPF via `META-INF/container.xml`, finds the existing cover image in the OPF manifest, removes it from the ZIP, and writes the new cover in the same directory — works for any EPUB directory structure (`OEBPS/`, `EPUB/`, etc.).

## 2026-03-25 (12)
- Fixed chapter images not loading in EPUBs that use a non-standard root directory (e.g. `EPUB/` instead of `OEBPS/`): image paths are now passed as full ZIP paths instead of stripping the root segment, and the image endpoint no longer hardcodes an `OEBPS/` prefix. Case-insensitive fallback retained for EPUBs with mismatched image folder casing.

## 2026-03-25 (11)
- Fixed reader progress bar jumping to 100% immediately when navigating to the second chapter in short EPUBs (2 chapters): changed progress formula denominator from `total - 1` to `total` so 100% is only reached after fully scrolling through the last chapter. Same fix applied to PDF page progress.

## 2026-03-25 (10)
- Added bookmark feature:
  - `bookmarks` table in DB (`migrate_create_bookmarks`): `filename`, `chapter_index`, `scroll_frac`, `chapter_title`, `note`, `created_at`
  - New API endpoints: `GET/POST /library/bookmarks/{filename}`, `PATCH/DELETE /library/bookmarks/{id}`, `GET /api/bookmarks`
  - Bookmark button in reader header (orange, bookmark icon): opens modal with optional note field
  - Bookmark modal closes on Escape, backdrop click, or Cancel
  - Reader supports `?bm_ch=N&bm_scroll=F` URL params to jump directly to bookmarked position (overrides saved progress)
  - Library sidebar: Bookmarks section in Library nav with live count badge
  - Library `#bookmarks` view: card list showing cover, book title, author, chapter, note, date, Go-to and Delete buttons

## 2026-03-22
- Added blueprint/technical documentation structure in `docs/`.
- Completed router split and bootstrap structure (`main.py`, routers, migrations, DB pool).
- Expanded media support to EPUB/PDF/CBR/CBZ in import and scan flows.
- Expanded Home UI with:
  - import dropzone for EPUB/PDF/CBR/CBZ
  - search functionality
  - alignment matching Library (search top-right, dropzone below)
- Updated Library import texts and drag/drop filtering for multi-format support.
- Expanded Library `New` view with:
  - `Grid`/`List` toggle
  - column visibility filter in `List`
  - multi-select + bulk `Remove from New`
  - selection only in `List` mode
  - `Shift+click` range selection on checkboxes
- Added route: `POST /library/new/mark-reviewed` (bulk set `needs_review=false`).
- Improved library performance:
  - `/api/library` fast-path (no full rescan on every page load)
  - optional `rescan=true` / `include_file_info=true`
  - SQL optimizations in `list_library_json()`
  - additional DB indexes for scale
- Restored `/api/home` full dataset output:
  - `continue_reading`
  - `shorts_unread`
  - `novels_unread`
  - `shorts_read`
  - `novels_read`
- Explicitly filtered series books out of Home sections.
- Corrected Home read ordering: `shorts_read` and `novels_read` now show oldest first (`ORDER BY MAX(read_at) ASC`).
- Restored Statistics page by returning the full `/api/stats` payload required for charts, favorites, top books, and reading history.
- Improved backup implementation:
  - Dropbox token stored encrypted in DB
  - Dropbox backup root configurable via web UI and stored encrypted in DB
  - versioned snapshots + object-store deduplication in Dropbox (`library_snapshots` / `library_objects`)
  - configurable snapshot retention (`snapshots to keep`) via backup settings
  - object pruning based on retained snapshots
  - scheduled backup (enable + interval in hours)
  - backup runs as a background process, so page navigation does not block execution
  - stale running state recovery after restart/crash (old running logs marked interrupted/error)
  - dry-run support in the new flow
- Updated Docker image with `postgresql-client` for `pg_dump`.
- Multiple test builds pushed to `gitea.oskamp.info/ivooskamp/novela:dev`.

## 2026-03-25 (9)
- Fixed backup progress not visible immediately after clicking Run Live Backup: sidebar `loadBackupProgress()` is now called directly from the backup page after a successful start
- Fixed backup status timestamp showing wrong relative time (e.g. "1h ago" for a recent backup): server UTC timestamps without timezone suffix are now correctly interpreted as UTC in the browser
- Fixed backup status bar using browser-default purple link color: now uses `var(--accent)` orange consistent with the rest of the UI
- Added password manager ignore attributes to App Key and App Secret fields: `data-1p-ignore` (1Password), `data-lpignore="true"` (LastPass), `data-form-type="other"` (1Password v8)

## 2026-03-25 (8)
- Added Dropbox OAuth2 refresh token support (no redirect URI needed):
  - New endpoints: `POST /api/backup/oauth/prepare` (generates auth URL) and `POST /api/backup/oauth/exchange` (exchanges code for refresh token)
  - `_dbx()` now prefers refresh token mode (app key + secret + refresh token) with automatic renewal; falls back to legacy access token for backwards compatibility
  - App key and secret stored encrypted in `credentials` table (`dropbox_app_key`, `dropbox_app_secret`)
  - `DELETE /api/backup/credentials` now also cleans up `dropbox_app_key` and `dropbox_app_secret`
  - Backup page updated with two-step OAuth flow UI (app key/secret → auth URL → paste code)
  - Settings status shows `• refresh token` or `• legacy token` indicator

## 2026-03-25 (7)
- Added backup progress tracking: `GET /api/backup/progress` returns `{running, done, total, phase}`; sidebar shows live file count (e.g. `312 / 652 · uploading`) polling every 3 s while running

## 2026-03-25 (6)
- Fixed `GET /api/backup/health` missing `schedule_enabled` and `schedule_interval_hours` in response (values were read but not returned; Health card showed "disabled" even when schedule was on)
- Added backup status indicator in sidebar above Rescan library button: coloured dot + status text (OK/running/failed/never), links to `/backup`, auto-refreshes every 8 s while running, shows time-ago for last successful backup

## 2026-03-25 (5)
- One-time path migration: all 571 existing library files moved to correct format-prefixed structure (`epub/`, `pdf/`, `comics/`) and all DB references updated (`migrate_paths.py --execute`)
- Empty old publisher root directories pruned after migration
- Recovery script (`recover_decock049.py`) written; confirmed file was added after last Dropbox backup so not recoverable — to be re-imported manually

## 2026-03-25 (4)
- Added {publisher} directory to PDF and CBR/CBZ paths: `pdf/{publisher}/{author}/{title}.pdf`, `comics/{publisher}/{author}/{title}.cbr/cbz` — consistent with EPUB structure

## 2026-03-25 (3)
- Fixed CBZ extension in import: `common.make_rel_path` always generated `.cbr` for CBZ files; now accepts `ext` parameter; `library.py` passes actual suffix so CBZ files land at `comics/{author}/{title}.cbz`
- Added missing `GET /download/{filename}` endpoint (referenced in book.html but was 404)
- TECHNICAL.md fully rewritten: added File Storage Paths section, complete endpoint lists for all routers including settings.py, corrected path documentation

## 2026-03-25 (2)
- Fixed PDF metadata editing (PATCH /library/book):
  - `_sync_epub_metadata` is now only called for `.epub` files; PDFs update DB only
  - `_make_rel_path` now includes the format prefix matching import: EPUB → `epub/{publisher}/{author}/…`, PDF → `pdf/{author}/{title}.pdf`, CBR/CBZ → `comics/{author}/{title}{ext}`; previously files were moved outside their format directory on metadata save
- Fixed PDF reader (infinite loading screen):
  - `reader_page` now passes `format` (epub/pdf/cbr/cbz) to `reader.html`
  - Added `GET /api/pdf/info/{filename}` endpoint returning `{"page_count": N}`
  - `reader.html` branches on `FORMAT`: PDFs render page images from `/library/pdf/{filename}?page=N`, EPUB flow unchanged
  - PDF progress tracked per page; keyboard and button navigation work identically to EPUB
- `Edit EPUB` button in Book Detail hidden for non-EPUB files

## 2026-03-23
- Added `All books` Grid/List toggle in Library:
  - same columns as `New` view (Publisher, Author, Series, Volume, Title, Has cover, Updated, Genres, Sub-genres, Tags, Status)
  - column visibility filter in `List` mode
  - no selection checkboxes or bulk actions
  - view mode and column visibility persisted separately in `localStorage` (`novela.all.viewMode`, `novela.all.visibleColumns`)
- Added 1–5 star rating for books:
  - stored in the database (`rating SMALLINT DEFAULT 0`)
  - written to EPUB OPF as `<meta name="novela:rating" content="N"/>` on rating change
  - written to CBZ `ComicInfo.xml` as `<NovelaRating>N</NovelaRating>` on rating change
  - CBR and PDF are DB-only (file format constraints)
  - rating is recovered from file metadata during rescan/DB rebuild (`upsert_book` preserves file rating over default 0)
  - stars displayed under the cover (outside `.book-info`) in all grid views (Library, Home)
  - stars in grid cards are display-only (no click) to prevent accidental taps while scrolling
  - stars in Book Detail are interactive (larger, 1.1rem), clicking same star removes rating
  - star color: amber (`#c8a03a`) for filled, `rgba(200, 160, 58, 0.25)` for unfilled — consistent across Library, Home, and Book Detail
- Added text colour setting to reader hamburger menu:
  - 5 warm-tone presets from bright (`#e8e2d9`) to dim (`#938d86`)
  - active preset shown with accent-coloured ring
  - choice persisted in `localStorage` (`reader-text-colour`) and restored on next open
- Increased spacing between hamburger button and back link in reader header (`margin-left: 1rem`) to prevent accidental taps
- Removed `Cover Missing` auto-tag:
  - tag is no longer added on import, rescan, or grabber download
  - `ensure_cover_missing_tag()` removed from `common.py`, `library.py`, and `grabber.py`
  - startup migration removes all existing `Cover Missing` tags from the database
- Fixed Tags/Genres/Sub-Genres not saving in book edit panel on desktop:
  - `,` (comma) now acts as a confirmation key alongside Enter in `PillInput`
  - `flush()` added to `PillInput`: any text still in the input field is auto-confirmed when Save is clicked
- Fixed tag/genre search and tag-pill navigation being broken:
  - `renderGenreView` was filtering on `b.genres` (non-existent field); now uses `bookGenres()`, `bookSubgenres()`, `bookPlainTags()`
  - `renderSearchResults` had the same bug; search now covers title, author, genres, sub-genres, and tags