novela/docs/changelog-develop.md

Develop Changelog

2026-03-29 (10)

• Duplicates: fixed updateCounts crashing with a TypeError (g.books.length → g.length); the crash prevented renderGrid from running, so the duplicates view never rendered and the counter was stale

2026-03-29 (9)

• Duplicates: volume-aware duplicate detection — a book is only a duplicate when title + author + volume all match
  • _duplicateGroups in library.js (Duplicates view): key now includes series_index when > 0, so different volumes of the same series are no longer grouped as duplicates
  • preload in grabber.py (Grabber): when the scraper returns a series_index_hint, the DB lookup only flags a match when title + author + volume all match; falls back to title + author when no volume is known — same logic as the bulk importer

2026-03-29 (8)

• Shared code: eliminated all remaining duplication across templates and JS files
  • CSS custom properties: extracted single :root { } block into static/theme.css; removed duplicate inline :root from all 15 templates and from library.css, book.css — editor.css keeps editor-specific vars (--danger, --header-h, --panel-w), reader.html keeps page-specific vars (--header-h, --footer-h, --content-w), backup.html keeps (--ok, --warn, --err)
  • Cover helpers: moved strHash, COVER_PALETTES, makePlaceholderCover, wrapText, truncate from library.js and book.js into books.js; removed from home.html and following.html
  • HTML escape: esc() added to books.js; removed from library.js, editor.js, and all 8 templates that defined it inline
  • SSE/EventSource: extracted shared connectConversionStream(job_id) and addLog() into new static/conversion.js; both index.html and grabber.html now call the shared function (removed ~70 duplicate lines)

2026-03-29 (7)

• Search: extracted shared book helpers and search logic into static/books.js
  • _filenameBase, bookTitle, bookAuthor, tagValuesByType, bookGenres, bookSubgenres, bookPlainTags, filterBooks, setupSearchInput moved from library.js and home.html to a single shared file
  • Both pages now use identical search behaviour: Enter to search, × to clear
  • Both pages now search across title, author, publisher, genre, sub-genre, and tag

2026-03-29 (6)

• Search: changed from search-as-you-type (250 ms debounce) to Enter-to-search on both Library and Home — prevents iPad keyboard from locking up on large collections

2026-03-29 (5)

• Accent colour updated to match logo orange (#ffa20e); secondary accent updated to #ffb840
  • Applied across all CSS files, templates, and inline styles

2026-03-29 (4)

• Branding: added logo, favicon, and Apple touch icon to all pages
  • Static assets: logo.png (sidebar), favicon.ico (16×16), favicon-32.png (32×32), favicon-256.png (256×256), apple-touch-icon.png (180×180 with #0f0e0c background)
  • Favicon <link> tags added to all 15 templates
  • Sidebar: image logo (logo.png) placed next to the existing "Novela" wordmark using flexbox
  • apple-touch-icon.png uses dark #0f0e0c background — renders as a native-looking iOS home screen icon

2026-03-29 (3)

• Dockerfile: replaced unrar-free with proprietary unrar (RARLAB v6.2.6) from Debian non-free — fixes "Failed to read enough data" errors on RAR archives using newer compression methods

2026-03-29 (2)

• CBR reader: detect archive format via magic bytes instead of file extension — .cbr files that are actually ZIP or 7-zip archives now open correctly; added py7zr dependency for 7-zip support

2026-03-29 (1)

• Bulk Import: duplicate check now volume-aware — books with the same title+author but a different volume number (e.g. recoloured reprints) are no longer flagged as duplicates; volume is included in the API call and matched against series_index in the DB

2026-03-28 (11)

• Bulk Import: duplicate detection against existing library
  • New POST /api/bulk-check-duplicates endpoint: case-insensitive title+author+volume match, single SQL query with OR conditions for all pairs
  • Duplicate rows highlighted in red in the preview table; a skip checkbox appears per duplicate row
  • Stats bar shows "X duplicates · Skip all · Import all" action buttons
  • Duplicate rows are skipped by default; user can toggle individual rows or use bulk actions
  • startImport() filters out skipped rows before batching; skipped files appear in the result summary as "Duplicate – skipped"
  • If all rows are skipped (nothing to import), result shown immediately without sending any request

2026-03-28 (10)

• After "Remove from New" succeeds, the library is now reloaded from the server (loadLibrary()) so the New list updates immediately without a manual refresh

2026-03-28 (9)

• Bulk delete is now batched: new POST /library/bulk-delete endpoint accepts a JSON list of filenames, deletes files and removes DB rows in one query per batch; JS sends 20 files per batch with a progress bar in the confirmation dialog

2026-03-28 (8)

• Disk usage warning in sidebar: GET /api/disk returns partition usage for the library directory; sidebar polls every 60 s and shows a warning bar (amber ≥ 85% or < 2 GB free, red ≥ 95% or < 500 MB free) above the backup status bar

2026-03-28 (7)

• Fixed mark-reviewed JS: UI (allBooks update + renderGrid) now only runs after confirmed server success; catch no longer fires a false error after a successful response where renderGrid throws

2026-03-28 (6)

• Bulk Import: rewrote pattern editor from token-pills to free-text %placeholder% syntax
  • Pattern input: free-text field; placeholders: %series% %volume% %title% %year% %month% %day% %author% %publisher% %ignore%
  • Colored chip row: click to insert at cursor position; drag onto input also supported
  • Colored live preview below the pattern input; regex-based parser replaces delimiter+token logic
  • Shared metadata now wins over filename-parsed values (previously filename won)
  • All UI text translated from Dutch to English

2026-03-28 (5)

• Status badge (top-right cover) and want-to-read star (top-left cover) now use dark fill rgba(15,14,12,0.82) + box-shadow: 0 0 0 2px #0f0e0c ring — always readable regardless of cover colour

2026-03-28 (4)

• Added Temporary Hold status; renamed Hiatus → Long-Term Hold
  • Startup migration migrate_rename_hiatus() converts existing DB records automatically
  • Status colours: Complete=green, Ongoing=blue, Temporary Hold=amber, Long-Term Hold=orange
  • statusBadgeHtml() helper in library.js replaces three identical inline badge blocks
  • Grabber: Temporary-Hold (gayauthors.org) now maps to Temporary Hold; Long-Term Hold passes through unchanged
  • Status dropdowns updated in Book Detail and Bulk Import

2026-03-28 (3)

• Added Bulk Import page (/bulk-import) for batch importing CBR/CBZ/EPUB/PDF files with filename-based metadata parsing
  • Configurable token pattern: Serie, Volume, Titel, Jaar, Auteur, Uitgever, Negeer (in any order)
  • Configurable delimiter (default -); year token right-anchored when last so title absorbs overflow segments
  • Live test-parse box: type any filename and see how it parses before selecting files
  • Shared metadata card: author, publisher, status, genres, tags applied to all files (overridden by pattern tokens or manual edits)
  • Preview table: all parsed fields editable via contenteditable cells; warning indicator for rows with fewer segments than tokens; sorted by filename
  • Batch upload in groups of 5 with progress bar (N / total files processed)
  • Result summary with list of skipped files and reasons
  • New routers/bulk_import.py: GET /bulk-import, POST /library/bulk-import
  • Sidebar link under Tools between Book Builder and Credentials

This file tracks changes on the develop line. changelog.md can later be used for release summaries.

2026-03-28 (2)

• Performance: library page now loads instantly for large collections (1000+ books)
  • IntersectionObserver defers both cover image loading and placeholder canvas drawing until cards enter the viewport — eliminates hundreds of upfront canvas ops that blocked the initial render
  • ETag caching on /library/list: server returns 304 Not Modified when nothing changed, client skips JSON parse and re-download
  • Single DOM pass in renderBooksGrid, renderDuplicatesView, renderSeriesDetail: canvas and img set up via card.querySelector immediately after innerHTML, removing a second iteration with document.getElementById per card
  • book_tags joined via json_agg in the main list_library_json() query, eliminating a separate SELECT * FROM book_tags query and Python merge loop
  • loadLibrary now shows an error message instead of staying stuck on "Loading…" when the fetch or render fails

2026-03-28 (1)

• Added Following page (/following): track external author URLs outside Library and Tools
  • New authors table: name (unique), url, created_at, updated_at
  • New routers/following.py: GET /following page, GET /api/following (all authors + URL + book count + last added), POST /api/following/{name} (set/clear URL)
  • Sidebar: new Following section between Library and Tools; counter shows number of followed authors
  • Following page: two tabs — Following (authors with URL) and All Authors; inline URL editing with Enter/Escape keyboard support; Visit button opens external URL in a new tab; author name links to library author view
• Added Incomplete view to Library (#incomplete): shows all non-archived books where publication_status ≠ Complete; sidebar counter included; entry placed after New in the Library section

2026-03-27 (1)

• Convert page: duplicate warning shown after loading metadata when a book with the same title+author already exists in the library; warning includes a link to the existing book; user can still proceed with conversion
• Library: added Duplicates section to sidebar (between Rated and Statistics); counter shows total number of books that are part of a duplicate group (same title+author, case-insensitive); Duplicates view groups books by title+author with a subheading per group
• Fixed Duplicates view not loading covers: card renderer now uses the same canvas + two-pass img/makePlaceholderCover pattern as renderBooksGrid

2026-03-26 (2)

• Fixed Book Builder page showing white background: library.css added to builder.html to load :root CSS variables and dark body background; all CSS variable references in builder.css aligned with library theme names (--text, --surface, --surface2, --text-dim, --border, --accent, --sidebar)

2026-03-26 (1)

• Added Book Builder: create EPUB books manually from scratch via a WYSIWYG editor
  • New builder_drafts table (id UUID, title, author, publisher, source_url, chapters JSONB)
  • build_epub() in epub.py: builds a standards-compliant EPUB 2.0 ZIP from title/author/publisher/chapters; embeds inline CSS and break.png if present
  • normalize_wysiwyg_html() in xhtml.py: converts contenteditable HTML to EPUB-safe XHTML; handles scene-breaks, <blockquote class="author-note">, inline formatting, and <hr> → break image
  • routers/builder.py: draft CRUD, chapter CRUD (GET/POST/PUT/DELETE), normalize preview endpoint, publish endpoint (normalizes all chapters → builds EPUB → writes to library/epub/ → upserts to DB → deletes draft → redirects to book detail)
  • templates/builder.html + static/builder.js + static/builder.css: index page (new draft form + draft list) and editor (chapter panel, contenteditable pane, toolbar with bold/italic/underline/blockquote/author-note/scene-break/normalize, autosave every 30 s, Ctrl+S, publish button)
  • Book Builder link added to sidebar Tools section (between Convert and Credentials)
  • .author-note blockquote style added to static/epub-style.css

2026-03-25 (20)

• Added Rated section to library sidebar: shows all non-archived books with rating > 0, sorted by rating descending then title alphabetically; badge displays total count; navigable via #rated URL hash

2026-03-25 (19)

• Fixed series index 0 not displaying in series slot view, grid cards, list volume column, and book detail:
  • Series slot labels now show #0 for all slots when the series has at least one positively-indexed sibling (consistent label height across all cards)
  • Grid card and list volume column use the same cross-book heuristic (indexedSeriesSet) to show [0] where appropriate
  • Book detail page queries whether any sibling in the same series has series_index > 0 (series_is_indexed) and shows [0] accordingly

2026-03-25 (18)

• Added autocomplete for Author, Publisher, and Series in the book edit panel: typing shows a filtered dropdown of existing values from the library; selecting a value fills the field (same dropdown styling as genres/tags)
• Status field now defaults to "Complete" when opening the edit panel for a book that has no status set yet

2026-03-25 (17)

• Fixed CBR reader showing only first page: cbr_page_count was missing from the cbr import in reader.py; /api/cbr/info/ was returning an error, causing page_count to fall back to 1 and the Next button to remain disabled

2026-03-25 (16)

• Fixed CBR/CBZ reader stuck on loading: added /api/cbr/info/{filename} endpoint returning {page_count}, and added a CBR/CBZ branch in reader.html that mirrors the PDF paged reader (page images served via /library/cbr/{filename}/{page})

2026-03-25 (15)

• Added series volume 0 and letter suffix support (e.g. "21a", "21b"):
  • New series_suffix VARCHAR(10) column added to library table via migrate_series_suffix
  • series_index lower bound changed from 1 to 0 throughout (index 0 = special/prequel edition)
  • Volume field in the book editor changed from type="number" to type="text" — accepts "0", "1", "21a", etc.
  • Server parses the combined volume string into series_index (INTEGER) + series_suffix (VARCHAR) via parse_volume_str
  • File naming includes suffix: 021a - Title.epub
  • novela:series_suffix meta tag written to/read from EPUB OPF for persistence across rescans
  • Series detail view: books sorted by (series_index, series_suffix); slot labels show "21a"; index 0 shown as slot when any sibling has index > 0
  • Grid cards, list view Volume column, author/publisher sort all include suffix

2026-03-25 (14)

• Added multi-select and bulk delete to All books List view:
  • Checkbox column added to the list table (List mode only; Grid mode unchanged)
  • Select all / Clear all / Clear selection controls in the controls bar
  • Shift+click range selection on checkboxes
  • Delete selected button (red, disabled when nothing is selected) triggers a confirmation dialog; confirmed deletions remove files from disk and database, then reload the library

2026-03-25 (13)

• Fixed EPUB cover replacement reverting to original after upload:
  • GET /library/cover/{filename} for EPUBs now checks the DB cover cache first (populated on upload) before falling back to extracting from the EPUB file; the cache is also warmed on cache-miss so subsequent requests are fast.
  • add_cover_to_epub fully rewritten: locates the OPF via META-INF/container.xml, finds the existing cover image in the OPF manifest, removes it from the ZIP, and writes the new cover in the same directory — works for any EPUB directory structure (OEBPS/, EPUB/, etc.).

2026-03-25 (12)

• Fixed chapter images not loading in EPUBs that use a non-standard root directory (e.g. EPUB/ instead of OEBPS/): image paths are now passed as full ZIP paths instead of stripping the root segment, and the image endpoint no longer hardcodes an OEBPS/ prefix. Case-insensitive fallback retained for EPUBs with mismatched image folder casing.

2026-03-25 (11)

• Fixed reader progress bar jumping to 100% immediately when navigating to the second chapter in short EPUBs (2 chapters): changed progress formula denominator from total - 1 to total so 100% is only reached after fully scrolling through the last chapter. Same fix applied to PDF page progress.

2026-03-25 (10)

• Added bookmark feature:
  • bookmarks table in DB (migrate_create_bookmarks): filename, chapter_index, scroll_frac, chapter_title, note, created_at
  • New API endpoints: GET/POST /library/bookmarks/{filename}, PATCH/DELETE /library/bookmarks/{id}, GET /api/bookmarks
  • Bookmark button in reader header (orange, bookmark icon): opens modal with optional note field
  • Bookmark modal closes on Escape, backdrop click, or Cancel
  • Reader supports ?bm_ch=N&bm_scroll=F URL params to jump directly to bookmarked position (overrides saved progress)
  • Library sidebar: Bookmarks section in Library nav with live count badge
  • Library #bookmarks view: card list showing cover, book title, author, chapter, note, date, Go-to and Delete buttons

2026-03-22

• Added blueprint/technical documentation structure in docs/.
• Completed router split and bootstrap structure (main.py, routers, migrations, DB pool).
• Expanded media support to EPUB/PDF/CBR/CBZ in import and scan flows.
• Expanded Home UI with:
  • import dropzone for EPUB/PDF/CBR/CBZ
  • search functionality
  • alignment matching Library (search top-right, dropzone below)
• Updated Library import texts and drag/drop filtering for multi-format support.
• Expanded Library New view with:
  • Grid/List toggle
  • column visibility filter in List
  • multi-select + bulk Remove from New
  • selection only in List mode
  • Shift+click range selection on checkboxes
• Added route: POST /library/new/mark-reviewed (bulk set needs_review=false).
• Improved library performance:
  • /api/library fast-path (no full rescan on every page load)
  • optional rescan=true / include_file_info=true
  • SQL optimizations in list_library_json()
  • additional DB indexes for scale
• Restored /api/home full dataset output:
  • continue_reading
  • shorts_unread
  • novels_unread
  • shorts_read
  • novels_read
• Explicitly filtered series books out of Home sections.
• Corrected Home read ordering: shorts_read and novels_read now show oldest first (ORDER BY MAX(read_at) ASC).
• Restored Statistics page by returning the full /api/stats payload required for charts, favorites, top books, and reading history.
• Improved backup implementation:
  • Dropbox token stored encrypted in DB
  • Dropbox backup root configurable via web UI and stored encrypted in DB
  • versioned snapshots + object-store deduplication in Dropbox (library_snapshots / library_objects)
  • configurable snapshot retention (snapshots to keep) via backup settings
  • object pruning based on retained snapshots
  • scheduled backup (enable + interval in hours)
  • backup runs as a background process, so page navigation does not block execution
  • stale running state recovery after restart/crash (old running logs marked interrupted/error)
  • dry-run support in the new flow
• Updated Docker image with postgresql-client for pg_dump.
• Multiple test builds pushed to gitea.oskamp.info/ivooskamp/novela:dev.

2026-03-25 (9)

• Fixed backup progress not visible immediately after clicking Run Live Backup: sidebar loadBackupProgress() is now called directly from the backup page after a successful start
• Fixed backup status timestamp showing wrong relative time (e.g. "1h ago" for a recent backup): server UTC timestamps without timezone suffix are now correctly interpreted as UTC in the browser
• Fixed backup status bar using browser-default purple link color: now uses var(--accent) orange consistent with the rest of the UI
• Added password manager ignore attributes to App Key and App Secret fields: data-1p-ignore (1Password), data-lpignore="true" (LastPass), data-form-type="other" (1Password v8)

2026-03-25 (8)

• Added Dropbox OAuth2 refresh token support (no redirect URI needed):
  • New endpoints: POST /api/backup/oauth/prepare (generates auth URL) and POST /api/backup/oauth/exchange (exchanges code for refresh token)
  • _dbx() now prefers refresh token mode (app key + secret + refresh token) with automatic renewal; falls back to legacy access token for backwards compatibility
  • App key and secret stored encrypted in credentials table (dropbox_app_key, dropbox_app_secret)
  • DELETE /api/backup/credentials now also cleans up dropbox_app_key and dropbox_app_secret
  • Backup page updated with two-step OAuth flow UI (app key/secret → auth URL → paste code)
  • Settings status shows • refresh token or • legacy token indicator

2026-03-25 (7)

• Added backup progress tracking: GET /api/backup/progress returns {running, done, total, phase}; sidebar shows live file count (e.g. 312 / 652 · uploading) polling every 3 s while running

2026-03-25 (6)

• Fixed GET /api/backup/health missing schedule_enabled and schedule_interval_hours in response (values were read but not returned; Health card showed "disabled" even when schedule was on)
• Added backup status indicator in sidebar above Rescan library button: coloured dot + status text (OK/running/failed/never), links to /backup, auto-refreshes every 8 s while running, shows time-ago for last successful backup

2026-03-25 (5)

• One-time path migration: all 571 existing library files moved to correct format-prefixed structure (epub/, pdf/, comics/) and all DB references updated (migrate_paths.py --execute)
• Empty old publisher root directories pruned after migration
• Recovery script (recover_decock049.py) written; confirmed file was added after last Dropbox backup so not recoverable — to be re-imported manually

2026-03-25 (4)

• Added {publisher} directory to PDF and CBR/CBZ paths: pdf/{publisher}/{author}/{title}.pdf, comics/{publisher}/{author}/{title}.cbr/cbz — consistent with EPUB structure

2026-03-25 (3)

• Fixed CBZ extension in import: common.make_rel_path always generated .cbr for CBZ files; now accepts ext parameter; library.py passes actual suffix so CBZ files land at comics/{author}/{title}.cbz
• Added missing GET /download/{filename} endpoint (referenced in book.html but was 404)
• TECHNICAL.md fully rewritten: added File Storage Paths section, complete endpoint lists for all routers including settings.py, corrected path documentation

2026-03-25 (2)

• Fixed PDF metadata editing (PATCH /library/book):
  • _sync_epub_metadata is now only called for .epub files; PDFs update DB only
  • _make_rel_path now includes the format prefix matching import: EPUB → epub/{publisher}/{author}/…, PDF → pdf/{author}/{title}.pdf, CBR/CBZ → comics/{author}/{title}{ext}; previously files were moved outside their format directory on metadata save
• Fixed PDF reader (infinite loading screen):
  • reader_page now passes format (epub/pdf/cbr/cbz) to reader.html
  • Added GET /api/pdf/info/{filename} endpoint returning {"page_count": N}
  • reader.html branches on FORMAT: PDFs render page images from /library/pdf/{filename}?page=N, EPUB flow unchanged
  • PDF progress tracked per page; keyboard and button navigation work identically to EPUB
• Edit EPUB button in Book Detail hidden for non-EPUB files

2026-03-23

• Added All books Grid/List toggle in Library:
  • same columns as New view (Publisher, Author, Series, Volume, Title, Has cover, Updated, Genres, Sub-genres, Tags, Status)
  • column visibility filter in List mode
  • no selection checkboxes or bulk actions
  • view mode and column visibility persisted separately in localStorage (novela.all.viewMode, novela.all.visibleColumns)
• Added 1–5 star rating for books:
  • stored in the database (rating SMALLINT DEFAULT 0)
  • written to EPUB OPF as <meta name="novela:rating" content="N"/> on rating change
  • written to CBZ ComicInfo.xml as <NovelaRating>N</NovelaRating> on rating change
  • CBR and PDF are DB-only (file format constraints)
  • rating is recovered from file metadata during rescan/DB rebuild (upsert_book preserves file rating over default 0)
  • stars displayed under the cover (outside .book-info) in all grid views (Library, Home)
  • stars in grid cards are display-only (no click) to prevent accidental taps while scrolling
  • stars in Book Detail are interactive (larger, 1.1rem), clicking same star removes rating
  • star color: amber (#c8a03a) for filled, rgba(200, 160, 58, 0.25) for unfilled — consistent across Library, Home, and Book Detail
• Added text colour setting to reader hamburger menu:
  • 5 warm-tone presets from bright (#e8e2d9) to dim (#938d86)
  • active preset shown with accent-coloured ring
  • choice persisted in localStorage (reader-text-colour) and restored on next open
• Increased spacing between hamburger button and back link in reader header (margin-left: 1rem) to prevent accidental taps
• Removed Cover Missing auto-tag:
  • tag is no longer added on import, rescan, or grabber download
  • ensure_cover_missing_tag() removed from common.py, library.py, and grabber.py
  • startup migration removes all existing Cover Missing tags from the database
• Fixed Tags/Genres/Sub-Genres not saving in book edit panel on desktop:
  • , (comma) now acts as a confirmation key alongside Enter in PillInput
  • flush() added to PillInput: any text still in the input field is auto-confirmed when Save is clicked
• Fixed tag/genre search and tag-pill navigation being broken:
  • renderGenreView was filtering on b.genres (non-existent field); now uses bookGenres(), bookSubgenres(), bookPlainTags()
  • renderSearchResults had the same bug; search now covers title, author, genres, sub-genres, and tags