ivooskamp/novela
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
6df47a8161
novela/docs/changelog-develop.md

72 KiB
Raw Blame History

Develop Changelog

2026-06-01 — Backup/Restore: database-stored books are now restorable

Fixed

  • Database-stored books (storage_type='db', synthetic db/... filenames) could not be restored through the Backup → Restore option. The restore UI only listed and restored files from the on-disk library object store, but db-books have no file on disk — their content lives entirely in PostgreSQL (book_chapters + library row + book_tags + library_cover_cache). They were captured only in the full pg_dump, which the UI offered no way to restore. As a result db-books never appeared in the restore list and were effectively unrecoverable per-book.

Added

  • Per-book db restore. The backup writer now serializes each database book to JSON (library row, all chapters, tags, and the cached cover) and stores it in the same content-addressed Dropbox object store used for file books, referenced from the snapshot with a "storage": "db" marker. Such books now appear in the Restore table (format DB) and can be restored individually; restore re-inserts the library row, chapters, tags and cover into PostgreSQL. Inline chapter images are unaffected — they already live on disk under library/images/ and are backed up as ordinary files.
    • routers/backup.py: new _db_book_filenames, _serialize_db_book, _restore_db_book, _entry_storage helpers; db-books serialized during _run_backup_internal; _download_and_restore branches on storage type; /api/backup/snapshots/{name}/files now reports storage and computes exists_locally for db-books from the library table.
    • Note: db-books only appear in snapshots created after this change. Older backups still contain them only in the pg_dump — recover those via Full Database Restore below.
  • Full Database Restore. New Backup-page card and endpoints to restore the entire PostgreSQL database from any Dropbox pg_dump. This recovers everything (all db-books, reading progress, tags, settings) from existing backups too. It is destructive: the public schema is dropped and recreated before the dump is applied, so any plain dump (with or without --clean) restores cleanly. Guarded behind a double confirmation in the UI.
    • routers/backup.py: _psql_base_args, _run_pg_restore, _list_pg_dump_paths; endpoints GET /api/backup/postgres/dumps and POST /api/backup/postgres/restore; health endpoint now also reports psql_available/psql_path.
    • templates/backup.html: db-books shown with DB format and "in library / not in library" status in the restore table; new "Full Database Restore" card with dump selector, double-confirm, and psql health row.

2026-06-01 — Reader: Sepia reading theme

Added

  • The reader now offers a Sepia theme alongside the existing Dark theme, for easier long-form reading (warm paper background with dark brown text instead of light-on-black). New Theme toggle (Dark / Sepia) at the top of the reading settings drawer.
    • static/theme.css: added a :root[data-theme="sepia"] palette overriding --bg, --surface, --surface2, --border, --accent, --accent2, --text, --text-dim, --text-faint, --success, --warning, --error. Also added a --shadow-ring variable (light hairline in dark, dark hairline in sepia) so swatch rings read correctly in both themes.
    • templates/reader.html: theme toggle buttons + .theme-btn styling; an inline head script applies the saved theme before paint to avoid a flash; the Text colour row now has two theme-specific swatch sets (light tints for dark, dark brown tints for sepia), toggled by visibility.
    • JS: applyTheme()/loadTheme() set the data-theme attribute and persist to localStorage (reader-theme). Text colour is now stored per theme (reader-text-colour-dark / reader-text-colour-sepia); the old single reader-text-colour key is migrated into the dark slot. init() calls loadTheme() (which loads the active theme's text colour).
    • Replaced two hard-coded rgba(255,255,255,...) values in the reader CSS (swatch ring, series-button border) with theme variables so they don't show as white halos in sepia.

2026-05-31 — Build version in the sidebar

Added

  • The sidebar now shows the running build version at the bottom (e.g. v0.2.11 for releases, v0.2.11.3 for dev builds), linking to the changelog page.
    • New containers/novela/version.py exposes display_version(). The semantic release version stays the single source in changelog.py (CHANGELOG[0]["version"]); a BUILD segment is appended for dev builds.
    • shared_templates.py registers app_version as a Jinja global; _sidebar.html renders it as a .sidebar-version link styled in sidebar.css.
    • main.py adds a /api/version endpoint returning {"version": display_version()}.

2026-05-31 — Find & Replace: scope option

Added

  • Editor Find & Replace: new Current chapter only checkbox in the modal options. When checked, the search/replace runs against the currently open chapter instead of every chapter in the book.
    • templates/editor.html: added the rp-current checkbox next to Regex/Case sensitive; modal title changed from "Find & Replace — all chapters" to "Find & Replace" since scope is now selectable.
    • static/editor.js (replaceInAllChapters()): reads rp-current; builds a targets list of either just currentCh() or all chapters, and iterates that. Shows an error ("No chapter open.") if Current-chapter-only is selected with no open chapter. Result message reads "… in current chapter" for the single-chapter case.
    • Default is unchecked, so the existing all-chapters behaviour is preserved.

Released as v0.2.12 on 2026-06-01

2026-05-10

  • Reader: subheading/chat styling now also wins when the wrapper contains block elements with their own color rule (e.g. <div class="subheading"><p>…</p></div>)
    • Previously the <p>/<h*> rules in reader.html (#chapter-content p { color: var(--text); }, etc.) had the same specificity as .subheading and applied more directly to the inner element, so the parent's color was effectively overridden — the wrapped paragraph stayed in the default text color
    • CSS now applies the rule to the wrapper and all descendants: #chapter-content .subheading, #chapter-content .subheading * { … } (and the same for .chat)
    • Reason: bug reported after v0.2.10 — .chat worked because it wraps plain inline text inside a <p>, but .subheading wrapping a <p> lost its color

Released as v0.2.11 on 2026-05-10

Released as v0.2.10 on 2026-05-10

2026-05-09 (2)

  • Editor / Reader: subheading (S) and chat (C) styling now also applies when the wrapper is a <div> instead of a <span>
    • wrapSpan() in editor.js falls back to <div class="…"> whenever the selection contains a block element (<h1..6>, <p>, <div>, etc.) so the resulting HTML stays valid; the previous CSS in reader.html (#chapter-content span.subheading / span.chat) only matched the <span> form, so anything wrapped around or inside a heading silently lost its color/weight
    • CSS selectors are now class-only (#chapter-content .subheading / .chat), matching both span and div wrappers

2026-05-09

  • Reader: progress is now monotonic across devices — saved position only advances, never rewinds
    • POST /library/progress/{filename} in routers/reader.py parses the incoming cfi as (chapterIndex, scrollFrac) and the currently stored row the same way, then skips the write when new_pos <= cur_pos
    • Same cfi format is used for EPUB, PDF and CBR/CBZ, so one tuple comparison covers all readers
    • Explicit Read/Unread actions still clear the row (mark-read / mark-unread delete from reading_progress), so users can deliberately reset and start over
    • Reason: when reading the same book on device A (chapter 12) and then continuing on device B (chapter 15), opening device A again previously sent its stale chapter-12 cfi back to the server and overwrote the further progress; now the older position is ignored

Released as v0.2.9 on 2026-05-09

2026-04-22 (5)

  • Grabber: newly converted books now appear in the New view again
    • Both the DB-storage and file-EPUB branches in routers/grabber.py set needs_review: True on upsert_book (was False); the New view filters on needs_review == True, so previously grabbed books never showed up there
    • Matches the behavior of disk-scanned imports (library.py already sets needs_review = True for freshly discovered files)

Released as v0.2.8 on 2026-04-22

2026-04-22 (4)

  • Break detection: runs of consecutive break images are now collapsed to a single break
    • New helper collapse_consecutive_breaks() in xhtml.py matches 2+ consecutive <center><img src=".../break.png" .../></center> occurrences (with optional whitespace in between) and replaces them with one
    • Applied in normalize_wysiwyg_html() (editor save path) and in routers/grabber.py on both the preview converted_xhtml and the per-chapter content_html produced during scraping
  • Docs: docs/TECHNICAL.md updated to cover previously missing changes — POST /api/edit/intro/{filename} and the title field on file-EPUB chapter save; FlareSolverr sidecar and BaseScraper.close(); AwesomeDudeScraper uses FlareSolverr; make_epub(include_intro=…) and epub_utils.build_book_info_body_html; grabber DB flow stores Book Info as chapter 0; "Book Info" h1-strip skip in reader; new env vars (FLARESOLVERR_URL, FLARESOLVERR_TIMEOUT_MS, NOVELA_PORT, ADMINER_PORT); collapse_consecutive_breaks() helper

Released as v0.2.7 on 2026-04-22

2026-04-22 (3)

  • Scrapers: Cloudflare "Just a moment…" challenges are now solved via a new FlareSolverr sidecar service so books on protected sites (awesomedude.org) can be scraped again
    • New service flaresolverr in stack/stack.yml (image ghcr.io/flaresolverr/flaresolverr:latest, internal-only, on novela-net); novela service gains FLARESOLVERR_URL=http://flaresolverr:8191/v1 and a depends_on: flaresolverr
    • New helpers in scrapers/base.py: flaresolverr_get(url, timeout_ms=None, session=None) posts cmd: request.get and returns a SimpleNamespace(text, url) (drop-in for httpx.Response attributes); flaresolverr_session_create() and flaresolverr_session_destroy(sid) manage browser sessions so Cloudflare cookies stay warm across per-chapter requests (first page solves the challenge, all further chapters reuse the same browser — much faster)
    • Configurable via FLARESOLVERR_URL and FLARESOLVERR_TIMEOUT_MS env vars (defaults: service DNS name and 60000 ms)
    • BaseScraper gained an async close() method (default no-op) for releasing scraper-scoped resources; scrapers/awesomedude.py creates a FlareSolverr session in fetch_book_info, reuses it in all fetch_chapter calls, and destroys it in close()
    • routers/grabber.py now wraps all scraper usage in try/finally: await scraper.close() so sessions are released even on errors
    • Stack uses ${NOVELA_PORT} and ${ADMINER_PORT} (defined in stack/novela.env as 8099 / 8098) so host-port values don't diverge between environments

Released as v0.2.6 on 2026-04-22

2026-04-22 (2)

  • Editor: chapter titles are now editable for file-EPUB books as well (DB books already supported this)
    • Frontend (editor.js): the chapter-title-input is always shown (the read-only header-chapter label is hidden for both storage types), title changes mark the chapter dirty, and the title is sent in both saveChapter and saveAllChapters
    • Backend (routers/editor.py): POST /api/edit/chapter/{index}/{filename} for file EPUBs now accepts title alongside content; if the title changed it calls new helper _update_epub_navpoint_title(path, href, new_title) which locates the matching NCX navPoint by content src and rewrites its <text> via rewrite_epub_entries

2026-04-22

  • Book Info page: new "Info page" button in the editor toolbar generates a gayauthors-style info page and inserts it as the first chapter
    • New endpoint POST /api/edit/intro/{filename} builds the page from stored metadata (author, genres, sub-genres, tags, description, source, updated) and prepends it
    • DB books: shifts existing chapter_index values up by one and inserts "Book Info" at index 0 (two-step negation to avoid unique-constraint violations)
    • File EPUBs: writes a new intro_<hex>.xhtml via make_intro_xhtml, adds a manifest item, places the itemref at the start of the spine, and inserts a navPoint at the top of the NCX with renumbered playOrder
    • Empty metadata fields are skipped
    • Option A: no duplicate detection — clicking the button on a book that already has one will add a second page
  • Grabber convert: DB-storage conversions now persist the Book Info page as a real stored chapter at index 0 (so it shows up in the editor and reader); EPUB-storage conversions continue to produce intro.xhtml via make_epub as before
  • DB → EPUB export (reader.py): no longer synthesises intro.xhtml (make_epub(..., include_intro=False)) — the stored chapter 0 is the single source of truth
    • Legacy DB books converted before this change have no stored info page; their exports will lack an intro page until the button is used
  • make_epub gained an include_intro: bool = True parameter; when false, the intro.xhtml file, its manifest item, its spine itemref, and its NCX navPoint are all omitted (remaining playOrder values start at 1)
  • Shared helper: epub_utils.build_book_info_body_html(title, author, info) — returns the inner-body HTML fragment used for DB storage, starting with <h1>{title}</h1>; skips empty fields and separates description and source/updated blocks with <hr/>
  • DB-storage info page: chapter title is "Book Info"; to preserve the leading <h1>{book title}</h1> in its body, the leading-h-tag stripping in reader.py (get_chapter_html and the DB→EPUB export) is now skipped when title == "Book Info"

Released as v0.2.5 on 2026-04-22

2026-04-21 (2)

  • Backup: separate "scanned" vs "uploaded" counters in backup_log and UI
    • New migration backup_log_scanned_files adds scanned_files INTEGER column to backup_log
    • _run_backup_internal now returns (scanned_files, uploaded_count, uploaded_size)
    • _finish_backup_log(...) accepts scanned_files=; _run_backup_job passes the value through
    • /api/backup/status and /api/backup/history return uploaded_files (old files_count) and scanned_files
    • backup.html: Latest Status now shows separate "Scanned" and "Uploaded" rows; History table renamed "Files" column to two columns "Scanned" / "Uploaded" (colspan of loading/empty states updated to 8)
  • Backup page: live phase indicator under the Run buttons while a backup is running
    • New #run-progress status line is filled by pollRunProgress() which polls /api/backup/progress every 3 s
    • Phase labels spelled out: scanning library, uploading library objects, uploading snapshot, uploading pg_dump (may take minutes) — so users understand why the counter sits at N / N during the final phases
    • Polling starts on page load (via refreshAll()) and is re-kicked when Run Live/Dry is clicked; it auto-stops when progress reports running=false

Released as v0.2.4 on 2026-04-21

2026-04-21

  • Backup: fix Dropbox Read timed out. (read timeout=120) error on large uploads
    • dropbox.Dropbox(...) timeout raised from 120 to 300 in routers/backup.py (_dbx(), both refresh-token and legacy-token branches)
    • _DROPBOX_UPLOAD_CHUNK reduced from 100 * 1024 * 1024 (100 MB) to 16 * 1024 * 1024 (16 MB)
    • _DROPBOX_UPLOAD_THRESHOLD lowered to match (16 * 1024 * 1024) so the session upload path is used earlier
    • Net effect: each chunk PUT finishes well within the socket timeout, and a stalled connection gets 5 minutes instead of 2 before erroring

Released as v0.2.3 on 2026-04-21

2026-04-16 (2)

  • Editor: four inline formatting buttons added to the chapter editor toolbar
    • S (subheading) — wraps selection in <span class="subheading"> (red, bold in reader)
    • C (chat) — wraps selection in <span class="chat"> (orange in reader)
    • →| (indent) — wraps selection in <p style="padding-left: 40px;"> (or <div> when the selection contains block elements)
    • [ ] (comment) — wraps selection in <div class="novela-comment"> (blue left border + subtle background in reader)
    • Without a selection: the tag is inserted at the cursor with the cursor positioned inside
    • All four buttons disabled when no chapter is loaded; enabled state mirrors the existing Break button
    • Wrap logic auto-detects block content in the selection: wrapSpan and insertIndent use a <div> wrapper instead of <span>/<p> when the selection contains <p>, <div>, <h*> etc. to keep the HTML valid
    • Reader CSS extended: span.subheading, span.chat, p[style*="padding-left"], and .novela-comment are styled in reader.html

2026-04-16 (1)

  • Startup: migration logging zichtbaar in Docker logs
    • logging.basicConfig(level=logging.INFO) toegevoegd aan main.py
    • migrations.py logt per migratie of deze overgeslagen of uitgevoerd wordt (met duur in ms)
    • Samenvattingsregel bij afsluiting: "all already applied" of "N executed"

Released as v0.2.1 on 2026-04-16

2026-04-15 (4)

  • Scraper: fix encoding for Codey's World
    • Pages are decoded with cp1252 (Windows-1252) instead of relying on r.text or html.parser auto-detection via iso-8859-1
    • cp1252 correctly maps the 0x80–0x9F byte range: …, ', ', ", ", –, — etc. now render correctly instead of producing replacement characters
  • XHTML: normalize non-breaking spaces globally
    • \xa0 (HTML &nbsp;) is now replaced with a regular space before HTML-escaping in both element_to_xhtml and normalize_wysiwyg_html
    • Consecutive spaces resulting from this substitution are collapsed to a single space
    • Applies to all scrapers

2026-04-15 (3)

  • Editor: chapter add/delete deferred until Save is clicked
    • Adding or deleting a chapter no longer triggers an immediate API call
    • All structural changes are collected in memory and applied (in the correct order) when the Save button is pressed
    • Deletes are applied in reverse server-index order to avoid index shifting errors; new chapters are appended afterwards
    • Fix: saving flag is set at the start of a delete operation to prevent a concurrent saveChapter from flushing pending changes during the async gap in loadChapter
  • Refactor: fix unclosed file handles in epub.py
    • make_epub and write_epub_file now use Path.read_text() / Path.read_bytes() instead of bare open() calls
  • Refactor: eliminate temp file in pdf_cover_thumb
    • Cover thumbnail is now generated fully in-memory via Image.frombytes() from the PyMuPDF pixmap — removes the race condition when multiple requests hit the same PDF simultaneously
  • Refactor: harden security.py
    • Hardcoded fallback encryption key removed; raises RuntimeError with a clear message when neither NOVELA_MASTER_KEY nor POSTGRES_PASSWORD is set
    • Fernet instance cached with @functools.lru_cache(maxsize=1) — key derivation runs only once per process
  • Add GET /health endpoint
    • Runs SELECT 1 against the database and returns {"ok": true/false}
  • Performance: cap in-memory job dicts at 50 entries
    • JOBS in grabber.py and BACKUP_TASKS/BACKUP_PROGRESS in backup.py are trimmed (oldest-first) whenever a new entry is added
  • Performance: improve ETag accuracy for the library API
    • ETag now includes MAX(updated_at) from reading_progress and MAX(id) from book_tags, so tag changes and progress updates correctly invalidate the client cache
  • Performance: cache CBR/CBZ page list
    • cbr_page_list is cached per (path, mtime) via lru_cache(maxsize=64) — avoids opening the archive twice per page request
  • Refactor: normalize transaction handling in builder.py
    • All conn.commit() calls replaced with with conn: context manager, consistent with the rest of the codebase

Released as v0.2.0 on 2026-04-15

2026-04-15 (2)

  • Editor: fix chapter add failing with UniqueViolation on DB-stored books
    • PostgreSQL checks the unique constraint on (filename, chapter_index) per row during UPDATE, so incrementing consecutive indices in a single statement (e.g. 1→2 while 2 exists) raised a UniqueViolation
    • Fixed by using a two-step approach: first shift affected rows to temporary negative values, then to their final positive values
  • Refactor: eliminate duplicated EPUB helper functions across reader.py, editor.py, common.py
    • New epub_utils.py with shared find_opf_path, norm_href, epub_spine, make_new_chapter_xhtml, rewrite_epub_entries
    • Fixes the double-escaped \\\\s* regex in the old _epub_spine OPF path lookup (was silently falling back to directory scan)
    • rewrite_epub_entries combines crash-safe .tmp.epub write with ZIP_STORED for the mimetype entry (EPUB spec requirement)
    • All private _epub_spine, _norm_href, _find_opf_path, _make_new_chapter_xhtml, _rewrite_epub_entries copies removed from reader.py, editor.py, common.py
  • Migrations: run each migration only once via schema_migrations tracking table
    • Eliminates heavy rebuild_chapter_tsv_with_title UPDATE running on every container restart
    • Reduces startup from 37 separate DB connections to 1

2026-04-15 (1)

  • Reader: font size control in reading settings
    • New "Font size" slider (80–150%, default 105%) in the settings drawer, between "Content width" and "Text colour"
    • Applies via CSS custom property --reader-font-size on #chapter-content
    • Persisted per-device in localStorage as reader-font-size — iPad and desktop each remember their own preference

2026-04-13 (1)

  • Edit metadata: comma-separated tag input fix
    • PillInput._add in book.js now splits the incoming value on commas before adding — each trimmed, non-empty, non-duplicate part is pushed individually
    • Applies to genres, subgenres and plain tags; pasting e.g. Fiction, Thriller, Adventure adds 3 separate pills instead of 1

Released as v0.1.11 on 2026-04-13

2026-04-12 (2)

  • Series navigation in the reader
    • New GET /api/series-nav/{filename} endpoint: returns {prev, next} with {filename, title, index, suffix} for adjacent books in the same series, ordered by series_index ASC, series_suffix ASC; returns {prev: null, next: null} for books without a series
    • Reader header now shows prev/next series buttons (skip-to-start/skip-to-end icons); hidden for books without a series, visible once loadSeriesNav() resolves
    • Hovering a series button shows a tooltip: #<index><suffix> <title> (e.g. #4 Batman: Year One)
    • markRead() in the reader redirects to /library/read/{next.filename} when a next volume exists, so reading continues without leaving the reader; falls back to the book detail page when the series is complete or the book has no series

2026-04-12 (1)

  • Comics: series_volume support for annual series (issue numbers restart each year)
    • New series_volume VARCHAR(20) column on library (migration migrate_series_volume); default ''
    • Stored in EPUB OPF as <meta name="novela:series_volume" content="…"/>; read back by scan_epub
    • upsert_book inserts/preserves series_volume with the same COALESCE strategy as series_suffix
    • list_library_json ORDER BY now: publisher → author → series → series_volume → series_index → title
    • PATCH /library/book/{filename} reads series_volume from request body; persists to DB and OPF for both file-based and DB-stored books
    • Book detail page: displays (year) after series name when series_volume is set (e.g. Donald Duck (1982) [15]); edit panel has a new "Year/Volume" input field
    • book.js: "Auto" button next to the Title field generates Series (Year/Vol) #Number from the current series fields
  • Bulk importer: series_volume support
    • New %series_volume% placeholder token (orange) for filename pattern parsing
    • New "Year/Vol." shared metadata field (applies to all files; overridden by per-row value or pattern)
    • Preview table has a new "Yr/Vol" column
    • New "Auto-generate titles from series info" checkbox: when enabled, rows without a parsed title get title Series (Year/Vol) #Number
    • Skip checkbox now always visible for every row (previously only shown when duplicates were detected); any file can be manually excluded before import; skipped rows are dimmed; stats bar shows "X skipped"
    • POST /api/bulk-check-duplicates extended: now also checks (series, series_index, author) as fallback — detects duplicates even when the title format has changed; items must include series field
  • Library front-end: sorting within a series now respects series_volume before series_index
    • groupBySeries, renderAuthorDetail, renderPublisherDetail all sort by series_volume → series_index → series_suffix
    • getSeriesSlots refactored: when any book in the series has series_volume set, gap-detection runs per volume (year) independently — prevents #5 (1982) and #5 (1983) from colliding in the same slot
    • Slot index label shows (year) #index for annual series (e.g. (1982) #5); unchanged for regular series

Released as v0.1.10 on 2026-04-12

2026-04-08 (13)

  • Library: archive series in one action
    • "Archive series" / "Unarchive series" button in the series detail view (#series/{name})
    • New endpoint POST /library/archive-series — sets archived for all books in the series via a single SQL UPDATE; body: {"series": "…", "archive": true|false}; returns {ok, archived, count}
    • Button label reflects current state: "Archive series" when any book is active, "Unarchive series" when all are archived
    • After the call, allBooks is updated in place and sidebar counters are recalculated without a full page reload

2026-04-08 (12)

  • TedLouisScraper: title extraction fix
    • <h2 class="story-page-title"> also contains a "Back" button (<a class="btn">) and the author byline (<span class="story-author-by-line">)
    • Fix: only direct NavigableString children of the h2 are used as the title, so link and span text is skipped

2026-04-08 (11)

  • New scraper: TedLouisScraper (scrapers/tedlouis.py) for tedlouis.com
    • Matches all tedlouis.com URLs; no login required
    • Entry point is the story index page (e.g. ?t=CWYSqpOryu2rQmT1); raises an error when a chapter URL is used as entry point
    • Title from direct text nodes of <h2 class="story-page-title">
    • Author from <span class="story-author-by-line"> <a>
    • Status from <span class="story-status-text"> (strips "Status: " prefix)
    • Updated date from <span class="story-last-updated"> → YYYY-MM-DD
    • Chapters from all <ul class="story-index-list"> elements (three columns); relative ?t=TOKEN links resolved to absolute URLs
    • fetch_chapter(): content from <div id="chapter">; removes story title, chapter title and copyright blocks

2026-04-08 (10)

  • Settings: break image upload added
    • New "Break image" card on the settings page: upload a PNG/JPG/WebP as the scene break image
    • Stored in the imagestore (sha256-addressed) and overwrites static/break.png so EPUB export uses the same image
    • app_settings extended with break_image_sha256 and break_image_ext columns (migration migrate_app_settings_break_image)
    • New endpoints: POST /api/app-settings/break-image; GET /api/app-settings now also returns break_image_url
    • Preview of the current break image visible on the settings page

2026-04-08 (9)

  • Grabber: break image fix for DB-stored books
    • Break images (<center><img src="../Images/break.png">) contain a relative EPUB path that does not exist in the DB context
    • Fix: storage_mode determined earlier in _run_scrape; for DB mode /static/break.png is passed as break_img_path to element_to_xhtml, for EPUB mode ../Images/break.png is kept
    • Additional fix: when a break image is detected in the image loop, the parent (<center>) is replaced with <hr> instead of decomposing the <img>; this allows element_to_xhtml to correctly detect the break after the image loop

2026-04-08 (8)

  • New scraper: IomfatsScraper (scrapers/iomfats.py) for iomfats.org
    • Matches all iomfats.org URLs; no login required
    • Entry point is a chapter URL (e.g. .../grasshopper/justhitsend-part1/justhitsend01.html); automatically navigates to the author page to fetch all metadata and chapters
    • Author page as entry point: raises a user-visible error message
    • Detects two structures on the author page:
      • Single story: outer <h3> = book title; chapters directly in <ul>
      • Multi-part series: outer <h3> = series name; nested <li><h3> = book title per part; chapters in sub-<ul>
    • Series index from folder name: *-part{N} or *-{N} → series_index_hint
    • Publication status from <p><small>[...]</small></p> after book title
    • fetch_chapter(): content via <div id="content">; removes <h2>/<h3> headings, chapternav divs, footer elements

2026-04-08 (7)

  • New scraper: CodeysWorldScraper (scrapers/codeysworld.py) for codeysworld.org
    • Matches all codeysworld.org URLs; no login required
    • Title from <h1>; author from <h2> ("by …"); fallback to URL slug (/{author}/{category}/filename)
    • Category from URL path → tag (e.g. "Remembrances")
    • Single-file stories (no chapter links): the page itself is the only chapter
    • Multi-chapter: links to .htm/.html files in the same directory (audio/image links skipped)
    • fetch_chapter(): removes all <h1>/<h2> headings, navigation links ("Back to …"), audio links (.mp3), mailto links
  • Nifty scraper: category/subcategory moved from genres/subgenres to tags

2026-04-08 (6)

  • NiftyNewScraper: chapter extraction made more robust
    • fetch_chapter() now first tries the standard chapter HTML ({url}) and reads <article><p> directly
    • Fallback added for Next payloads with escaped paragraphs (\u003cp...\u003c/p) via _extract_escaped_html_paragraphs()
    • Last fallback remains ?_rsc=1: first _parse_rsc_paragraphs(), then escaped-paragraph fallback again
  • Nifty (classic + new): standard boilerplate no longer visible in reader, but preserved in chapter
    • Lead/tail detection added for common blocks (e.g. NOTICE This is a work of fiction…, If you enjoy this story…, donate text)
    • Detected intro/closing boilerplate removed from visible paragraphs
    • Removed text stored as invisible HTML comment:
      • <!-- NIFTY_HIDDEN_LEAD: ... -->
      • <!-- NIFTY_HIDDEN_TAIL: ... -->
    • Detection also works when text contains inline HTML (e.g. <a> or <svg> in donate links)

2026-04-08 (5)

  • NiftyNewScraper: chapter content fix
    • new.nifty.org uses Next.js RSC — chapter content is not present in the static HTML but in the RSC payload (React component tree)
    • fetch_chapter() now fetches {url}?_rsc=1 instead of the regular HTML page
    • RSC parser added: _parse_rsc_paragraphs(), _rsc_find_paragraphs(), _rsc_text() — parse the RSC stream line by line (format: {hex_id}:{json}), recursively search for ["$","p",null,{...}] nodes and extract text from children

2026-04-08 (4)

  • NiftyNewScraper added (scrapers/nifty_new.py) for new.nifty.org
    • Matches all new.nifty.org URLs; no login required
    • _to_index_url(): strips trailing /N (chapter number) so both index and chapter URLs can be used as entry point
    • fetch_book_info(): title from <h1>, author from <strong itemprop="name"> in author link, dates from <time itemprop="datePublished/dateModified">, tags from all <ul aria-label="Tags"> containers (category links and generated tags, deduplicated), description from <meta name="description">, chapter list via <a> links matching /stories/{slug}/N (RSC stream regex as fallback)
    • fetch_chapter(): title from JSON-LD @type: "Chapter", content from <article>; no email header stripping, no line joining, no boilerplate detection
    • NiftyScraper.matches() updated: excludes new.nifty.org ("nifty.org" in url and "new.nifty.org" not in url)
    • NiftyNewScraper registered before NiftyScraper in scrapers/__init__.py

2026-04-08 (3)

  • Settings: develop mode added
    • New app_settings table (single row, id = 1) with develop_mode boolean; created via migrate_create_app_settings()
    • shared_templates.py: shared Jinja2Templates instance for all routers; develop_mode() registered as Jinja2 global so all templates can access it without explicit context injection
    • All 11 routers now import templates from shared_templates instead of each creating their own instance
    • New endpoints in routers/settings.py: GET /api/app-settings and PATCH /api/app-settings
    • Diagonal DEVELOP banner in the top-left of every page (CSS in static/sidebar.css, HTML in templates/_sidebar.html); only visible when develop mode is enabled
    • All 17 HTML templates: <title> shows Novela Develop — … when develop mode is active
    • Settings page: new card with checkbox toggle; page reloads after saving so banner and title apply immediately

2026-04-08 (2)

  • Nifty scraper: fix _strip_email_headers now tolerates blank lines between header fields
    • Some Nifty pages place Subject: after a blank line (Date:\nFrom:\n\nSubject:\n) — the previous implementation stopped at the first blank line causing Subject: to appear as a paragraph in chapter text
    • Fix: on a blank line, look ahead to check whether a header field still follows; if so, skip the blank line(s) and continue stripping

2026-04-08 (1)

  • Nifty scraper added (scrapers/nifty.py)
    • Matches all nifty.org URLs; no login required
    • fetch_book_info: accepts index or chapter URL; normalises to index; title from URL slug, author and publication date from email headers of chapter 1, updated_date from email headers of the last chapter; genres/subgenres from URL path (/nifty/{category}/{subcategory}/…)
    • Boilerplate detection: compares first paragraphs of chapters 1 and 2 after header stripping; matching paragraphs are skipped in all chapters (preamble_count per chapter dict)
    • fetch_chapter: retrieves <pre> content (fallback: body); subject header stored as invisible HTML comment <!-- Subject: … --> at the top of chapter content; email headers stripped; hard line breaks within paragraphs joined into a single line; scene breaks (***, ---, etc.) → <hr/>
    • Date parsing via email.utils.parsedate → YYYY-MM-DD
  • xhtml.py: element_to_xhtml now handles bs4.Comment objects as XML comments (<!-- … -->); -- in the body is sanitised to - - (illegal in XML comments)

2026-04-06 (3)

  • Book detail: rating moved from clickable stars to dropdown in the Edit metadata panel
    • Stars on the detail page are now purely visual (no longer clickable)
    • New <select id="ed-rating"> field in the edit panel, directly below Status
    • openEdit() populates the select with the current rating; saveEdit() calls POST /library/rating/… if the value has changed
    • rateBook() function removed from book.js
    • Overview pages unchanged

2026-04-06 (2)

  • Library: cover upload now also supported for DB-stored books
    • POST /library/cover/{filename} previously returned an error for DB books ("File not found") because no physical file exists
    • Fix: DB books are now detected via is_db_filename; cover is stored directly in library_cover_cache and has_cover = TRUE set in the library table — add_cover_to_epub is not called

2026-04-06 (1)

  • Search: filter for unread novels / unread shorts
    • New filter parameter on GET /api/search?q=…&filter=all|unread_novels|unread_shorts (default: all)
    • unread_novels: restricts to books with no reading sessions/progress and no Shorts tag
    • unread_shorts: restricts to books with no reading sessions/progress and with a Shorts tag
    • UI: second toggle row (All / Unread novels / Unread shorts) below the Phrase/All words toggle
    • Filter persisted in URL (?filter=…) and restored on page load

2026-04-05 (4)

  • Backup: large files (> 148 MB) now uploaded via Dropbox upload session in 100 MB chunks
    • _dropbox_upload_bytes: files ≤ 148 MB go via files_upload (unchanged); larger files via upload_session_start → upload_session_append_v2 → upload_session_finish
    • Fixes ApiError: UploadError('payload_too_large')

2026-04-05 (3)

  • Filenames: spaces now replaced with underscores when saving new files
    • clean_segment (common.py) and _clean_segment (reader.py): \s+ → _ instead of space
    • Series separator changed from - to _-_ (applies to EPUB, CBR, CBZ and DB filenames)
    • Existing files are not renamed

2026-04-05 (2)

  • Search: removed result limit; added Phrase / All words mode toggle
    • LIMIT 30 removed — all matching chapters are returned
    • New mode parameter on GET /api/search?q=…&mode=phrase|words: phrase (default) requires words in order (phraseto_tsquery); words requires all words present in any order (plainto_tsquery)
    • Toggle in the UI (Phrase / All words) above the results; mode is included in the URL

2026-04-05 (1)

  • Export EPUB: double chapter titles fixed — same heading-stripping logic as the reader now applied before passing content to make_chapter_xhtml
  • Library: authors and publishers with only archived books now remain visible in the Authors and Publishers list views
    • renderAuthorsView and renderPublishersView switched from activeBooks() to allBooks — consistent with renderAuthorDetail and renderPublisherDetail

2026-04-04 (2)

  • Reader: fixed double chapter titles for books where the heading is wrapped in a <section> element (pandoc-style)
    • Previous regex only stripped <h1>–<h4> at the very start of content; pandoc-converted EPUBs wrap headings as <section …><h1>…</h1> — heading was not at position 0 so the regex didn't fire
    • Added a second re.sub pass that removes the first <h1>–<h4> found directly after an opening <section> or <div> tag, preserving the wrapper element
    • Same stripping applied in the DB→EPUB export (export_epub) before passing content to make_chapter_xhtml
  • Search: switched from plainto_tsquery to phraseto_tsquery in the FTS WHERE clause
    • plainto_tsquery ANDs all words but treats them as independent terms (any order, any distance) — multi-word queries like "4 years later" matched chapters where the words appeared far apart
    • phraseto_tsquery requires all words to appear in sequence; ts_rank and ts_headline still use plainto_tsquery for correct scoring and highlighting

2026-04-04 (1)

  • Reader: fixed double chapter titles in DB-stored books
    • Chapter endpoint (GET /library/chapter/{index}/{filename}) now strips all leading <h1>–<h4> tags from stored content before prepending its own <h2 class="chapter-title"> — books scraped before front-matter stripping was added to the scraper showed the title (and chapter heading) twice
  • Library: archived books now visible in author and publisher detail views, with an indicator badge on the cover
    • renderAuthorDetail and renderPublisherDetail reverted to use allBooks (including archived) so archived books remain accessible from the author/publisher pages
    • New .badge-archived overlay (bottom-left of cover): dark circle with archive icon, consistent with existing badge-status and read-pill overlays; added to all three card-rendering code paths

2026-04-03 (3)

  • DB chapter editor: Monaco-based editor now supports DB-stored books
    • GET /library/editor/{filename} handles db/… filenames; is_db flag passed to template
    • GET /api/edit/chapter/{index}/{filename} and POST …: DB branches query/update book_chapters directly; save calls upsert_chapter (updates content_tsv too)
    • POST /api/edit/chapter/add/{filename} and DELETE …: DB branches insert/delete with chapter_index shift via UPDATE … SET chapter_index = chapter_index ± 1
    • Title editing: header chapter-name replaced with a text input for DB books; pendingTitles map preserves unsaved titles across chapter switches (parallel to pendingContent); title-only dirty chapters correctly saved in Save All
    • insertBreak: scene-break image path is /static/break.png for DB books (vs ../Images/break.png for EPUB)
    • Fix: editor.focus() called after content load so Monaco receives keyboard focus immediately
    • Fix: header-chapter "Loading…" text suppressed for DB books where that element is hidden
    • book.html: "Edit chapters" button shown for storage_type = 'db' books
  • Search: chapter titles now included in FTS
    • upsert_chapter prepends title to the plain-text input for to_tsvector: title + " " + stripped_html
    • GET /api/search: added OR LOWER(bc.title) LIKE LOWER('%…%') fallback for chapters whose title matches but content doesn't
    • Startup migration migrate_rebuild_chapter_tsv_with_title() rebuilds existing content_tsv values to include titles
  • Grabber: added DB/EPUB storage toggle on the Convert page
    • UI toggle above Convert button ("Save as: DB | EPUB file"); storageMode JS variable sent in POST body
    • POST /convert: reads storage_mode from body; stored in job as 'db' or 'epub'
    • _run_scrape: EPUB path builds chapters via make_chapter_xhtml, calls make_epub, writes file, calls upsert_book(storage_type='file'); DB path unchanged
    • done SSE event includes storage_type; conversion.js updates the download button label/action accordingly
  • EPUB → DB conversion: fixed double chapter title
    • _epub_body_inner strips the first <h1>/<h2>/<h3> heading from each chapter body before storing; the editor prepends its own heading, so storing the EPUB heading too caused it to appear twice
    • Fix for NavigableString crash: getattr(child, "name", None) is None used instead of hasattr(child, "name") — NavigableString has name = None but no decompose() method
  • Sidebar: Search link styling fixed
    • Stray <li>Search</li> moved inside the Library <ul class="sidebar-nav"> (was outside, causing incorrect HTML structure)
    • sidebar.css: added a:visited { color: var(--text-dim) } and a.active:visited { color: var(--accent) } to prevent the browser's default purple visited color

2026-04-03 (2)

  • DB-stored books (Fase 4–6): EPUB→DB conversion, DB→EPUB export, full-text search
    • Fase 4 — EPUB-to-DB conversion: POST /api/library/convert-to-db/{filename} converts an existing on-disk EPUB to a DB-stored book; extracts chapters via _epub_body_inner (rewrites img src to imagestore URLs), migrates all child rows (book_tags, reading_progress, reading_sessions, bookmarks, library_cover_cache) to the new db/… filename using INSERT→UPDATE→DELETE to respect FK constraints, then deletes the EPUB file
    • Fase 5 — DB→EPUB export: GET /api/library/export-epub/{filename} builds and streams an EPUB from DB content; _rewrite_db_images_for_epub rewrites /library/db-images/… URLs back to OEBPS/Images/… paths, deduplicating by sha256; Content-Disposition: attachment response
    • Fase 6 — Full-text search: new routers/search.py with GET /search (page) and GET /api/search?q=… (FTS over book_chapters.content_tsv via plainto_tsquery('simple', q), ts_headline for snippets, ts_rank for ordering, LIMIT 30, excludes archived); new templates/search.html with highlight (<mark>), "Read here" link (?bm_ch=N&bm_scroll=0), and "Book detail" link; Search entry added to sidebar
    • book.html: DB books show "Export EPUB" instead of "Download"; "Edit EPUB" and "Convert to DB" buttons only shown for .epub files; delete modal text differs for DB vs file books
    • PATCH /library/book/{filename}: DB book branch added — skips file move, recomputes synthetic db/… filename via make_rel_path, applies same FK-safe rename pattern, updates book_chapters and bookmarks in addition to standard child tables

2026-04-03 (1)

  • DB-stored books (Fase 1–3): grabber now stores scraped books in PostgreSQL instead of EPUB files on disk
    • New book_chapters table: filename FK, chapter_index, title, content TEXT, content_tsv TSVECTOR; GIN index on content_tsv for future FTS
    • New book_images table: sha256 PK, ext, media_type, size_bytes; content-addressed imagestore at library/images/{sha2}/{sha256}{ext}
    • New storage_type VARCHAR(10) DEFAULT 'file' column on library; DB-stored books use 'db'
    • New utilities in common.py: is_db_filename, write_image_file, store_db_image, html_to_plain, upsert_chapter, ensure_unique_db_filename; make_rel_path now handles media_type="db" → synthetic db/{pub}/{auth}/... filename
    • upsert_book and list_library_json updated to include storage_type
    • Grabber: _run_scrape stores chapters in book_chapters, chapter images in imagestore (absolute /library/db-images/ URLs embedded in HTML), cover in library_cover_cache; no EPUB file written
    • New GET /library/db-images/{path:path} endpoint serves imagestore files
    • Reader: GET /library/chapters/ and GET /library/chapter/ have DB branches for storage_type='db' books (query book_chapters directly)
    • Reader page (/library/read/), book detail page, mark-read, and rating endpoints all handle DB filenames (no file existence required)
    • Cover endpoints (/library/cover/, /library/cover-cached/) serve DB books from library_cover_cache

2026-04-02 (1)

  • Added Restore functionality to the Backup page
    • New GET /api/backup/snapshots endpoint: lists available Dropbox snapshots (name + date parsed from filename, no downloads needed)
    • New GET /api/backup/snapshots/{snapshot_name}/files endpoint: loads a snapshot from Dropbox and returns all files with path, size, sha256, and whether the file currently exists locally
    • New POST /api/backup/restore endpoint: downloads file objects from Dropbox, writes to disk, and re-indexes via scan_media + upsert_book; returns per-file result with errors
    • New "Restore" card on the backup page: snapshot dropdown (auto-loaded on page open), file list with filter/search, per-file "Restore" button, multi-select + "Restore selected", on-disk indicator, inline status feedback
    • After restore, the file list refreshes to reflect updated on-disk state

2026-03-29 (10)

  • Duplicates: fixed updateCounts crashing with a TypeError (g.books.length → g.length); the crash prevented renderGrid from running, so the duplicates view never rendered and the counter was stale

2026-03-29 (9)

  • Duplicates: volume-aware duplicate detection — a book is only a duplicate when title + author + volume all match
    • _duplicateGroups in library.js (Duplicates view): key now includes series_index when > 0, so different volumes of the same series are no longer grouped as duplicates
    • preload in grabber.py (Grabber): when the scraper returns a series_index_hint, the DB lookup only flags a match when title + author + volume all match; falls back to title + author when no volume is known — same logic as the bulk importer

2026-03-29 (8)

  • Shared code: eliminated all remaining duplication across templates and JS files
    • CSS custom properties: extracted single :root { } block into static/theme.css; removed duplicate inline :root from all 15 templates and from library.css, book.css — editor.css keeps editor-specific vars (--danger, --header-h, --panel-w), reader.html keeps page-specific vars (--header-h, --footer-h, --content-w), backup.html keeps (--ok, --warn, --err)
    • Cover helpers: moved strHash, COVER_PALETTES, makePlaceholderCover, wrapText, truncate from library.js and book.js into books.js; removed from home.html and following.html
    • HTML escape: esc() added to books.js; removed from library.js, editor.js, and all 8 templates that defined it inline
    • SSE/EventSource: extracted shared connectConversionStream(job_id) and addLog() into new static/conversion.js; both index.html and grabber.html now call the shared function (removed ~70 duplicate lines)

2026-03-29 (7)

  • Search: extracted shared book helpers and search logic into static/books.js
    • _filenameBase, bookTitle, bookAuthor, tagValuesByType, bookGenres, bookSubgenres, bookPlainTags, filterBooks, setupSearchInput moved from library.js and home.html to a single shared file
    • Both pages now use identical search behaviour: Enter to search, × to clear
    • Both pages now search across title, author, publisher, genre, sub-genre, and tag

2026-03-29 (6)

  • Search: changed from search-as-you-type (250 ms debounce) to Enter-to-search on both Library and Home — prevents iPad keyboard from locking up on large collections

2026-03-29 (5)

  • Accent colour updated to match logo orange (#ffa20e); secondary accent updated to #ffb840
    • Applied across all CSS files, templates, and inline styles

2026-03-29 (4)

  • Branding: added logo, favicon, and Apple touch icon to all pages
    • Static assets: logo.png (sidebar), favicon.ico (16×16), favicon-32.png (32×32), favicon-256.png (256×256), apple-touch-icon.png (180×180 with #0f0e0c background)
    • Favicon <link> tags added to all 15 templates
    • Sidebar: image logo (logo.png) placed next to the existing "Novela" wordmark using flexbox
    • apple-touch-icon.png uses dark #0f0e0c background — renders as a native-looking iOS home screen icon

2026-03-29 (3)

  • Dockerfile: replaced unrar-free with proprietary unrar (RARLAB v6.2.6) from Debian non-free — fixes "Failed to read enough data" errors on RAR archives using newer compression methods

2026-03-29 (2)

  • CBR reader: detect archive format via magic bytes instead of file extension — .cbr files that are actually ZIP or 7-zip archives now open correctly; added py7zr dependency for 7-zip support

2026-03-29 (1)

  • Bulk Import: duplicate check now volume-aware — books with the same title+author but a different volume number (e.g. recoloured reprints) are no longer flagged as duplicates; volume is included in the API call and matched against series_index in the DB

2026-03-28 (11)

  • Bulk Import: duplicate detection against existing library
    • New POST /api/bulk-check-duplicates endpoint: case-insensitive title+author+volume match, single SQL query with OR conditions for all pairs
    • Duplicate rows highlighted in red in the preview table; a skip checkbox appears per duplicate row
    • Stats bar shows "X duplicates · Skip all · Import all" action buttons
    • Duplicate rows are skipped by default; user can toggle individual rows or use bulk actions
    • startImport() filters out skipped rows before batching; skipped files appear in the result summary as "Duplicate – skipped"
    • If all rows are skipped (nothing to import), result shown immediately without sending any request

2026-03-28 (10)

  • After "Remove from New" succeeds, the library is now reloaded from the server (loadLibrary()) so the New list updates immediately without a manual refresh

2026-03-28 (9)

  • Bulk delete is now batched: new POST /library/bulk-delete endpoint accepts a JSON list of filenames, deletes files and removes DB rows in one query per batch; JS sends 20 files per batch with a progress bar in the confirmation dialog

2026-03-28 (8)

  • Disk usage warning in sidebar: GET /api/disk returns partition usage for the library directory; sidebar polls every 60 s and shows a warning bar (amber ≥ 85% or < 2 GB free, red ≥ 95% or < 500 MB free) above the backup status bar

2026-03-28 (7)

  • Fixed mark-reviewed JS: UI (allBooks update + renderGrid) now only runs after confirmed server success; catch no longer fires a false error after a successful response where renderGrid throws

2026-03-28 (6)

  • Bulk Import: rewrote pattern editor from token-pills to free-text %placeholder% syntax
    • Pattern input: free-text field; placeholders: %series% %volume% %title% %year% %month% %day% %author% %publisher% %ignore%
    • Colored chip row: click to insert at cursor position; drag onto input also supported
    • Colored live preview below the pattern input; regex-based parser replaces delimiter+token logic
    • Shared metadata now wins over filename-parsed values (previously filename won)
    • All UI text translated from Dutch to English

2026-03-28 (5)

  • Status badge (top-right cover) and want-to-read star (top-left cover) now use dark fill rgba(15,14,12,0.82) + box-shadow: 0 0 0 2px #0f0e0c ring — always readable regardless of cover colour

2026-03-28 (4)

  • Added Temporary Hold status; renamed Hiatus → Long-Term Hold
    • Startup migration migrate_rename_hiatus() converts existing DB records automatically
    • Status colours: Complete=green, Ongoing=blue, Temporary Hold=amber, Long-Term Hold=orange
    • statusBadgeHtml() helper in library.js replaces three identical inline badge blocks
    • Grabber: Temporary-Hold (gayauthors.org) now maps to Temporary Hold; Long-Term Hold passes through unchanged
    • Status dropdowns updated in Book Detail and Bulk Import

2026-03-28 (3)

  • Added Bulk Import page (/bulk-import) for batch importing CBR/CBZ/EPUB/PDF files with filename-based metadata parsing
    • Configurable token pattern: Serie, Volume, Titel, Jaar, Auteur, Uitgever, Negeer (in any order)
    • Configurable delimiter (default -); year token right-anchored when last so title absorbs overflow segments
    • Live test-parse box: type any filename and see how it parses before selecting files
    • Shared metadata card: author, publisher, status, genres, tags applied to all files (overridden by pattern tokens or manual edits)
    • Preview table: all parsed fields editable via contenteditable cells; warning indicator for rows with fewer segments than tokens; sorted by filename
    • Batch upload in groups of 5 with progress bar (N / total files processed)
    • Result summary with list of skipped files and reasons
    • New routers/bulk_import.py: GET /bulk-import, POST /library/bulk-import
    • Sidebar link under Tools between Book Builder and Credentials

This file tracks changes on the develop line. changelog.md can later be used for release summaries.

2026-03-28 (2)

  • Performance: library page now loads instantly for large collections (1000+ books)
    • IntersectionObserver defers both cover image loading and placeholder canvas drawing until cards enter the viewport — eliminates hundreds of upfront canvas ops that blocked the initial render
    • ETag caching on /library/list: server returns 304 Not Modified when nothing changed, client skips JSON parse and re-download
    • Single DOM pass in renderBooksGrid, renderDuplicatesView, renderSeriesDetail: canvas and img set up via card.querySelector immediately after innerHTML, removing a second iteration with document.getElementById per card
    • book_tags joined via json_agg in the main list_library_json() query, eliminating a separate SELECT * FROM book_tags query and Python merge loop
    • loadLibrary now shows an error message instead of staying stuck on "Loading…" when the fetch or render fails

2026-03-28 (1)

  • Added Following page (/following): track external author URLs outside Library and Tools
    • New authors table: name (unique), url, created_at, updated_at
    • New routers/following.py: GET /following page, GET /api/following (all authors + URL + book count + last added), POST /api/following/{name} (set/clear URL)
    • Sidebar: new Following section between Library and Tools; counter shows number of followed authors
    • Following page: two tabs — Following (authors with URL) and All Authors; inline URL editing with Enter/Escape keyboard support; Visit button opens external URL in a new tab; author name links to library author view
  • Added Incomplete view to Library (#incomplete): shows all non-archived books where publication_status ≠ Complete; sidebar counter included; entry placed after New in the Library section

2026-03-27 (1)

  • Convert page: duplicate warning shown after loading metadata when a book with the same title+author already exists in the library; warning includes a link to the existing book; user can still proceed with conversion
  • Library: added Duplicates section to sidebar (between Rated and Statistics); counter shows total number of books that are part of a duplicate group (same title+author, case-insensitive); Duplicates view groups books by title+author with a subheading per group
  • Fixed Duplicates view not loading covers: card renderer now uses the same canvas + two-pass img/makePlaceholderCover pattern as renderBooksGrid

2026-03-26 (2)

  • Fixed Book Builder page showing white background: library.css added to builder.html to load :root CSS variables and dark body background; all CSS variable references in builder.css aligned with library theme names (--text, --surface, --surface2, --text-dim, --border, --accent, --sidebar)

2026-03-26 (1)

  • Added Book Builder: create EPUB books manually from scratch via a WYSIWYG editor
    • New builder_drafts table (id UUID, title, author, publisher, source_url, chapters JSONB)
    • build_epub() in epub.py: builds a standards-compliant EPUB 2.0 ZIP from title/author/publisher/chapters; embeds inline CSS and break.png if present
    • normalize_wysiwyg_html() in xhtml.py: converts contenteditable HTML to EPUB-safe XHTML; handles scene-breaks, <blockquote class="author-note">, inline formatting, and <hr> → break image
    • routers/builder.py: draft CRUD, chapter CRUD (GET/POST/PUT/DELETE), normalize preview endpoint, publish endpoint (normalizes all chapters → builds EPUB → writes to library/epub/ → upserts to DB → deletes draft → redirects to book detail)
    • templates/builder.html + static/builder.js + static/builder.css: index page (new draft form + draft list) and editor (chapter panel, contenteditable pane, toolbar with bold/italic/underline/blockquote/author-note/scene-break/normalize, autosave every 30 s, Ctrl+S, publish button)
    • Book Builder link added to sidebar Tools section (between Convert and Credentials)
    • .author-note blockquote style added to static/epub-style.css

2026-03-25 (20)

  • Added Rated section to library sidebar: shows all non-archived books with rating > 0, sorted by rating descending then title alphabetically; badge displays total count; navigable via #rated URL hash

2026-03-25 (19)

  • Fixed series index 0 not displaying in series slot view, grid cards, list volume column, and book detail:
    • Series slot labels now show #0 for all slots when the series has at least one positively-indexed sibling (consistent label height across all cards)
    • Grid card and list volume column use the same cross-book heuristic (indexedSeriesSet) to show [0] where appropriate
    • Book detail page queries whether any sibling in the same series has series_index > 0 (series_is_indexed) and shows [0] accordingly

2026-03-25 (18)

  • Added autocomplete for Author, Publisher, and Series in the book edit panel: typing shows a filtered dropdown of existing values from the library; selecting a value fills the field (same dropdown styling as genres/tags)
  • Status field now defaults to "Complete" when opening the edit panel for a book that has no status set yet

2026-03-25 (17)

  • Fixed CBR reader showing only first page: cbr_page_count was missing from the cbr import in reader.py; /api/cbr/info/ was returning an error, causing page_count to fall back to 1 and the Next button to remain disabled

2026-03-25 (16)

  • Fixed CBR/CBZ reader stuck on loading: added /api/cbr/info/{filename} endpoint returning {page_count}, and added a CBR/CBZ branch in reader.html that mirrors the PDF paged reader (page images served via /library/cbr/{filename}/{page})

2026-03-25 (15)

  • Added series volume 0 and letter suffix support (e.g. "21a", "21b"):
    • New series_suffix VARCHAR(10) column added to library table via migrate_series_suffix
    • series_index lower bound changed from 1 to 0 throughout (index 0 = special/prequel edition)
    • Volume field in the book editor changed from type="number" to type="text" — accepts "0", "1", "21a", etc.
    • Server parses the combined volume string into series_index (INTEGER) + series_suffix (VARCHAR) via parse_volume_str
    • File naming includes suffix: 021a - Title.epub
    • novela:series_suffix meta tag written to/read from EPUB OPF for persistence across rescans
    • Series detail view: books sorted by (series_index, series_suffix); slot labels show "21a"; index 0 shown as slot when any sibling has index > 0
    • Grid cards, list view Volume column, author/publisher sort all include suffix

2026-03-25 (14)

  • Added multi-select and bulk delete to All books List view:
    • Checkbox column added to the list table (List mode only; Grid mode unchanged)
    • Select all / Clear all / Clear selection controls in the controls bar
    • Shift+click range selection on checkboxes
    • Delete selected button (red, disabled when nothing is selected) triggers a confirmation dialog; confirmed deletions remove files from disk and database, then reload the library

2026-03-25 (13)

  • Fixed EPUB cover replacement reverting to original after upload:
    • GET /library/cover/{filename} for EPUBs now checks the DB cover cache first (populated on upload) before falling back to extracting from the EPUB file; the cache is also warmed on cache-miss so subsequent requests are fast.
    • add_cover_to_epub fully rewritten: locates the OPF via META-INF/container.xml, finds the existing cover image in the OPF manifest, removes it from the ZIP, and writes the new cover in the same directory — works for any EPUB directory structure (OEBPS/, EPUB/, etc.).

2026-03-25 (12)

  • Fixed chapter images not loading in EPUBs that use a non-standard root directory (e.g. EPUB/ instead of OEBPS/): image paths are now passed as full ZIP paths instead of stripping the root segment, and the image endpoint no longer hardcodes an OEBPS/ prefix. Case-insensitive fallback retained for EPUBs with mismatched image folder casing.

2026-03-25 (11)

  • Fixed reader progress bar jumping to 100% immediately when navigating to the second chapter in short EPUBs (2 chapters): changed progress formula denominator from total - 1 to total so 100% is only reached after fully scrolling through the last chapter. Same fix applied to PDF page progress.

2026-03-25 (10)

  • Added bookmark feature:
    • bookmarks table in DB (migrate_create_bookmarks): filename, chapter_index, scroll_frac, chapter_title, note, created_at
    • New API endpoints: GET/POST /library/bookmarks/{filename}, PATCH/DELETE /library/bookmarks/{id}, GET /api/bookmarks
    • Bookmark button in reader header (orange, bookmark icon): opens modal with optional note field
    • Bookmark modal closes on Escape, backdrop click, or Cancel
    • Reader supports ?bm_ch=N&bm_scroll=F URL params to jump directly to bookmarked position (overrides saved progress)
    • Library sidebar: Bookmarks section in Library nav with live count badge
    • Library #bookmarks view: card list showing cover, book title, author, chapter, note, date, Go-to and Delete buttons

2026-03-22

  • Added blueprint/technical documentation structure in docs/.
  • Completed router split and bootstrap structure (main.py, routers, migrations, DB pool).
  • Expanded media support to EPUB/PDF/CBR/CBZ in import and scan flows.
  • Expanded Home UI with:
    • import dropzone for EPUB/PDF/CBR/CBZ
    • search functionality
    • alignment matching Library (search top-right, dropzone below)
  • Updated Library import texts and drag/drop filtering for multi-format support.
  • Expanded Library New view with:
    • Grid/List toggle
    • column visibility filter in List
    • multi-select + bulk Remove from New
    • selection only in List mode
    • Shift+click range selection on checkboxes
  • Added route: POST /library/new/mark-reviewed (bulk set needs_review=false).
  • Improved library performance:
    • /api/library fast-path (no full rescan on every page load)
    • optional rescan=true / include_file_info=true
    • SQL optimizations in list_library_json()
    • additional DB indexes for scale
  • Restored /api/home full dataset output:
    • continue_reading
    • shorts_unread
    • novels_unread
    • shorts_read
    • novels_read
  • Explicitly filtered series books out of Home sections.
  • Corrected Home read ordering: shorts_read and novels_read now show oldest first (ORDER BY MAX(read_at) ASC).
  • Restored Statistics page by returning the full /api/stats payload required for charts, favorites, top books, and reading history.
  • Improved backup implementation:
    • Dropbox token stored encrypted in DB
    • Dropbox backup root configurable via web UI and stored encrypted in DB
    • versioned snapshots + object-store deduplication in Dropbox (library_snapshots / library_objects)
    • configurable snapshot retention (snapshots to keep) via backup settings
    • object pruning based on retained snapshots
    • scheduled backup (enable + interval in hours)
    • backup runs as a background process, so page navigation does not block execution
    • stale running state recovery after restart/crash (old running logs marked interrupted/error)
    • dry-run support in the new flow
  • Updated Docker image with postgresql-client for pg_dump.
  • Multiple test builds pushed to gitea.oskamp.info/ivooskamp/novela:dev.

2026-03-25 (9)

  • Fixed backup progress not visible immediately after clicking Run Live Backup: sidebar loadBackupProgress() is now called directly from the backup page after a successful start
  • Fixed backup status timestamp showing wrong relative time (e.g. "1h ago" for a recent backup): server UTC timestamps without timezone suffix are now correctly interpreted as UTC in the browser
  • Fixed backup status bar using browser-default purple link color: now uses var(--accent) orange consistent with the rest of the UI
  • Added password manager ignore attributes to App Key and App Secret fields: data-1p-ignore (1Password), data-lpignore="true" (LastPass), data-form-type="other" (1Password v8)

2026-03-25 (8)

  • Added Dropbox OAuth2 refresh token support (no redirect URI needed):
    • New endpoints: POST /api/backup/oauth/prepare (generates auth URL) and POST /api/backup/oauth/exchange (exchanges code for refresh token)
    • _dbx() now prefers refresh token mode (app key + secret + refresh token) with automatic renewal; falls back to legacy access token for backwards compatibility
    • App key and secret stored encrypted in credentials table (dropbox_app_key, dropbox_app_secret)
    • DELETE /api/backup/credentials now also cleans up dropbox_app_key and dropbox_app_secret
    • Backup page updated with two-step OAuth flow UI (app key/secret → auth URL → paste code)
    • Settings status shows • refresh token or • legacy token indicator

2026-03-25 (7)

  • Added backup progress tracking: GET /api/backup/progress returns {running, done, total, phase}; sidebar shows live file count (e.g. 312 / 652 · uploading) polling every 3 s while running

2026-03-25 (6)

  • Fixed GET /api/backup/health missing schedule_enabled and schedule_interval_hours in response (values were read but not returned; Health card showed "disabled" even when schedule was on)
  • Added backup status indicator in sidebar above Rescan library button: coloured dot + status text (OK/running/failed/never), links to /backup, auto-refreshes every 8 s while running, shows time-ago for last successful backup

2026-03-25 (5)

  • One-time path migration: all 571 existing library files moved to correct format-prefixed structure (epub/, pdf/, comics/) and all DB references updated (migrate_paths.py --execute)
  • Empty old publisher root directories pruned after migration
  • Recovery script (recover_decock049.py) written; confirmed file was added after last Dropbox backup so not recoverable — to be re-imported manually

2026-03-25 (4)

  • Added {publisher} directory to PDF and CBR/CBZ paths: pdf/{publisher}/{author}/{title}.pdf, comics/{publisher}/{author}/{title}.cbr/cbz — consistent with EPUB structure

2026-03-25 (3)

  • Fixed CBZ extension in import: common.make_rel_path always generated .cbr for CBZ files; now accepts ext parameter; library.py passes actual suffix so CBZ files land at comics/{author}/{title}.cbz
  • Added missing GET /download/{filename} endpoint (referenced in book.html but was 404)
  • TECHNICAL.md fully rewritten: added File Storage Paths section, complete endpoint lists for all routers including settings.py, corrected path documentation

2026-03-25 (2)

  • Fixed PDF metadata editing (PATCH /library/book):
    • _sync_epub_metadata is now only called for .epub files; PDFs update DB only
    • _make_rel_path now includes the format prefix matching import: EPUB → epub/{publisher}/{author}/…, PDF → pdf/{author}/{title}.pdf, CBR/CBZ → comics/{author}/{title}{ext}; previously files were moved outside their format directory on metadata save
  • Fixed PDF reader (infinite loading screen):
    • reader_page now passes format (epub/pdf/cbr/cbz) to reader.html
    • Added GET /api/pdf/info/{filename} endpoint returning {"page_count": N}
    • reader.html branches on FORMAT: PDFs render page images from /library/pdf/{filename}?page=N, EPUB flow unchanged
    • PDF progress tracked per page; keyboard and button navigation work identically to EPUB
  • Edit EPUB button in Book Detail hidden for non-EPUB files

2026-03-23

  • Added All books Grid/List toggle in Library:
    • same columns as New view (Publisher, Author, Series, Volume, Title, Has cover, Updated, Genres, Sub-genres, Tags, Status)
    • column visibility filter in List mode
    • no selection checkboxes or bulk actions
    • view mode and column visibility persisted separately in localStorage (novela.all.viewMode, novela.all.visibleColumns)
  • Added 1–5 star rating for books:
    • stored in the database (rating SMALLINT DEFAULT 0)
    • written to EPUB OPF as <meta name="novela:rating" content="N"/> on rating change
    • written to CBZ ComicInfo.xml as <NovelaRating>N</NovelaRating> on rating change
    • CBR and PDF are DB-only (file format constraints)
    • rating is recovered from file metadata during rescan/DB rebuild (upsert_book preserves file rating over default 0)
    • stars displayed under the cover (outside .book-info) in all grid views (Library, Home)
    • stars in grid cards are display-only (no click) to prevent accidental taps while scrolling
    • stars in Book Detail are interactive (larger, 1.1rem), clicking same star removes rating
    • star color: amber (#c8a03a) for filled, rgba(200, 160, 58, 0.25) for unfilled — consistent across Library, Home, and Book Detail
  • Added text colour setting to reader hamburger menu:
    • 5 warm-tone presets from bright (#e8e2d9) to dim (#938d86)
    • active preset shown with accent-coloured ring
    • choice persisted in localStorage (reader-text-colour) and restored on next open
  • Increased spacing between hamburger button and back link in reader header (margin-left: 1rem) to prevent accidental taps
  • Removed Cover Missing auto-tag:
    • tag is no longer added on import, rescan, or grabber download
    • ensure_cover_missing_tag() removed from common.py, library.py, and grabber.py
    • startup migration removes all existing Cover Missing tags from the database
  • Fixed Tags/Genres/Sub-Genres not saving in book edit panel on desktop:
    • , (comma) now acts as a confirmation key alongside Enter in PillInput
    • flush() added to PillInput: any text still in the input field is auto-confirmed when Save is clicked
  • Fixed tag/genre search and tag-pill navigation being broken:
    • renderGenreView was filtering on b.genres (non-existent field); now uses bookGenres(), bookSubgenres(), bookPlainTags()
    • renderSearchResults had the same bug; search now covers title, author, genres, sub-genres, and tags
  • Made reading progress monotonic across devices:
    • POST /library/progress/{filename} now rejects any save whose (chapter_index, scrollFrac) is not strictly ahead of the stored position (returns {"ok": true, "skipped": true})
    • prevents device A from overwriting further progress saved by device B when switching between devices without closing the book
    • also prevents bookmark-based backward navigation (e.g. jumping back to correct an earlier chapter) from clobbering the furthest-read position
    • progress reset remains via the explicit Read/Unread actions, which clear the reading_progress row