novela/docs/changelog.md

Changelog

v0.2.10 — 2026-05-10

Bug fixes

• Editor / Reader: subheading (S) and chat (C) styling now also renders correctly when the wrapper is a <div> instead of a <span>. When the selection contained a block element (e.g. an <h1..6>, <p> or <div>), wrapSpan() in editor.js falls back to <div class="…"> to keep the HTML valid; the previous CSS in reader.html only targeted span.subheading / span.chat, so anything wrapped around or inside a heading silently lost its color/weight. CSS selectors are now class-only (#chapter-content .subheading / .chat), matching both span and div wrappers.

---

v0.2.9 — 2026-05-09

Bug fixes

• Reader: reading position is now monotonic across devices — saved position only advances, never rewinds. Previously, reading the same book on a second device (e.g. continuing from chapter 12 to chapter 15 on an iPad) and then opening the book again on the original device would overwrite the further position with the older one, dropping the user back to chapter 12. The progress endpoint now compares the incoming (chapterIndex, scrollFrac) to the stored value and only writes when the new position is strictly further. Explicit Mark as read / Mark as unread still clears the row, so deliberate restarts still work.

---

v0.2.8 — 2026-04-22

Bug fixes

• Grabber: newly converted books now appear in the New view again. Both the DB-storage and file-EPUB branches in routers/grabber.py now persist needs_review = True on upsert_book (was False); the New view filters on needs_review, so previously grabbed books never showed up there. Disk-scanned imports already behaved correctly.

---

v0.2.7 — 2026-04-22

Bug fixes

• Break detection: runs of two or more consecutive scene-break images are now collapsed to a single break. Previously some books ended up with several identical <center><img src=".../break.png"/></center> lines directly after each other; the reader and exported EPUBs now show only one.

Internal

• New helper collapse_consecutive_breaks() in xhtml.py matches 2+ consecutive break-image <center> lines (with optional whitespace between) and replaces them with a single break. Applied in normalize_wysiwyg_html() (editor save path) and in routers/grabber.py on both the preview converted_xhtml and the per-chapter content_html produced during scraping.
• docs/TECHNICAL.md updated to cover previously missing changes: POST /api/edit/intro/{filename} and the title field on file-EPUB chapter save; FlareSolverr sidecar and BaseScraper.close(); AwesomeDudeScraper uses FlareSolverr; make_epub(include_intro=…) and epub_utils.build_book_info_body_html; grabber DB flow stores Book Info as chapter 0; "Book Info" h1-strip skip in reader; new env vars (FLARESOLVERR_URL, FLARESOLVERR_TIMEOUT_MS, NOVELA_PORT, ADMINER_PORT); collapse_consecutive_breaks() helper.

---

v0.2.6 — 2026-04-22

New features

• Scrapers: Cloudflare-protected sites (e.g. awesomedude.org, which moved fully behind a "Just a moment…" JavaScript challenge) can be scraped again via a new FlareSolverr sidecar service that solves the challenge in a headless browser; the novela container uses FlareSolverr for both the book-info page and every chapter fetch
• Per-book FlareSolverr sessions: the scraper creates one browser session at the start of a book, reuses it across all chapters (Cloudflare cookies stay warm), and destroys it on completion — so only the first request pays the full challenge-solve cost and subsequent chapters are much faster

Internal

• stack/stack.yml adds a flaresolverr service (image ghcr.io/flaresolverr/flaresolverr:latest, internal-only, on novela-net); novela gains FLARESOLVERR_URL=http://flaresolverr:8191/v1 and a depends_on: flaresolverr
• Host port mappings in stack/stack.yml are now driven by ${NOVELA_PORT} and ${ADMINER_PORT}, defaulted in stack/novela.env to 8099 / 8098; production stacks can override without diverging from the repo
• New helpers in scrapers/base.py: flaresolverr_get(url, timeout_ms=None, session=None) returns a SimpleNamespace(text, url) (drop-in for httpx.Response attributes); flaresolverr_session_create() / flaresolverr_session_destroy(sid) manage browser sessions; configurable via FLARESOLVERR_URL and FLARESOLVERR_TIMEOUT_MS env vars
• BaseScraper gained an async close() method (default no-op) so scrapers can release scoped resources
• scrapers/awesomedude.py creates a FlareSolverr session in fetch_book_info, reuses it in every fetch_chapter call, and destroys it in close()
• routers/grabber.py now wraps all three scraper usages (debug_run, preview, _run_scrape) in try/finally: await scraper.close() so FlareSolverr sessions are always released, even on errors

---

v0.2.5 — 2026-04-22

New features

• Editor: Info page button in the chapter editor toolbar generates a gayauthors-style book-info page (title, author, genres, sub-genres, tags, description, source, updated) and inserts it as the first chapter; empty metadata fields are skipped; no duplicate detection — clicking it again will add another page
• Editor: chapter titles are now editable for file-EPUB books (DB books already supported this); the chapter-title input in the header works for both storage types, and for file EPUBs the matching NCX navPoint is updated on save so the table of contents reflects the new title
• Grabber: DB-storage conversions now persist the Book Info page as a real stored chapter at index 0, so it is visible in the editor and reader (EPUB-storage conversions continue to produce intro.xhtml via make_epub as before)

Internal

• New endpoint POST /api/edit/intro/{filename} — for DB books, shifts existing chapter_index values up by one via a two-step negation (to avoid unique-constraint violations) and inserts "Book Info" at index 0; for file EPUBs, writes a new intro_<hex>.xhtml via make_intro_xhtml, adds a manifest item, places the itemref at the start of the spine, and inserts a navPoint at the top of the NCX with renumbered playOrder
• POST /api/edit/chapter/{index}/{filename} for file EPUBs now accepts a title field alongside content and updates the matching NCX navPoint text when it changes
• make_epub gained an include_intro: bool = True parameter; DB → EPUB export (reader.py) calls it with include_intro=False because the stored chapter 0 is now the single source of truth for the info page
• reader.py leading-h-tag stripping (get_chapter_html and the DB→EPUB export) is skipped when title == "Book Info", so the <h1>{book title}</h1> in that chapter's body survives
• New helper epub_utils.build_book_info_body_html(title, author, info) returns the inner-body HTML fragment for DB storage; skips empty fields and separates description and source/updated blocks with <hr/>

---

v0.2.4 — 2026-04-21

Improvements

• Backup: status and history now clearly distinguish Scanned (number of library files inspected) from Uploaded (objects actually sent to Dropbox — library + snapshot + pg_dump); previously only the upload count was shown, which caused confusion when most files were already deduplicated and the number looked suspiciously low (e.g. files=2 while the library contained 952 books)
• Backup page: a live phase indicator is shown under the Run buttons while a backup is running (scanning library, uploading library objects, uploading snapshot, uploading pg_dump), so it is clear the process is not stuck at N / N while the snapshot and pg_dump are being uploaded

Internal

• New migration backup_log_scanned_files adds a scanned_files column to backup_log; historical rows keep NULL for this column
• /api/backup/status and /api/backup/history now return uploaded_files and scanned_files; the old files_count key was renamed to uploaded_files — external consumers (if any) should update accordingly

---

v0.2.3 — 2026-04-21

Bug fixes

• Backup: Dropbox uploads no longer fail with HTTPSConnectionPool ... Read timed out. (read timeout=120) — the Dropbox client timeout was raised from 120s to 300s and the upload chunk size was reduced from 100 MB to 16 MB so each chunk completes comfortably within the timeout window

---

v0.2.2 — 2026-04-16

New features

• Editor: four inline formatting buttons added to the chapter editor toolbar
  • S — wraps selection in <span class="subheading"> (red bold text in the reader)
  • C — wraps selection in <span class="chat"> (orange text in the reader)
  • →| — wraps selection in an indented paragraph (padding-left: 40px)
  • [ ] — wraps selection in a comment block with a blue left border and tinted background
  • Without a selection each button inserts an empty tag with the cursor placed inside
  • Wrap logic automatically uses a <div> when the selection contains block elements, keeping the HTML valid

---

v0.2.1 — 2026-04-16

Improvements

• Startup: migration progress is now visible in Docker logs — each migration logs whether it was skipped or executed (with duration in ms); a summary line at the end shows either "all already applied" or how many were executed

---

v0.2.0 — 2026-04-15

New features

• Editor: chapter add and delete are now deferred — structural changes are no longer saved immediately; they are applied in the correct order when the Save button is pressed
• Operations: GET /health endpoint — returns {"ok": true} when the database is reachable; suitable for container health checks and monitoring

Bug fixes

• Editor: adding a chapter to a DB-stored book no longer fails with a UniqueViolation — PostgreSQL was checking the unique constraint on (filename, chapter_index) mid-update; fixed with a two-step index shift
• Scraper: Codey's World pages now decode correctly — pages are read as Windows-1252 (cp1252), which correctly maps the 0x80–0x9F byte range; characters like …, ', ", — no longer appear as replacement characters
• XHTML conversion:   followed by a regular space no longer produces a double space — non-breaking spaces are normalized to regular spaces and consecutive spaces are collapsed; applies to all scrapers

Improvements

• Startup: each database migration now runs only once — a schema_migrations tracking table prevents heavy migrations (such as the chapter TSV rebuild) from re-running on every container restart; startup connection overhead reduced from 37 separate connections to 1
• Library API: ETag now reflects changes to tags and reading progress — MAX(updated_at) from reading_progress and MAX(id) from book_tags are included; tag edits and progress updates now correctly invalidate the client cache
• CBR/CBZ reader: page list is cached per file and modification time — avoids opening the archive twice per page request
• Grabber and backup: in-memory job dicts are capped at 50 entries to prevent unbounded memory growth

Internal

• Shared epub_utils.py module with deduplicated EPUB helper functions — eliminates near-identical copies of epub_spine, find_opf_path, norm_href, make_new_chapter_xhtml, and rewrite_epub_entries that existed across reader.py, editor.py, and common.py; fixes a double-escaped regex in the old OPF path lookup
• pdf_cover_thumb no longer writes a temporary file — the cover thumbnail is generated fully in-memory via Image.frombytes(), eliminating a race condition under concurrent requests
• security.py: hardcoded fallback encryption key removed; raises a clear error at startup when neither NOVELA_MASTER_KEY nor POSTGRES_PASSWORD is configured; Fernet instance cached per process
• builder.py: all explicit conn.commit() calls replaced with with conn: context manager, consistent with the rest of the codebase

---

v0.1.12 — 2026-04-15

New features

• Reader: font size slider in the reading settings drawer — adjust text size from 80% to 150%; setting is saved per device so iPad and desktop each remember their own preference

---

v0.1.11 — 2026-04-13

Bug fixes

• Edit metadata: pasting or typing a comma-separated list in the genre, subgenre or tag input now adds each value as a separate tag instead of one combined tag

---

v0.1.10 — 2026-04-12

New features

• Reader: prev/next volume buttons in the header for books that are part of a series — buttons appear automatically when the book has adjacent volumes; tooltip shows the volume number and title; marking a book as read redirects directly to the next volume in the reader instead of the book detail page
• Comics: series_volume field for annual series where issue numbers restart each year (e.g. Donald Duck (1982) [15]) — stored in the database and EPUB OPF; displayed as (year) after the series name on the book detail page; sorting respects series_volume before series_index; supported in Bulk Import via %series_volume% placeholder and a "Year/Vol." shared field
• Library: archive a series in one click — "Archive series" / "Unarchive series" button in the series detail view; updates all books in the series via a single SQL UPDATE and recalculates sidebar counters without a page reload

Bug fixes

• TedLouis scraper: title extraction no longer includes the "Back" button text or the author byline — only direct text nodes of the title heading are used

---

v0.1.9 — 2026-04-08

New features

• New scraper: Nifty.org (classic) — scrapes plain-text email-format stories; email headers stripped, boilerplate paragraphs auto-detected and hidden, scene-break patterns converted to break images
• New scraper: new.nifty.org — scrapes the Next.js version of Nifty; reads chapter content from the RSC payload when the static HTML does not include it; boilerplate detection shared with classic Nifty
• New scraper: codeysworld.org — single-file and multi-chapter stories; title and author extracted from heading elements; category from URL path stored as tag; navigation links and audio links stripped from chapter content
• New scraper: iomfats.org — all stories are listed on a single author page; provide any chapter URL and the scraper finds the correct story automatically; supports single stories and multi-part series (series name, book title, and series index derived from the page structure)
• New scraper: tedlouis.com — all pages use opaque token-based routing (?t=TOKEN); provide the story index URL and the scraper collects all chapter links from the three-column chapter list
• Settings: break image upload — upload a custom PNG/JPG/WebP to use as the scene break image in all converted books; stored in the imagestore and applied to both DB-stored and EPUB-format books
• Settings: develop mode toggle — shows a DEVELOP banner and updates the page title across all pages when enabled

Bug fixes

• Break images were not displayed in DB-stored books — the image path ../Images/break.png is a relative EPUB path that does not exist for DB content; DB mode now uses /static/break.png
• Break images were silently lost during import — the image was decomposed before element_to_xhtml ran, leaving an empty wrapper; the wrapper is now replaced with <hr> so the break is correctly rendered

---

v0.1.8 — 2026-04-06

Bug fixes

• Library: cover upload now works for DB-stored books — the upload endpoint previously returned "File not found" because DB books have no file on disk; the cover is now stored directly in the cover cache

Improvements

• Book detail: rating moved from clickable stars to a dropdown in the Edit metadata panel — avoids touch-input issues on iPad where hover state caused all stars to appear filled

---

v0.1.7 — 2026-04-06

New feature

• Search: filter on unread novels or unread shorts — a second toggle row (All / Unread novels / Unread shorts) restricts results to books with no reading history; filter is preserved in the URL

Bug fixes

• Backup: files larger than 148 MB now upload correctly — chunked upload session (100 MB per chunk) replaces the single-call upload that hit Dropbox's payload size limit

Improvements

• File paths: spaces in new filenames are now replaced with underscores (publisher, author, title, series segments); series separator changed from - to _-_

---

v0.1.6 — 2026-04-05

Bug fixes

• Export EPUB: double chapter titles fixed — same heading-stripping logic as the reader now applied before passing content to make_chapter_xhtml
• Library: authors and publishers with only archived books now remain visible in the Authors and Publishers list views

---

v0.1.5 — 2026-04-04

Bug fixes

• Reader: double chapter titles for pandoc-converted books — headings wrapped in a <section> element were not stripped by the previous regex; now also removes the first heading found directly inside an opening <section> or <div>
• Search: multi-word queries no longer match chapters where the words appear far apart — switched to phraseto_tsquery so all words must appear in order

---

v0.1.4 — 2026-04-04

Bug fixes

• Reader: double chapter titles in DB-stored books — the chapter endpoint now strips all leading headings from stored content before prepending its own chapter title; affects books scraped before front-matter stripping was added
• Library: archived books were missing from author and publisher detail views — detail views now include all books (active and archived); archived books have a badge on their cover so they remain distinguishable

---

v0.1.3 — 2026-04-03

New feature

• DB-stored books: scraped books are now stored as chapters in PostgreSQL instead of EPUB files on disk — full-text search, content deduplication, and backup coverage are all handled automatically
  • Grabber stores chapters in a book_chapters table and images in a content-addressed imagestore (sha256-based, automatic deduplication)
  • EPUB → DB conversion: "Convert to DB" button on any EPUB book detail page — extracts chapters, migrates all metadata and child rows (tags, progress, bookmarks, cover), removes the EPUB file
  • DB → EPUB export: "Export EPUB" button on DB-stored books — builds and streams a standards-compliant EPUB without writing a file to disk
  • Full-text search (/search): searches across all DB-stored chapter content via PostgreSQL FTS (tsvector / plainto_tsquery), returns highlighted snippets with direct links to the chapter position in the reader
  • Chapter editor supports DB-stored books: Monaco-based editor reads and writes book_chapters directly; chapter titles editable inline; title-only changes correctly included in Save All
  • Grabber: storage toggle on the Convert page — choose between DB storage and EPUB file before converting

---

v0.1.2 — 2026-04-02

New feature

• Restore functionality on the Backup page: browse any available Dropbox snapshot, see which files are currently missing from disk, and restore individual books or a selection back to the library — file is written to disk and immediately re-indexed

---

v0.1.1 — 2026-03-31

Bug fixes, volume-aware duplicate detection, shared code cleanup, and a new Changelog page.

Bug fixes

• Duplicates view crashed on load due to a TypeError (g.books.length was undefined); counter was stale and the view never rendered
• Duplicate detection was too aggressive: different volumes of the same series (same title + author, different volume) were incorrectly grouped as duplicates — now keyed on title + author + volume
• Grabber preload: same volume-aware fix — only flags a duplicate when title, author, and volume all match; falls back to title + author when no volume is known
• Bulk Import duplicate check: different volumes of the same series are no longer flagged as duplicates

Improvements

• Search changed from search-as-you-type (250 ms debounce) to Enter-to-search — prevents the iPad keyboard from locking up on large collections
• CBR reader: archive format now detected via magic bytes instead of file extension — .cbr files that are actually ZIP or 7-zip archives open correctly; added 7-zip support via py7zr
• Docker: replaced unrar-free with proprietary unrar (RARLAB v6.2.6) — fixes failures on RAR archives using newer compression methods

New feature

• Changelog page (/changelog): structured release history with version, date, and categorised change lists

Code quality

• Shared CSS (theme.css): single :root block with all global CSS custom properties; loaded on every page — no more duplicate inline :root blocks across templates
• Shared JS (books.js): book helpers (bookTitle, bookAuthor, bookGenres, bookSubgenres, bookPlainTags, filterBooks) and search input wiring extracted into one shared file
• Shared JS (conversion.js): SSE/EventSource logic (connectConversionStream, addLog) extracted from Convert and Grabber pages into one shared file

---

v0.1.0 — 2026-03-29

First release of Novela: a self-hosted personal library for EPUB, PDF, CBR, and CBZ files.

Library

• Grid and List view for all books and New books, with column visibility filter and persistent view mode
• Sidebar navigation: All books, Want to Read, New, Incomplete, Series, Authors, Publishers, Archived, Bookmarks, Rated, Duplicates, Statistics
• Sidebar counters for all sections, live-updated without page reload
• 1–5 star ratings stored in DB and written back to EPUB OPF / CBZ ComicInfo.xml
• Publication status: Complete, Ongoing, Temporary Hold, Long-Term Hold
• Status and want-to-read badges on grid covers, always readable regardless of cover colour
• Duplicate detection: groups books by title+author; counter in sidebar
• Incomplete view: all non-archived books where publication status is not Complete
• Rated view: non-archived books with a star rating, sorted by rating
• Bulk delete in All books List view with multi-select and Shift+click range selection
• Disk usage warning in sidebar (amber ≥ 85%, red ≥ 95% or low free space)
• Autocomplete for Author, Publisher, and Series in the book edit panel
• Series volume suffix support (e.g. "21a", "21b") and volume 0 for prequels/specials
• Cover upload for EPUB books; cover cache for fast subsequent loads

Reader

• EPUB reader with chapter navigation, scroll progress, and bookmarks
• PDF reader with page-image rendering and page navigation
• CBR/CBZ reader with page-image rendering; format detection via magic bytes (supports ZIP, RAR, 7-zip archives)
• Reader text colour: 5 warm-tone presets, persisted per browser
• Content width slider (30–100 vw), persisted per browser
• Bookmarks: save position with optional note; navigate back via sidebar or bookmark list

Import & Convert

• Single-file import: drag-and-drop or file picker for EPUB, PDF, CBR, CBZ
• Bulk Import (/bulk-import): batch import with %placeholder% filename pattern parsing, shared metadata, live preview table, and duplicate detection
• Convert (/convert): scrape web fiction and convert to EPUB; warns if title+author already exists in library
• Grabber with credentials manager for site-specific login

Book Builder

• Create EPUB books from scratch via a WYSIWYG editor (/builder)
• Chapters with contenteditable editing; toolbar: bold, italic, underline, blockquote, author note, scene break, normalize
• Autosave every 30 s and Ctrl+S; publish produces a standards-compliant EPUB 2.0 added directly to the library

Following

• Following page (/following): track external author URLs
• Two tabs: Following (authors with URL set) and All Authors
• Inline URL editing with Enter/Escape support; Visit opens URL in new tab
• Sidebar counter shows number of followed authors

Backup

• Dropbox backup with versioned snapshots and object-store deduplication
• OAuth2 refresh token flow (does not expire); legacy access token supported as fallback
• Configurable backup root, snapshot retention, and scheduled interval
• Live backup progress in sidebar (file count + phase); backup status dot with time-ago
• PostgreSQL dump included in each backup run

Performance

• Library loads instantly for large collections: ETag 304 Not Modified, IntersectionObserver lazy covers, single DOM pass rendering, json_agg SQL tag aggregation
• Fast-path /api/library (DB-only); full rescan only on demand

Branding

• Favicon for browser tabs (16×16, 32×32, 256×256)
• Apple touch icon (180×180) with dark background for iOS home screen
• Logo in sidebar alongside the Novela wordmark