891 lines
85 KiB
Markdown
891 lines
85 KiB
Markdown
# Develop Changelog
|
||
|
||
## 2026-06-13 — Visual editor: drop H2/H3 buttons, add superscript/subscript
|
||
|
||
### Changed
|
||
- Replaced the H2 and H3 heading buttons in the visual (WYSIWYG) toolbar with **superscript (x²)** and **subscript (x₂)** buttons. The heading buttons weren't part of the novela-ng editor and aren't used; superscript/subscript were missing. Heading levels remain in the schema so existing headings in other books still round-trip — only the authoring buttons changed. `templates/editor.html` (toolbar buttons), `static/editor.js` (`updateVisualButtons` active-state, removed unused `vHeading`). No bundle rebuild (sup/sub extensions were already bundled).
|
||
|
||
## 2026-06-12 — Visual editor: full-width editing column
|
||
|
||
### Changed
|
||
- Made the Visual (WYSIWYG) editing column fill the full editor pane width: dropped the max-width cap entirely (now `width: 100%`, margin 0, with 3rem horizontal padding) — 70rem was still too narrow. `static/editor.css` (`.visual-pane .ProseMirror`). CSS only; no bundle rebuild.
|
||
|
||
## 2026-06-12 — Visual editor: wider editing column
|
||
|
||
### Changed
|
||
- Widened the Visual (WYSIWYG) editing column from the reader's 42rem reading width to 70rem with more horizontal padding — the reading width was comfortable to read but too narrow to edit in. `static/editor.css` (`.visual-pane .ProseMirror`). CSS only; no bundle rebuild.
|
||
|
||
## 2026-06-12 — Visual editor: broaden schema & fidelity check against real library content
|
||
|
||
### Changed
|
||
- Tested the round-trip safety gate against every chapter of the actual DB library and broadened the visual editor so Visual mode is available for the overwhelming majority of real content instead of being refused on common, lossless markup. Across the sampled DB books (~185 chapters) Visual mode now opens for 100% of chapters; only genuinely lossy or malformed markup is still blocked.
|
||
- **Schema broadened** to cover conventions that real chapters actually use (all now round-trip losslessly): the **block** variants of subheading and chat (`<div class="subheading">…</div>`, `<div class="chat">…</div>` — previously only the inline `<span>` marks were modelled), **block indent** (`<div style="padding-left: 40px;">…</div>`), all **heading levels** h1–h6 (was h2/h3 only; the Book Info page uses `<h1>`, and some books use `<h4>` separators), plain **`<hr>`** rules (re-enabled alongside the `<center><img>` scene break), **links** (`<a href>`, with rel/target injection disabled so links stay byte-identical), and preservation of arbitrary **`class`** attributes on paragraphs/headings (e.g. the generated `<p class="author">`).
|
||
- **Round-trip comparison made tolerant of the editor's lossless normalisations**, so they no longer read as "unsafe": inline formatting is now compared as an unordered **mark set** per text run (so `<em><strong>x</strong></em>` and `<strong><em>x</em></strong>` are equal, empty marks like `<strong></strong>` are ignored, and adjacent identical marks merge), and loose inline content sitting directly under `<body>` is compared as if wrapped in a paragraph (matching how the editor wraps it). Genuine loss is still blocked — verified by guardrail tests that unknown tags (`<table>`, `<font color>`), standalone `<img>`, unknown `<div class>` wrappers, and whole-xhtml EPUB documents all remain "unsafe".
|
||
- Files: `containers/novela/editor-src/extensions.js` (added `SubheadingBlock`, `ChatBlock`, `IndentBlock`, `ClassPreserve`); `containers/novela/editor-src/index.js` (extension list, heading levels, Link, mark-set canonicaliser with body-level loose-inline wrapping, `roundtripDebug` diagnostic export); `containers/novela/editor-src/package.json` (+`@tiptap/extension-link`); rebuilt `static/editor-bundle.js`; new dev harnesses `editor-src/{diagnose.mjs,test-commands.mjs}` and expanded `editor-src/test.mjs`. No backend or `editor.js`/`editor.html`/`editor.css` changes in this round.
|
||
|
||
## 2026-06-12 — Chapter editor: optional visual (WYSIWYG) mode alongside Monaco
|
||
|
||
### Added
|
||
- The chapter editor now has an optional **Visual (WYSIWYG) mode** next to the existing Monaco HTML-source editor, ported from the Tiptap editor built in novela-ng and adapted to Novela's own markup conventions. A **Visual / HTML** toggle button in the toolbar switches between the two; Monaco remains the default and the backing source of truth.
|
||
- **Tiptap is bundled, not CDN-loaded.** A new `containers/novela/editor-src/` npm project bundles Tiptap + the custom extensions into a single committed file, `static/editor-bundle.js` (`window.NovelaVisual`), loaded by the editor page with a plain `<script>` tag — the runtime frontend stays build-less. Rebuild with `cd containers/novela/editor-src && npm install && npm run build`. `node_modules` is git-ignored; the bundle is committed.
|
||
- **Novela conventions modelled as a real ProseMirror schema** so they apply and round-trip as formatting instead of raw tags: subheading (`span.subheading`), chat (`span.chat`), author comment (`div.novela-comment`), indented paragraph (`<p style="padding-left: 40px;">`), and scene break (`<center><img src="…break.png" style="height:15px;"/></center>` — the original img `src` is preserved, so both DB `/static/break.png` and on-disk EPUB `../Images/break.png` breaks round-trip exactly). Standard bold/italic/underline/superscript/subscript, H2/H3 headings, and bullet/numbered lists are also available as visual-only toolbar buttons.
|
||
- **Detect & block (no silent data loss).** ProseMirror drops any markup its schema doesn't model, which would silently strip formatting from arbitrary EPUB/DB HTML. Switching into Visual mode is therefore gated by a round-trip safety check (`NovelaVisual.roundtripSafe`): the chapter is fed through the editor and the result is compared, canonicalised, against the original (ignoring insignificant formatting differences such as attribute order, whitespace, self-closing style, and `b`/`strong` & `i`/`em` equivalence). If anything — a tag, attribute, or text — would be altered or dropped, Visual mode is refused for that chapter and editing stays in HTML mode. In practice this means Visual mode is offered for clean, convention-conforming content (typically DB books) and declined for heterogeneous EPUB markup or whole-xhtml documents, so a book's existing formatting can never be quietly destroyed.
|
||
- The semantic toolbar buttons (Break, S, C, →|, [ ]) route to the corresponding Tiptap commands in Visual mode and to the existing Monaco text-wrapping in HTML mode. Visual edits participate in the existing per-chapter dirty-tracking, Save / Save-all, chapter switching, add/delete, and Find & Replace flows; the visual content is synced back into Monaco (the backing store) at every save/switch/replace boundary, and the safety gate is re-evaluated whenever a chapter is loaded. Ctrl/Cmd+S also saves while editing visually.
|
||
- Files: new `containers/novela/editor-src/{package.json,build.mjs,index.js,extensions.js,test.mjs,test-commands.mjs,test.html,.gitignore}`; new committed `static/editor-bundle.js`; `templates/editor.html` (bundle script, toggle button, visual-only toolbar group, `#visual-pane`); `static/editor.css` (visual pane, toolbar and convention styling); `static/editor.js` (pane abstraction, `toggleVisual`/`mountVisual`/`syncVisualToMonaco`/`refreshVisualAfterLoad`, boundary syncing, toolbar routing). No backend changes — chapter save is unchanged and does not sanitise, so the editor's clean output is stored as-is.
|
||
|
||
## 2026-06-03 — Reader: Page Up/Down scroll within the page
|
||
|
||
### Changed
|
||
- In the reader, **Page Up / Page Down no longer switch chapters** — they now fall through to the browser's default in-page scrolling. Chapter navigation is still bound to the **arrow keys** (Left/Right). Previously both the arrow keys and the page keys triggered chapter navigation, so accidentally pressing Page Down while reading jumped to the next chapter and lost the reading position.
|
||
- `templates/reader.html`: the `keydown` handler now matches only `ArrowRight`/`ArrowLeft` for `navigate(±1)`; `PageDown`/`PageUp` removed from the bindings.
|
||
|
||
*Released as v0.2.14 on 2026-06-03*
|
||
|
||
## 2026-06-01 — Full Database Restore: live progress + download-to-local first
|
||
|
||
### Added
|
||
- Full database restores (Dropbox, local and upload) now run as a **background task with live byte-level progress**, polled by the Backup page, instead of a silent blocking request. This matters for large (e.g. ~1 GB) production databases where the restore previously showed nothing for minutes.
|
||
- Phases reported and shown with a percentage / size: **downloading** (Dropbox only) → **safety snapshot** → **resetting schema** → **loading** → done / rolled back / failed.
|
||
- Load progress is derived from feeding the dump to `psql` over stdin in chunks; pipe back-pressure makes the byte counter track psql's real progress closely.
|
||
- **Dropbox restore now downloads the dump to the local store first**, then restores from that file — so the restore no longer depends on the network connection mid-load, and a reusable local copy is kept for token-free restores. (Previously the Dropbox dump was streamed straight into memory and not persisted.)
|
||
- `routers/backup.py`: `RESTORE_PROGRESS` state + `_restore_prog`; `GET /api/backup/restore/progress`; `_apply_pg_dump_file` (stdin-streamed load with progress, stderr captured to a temp file to avoid pipe deadlock), `_pg_dump_safety_to_local` (safety snapshot with live size), `_download_dropbox_to_file`, `_restore_worker_sync`, `_start_restore` (single restore at a time). The three restore endpoints now start the background job and return `{started: true}`; `_run_pg_restore`/`_apply_pg_dump` (bytes-based) were replaced.
|
||
- Local dump retention now sorts by mtime (robust across `pre-restore-…` and downloaded dump names).
|
||
- `templates/backup.html`: shared `pollRestoreProgress` with phase labels and percentage/size; the three restore buttons start the job then poll; progress resumes on page reload if a restore is still running.
|
||
|
||
## 2026-06-01 — Sidebar version is display-only
|
||
|
||
### Changed
|
||
- The build version at the bottom of the sidebar is no longer a clickable link to the changelog — it is now a plain text display. It exists only to show that the running build number has incremented (so it's clear the new code is active).
|
||
- `templates/_sidebar.html`: `<a href="/changelog" class="sidebar-version">` → `<span class="sidebar-version">`.
|
||
- `static/sidebar.css`: dropped the link/hover styling; added `cursor: default` and `user-select: none`.
|
||
|
||
## 2026-06-01 — Local pre-restore snapshots + token-free / upload restore
|
||
|
||
### Added
|
||
- Every restore now writes its **pre-restore safety dump to a local store** on the config volume (`CONFIG_DIR/postgres_dumps/`, named `pre-restore-…`, with snapshot-equal retention), so the database can be restored or rolled back **without a Dropbox token**. Regular backups are intentionally left **Dropbox-only** — production backs up hourly, so writing every backup to disk would fill the volume.
|
||
- `routers/backup.py`: `LOCAL_DUMP_DIR`, helpers `_save_local_dump`, `_list_local_dumps`, `_enforce_local_dump_retention`, `_resolve_local_dump`; `_run_pg_restore` persists the safety dump locally (and prunes by retention) before the destructive load. `_run_backup_internal` is unchanged (no local copy).
|
||
- New **Local Database Restore** card on the Backup page with two token-free paths:
|
||
- Restore from a locally stored dump — `GET /api/backup/local/dumps`, `POST /api/backup/local/restore`.
|
||
- Upload a `.sql` dump (e.g. downloaded manually from dropbox.com) and restore it — `POST /api/backup/upload-restore`. The upload itself is not persisted; the automatic pre-restore safety snapshot covers the local copy.
|
||
- Both reuse the safe restore path (pre-restore safety dump + automatic rollback) and require `psql`.
|
||
- `templates/backup.html`: new card with a local-dump selector, an upload field, double-guarded restore buttons, and `fmtBytes` sizes.
|
||
|
||
## 2026-06-01 — Full Database Restore: safety dump + automatic rollback
|
||
|
||
### Fixed
|
||
- Full Database Restore could leave the database **empty** if the dump failed to load: the restore dropped and recreated the public schema first and only then applied the dump, so any failure during the load (e.g. a header `transaction_timeout` error with `ON_ERROR_STOP`) wiped all data with nothing to fall back to.
|
||
- `routers/backup.py`: split the load into `_apply_pg_dump`; `_run_pg_restore` now takes a safety `pg_dump` of the current database before the destructive load and, if applying the requested dump fails, automatically rolls back to that safety snapshot. A failed restore therefore no longer leaves an empty or broken database.
|
||
|
||
## 2026-06-01 — Full Database Restore: tolerate PostgreSQL version mismatch
|
||
|
||
### Fixed
|
||
- Full Database Restore failed with `ERROR: unrecognized configuration parameter "transaction_timeout"` when the dump was produced by a newer `pg_dump` (PostgreSQL 17+, which emits `SET transaction_timeout = 0;` in the header) but restored into an older server (<17) that doesn't know that session parameter. The restore ran with `ON_ERROR_STOP=1` and aborted on that harmless header line.
|
||
- `routers/backup.py` (`_run_pg_restore`): the dump is now applied without `ON_ERROR_STOP`; stderr is inspected afterwards via `_real_restore_errors`, which ignores benign "unrecognized configuration parameter" errors but still fails the restore on any other `ERROR:` line. The schema-reset step keeps `ON_ERROR_STOP=1` (it is our own controlled SQL).
|
||
|
||
## 2026-06-01 — Backup/Restore: database-stored books are now restorable
|
||
|
||
### Fixed
|
||
- Database-stored books (`storage_type='db'`, synthetic `db/...` filenames) could not be restored through the Backup → Restore option. The restore UI only listed and restored files from the on-disk library object store, but db-books have no file on disk — their content lives entirely in PostgreSQL (`book_chapters` + `library` row + `book_tags` + `library_cover_cache`). They were captured only in the full `pg_dump`, which the UI offered no way to restore. As a result db-books never appeared in the restore list and were effectively unrecoverable per-book.
|
||
|
||
### Added
|
||
- **Per-book db restore.** The backup writer now serializes each database book to JSON (library row, all chapters, tags, and the cached cover) and stores it in the same content-addressed Dropbox object store used for file books, referenced from the snapshot with a `"storage": "db"` marker. Such books now appear in the Restore table (format **DB**) and can be restored individually; restore re-inserts the library row, chapters, tags and cover into PostgreSQL. Inline chapter images are unaffected — they already live on disk under `library/images/` and are backed up as ordinary files.
|
||
- `routers/backup.py`: new `_db_book_filenames`, `_serialize_db_book`, `_restore_db_book`, `_entry_storage` helpers; db-books serialized during `_run_backup_internal`; `_download_and_restore` branches on storage type; `/api/backup/snapshots/{name}/files` now reports `storage` and computes `exists_locally` for db-books from the `library` table.
|
||
- Note: db-books only appear in snapshots created *after* this change. Older backups still contain them only in the `pg_dump` — recover those via Full Database Restore below.
|
||
- **Full Database Restore.** New Backup-page card and endpoints to restore the entire PostgreSQL database from any Dropbox `pg_dump`. This recovers everything (all db-books, reading progress, tags, settings) from existing backups too. It is destructive: the public schema is dropped and recreated before the dump is applied, so any plain dump (with or without `--clean`) restores cleanly. Guarded behind a double confirmation in the UI.
|
||
- `routers/backup.py`: `_psql_base_args`, `_run_pg_restore`, `_list_pg_dump_paths`; endpoints `GET /api/backup/postgres/dumps` and `POST /api/backup/postgres/restore`; health endpoint now also reports `psql_available`/`psql_path`.
|
||
- `templates/backup.html`: db-books shown with **DB** format and "in library / not in library" status in the restore table; new "Full Database Restore" card with dump selector, double-confirm, and `psql` health row.
|
||
|
||
---
|
||
*Released as v0.2.13 on 2026-06-01*
|
||
|
||
## 2026-06-01 — Reader: Sepia reading theme
|
||
|
||
### Added
|
||
- The reader now offers a **Sepia** theme alongside the existing Dark theme, for easier long-form reading (warm paper background with dark brown text instead of light-on-black). New **Theme** toggle (Dark / Sepia) at the top of the reading settings drawer.
|
||
- `static/theme.css`: added a `:root[data-theme="sepia"]` palette overriding `--bg`, `--surface`, `--surface2`, `--border`, `--accent`, `--accent2`, `--text`, `--text-dim`, `--text-faint`, `--success`, `--warning`, `--error`. Also added a `--shadow-ring` variable (light hairline in dark, dark hairline in sepia) so swatch rings read correctly in both themes.
|
||
- `templates/reader.html`: theme toggle buttons + `.theme-btn` styling; an inline head script applies the saved theme before paint to avoid a flash; the Text colour row now has two theme-specific swatch sets (light tints for dark, dark brown tints for sepia), toggled by visibility.
|
||
- JS: `applyTheme()`/`loadTheme()` set the `data-theme` attribute and persist to `localStorage` (`reader-theme`). Text colour is now stored per theme (`reader-text-colour-dark` / `reader-text-colour-sepia`); the old single `reader-text-colour` key is migrated into the dark slot. `init()` calls `loadTheme()` (which loads the active theme's text colour).
|
||
- Replaced two hard-coded `rgba(255,255,255,...)` values in the reader CSS (swatch ring, series-button border) with theme variables so they don't show as white halos in sepia.
|
||
|
||
## 2026-05-31 — Build version in the sidebar
|
||
|
||
### Added
|
||
- The sidebar now shows the running build version at the bottom (e.g. `v0.2.11` for releases, `v0.2.11.3` for dev builds), linking to the changelog page.
|
||
- New `containers/novela/version.py` exposes `display_version()`. The semantic release version stays the single source in `changelog.py` (`CHANGELOG[0]["version"]`); a `BUILD` segment is appended for dev builds.
|
||
- `shared_templates.py` registers `app_version` as a Jinja global; `_sidebar.html` renders it as a `.sidebar-version` link styled in `sidebar.css`.
|
||
- `main.py` adds a `/api/version` endpoint returning `{"version": display_version()}`.
|
||
|
||
## 2026-05-31 — Find & Replace: scope option
|
||
|
||
### Added
|
||
- Editor Find & Replace: new **Current chapter only** checkbox in the modal options. When checked, the search/replace runs against the currently open chapter instead of every chapter in the book.
|
||
- `templates/editor.html`: added the `rp-current` checkbox next to Regex/Case sensitive; modal title changed from "Find & Replace — all chapters" to "Find & Replace" since scope is now selectable.
|
||
- `static/editor.js` (`replaceInAllChapters()`): reads `rp-current`; builds a `targets` list of either just `currentCh()` or all `chapters`, and iterates that. Shows an error ("No chapter open.") if Current-chapter-only is selected with no open chapter. Result message reads "… in current chapter" for the single-chapter case.
|
||
- Default is unchecked, so the existing all-chapters behaviour is preserved.
|
||
|
||
---
|
||
*Released as v0.2.12 on 2026-06-01*
|
||
|
||
## 2026-05-10
|
||
- Reader: subheading/chat styling now also wins when the wrapper contains block elements with their own color rule (e.g. `<div class="subheading"><p>…</p></div>`)
|
||
- Previously the `<p>`/`<h*>` rules in `reader.html` (`#chapter-content p { color: var(--text); }`, etc.) had the same specificity as `.subheading` and applied more directly to the inner element, so the parent's color was effectively overridden — the wrapped paragraph stayed in the default text color
|
||
- CSS now applies the rule to the wrapper *and* all descendants: `#chapter-content .subheading, #chapter-content .subheading * { … }` (and the same for `.chat`)
|
||
- Reason: bug reported after v0.2.10 — `.chat` worked because it wraps plain inline text inside a `<p>`, but `.subheading` wrapping a `<p>` lost its color
|
||
|
||
---
|
||
*Released as v0.2.11 on 2026-05-10*
|
||
|
||
---
|
||
*Released as v0.2.10 on 2026-05-10*
|
||
|
||
## 2026-05-09 (2)
|
||
- Editor / Reader: subheading (**S**) and chat (**C**) styling now also applies when the wrapper is a `<div>` instead of a `<span>`
|
||
- `wrapSpan()` in `editor.js` falls back to `<div class="…">` whenever the selection contains a block element (`<h1..6>`, `<p>`, `<div>`, etc.) so the resulting HTML stays valid; the previous CSS in `reader.html` (`#chapter-content span.subheading` / `span.chat`) only matched the `<span>` form, so anything wrapped around or inside a heading silently lost its color/weight
|
||
- CSS selectors are now class-only (`#chapter-content .subheading` / `.chat`), matching both span and div wrappers
|
||
|
||
---
|
||
|
||
## 2026-05-09
|
||
- Reader: progress is now monotonic across devices — saved position only advances, never rewinds
|
||
- `POST /library/progress/{filename}` in `routers/reader.py` parses the incoming `cfi` as `(chapterIndex, scrollFrac)` and the currently stored row the same way, then skips the write when `new_pos <= cur_pos`
|
||
- Same `cfi` format is used for EPUB, PDF and CBR/CBZ, so one tuple comparison covers all readers
|
||
- Explicit Read/Unread actions still clear the row (`mark-read` / `mark-unread` delete from `reading_progress`), so users can deliberately reset and start over
|
||
- Reason: when reading the same book on device A (chapter 12) and then continuing on device B (chapter 15), opening device A again previously sent its stale chapter-12 cfi back to the server and overwrote the further progress; now the older position is ignored
|
||
|
||
---
|
||
*Released as v0.2.9 on 2026-05-09*
|
||
|
||
## 2026-04-22 (5)
|
||
- Grabber: newly converted books now appear in the **New** view again
|
||
- Both the DB-storage and file-EPUB branches in `routers/grabber.py` set `needs_review: True` on `upsert_book` (was `False`); the New view filters on `needs_review == True`, so previously grabbed books never showed up there
|
||
- Matches the behavior of disk-scanned imports (`library.py` already sets `needs_review = True` for freshly discovered files)
|
||
|
||
---
|
||
*Released as v0.2.8 on 2026-04-22*
|
||
|
||
## 2026-04-22 (4)
|
||
- Break detection: runs of consecutive break images are now collapsed to a single break
|
||
- New helper `collapse_consecutive_breaks()` in `xhtml.py` matches 2+ consecutive `<center><img src=".../break.png" .../></center>` occurrences (with optional whitespace in between) and replaces them with one
|
||
- Applied in `normalize_wysiwyg_html()` (editor save path) and in `routers/grabber.py` on both the preview `converted_xhtml` and the per-chapter `content_html` produced during scraping
|
||
- Docs: `docs/TECHNICAL.md` updated to cover previously missing changes — `POST /api/edit/intro/{filename}` and the `title` field on file-EPUB chapter save; FlareSolverr sidecar and `BaseScraper.close()`; `AwesomeDudeScraper` uses FlareSolverr; `make_epub(include_intro=…)` and `epub_utils.build_book_info_body_html`; grabber DB flow stores Book Info as chapter 0; `"Book Info"` h1-strip skip in reader; new env vars (`FLARESOLVERR_URL`, `FLARESOLVERR_TIMEOUT_MS`, `NOVELA_PORT`, `ADMINER_PORT`); `collapse_consecutive_breaks()` helper
|
||
|
||
---
|
||
*Released as v0.2.7 on 2026-04-22*
|
||
|
||
## 2026-04-22 (3)
|
||
- Scrapers: Cloudflare "Just a moment…" challenges are now solved via a new FlareSolverr sidecar service so books on protected sites (awesomedude.org) can be scraped again
|
||
- New service `flaresolverr` in `stack/stack.yml` (image `ghcr.io/flaresolverr/flaresolverr:latest`, internal-only, on `novela-net`); `novela` service gains `FLARESOLVERR_URL=http://flaresolverr:8191/v1` and a `depends_on: flaresolverr`
|
||
- New helpers in `scrapers/base.py`: `flaresolverr_get(url, timeout_ms=None, session=None)` posts `cmd: request.get` and returns a `SimpleNamespace(text, url)` (drop-in for `httpx.Response` attributes); `flaresolverr_session_create()` and `flaresolverr_session_destroy(sid)` manage browser sessions so Cloudflare cookies stay warm across per-chapter requests (first page solves the challenge, all further chapters reuse the same browser — much faster)
|
||
- Configurable via `FLARESOLVERR_URL` and `FLARESOLVERR_TIMEOUT_MS` env vars (defaults: service DNS name and 60000 ms)
|
||
- `BaseScraper` gained an async `close()` method (default no-op) for releasing scraper-scoped resources; `scrapers/awesomedude.py` creates a FlareSolverr session in `fetch_book_info`, reuses it in all `fetch_chapter` calls, and destroys it in `close()`
|
||
- `routers/grabber.py` now wraps all scraper usage in `try/finally: await scraper.close()` so sessions are released even on errors
|
||
- Stack uses `${NOVELA_PORT}` and `${ADMINER_PORT}` (defined in `stack/novela.env` as 8099 / 8098) so host-port values don't diverge between environments
|
||
|
||
---
|
||
*Released as v0.2.6 on 2026-04-22*
|
||
|
||
## 2026-04-22 (2)
|
||
- Editor: chapter titles are now editable for file-EPUB books as well (DB books already supported this)
|
||
- Frontend (`editor.js`): the `chapter-title-input` is always shown (the read-only `header-chapter` label is hidden for both storage types), title changes mark the chapter dirty, and the title is sent in both `saveChapter` and `saveAllChapters`
|
||
- Backend (`routers/editor.py`): `POST /api/edit/chapter/{index}/{filename}` for file EPUBs now accepts `title` alongside `content`; if the title changed it calls new helper `_update_epub_navpoint_title(path, href, new_title)` which locates the matching NCX `navPoint` by content src and rewrites its `<text>` via `rewrite_epub_entries`
|
||
|
||
---
|
||
|
||
## 2026-04-22
|
||
- Book Info page: new "Info page" button in the editor toolbar generates a gayauthors-style info page and inserts it as the first chapter
|
||
- New endpoint `POST /api/edit/intro/{filename}` builds the page from stored metadata (author, genres, sub-genres, tags, description, source, updated) and prepends it
|
||
- DB books: shifts existing `chapter_index` values up by one and inserts `"Book Info"` at index 0 (two-step negation to avoid unique-constraint violations)
|
||
- File EPUBs: writes a new `intro_<hex>.xhtml` via `make_intro_xhtml`, adds a manifest item, places the `itemref` at the start of the spine, and inserts a navPoint at the top of the NCX with renumbered `playOrder`
|
||
- Empty metadata fields are skipped
|
||
- Option A: no duplicate detection — clicking the button on a book that already has one will add a second page
|
||
- Grabber convert: DB-storage conversions now persist the Book Info page as a real stored chapter at index 0 (so it shows up in the editor and reader); EPUB-storage conversions continue to produce `intro.xhtml` via `make_epub` as before
|
||
- DB → EPUB export (`reader.py`): no longer synthesises `intro.xhtml` (`make_epub(..., include_intro=False)`) — the stored chapter 0 is the single source of truth
|
||
- Legacy DB books converted before this change have no stored info page; their exports will lack an intro page until the button is used
|
||
- `make_epub` gained an `include_intro: bool = True` parameter; when false, the `intro.xhtml` file, its manifest item, its spine `itemref`, and its NCX navPoint are all omitted (remaining `playOrder` values start at 1)
|
||
- Shared helper: `epub_utils.build_book_info_body_html(title, author, info)` — returns the inner-body HTML fragment used for DB storage, starting with `<h1>{title}</h1>`; skips empty fields and separates description and source/updated blocks with `<hr/>`
|
||
- DB-storage info page: chapter title is `"Book Info"`; to preserve the leading `<h1>{book title}</h1>` in its body, the leading-h-tag stripping in `reader.py` (`get_chapter_html` and the DB→EPUB export) is now skipped when `title == "Book Info"`
|
||
|
||
---
|
||
*Released as v0.2.5 on 2026-04-22*
|
||
|
||
## 2026-04-21 (2)
|
||
- Backup: separate "scanned" vs "uploaded" counters in backup_log and UI
|
||
- New migration `backup_log_scanned_files` adds `scanned_files INTEGER` column to `backup_log`
|
||
- `_run_backup_internal` now returns `(scanned_files, uploaded_count, uploaded_size)`
|
||
- `_finish_backup_log(...)` accepts `scanned_files=`; `_run_backup_job` passes the value through
|
||
- `/api/backup/status` and `/api/backup/history` return `uploaded_files` (old `files_count`) and `scanned_files`
|
||
- `backup.html`: Latest Status now shows separate "Scanned" and "Uploaded" rows; History table renamed "Files" column to two columns "Scanned" / "Uploaded" (colspan of loading/empty states updated to 8)
|
||
- Backup page: live phase indicator under the Run buttons while a backup is running
|
||
- New `#run-progress` status line is filled by `pollRunProgress()` which polls `/api/backup/progress` every 3 s
|
||
- Phase labels spelled out: `scanning library`, `uploading library objects`, `uploading snapshot`, `uploading pg_dump (may take minutes)` — so users understand why the counter sits at `N / N` during the final phases
|
||
- Polling starts on page load (via `refreshAll()`) and is re-kicked when Run Live/Dry is clicked; it auto-stops when progress reports `running=false`
|
||
|
||
---
|
||
*Released as v0.2.4 on 2026-04-21*
|
||
|
||
## 2026-04-21
|
||
- Backup: fix Dropbox `Read timed out. (read timeout=120)` error on large uploads
|
||
- `dropbox.Dropbox(...)` `timeout` raised from `120` to `300` in `routers/backup.py` (`_dbx()`, both refresh-token and legacy-token branches)
|
||
- `_DROPBOX_UPLOAD_CHUNK` reduced from `100 * 1024 * 1024` (100 MB) to `16 * 1024 * 1024` (16 MB)
|
||
- `_DROPBOX_UPLOAD_THRESHOLD` lowered to match (`16 * 1024 * 1024`) so the session upload path is used earlier
|
||
- Net effect: each chunk PUT finishes well within the socket timeout, and a stalled connection gets 5 minutes instead of 2 before erroring
|
||
|
||
---
|
||
*Released as v0.2.3 on 2026-04-21*
|
||
|
||
## 2026-04-16 (2)
|
||
- Editor: four inline formatting buttons added to the chapter editor toolbar
|
||
- **S** (subheading) — wraps selection in `<span class="subheading">` (red, bold in reader)
|
||
- **C** (chat) — wraps selection in `<span class="chat">` (orange in reader)
|
||
- **→|** (indent) — wraps selection in `<p style="padding-left: 40px;">` (or `<div>` when the selection contains block elements)
|
||
- **[ ]** (comment) — wraps selection in `<div class="novela-comment">` (blue left border + subtle background in reader)
|
||
- Without a selection: the tag is inserted at the cursor with the cursor positioned inside
|
||
- All four buttons disabled when no chapter is loaded; enabled state mirrors the existing Break button
|
||
- Wrap logic auto-detects block content in the selection: `wrapSpan` and `insertIndent` use a `<div>` wrapper instead of `<span>`/`<p>` when the selection contains `<p>`, `<div>`, `<h*>` etc. to keep the HTML valid
|
||
- Reader CSS extended: `span.subheading`, `span.chat`, `p[style*="padding-left"]`, and `.novela-comment` are styled in `reader.html`
|
||
|
||
## 2026-04-16 (1)
|
||
- Startup: migration logging zichtbaar in Docker logs
|
||
- `logging.basicConfig(level=logging.INFO)` toegevoegd aan `main.py`
|
||
- `migrations.py` logt per migratie of deze overgeslagen of uitgevoerd wordt (met duur in ms)
|
||
- Samenvattingsregel bij afsluiting: "all already applied" of "N executed"
|
||
|
||
---
|
||
*Released as v0.2.1 on 2026-04-16*
|
||
|
||
## 2026-04-15 (4)
|
||
- Scraper: fix encoding for Codey's World
|
||
- Pages are decoded with `cp1252` (Windows-1252) instead of relying on `r.text` or `html.parser` auto-detection via `iso-8859-1`
|
||
- `cp1252` correctly maps the 0x80–0x9F byte range: `…`, `'`, `'`, `"`, `"`, `–`, `—` etc. now render correctly instead of producing replacement characters
|
||
- XHTML: normalize non-breaking spaces globally
|
||
- `\xa0` (HTML ` `) is now replaced with a regular space before HTML-escaping in both `element_to_xhtml` and `normalize_wysiwyg_html`
|
||
- Consecutive spaces resulting from this substitution are collapsed to a single space
|
||
- Applies to all scrapers
|
||
|
||
## 2026-04-15 (3)
|
||
- Editor: chapter add/delete deferred until Save is clicked
|
||
- Adding or deleting a chapter no longer triggers an immediate API call
|
||
- All structural changes are collected in memory and applied (in the correct order) when the Save button is pressed
|
||
- Deletes are applied in reverse server-index order to avoid index shifting errors; new chapters are appended afterwards
|
||
- Fix: `saving` flag is set at the start of a delete operation to prevent a concurrent `saveChapter` from flushing pending changes during the async gap in `loadChapter`
|
||
- Refactor: fix unclosed file handles in `epub.py`
|
||
- `make_epub` and `write_epub_file` now use `Path.read_text()` / `Path.read_bytes()` instead of bare `open()` calls
|
||
- Refactor: eliminate temp file in `pdf_cover_thumb`
|
||
- Cover thumbnail is now generated fully in-memory via `Image.frombytes()` from the PyMuPDF pixmap — removes the race condition when multiple requests hit the same PDF simultaneously
|
||
- Refactor: harden `security.py`
|
||
- Hardcoded fallback encryption key removed; raises `RuntimeError` with a clear message when neither `NOVELA_MASTER_KEY` nor `POSTGRES_PASSWORD` is set
|
||
- Fernet instance cached with `@functools.lru_cache(maxsize=1)` — key derivation runs only once per process
|
||
- Add `GET /health` endpoint
|
||
- Runs `SELECT 1` against the database and returns `{"ok": true/false}`
|
||
- Performance: cap in-memory job dicts at 50 entries
|
||
- `JOBS` in `grabber.py` and `BACKUP_TASKS`/`BACKUP_PROGRESS` in `backup.py` are trimmed (oldest-first) whenever a new entry is added
|
||
- Performance: improve ETag accuracy for the library API
|
||
- ETag now includes `MAX(updated_at)` from `reading_progress` and `MAX(id)` from `book_tags`, so tag changes and progress updates correctly invalidate the client cache
|
||
- Performance: cache CBR/CBZ page list
|
||
- `cbr_page_list` is cached per `(path, mtime)` via `lru_cache(maxsize=64)` — avoids opening the archive twice per page request
|
||
- Refactor: normalize transaction handling in `builder.py`
|
||
- All `conn.commit()` calls replaced with `with conn:` context manager, consistent with the rest of the codebase
|
||
|
||
---
|
||
*Released as v0.2.0 on 2026-04-15*
|
||
|
||
## 2026-04-15 (2)
|
||
- Editor: fix chapter add failing with UniqueViolation on DB-stored books
|
||
- PostgreSQL checks the unique constraint on `(filename, chapter_index)` per row during `UPDATE`, so incrementing consecutive indices in a single statement (e.g. 1→2 while 2 exists) raised a `UniqueViolation`
|
||
- Fixed by using a two-step approach: first shift affected rows to temporary negative values, then to their final positive values
|
||
- Refactor: eliminate duplicated EPUB helper functions across `reader.py`, `editor.py`, `common.py`
|
||
- New `epub_utils.py` with shared `find_opf_path`, `norm_href`, `epub_spine`, `make_new_chapter_xhtml`, `rewrite_epub_entries`
|
||
- Fixes the double-escaped `\\\\s*` regex in the old `_epub_spine` OPF path lookup (was silently falling back to directory scan)
|
||
- `rewrite_epub_entries` combines crash-safe `.tmp.epub` write with `ZIP_STORED` for the `mimetype` entry (EPUB spec requirement)
|
||
- All private `_epub_spine`, `_norm_href`, `_find_opf_path`, `_make_new_chapter_xhtml`, `_rewrite_epub_entries` copies removed from `reader.py`, `editor.py`, `common.py`
|
||
- Migrations: run each migration only once via `schema_migrations` tracking table
|
||
- Eliminates heavy `rebuild_chapter_tsv_with_title` UPDATE running on every container restart
|
||
- Reduces startup from 37 separate DB connections to 1
|
||
|
||
## 2026-04-15 (1)
|
||
- Reader: font size control in reading settings
|
||
- New "Font size" slider (80–150%, default 105%) in the settings drawer, between "Content width" and "Text colour"
|
||
- Applies via CSS custom property `--reader-font-size` on `#chapter-content`
|
||
- Persisted per-device in `localStorage` as `reader-font-size` — iPad and desktop each remember their own preference
|
||
|
||
## 2026-04-13 (1)
|
||
- Edit metadata: comma-separated tag input fix
|
||
- `PillInput._add` in `book.js` now splits the incoming value on commas before adding — each trimmed, non-empty, non-duplicate part is pushed individually
|
||
- Applies to genres, subgenres and plain tags; pasting e.g. `Fiction, Thriller, Adventure` adds 3 separate pills instead of 1
|
||
|
||
---
|
||
*Released as v0.1.11 on 2026-04-13*
|
||
|
||
## 2026-04-12 (2)
|
||
- Series navigation in the reader
|
||
- New `GET /api/series-nav/{filename}` endpoint: returns `{prev, next}` with `{filename, title, index, suffix}` for adjacent books in the same series, ordered by `series_index ASC, series_suffix ASC`; returns `{prev: null, next: null}` for books without a series
|
||
- Reader header now shows prev/next series buttons (skip-to-start/skip-to-end icons); hidden for books without a series, visible once `loadSeriesNav()` resolves
|
||
- Hovering a series button shows a tooltip: `#<index><suffix> <title>` (e.g. `#4 Batman: Year One`)
|
||
- `markRead()` in the reader redirects to `/library/read/{next.filename}` when a next volume exists, so reading continues without leaving the reader; falls back to the book detail page when the series is complete or the book has no series
|
||
|
||
## 2026-04-12 (1)
|
||
- Comics: series_volume support for annual series (issue numbers restart each year)
|
||
- New `series_volume VARCHAR(20)` column on `library` (migration `migrate_series_volume`); default `''`
|
||
- Stored in EPUB OPF as `<meta name="novela:series_volume" content="…"/>`; read back by `scan_epub`
|
||
- `upsert_book` inserts/preserves `series_volume` with the same COALESCE strategy as `series_suffix`
|
||
- `list_library_json` ORDER BY now: `publisher → author → series → series_volume → series_index → title`
|
||
- `PATCH /library/book/{filename}` reads `series_volume` from request body; persists to DB and OPF for both file-based and DB-stored books
|
||
- Book detail page: displays `(year)` after series name when `series_volume` is set (e.g. `Donald Duck (1982) [15]`); edit panel has a new "Year/Volume" input field
|
||
- `book.js`: "Auto" button next to the Title field generates `Series (Year/Vol) #Number` from the current series fields
|
||
- Bulk importer: `series_volume` support
|
||
- New `%series_volume%` placeholder token (orange) for filename pattern parsing
|
||
- New "Year/Vol." shared metadata field (applies to all files; overridden by per-row value or pattern)
|
||
- Preview table has a new "Yr/Vol" column
|
||
- New "Auto-generate titles from series info" checkbox: when enabled, rows without a parsed title get title `Series (Year/Vol) #Number`
|
||
- Skip checkbox now always visible for every row (previously only shown when duplicates were detected); any file can be manually excluded before import; skipped rows are dimmed; stats bar shows "X skipped"
|
||
- `POST /api/bulk-check-duplicates` extended: now also checks `(series, series_index, author)` as fallback — detects duplicates even when the title format has changed; items must include `series` field
|
||
- Library front-end: sorting within a series now respects `series_volume` before `series_index`
|
||
- `groupBySeries`, `renderAuthorDetail`, `renderPublisherDetail` all sort by `series_volume → series_index → series_suffix`
|
||
- `getSeriesSlots` refactored: when any book in the series has `series_volume` set, gap-detection runs per volume (year) independently — prevents `#5 (1982)` and `#5 (1983)` from colliding in the same slot
|
||
- Slot index label shows `(year) #index` for annual series (e.g. `(1982) #5`); unchanged for regular series
|
||
|
||
---
|
||
*Released as v0.1.10 on 2026-04-12*
|
||
|
||
## 2026-04-08 (13)
|
||
- Library: archive series in one action
|
||
- "Archive series" / "Unarchive series" button in the series detail view (`#series/{name}`)
|
||
- New endpoint `POST /library/archive-series` — sets `archived` for all books in the series via a single SQL UPDATE; body: `{"series": "…", "archive": true|false}`; returns `{ok, archived, count}`
|
||
- Button label reflects current state: "Archive series" when any book is active, "Unarchive series" when all are archived
|
||
- After the call, `allBooks` is updated in place and sidebar counters are recalculated without a full page reload
|
||
|
||
## 2026-04-08 (12)
|
||
- TedLouisScraper: title extraction fix
|
||
- `<h2 class="story-page-title">` also contains a "Back" button (`<a class="btn">`) and the author byline (`<span class="story-author-by-line">`)
|
||
- Fix: only direct `NavigableString` children of the h2 are used as the title, so link and span text is skipped
|
||
|
||
## 2026-04-08 (11)
|
||
- New scraper: `TedLouisScraper` (`scrapers/tedlouis.py`) for `tedlouis.com`
|
||
- Matches all `tedlouis.com` URLs; no login required
|
||
- Entry point is the story index page (e.g. `?t=CWYSqpOryu2rQmT1`); raises an error when a chapter URL is used as entry point
|
||
- Title from direct text nodes of `<h2 class="story-page-title">`
|
||
- Author from `<span class="story-author-by-line"> <a>`
|
||
- Status from `<span class="story-status-text">` (strips "Status: " prefix)
|
||
- Updated date from `<span class="story-last-updated">` → `YYYY-MM-DD`
|
||
- Chapters from all `<ul class="story-index-list">` elements (three columns); relative `?t=TOKEN` links resolved to absolute URLs
|
||
- `fetch_chapter()`: content from `<div id="chapter">`; removes story title, chapter title and copyright blocks
|
||
|
||
## 2026-04-08 (10)
|
||
- Settings: break image upload added
|
||
- New "Break image" card on the settings page: upload a PNG/JPG/WebP as the scene break image
|
||
- Stored in the imagestore (sha256-addressed) and overwrites `static/break.png` so EPUB export uses the same image
|
||
- `app_settings` extended with `break_image_sha256` and `break_image_ext` columns (migration `migrate_app_settings_break_image`)
|
||
- New endpoints: `POST /api/app-settings/break-image`; `GET /api/app-settings` now also returns `break_image_url`
|
||
- Preview of the current break image visible on the settings page
|
||
|
||
## 2026-04-08 (9)
|
||
- Grabber: break image fix for DB-stored books
|
||
- Break images (`<center><img src="../Images/break.png">`) contain a relative EPUB path that does not exist in the DB context
|
||
- Fix: `storage_mode` determined earlier in `_run_scrape`; for DB mode `/static/break.png` is passed as `break_img_path` to `element_to_xhtml`, for EPUB mode `../Images/break.png` is kept
|
||
- Additional fix: when a break image is detected in the image loop, the parent (`<center>`) is replaced with `<hr>` instead of decomposing the `<img>`; this allows `element_to_xhtml` to correctly detect the break after the image loop
|
||
|
||
## 2026-04-08 (8)
|
||
- New scraper: `IomfatsScraper` (`scrapers/iomfats.py`) for `iomfats.org`
|
||
- Matches all `iomfats.org` URLs; no login required
|
||
- Entry point is a chapter URL (e.g. `.../grasshopper/justhitsend-part1/justhitsend01.html`); automatically navigates to the author page to fetch all metadata and chapters
|
||
- Author page as entry point: raises a user-visible error message
|
||
- Detects two structures on the author page:
|
||
- Single story: outer `<h3>` = book title; chapters directly in `<ul>`
|
||
- Multi-part series: outer `<h3>` = series name; nested `<li><h3>` = book title per part; chapters in sub-`<ul>`
|
||
- Series index from folder name: `*-part{N}` or `*-{N}` → `series_index_hint`
|
||
- Publication status from `<p><small>[...]</small></p>` after book title
|
||
- `fetch_chapter()`: content via `<div id="content">`; removes `<h2>`/`<h3>` headings, chapternav divs, footer elements
|
||
|
||
## 2026-04-08 (7)
|
||
- New scraper: `CodeysWorldScraper` (`scrapers/codeysworld.py`) for `codeysworld.org`
|
||
- Matches all `codeysworld.org` URLs; no login required
|
||
- Title from `<h1>`; author from `<h2>` ("by …"); fallback to URL slug (`/{author}/{category}/filename`)
|
||
- Category from URL path → tag (e.g. "Remembrances")
|
||
- Single-file stories (no chapter links): the page itself is the only chapter
|
||
- Multi-chapter: links to `.htm`/`.html` files in the same directory (audio/image links skipped)
|
||
- `fetch_chapter()`: removes all `<h1>`/`<h2>` headings, navigation links ("Back to …"), audio links (`.mp3`), mailto links
|
||
- Nifty scraper: category/subcategory moved from `genres`/`subgenres` to `tags`
|
||
|
||
## 2026-04-08 (6)
|
||
- NiftyNewScraper: chapter extraction made more robust
|
||
- `fetch_chapter()` now first tries the standard chapter HTML (`{url}`) and reads `<article><p>` directly
|
||
- Fallback added for Next payloads with escaped paragraphs (`\u003cp...\u003c/p`) via `_extract_escaped_html_paragraphs()`
|
||
- Last fallback remains `?_rsc=1`: first `_parse_rsc_paragraphs()`, then escaped-paragraph fallback again
|
||
- Nifty (classic + new): standard boilerplate no longer visible in reader, but preserved in chapter
|
||
- Lead/tail detection added for common blocks (e.g. `NOTICE This is a work of fiction…`, `If you enjoy this story…`, donate text)
|
||
- Detected intro/closing boilerplate removed from visible paragraphs
|
||
- Removed text stored as invisible HTML comment:
|
||
- `<!-- NIFTY_HIDDEN_LEAD: ... -->`
|
||
- `<!-- NIFTY_HIDDEN_TAIL: ... -->`
|
||
- Detection also works when text contains inline HTML (e.g. `<a>` or `<svg>` in donate links)
|
||
|
||
## 2026-04-08 (5)
|
||
- NiftyNewScraper: chapter content fix
|
||
- `new.nifty.org` uses Next.js RSC — chapter content is not present in the static HTML but in the RSC payload (React component tree)
|
||
- `fetch_chapter()` now fetches `{url}?_rsc=1` instead of the regular HTML page
|
||
- RSC parser added: `_parse_rsc_paragraphs()`, `_rsc_find_paragraphs()`, `_rsc_text()` — parse the RSC stream line by line (format: `{hex_id}:{json}`), recursively search for `["$","p",null,{...}]` nodes and extract text from `children`
|
||
|
||
## 2026-04-08 (4)
|
||
- NiftyNewScraper added (`scrapers/nifty_new.py`) for `new.nifty.org`
|
||
- Matches all `new.nifty.org` URLs; no login required
|
||
- `_to_index_url()`: strips trailing `/N` (chapter number) so both index and chapter URLs can be used as entry point
|
||
- `fetch_book_info()`: title from `<h1>`, author from `<strong itemprop="name">` in author link, dates from `<time itemprop="datePublished/dateModified">`, tags from all `<ul aria-label="Tags">` containers (category links and generated tags, deduplicated), description from `<meta name="description">`, chapter list via `<a>` links matching `/stories/{slug}/N` (RSC stream regex as fallback)
|
||
- `fetch_chapter()`: title from JSON-LD `@type: "Chapter"`, content from `<article>`; no email header stripping, no line joining, no boilerplate detection
|
||
- `NiftyScraper.matches()` updated: excludes `new.nifty.org` (`"nifty.org" in url and "new.nifty.org" not in url`)
|
||
- `NiftyNewScraper` registered before `NiftyScraper` in `scrapers/__init__.py`
|
||
|
||
## 2026-04-08 (3)
|
||
- Settings: develop mode added
|
||
- New `app_settings` table (single row, `id = 1`) with `develop_mode` boolean; created via `migrate_create_app_settings()`
|
||
- `shared_templates.py`: shared `Jinja2Templates` instance for all routers; `develop_mode()` registered as Jinja2 global so all templates can access it without explicit context injection
|
||
- All 11 routers now import `templates` from `shared_templates` instead of each creating their own instance
|
||
- New endpoints in `routers/settings.py`: `GET /api/app-settings` and `PATCH /api/app-settings`
|
||
- Diagonal **DEVELOP** banner in the top-left of every page (CSS in `static/sidebar.css`, HTML in `templates/_sidebar.html`); only visible when develop mode is enabled
|
||
- All 17 HTML templates: `<title>` shows **Novela Develop — …** when develop mode is active
|
||
- Settings page: new card with checkbox toggle; page reloads after saving so banner and title apply immediately
|
||
|
||
## 2026-04-08 (2)
|
||
- Nifty scraper: fix `_strip_email_headers` now tolerates blank lines between header fields
|
||
- Some Nifty pages place `Subject:` after a blank line (`Date:\nFrom:\n\nSubject:\n`) — the previous implementation stopped at the first blank line causing `Subject:` to appear as a paragraph in chapter text
|
||
- Fix: on a blank line, look ahead to check whether a header field still follows; if so, skip the blank line(s) and continue stripping
|
||
|
||
## 2026-04-08 (1)
|
||
- Nifty scraper added (`scrapers/nifty.py`)
|
||
- Matches all `nifty.org` URLs; no login required
|
||
- `fetch_book_info`: accepts index or chapter URL; normalises to index; title from URL slug, author and publication date from email headers of chapter 1, `updated_date` from email headers of the last chapter; genres/subgenres from URL path (`/nifty/{category}/{subcategory}/…`)
|
||
- Boilerplate detection: compares first paragraphs of chapters 1 and 2 after header stripping; matching paragraphs are skipped in all chapters (`preamble_count` per chapter dict)
|
||
- `fetch_chapter`: retrieves `<pre>` content (fallback: body); subject header stored as invisible HTML comment `<!-- Subject: … -->` at the top of chapter content; email headers stripped; hard line breaks within paragraphs joined into a single line; scene breaks (`***`, `---`, etc.) → `<hr/>`
|
||
- Date parsing via `email.utils.parsedate` → `YYYY-MM-DD`
|
||
- `xhtml.py`: `element_to_xhtml` now handles `bs4.Comment` objects as XML comments (`<!-- … -->`); `--` in the body is sanitised to `- -` (illegal in XML comments)
|
||
|
||
## 2026-04-06 (3)
|
||
- Book detail: rating moved from clickable stars to dropdown in the Edit metadata panel
|
||
- Stars on the detail page are now purely visual (no longer clickable)
|
||
- New `<select id="ed-rating">` field in the edit panel, directly below Status
|
||
- `openEdit()` populates the select with the current rating; `saveEdit()` calls `POST /library/rating/…` if the value has changed
|
||
- `rateBook()` function removed from `book.js`
|
||
- Overview pages unchanged
|
||
|
||
## 2026-04-06 (2)
|
||
- Library: cover upload now also supported for DB-stored books
|
||
- `POST /library/cover/{filename}` previously returned an error for DB books (`"File not found"`) because no physical file exists
|
||
- Fix: DB books are now detected via `is_db_filename`; cover is stored directly in `library_cover_cache` and `has_cover = TRUE` set in the `library` table — `add_cover_to_epub` is not called
|
||
|
||
## 2026-04-06 (1)
|
||
- Search: filter for unread novels / unread shorts
|
||
- New `filter` parameter on `GET /api/search?q=…&filter=all|unread_novels|unread_shorts` (default: `all`)
|
||
- `unread_novels`: restricts to books with no reading sessions/progress and no `Shorts` tag
|
||
- `unread_shorts`: restricts to books with no reading sessions/progress and with a `Shorts` tag
|
||
- UI: second toggle row (All / Unread novels / Unread shorts) below the Phrase/All words toggle
|
||
- Filter persisted in URL (`?filter=…`) and restored on page load
|
||
|
||
## 2026-04-05 (4)
|
||
- Backup: large files (> 148 MB) now uploaded via Dropbox upload session in 100 MB chunks
|
||
- `_dropbox_upload_bytes`: files ≤ 148 MB go via `files_upload` (unchanged); larger files via `upload_session_start` → `upload_session_append_v2` → `upload_session_finish`
|
||
- Fixes `ApiError: UploadError('payload_too_large')`
|
||
|
||
## 2026-04-05 (3)
|
||
- Filenames: spaces now replaced with underscores when saving new files
|
||
- `clean_segment` (common.py) and `_clean_segment` (reader.py): `\s+` → `_` instead of space
|
||
- Series separator changed from ` - ` to `_-_` (applies to EPUB, CBR, CBZ and DB filenames)
|
||
- Existing files are not renamed
|
||
|
||
## 2026-04-05 (2)
|
||
- Search: removed result limit; added Phrase / All words mode toggle
|
||
- `LIMIT 30` removed — all matching chapters are returned
|
||
- New `mode` parameter on `GET /api/search?q=…&mode=phrase|words`: `phrase` (default) requires words in order (`phraseto_tsquery`); `words` requires all words present in any order (`plainto_tsquery`)
|
||
- Toggle in the UI (Phrase / All words) above the results; mode is included in the URL
|
||
|
||
## 2026-04-05 (1)
|
||
- Export EPUB: double chapter titles fixed — same heading-stripping logic as the reader now applied before passing content to `make_chapter_xhtml`
|
||
- Library: authors and publishers with only archived books now remain visible in the Authors and Publishers list views
|
||
- `renderAuthorsView` and `renderPublishersView` switched from `activeBooks()` to `allBooks` — consistent with `renderAuthorDetail` and `renderPublisherDetail`
|
||
|
||
## 2026-04-04 (2)
|
||
- Reader: fixed double chapter titles for books where the heading is wrapped in a `<section>` element (pandoc-style)
|
||
- Previous regex only stripped `<h1>`–`<h4>` at the very start of content; pandoc-converted EPUBs wrap headings as `<section …><h1>…</h1>` — heading was not at position 0 so the regex didn't fire
|
||
- Added a second `re.sub` pass that removes the first `<h1>`–`<h4>` found directly after an opening `<section>` or `<div>` tag, preserving the wrapper element
|
||
- Same stripping applied in the DB→EPUB export (`export_epub`) before passing content to `make_chapter_xhtml`
|
||
- Search: switched from `plainto_tsquery` to `phraseto_tsquery` in the FTS WHERE clause
|
||
- `plainto_tsquery` ANDs all words but treats them as independent terms (any order, any distance) — multi-word queries like "4 years later" matched chapters where the words appeared far apart
|
||
- `phraseto_tsquery` requires all words to appear in sequence; `ts_rank` and `ts_headline` still use `plainto_tsquery` for correct scoring and highlighting
|
||
|
||
## 2026-04-04 (1)
|
||
- Reader: fixed double chapter titles in DB-stored books
|
||
- Chapter endpoint (`GET /library/chapter/{index}/{filename}`) now strips all leading `<h1>`–`<h4>` tags from stored content before prepending its own `<h2 class="chapter-title">` — books scraped before front-matter stripping was added to the scraper showed the title (and chapter heading) twice
|
||
- Library: archived books now visible in author and publisher detail views, with an indicator badge on the cover
|
||
- `renderAuthorDetail` and `renderPublisherDetail` reverted to use `allBooks` (including archived) so archived books remain accessible from the author/publisher pages
|
||
- New `.badge-archived` overlay (bottom-left of cover): dark circle with archive icon, consistent with existing `badge-status` and `read-pill` overlays; added to all three card-rendering code paths
|
||
|
||
## 2026-04-03 (3)
|
||
- DB chapter editor: Monaco-based editor now supports DB-stored books
|
||
- `GET /library/editor/{filename}` handles `db/…` filenames; `is_db` flag passed to template
|
||
- `GET /api/edit/chapter/{index}/{filename}` and `POST …`: DB branches query/update `book_chapters` directly; save calls `upsert_chapter` (updates `content_tsv` too)
|
||
- `POST /api/edit/chapter/add/{filename}` and `DELETE …`: DB branches insert/delete with `chapter_index` shift via `UPDATE … SET chapter_index = chapter_index ± 1`
|
||
- Title editing: header chapter-name replaced with a text input for DB books; `pendingTitles` map preserves unsaved titles across chapter switches (parallel to `pendingContent`); title-only dirty chapters correctly saved in Save All
|
||
- `insertBreak`: scene-break image path is `/static/break.png` for DB books (vs `../Images/break.png` for EPUB)
|
||
- Fix: `editor.focus()` called after content load so Monaco receives keyboard focus immediately
|
||
- Fix: `header-chapter` "Loading…" text suppressed for DB books where that element is hidden
|
||
- `book.html`: "Edit chapters" button shown for `storage_type = 'db'` books
|
||
- Search: chapter titles now included in FTS
|
||
- `upsert_chapter` prepends title to the plain-text input for `to_tsvector`: `title + " " + stripped_html`
|
||
- `GET /api/search`: added `OR LOWER(bc.title) LIKE LOWER('%…%')` fallback for chapters whose title matches but content doesn't
|
||
- Startup migration `migrate_rebuild_chapter_tsv_with_title()` rebuilds existing `content_tsv` values to include titles
|
||
- Grabber: added DB/EPUB storage toggle on the Convert page
|
||
- UI toggle above Convert button ("Save as: DB | EPUB file"); `storageMode` JS variable sent in POST body
|
||
- `POST /convert`: reads `storage_mode` from body; stored in job as `'db'` or `'epub'`
|
||
- `_run_scrape`: EPUB path builds chapters via `make_chapter_xhtml`, calls `make_epub`, writes file, calls `upsert_book(storage_type='file')`; DB path unchanged
|
||
- `done` SSE event includes `storage_type`; `conversion.js` updates the download button label/action accordingly
|
||
- EPUB → DB conversion: fixed double chapter title
|
||
- `_epub_body_inner` strips the first `<h1>`/`<h2>`/`<h3>` heading from each chapter body before storing; the editor prepends its own heading, so storing the EPUB heading too caused it to appear twice
|
||
- Fix for `NavigableString` crash: `getattr(child, "name", None) is None` used instead of `hasattr(child, "name")` — `NavigableString` has `name = None` but no `decompose()` method
|
||
- Sidebar: Search link styling fixed
|
||
- Stray `<li>Search</li>` moved inside the Library `<ul class="sidebar-nav">` (was outside, causing incorrect HTML structure)
|
||
- `sidebar.css`: added `a:visited { color: var(--text-dim) }` and `a.active:visited { color: var(--accent) }` to prevent the browser's default purple visited color
|
||
|
||
## 2026-04-03 (2)
|
||
- DB-stored books (Fase 4–6): EPUB→DB conversion, DB→EPUB export, full-text search
|
||
- **Fase 4** — EPUB-to-DB conversion: `POST /api/library/convert-to-db/{filename}` converts an existing on-disk EPUB to a DB-stored book; extracts chapters via `_epub_body_inner` (rewrites img src to imagestore URLs), migrates all child rows (book_tags, reading_progress, reading_sessions, bookmarks, library_cover_cache) to the new `db/…` filename using INSERT→UPDATE→DELETE to respect FK constraints, then deletes the EPUB file
|
||
- **Fase 5** — DB→EPUB export: `GET /api/library/export-epub/{filename}` builds and streams an EPUB from DB content; `_rewrite_db_images_for_epub` rewrites `/library/db-images/…` URLs back to `OEBPS/Images/…` paths, deduplicating by sha256; `Content-Disposition: attachment` response
|
||
- **Fase 6** — Full-text search: new `routers/search.py` with `GET /search` (page) and `GET /api/search?q=…` (FTS over `book_chapters.content_tsv` via `plainto_tsquery('simple', q)`, `ts_headline` for snippets, `ts_rank` for ordering, LIMIT 30, excludes archived); new `templates/search.html` with highlight (`<mark>`), "Read here" link (`?bm_ch=N&bm_scroll=0`), and "Book detail" link; Search entry added to sidebar
|
||
- `book.html`: DB books show "Export EPUB" instead of "Download"; "Edit EPUB" and "Convert to DB" buttons only shown for `.epub` files; delete modal text differs for DB vs file books
|
||
- `PATCH /library/book/{filename}`: DB book branch added — skips file move, recomputes synthetic `db/…` filename via `make_rel_path`, applies same FK-safe rename pattern, updates `book_chapters` and `bookmarks` in addition to standard child tables
|
||
|
||
## 2026-04-03 (1)
|
||
- DB-stored books (Fase 1–3): grabber now stores scraped books in PostgreSQL instead of EPUB files on disk
|
||
- New `book_chapters` table: `filename FK, chapter_index, title, content TEXT, content_tsv TSVECTOR`; GIN index on `content_tsv` for future FTS
|
||
- New `book_images` table: `sha256 PK, ext, media_type, size_bytes`; content-addressed imagestore at `library/images/{sha2}/{sha256}{ext}`
|
||
- New `storage_type VARCHAR(10) DEFAULT 'file'` column on `library`; DB-stored books use `'db'`
|
||
- New utilities in `common.py`: `is_db_filename`, `write_image_file`, `store_db_image`, `html_to_plain`, `upsert_chapter`, `ensure_unique_db_filename`; `make_rel_path` now handles `media_type="db"` → synthetic `db/{pub}/{auth}/...` filename
|
||
- `upsert_book` and `list_library_json` updated to include `storage_type`
|
||
- Grabber: `_run_scrape` stores chapters in `book_chapters`, chapter images in imagestore (absolute `/library/db-images/` URLs embedded in HTML), cover in `library_cover_cache`; no EPUB file written
|
||
- New `GET /library/db-images/{path:path}` endpoint serves imagestore files
|
||
- Reader: `GET /library/chapters/` and `GET /library/chapter/` have DB branches for `storage_type='db'` books (query `book_chapters` directly)
|
||
- Reader page (`/library/read/`), book detail page, mark-read, and rating endpoints all handle DB filenames (no file existence required)
|
||
- Cover endpoints (`/library/cover/`, `/library/cover-cached/`) serve DB books from `library_cover_cache`
|
||
|
||
## 2026-04-02 (1)
|
||
- Added Restore functionality to the Backup page
|
||
- New `GET /api/backup/snapshots` endpoint: lists available Dropbox snapshots (name + date parsed from filename, no downloads needed)
|
||
- New `GET /api/backup/snapshots/{snapshot_name}/files` endpoint: loads a snapshot from Dropbox and returns all files with path, size, sha256, and whether the file currently exists locally
|
||
- New `POST /api/backup/restore` endpoint: downloads file objects from Dropbox, writes to disk, and re-indexes via `scan_media` + `upsert_book`; returns per-file result with errors
|
||
- New "Restore" card on the backup page: snapshot dropdown (auto-loaded on page open), file list with filter/search, per-file "Restore" button, multi-select + "Restore selected", on-disk indicator, inline status feedback
|
||
- After restore, the file list refreshes to reflect updated on-disk state
|
||
|
||
## 2026-03-29 (10)
|
||
- Duplicates: fixed `updateCounts` crashing with a TypeError (`g.books.length` → `g.length`); the crash prevented `renderGrid` from running, so the duplicates view never rendered and the counter was stale
|
||
|
||
## 2026-03-29 (9)
|
||
- Duplicates: volume-aware duplicate detection — a book is only a duplicate when title + author + volume all match
|
||
- `_duplicateGroups` in `library.js` (Duplicates view): key now includes `series_index` when > 0, so different volumes of the same series are no longer grouped as duplicates
|
||
- `preload` in `grabber.py` (Grabber): when the scraper returns a `series_index_hint`, the DB lookup only flags a match when title + author + volume all match; falls back to title + author when no volume is known — same logic as the bulk importer
|
||
|
||
## 2026-03-29 (8)
|
||
- Shared code: eliminated all remaining duplication across templates and JS files
|
||
- CSS custom properties: extracted single `:root { }` block into `static/theme.css`; removed duplicate inline `:root` from all 15 templates and from `library.css`, `book.css` — `editor.css` keeps editor-specific vars (`--danger`, `--header-h`, `--panel-w`), `reader.html` keeps page-specific vars (`--header-h`, `--footer-h`, `--content-w`), `backup.html` keeps (`--ok`, `--warn`, `--err`)
|
||
- Cover helpers: moved `strHash`, `COVER_PALETTES`, `makePlaceholderCover`, `wrapText`, `truncate` from `library.js` and `book.js` into `books.js`; removed from `home.html` and `following.html`
|
||
- HTML escape: `esc()` added to `books.js`; removed from `library.js`, `editor.js`, and all 8 templates that defined it inline
|
||
- SSE/EventSource: extracted shared `connectConversionStream(job_id)` and `addLog()` into new `static/conversion.js`; both `index.html` and `grabber.html` now call the shared function (removed ~70 duplicate lines)
|
||
|
||
## 2026-03-29 (7)
|
||
- Search: extracted shared book helpers and search logic into `static/books.js`
|
||
- `_filenameBase`, `bookTitle`, `bookAuthor`, `tagValuesByType`, `bookGenres`, `bookSubgenres`, `bookPlainTags`, `filterBooks`, `setupSearchInput` moved from `library.js` and `home.html` to a single shared file
|
||
- Both pages now use identical search behaviour: Enter to search, × to clear
|
||
- Both pages now search across title, author, publisher, genre, sub-genre, and tag
|
||
|
||
## 2026-03-29 (6)
|
||
- Search: changed from search-as-you-type (250 ms debounce) to Enter-to-search on both Library and Home — prevents iPad keyboard from locking up on large collections
|
||
|
||
## 2026-03-29 (5)
|
||
- Accent colour updated to match logo orange (`#ffa20e`); secondary accent updated to `#ffb840`
|
||
- Applied across all CSS files, templates, and inline styles
|
||
|
||
## 2026-03-29 (4)
|
||
- Branding: added logo, favicon, and Apple touch icon to all pages
|
||
- Static assets: `logo.png` (sidebar), `favicon.ico` (16×16), `favicon-32.png` (32×32), `favicon-256.png` (256×256), `apple-touch-icon.png` (180×180 with `#0f0e0c` background)
|
||
- Favicon `<link>` tags added to all 15 templates
|
||
- Sidebar: image logo (`logo.png`) placed next to the existing "No**vela**" wordmark using flexbox
|
||
- `apple-touch-icon.png` uses dark `#0f0e0c` background — renders as a native-looking iOS home screen icon
|
||
|
||
## 2026-03-29 (3)
|
||
- Dockerfile: replaced `unrar-free` with proprietary `unrar` (RARLAB v6.2.6) from Debian non-free — fixes "Failed to read enough data" errors on RAR archives using newer compression methods
|
||
|
||
## 2026-03-29 (2)
|
||
- CBR reader: detect archive format via magic bytes instead of file extension — `.cbr` files that are actually ZIP or 7-zip archives now open correctly; added `py7zr` dependency for 7-zip support
|
||
|
||
## 2026-03-29 (1)
|
||
- Bulk Import: duplicate check now volume-aware — books with the same title+author but a different volume number (e.g. recoloured reprints) are no longer flagged as duplicates; `volume` is included in the API call and matched against `series_index` in the DB
|
||
|
||
## 2026-03-28 (11)
|
||
- Bulk Import: duplicate detection against existing library
|
||
- New `POST /api/bulk-check-duplicates` endpoint: case-insensitive title+author+volume match, single SQL query with OR conditions for all pairs
|
||
- Duplicate rows highlighted in red in the preview table; a skip checkbox appears per duplicate row
|
||
- Stats bar shows "X duplicates · Skip all · Import all" action buttons
|
||
- Duplicate rows are skipped by default; user can toggle individual rows or use bulk actions
|
||
- `startImport()` filters out skipped rows before batching; skipped files appear in the result summary as "Duplicate – skipped"
|
||
- If all rows are skipped (nothing to import), result shown immediately without sending any request
|
||
|
||
## 2026-03-28 (10)
|
||
- After "Remove from New" succeeds, the library is now reloaded from the server (`loadLibrary()`) so the New list updates immediately without a manual refresh
|
||
|
||
## 2026-03-28 (9)
|
||
- Bulk delete is now batched: new `POST /library/bulk-delete` endpoint accepts a JSON list of filenames, deletes files and removes DB rows in one query per batch; JS sends 20 files per batch with a progress bar in the confirmation dialog
|
||
|
||
## 2026-03-28 (8)
|
||
- Disk usage warning in sidebar: `GET /api/disk` returns partition usage for the library directory; sidebar polls every 60 s and shows a warning bar (amber ≥ 85% or < 2 GB free, red ≥ 95% or < 500 MB free) above the backup status bar
|
||
|
||
## 2026-03-28 (7)
|
||
- Fixed `mark-reviewed` JS: UI (allBooks update + renderGrid) now only runs after confirmed server success; `catch` no longer fires a false error after a successful response where renderGrid throws
|
||
|
||
## 2026-03-28 (6)
|
||
- Bulk Import: rewrote pattern editor from token-pills to free-text `%placeholder%` syntax
|
||
- Pattern input: free-text field; placeholders: `%series%` `%volume%` `%title%` `%year%` `%month%` `%day%` `%author%` `%publisher%` `%ignore%`
|
||
- Colored chip row: click to insert at cursor position; drag onto input also supported
|
||
- Colored live preview below the pattern input; regex-based parser replaces delimiter+token logic
|
||
- Shared metadata now wins over filename-parsed values (previously filename won)
|
||
- All UI text translated from Dutch to English
|
||
|
||
## 2026-03-28 (5)
|
||
- Status badge (top-right cover) and want-to-read star (top-left cover) now use dark fill `rgba(15,14,12,0.82)` + `box-shadow: 0 0 0 2px #0f0e0c` ring — always readable regardless of cover colour
|
||
|
||
## 2026-03-28 (4)
|
||
- Added `Temporary Hold` status; renamed `Hiatus` → `Long-Term Hold`
|
||
- Startup migration `migrate_rename_hiatus()` converts existing DB records automatically
|
||
- Status colours: Complete=green, Ongoing=blue, Temporary Hold=amber, Long-Term Hold=orange
|
||
- `statusBadgeHtml()` helper in `library.js` replaces three identical inline badge blocks
|
||
- Grabber: `Temporary-Hold` (gayauthors.org) now maps to `Temporary Hold`; `Long-Term Hold` passes through unchanged
|
||
- Status dropdowns updated in Book Detail and Bulk Import
|
||
|
||
## 2026-03-28 (3)
|
||
- Added Bulk Import page (`/bulk-import`) for batch importing CBR/CBZ/EPUB/PDF files with filename-based metadata parsing
|
||
- Configurable token pattern: Serie, Volume, Titel, Jaar, Auteur, Uitgever, Negeer (in any order)
|
||
- Configurable delimiter (default ` - `); year token right-anchored when last so title absorbs overflow segments
|
||
- Live test-parse box: type any filename and see how it parses before selecting files
|
||
- Shared metadata card: author, publisher, status, genres, tags applied to all files (overridden by pattern tokens or manual edits)
|
||
- Preview table: all parsed fields editable via contenteditable cells; warning indicator for rows with fewer segments than tokens; sorted by filename
|
||
- Batch upload in groups of 5 with progress bar (N / total files processed)
|
||
- Result summary with list of skipped files and reasons
|
||
- New `routers/bulk_import.py`: `GET /bulk-import`, `POST /library/bulk-import`
|
||
- Sidebar link under Tools between Book Builder and Credentials
|
||
|
||
This file tracks changes on the `develop` line.
|
||
`changelog.md` can later be used for release summaries.
|
||
|
||
## 2026-03-28 (2)
|
||
- Performance: library page now loads instantly for large collections (1000+ books)
|
||
- `IntersectionObserver` defers both cover image loading and placeholder canvas drawing until cards enter the viewport — eliminates hundreds of upfront canvas ops that blocked the initial render
|
||
- `ETag` caching on `/library/list`: server returns `304 Not Modified` when nothing changed, client skips JSON parse and re-download
|
||
- Single DOM pass in `renderBooksGrid`, `renderDuplicatesView`, `renderSeriesDetail`: canvas and img set up via `card.querySelector` immediately after `innerHTML`, removing a second iteration with `document.getElementById` per card
|
||
- `book_tags` joined via `json_agg` in the main `list_library_json()` query, eliminating a separate `SELECT * FROM book_tags` query and Python merge loop
|
||
- `loadLibrary` now shows an error message instead of staying stuck on "Loading…" when the fetch or render fails
|
||
|
||
## 2026-03-28 (1)
|
||
- Added Following page (`/following`): track external author URLs outside Library and Tools
|
||
- New `authors` table: `name` (unique), `url`, `created_at`, `updated_at`
|
||
- New `routers/following.py`: `GET /following` page, `GET /api/following` (all authors + URL + book count + last added), `POST /api/following/{name}` (set/clear URL)
|
||
- Sidebar: new Following section between Library and Tools; counter shows number of followed authors
|
||
- Following page: two tabs — Following (authors with URL) and All Authors; inline URL editing with Enter/Escape keyboard support; Visit button opens external URL in a new tab; author name links to library author view
|
||
- Added Incomplete view to Library (`#incomplete`): shows all non-archived books where `publication_status ≠ Complete`; sidebar counter included; entry placed after New in the Library section
|
||
|
||
## 2026-03-27 (1)
|
||
- Convert page: duplicate warning shown after loading metadata when a book with the same title+author already exists in the library; warning includes a link to the existing book; user can still proceed with conversion
|
||
- Library: added Duplicates section to sidebar (between Rated and Statistics); counter shows total number of books that are part of a duplicate group (same title+author, case-insensitive); Duplicates view groups books by title+author with a subheading per group
|
||
- Fixed Duplicates view not loading covers: card renderer now uses the same canvas + two-pass img/`makePlaceholderCover` pattern as `renderBooksGrid`
|
||
|
||
## 2026-03-26 (2)
|
||
- Fixed Book Builder page showing white background: `library.css` added to `builder.html` to load `:root` CSS variables and dark `body` background; all CSS variable references in `builder.css` aligned with library theme names (`--text`, `--surface`, `--surface2`, `--text-dim`, `--border`, `--accent`, `--sidebar`)
|
||
|
||
## 2026-03-26 (1)
|
||
- Added Book Builder: create EPUB books manually from scratch via a WYSIWYG editor
|
||
- New `builder_drafts` table (`id UUID`, `title`, `author`, `publisher`, `source_url`, `chapters JSONB`)
|
||
- `build_epub()` in `epub.py`: builds a standards-compliant EPUB 2.0 ZIP from title/author/publisher/chapters; embeds inline CSS and `break.png` if present
|
||
- `normalize_wysiwyg_html()` in `xhtml.py`: converts contenteditable HTML to EPUB-safe XHTML; handles scene-breaks, `<blockquote class="author-note">`, inline formatting, and `<hr>` → break image
|
||
- `routers/builder.py`: draft CRUD, chapter CRUD (`GET/POST/PUT/DELETE`), normalize preview endpoint, publish endpoint (normalizes all chapters → builds EPUB → writes to `library/epub/` → upserts to DB → deletes draft → redirects to book detail)
|
||
- `templates/builder.html` + `static/builder.js` + `static/builder.css`: index page (new draft form + draft list) and editor (chapter panel, contenteditable pane, toolbar with bold/italic/underline/blockquote/author-note/scene-break/normalize, autosave every 30 s, Ctrl+S, publish button)
|
||
- Book Builder link added to sidebar Tools section (between Convert and Credentials)
|
||
- `.author-note` blockquote style added to `static/epub-style.css`
|
||
|
||
## 2026-03-25 (20)
|
||
- Added Rated section to library sidebar: shows all non-archived books with `rating > 0`, sorted by rating descending then title alphabetically; badge displays total count; navigable via `#rated` URL hash
|
||
|
||
## 2026-03-25 (19)
|
||
- Fixed series index 0 not displaying in series slot view, grid cards, list volume column, and book detail:
|
||
- Series slot labels now show `#0` for all slots when the series has at least one positively-indexed sibling (consistent label height across all cards)
|
||
- Grid card and list volume column use the same cross-book heuristic (`indexedSeriesSet`) to show `[0]` where appropriate
|
||
- Book detail page queries whether any sibling in the same series has `series_index > 0` (`series_is_indexed`) and shows `[0]` accordingly
|
||
|
||
## 2026-03-25 (18)
|
||
- Added autocomplete for Author, Publisher, and Series in the book edit panel: typing shows a filtered dropdown of existing values from the library; selecting a value fills the field (same dropdown styling as genres/tags)
|
||
- Status field now defaults to "Complete" when opening the edit panel for a book that has no status set yet
|
||
|
||
## 2026-03-25 (17)
|
||
- Fixed CBR reader showing only first page: `cbr_page_count` was missing from the `cbr` import in `reader.py`; `/api/cbr/info/` was returning an error, causing `page_count` to fall back to 1 and the Next button to remain disabled
|
||
|
||
## 2026-03-25 (16)
|
||
- Fixed CBR/CBZ reader stuck on loading: added `/api/cbr/info/{filename}` endpoint returning `{page_count}`, and added a CBR/CBZ branch in `reader.html` that mirrors the PDF paged reader (page images served via `/library/cbr/{filename}/{page}`)
|
||
|
||
## 2026-03-25 (15)
|
||
- Added series volume 0 and letter suffix support (e.g. "21a", "21b"):
|
||
- New `series_suffix VARCHAR(10)` column added to `library` table via `migrate_series_suffix`
|
||
- `series_index` lower bound changed from 1 to 0 throughout (index 0 = special/prequel edition)
|
||
- Volume field in the book editor changed from `type="number"` to `type="text"` — accepts "0", "1", "21a", etc.
|
||
- Server parses the combined volume string into `series_index` (INTEGER) + `series_suffix` (VARCHAR) via `parse_volume_str`
|
||
- File naming includes suffix: `021a - Title.epub`
|
||
- `novela:series_suffix` meta tag written to/read from EPUB OPF for persistence across rescans
|
||
- Series detail view: books sorted by `(series_index, series_suffix)`; slot labels show "21a"; index 0 shown as slot when any sibling has index > 0
|
||
- Grid cards, list view Volume column, author/publisher sort all include suffix
|
||
|
||
## 2026-03-25 (14)
|
||
- Added multi-select and bulk delete to `All books` List view:
|
||
- Checkbox column added to the list table (List mode only; Grid mode unchanged)
|
||
- Select all / Clear all / Clear selection controls in the controls bar
|
||
- `Shift+click` range selection on checkboxes
|
||
- `Delete selected` button (red, disabled when nothing is selected) triggers a confirmation dialog; confirmed deletions remove files from disk and database, then reload the library
|
||
|
||
## 2026-03-25 (13)
|
||
- Fixed EPUB cover replacement reverting to original after upload:
|
||
- `GET /library/cover/{filename}` for EPUBs now checks the DB cover cache first (populated on upload) before falling back to extracting from the EPUB file; the cache is also warmed on cache-miss so subsequent requests are fast.
|
||
- `add_cover_to_epub` fully rewritten: locates the OPF via `META-INF/container.xml`, finds the existing cover image in the OPF manifest, removes it from the ZIP, and writes the new cover in the same directory — works for any EPUB directory structure (`OEBPS/`, `EPUB/`, etc.).
|
||
|
||
## 2026-03-25 (12)
|
||
- Fixed chapter images not loading in EPUBs that use a non-standard root directory (e.g. `EPUB/` instead of `OEBPS/`): image paths are now passed as full ZIP paths instead of stripping the root segment, and the image endpoint no longer hardcodes an `OEBPS/` prefix. Case-insensitive fallback retained for EPUBs with mismatched image folder casing.
|
||
|
||
## 2026-03-25 (11)
|
||
- Fixed reader progress bar jumping to 100% immediately when navigating to the second chapter in short EPUBs (2 chapters): changed progress formula denominator from `total - 1` to `total` so 100% is only reached after fully scrolling through the last chapter. Same fix applied to PDF page progress.
|
||
|
||
## 2026-03-25 (10)
|
||
- Added bookmark feature:
|
||
- `bookmarks` table in DB (`migrate_create_bookmarks`): `filename`, `chapter_index`, `scroll_frac`, `chapter_title`, `note`, `created_at`
|
||
- New API endpoints: `GET/POST /library/bookmarks/{filename}`, `PATCH/DELETE /library/bookmarks/{id}`, `GET /api/bookmarks`
|
||
- Bookmark button in reader header (orange, bookmark icon): opens modal with optional note field
|
||
- Bookmark modal closes on Escape, backdrop click, or Cancel
|
||
- Reader supports `?bm_ch=N&bm_scroll=F` URL params to jump directly to bookmarked position (overrides saved progress)
|
||
- Library sidebar: Bookmarks section in Library nav with live count badge
|
||
- Library `#bookmarks` view: card list showing cover, book title, author, chapter, note, date, Go-to and Delete buttons
|
||
|
||
## 2026-03-22
|
||
- Added blueprint/technical documentation structure in `docs/`.
|
||
- Completed router split and bootstrap structure (`main.py`, routers, migrations, DB pool).
|
||
- Expanded media support to EPUB/PDF/CBR/CBZ in import and scan flows.
|
||
- Expanded Home UI with:
|
||
- import dropzone for EPUB/PDF/CBR/CBZ
|
||
- search functionality
|
||
- alignment matching Library (search top-right, dropzone below)
|
||
- Updated Library import texts and drag/drop filtering for multi-format support.
|
||
- Expanded Library `New` view with:
|
||
- `Grid`/`List` toggle
|
||
- column visibility filter in `List`
|
||
- multi-select + bulk `Remove from New`
|
||
- selection only in `List` mode
|
||
- `Shift+click` range selection on checkboxes
|
||
- Added route: `POST /library/new/mark-reviewed` (bulk set `needs_review=false`).
|
||
- Improved library performance:
|
||
- `/api/library` fast-path (no full rescan on every page load)
|
||
- optional `rescan=true` / `include_file_info=true`
|
||
- SQL optimizations in `list_library_json()`
|
||
- additional DB indexes for scale
|
||
- Restored `/api/home` full dataset output:
|
||
- `continue_reading`
|
||
- `shorts_unread`
|
||
- `novels_unread`
|
||
- `shorts_read`
|
||
- `novels_read`
|
||
- Explicitly filtered series books out of Home sections.
|
||
- Corrected Home read ordering: `shorts_read` and `novels_read` now show oldest first (`ORDER BY MAX(read_at) ASC`).
|
||
- Restored Statistics page by returning the full `/api/stats` payload required for charts, favorites, top books, and reading history.
|
||
- Improved backup implementation:
|
||
- Dropbox token stored encrypted in DB
|
||
- Dropbox backup root configurable via web UI and stored encrypted in DB
|
||
- versioned snapshots + object-store deduplication in Dropbox (`library_snapshots` / `library_objects`)
|
||
- configurable snapshot retention (`snapshots to keep`) via backup settings
|
||
- object pruning based on retained snapshots
|
||
- scheduled backup (enable + interval in hours)
|
||
- backup runs as a background process, so page navigation does not block execution
|
||
- stale running state recovery after restart/crash (old running logs marked interrupted/error)
|
||
- dry-run support in the new flow
|
||
- Updated Docker image with `postgresql-client` for `pg_dump`.
|
||
- Multiple test builds pushed to `gitea.oskamp.info/ivooskamp/novela:dev`.
|
||
|
||
## 2026-03-25 (9)
|
||
- Fixed backup progress not visible immediately after clicking Run Live Backup: sidebar `loadBackupProgress()` is now called directly from the backup page after a successful start
|
||
- Fixed backup status timestamp showing wrong relative time (e.g. "1h ago" for a recent backup): server UTC timestamps without timezone suffix are now correctly interpreted as UTC in the browser
|
||
- Fixed backup status bar using browser-default purple link color: now uses `var(--accent)` orange consistent with the rest of the UI
|
||
- Added password manager ignore attributes to App Key and App Secret fields: `data-1p-ignore` (1Password), `data-lpignore="true"` (LastPass), `data-form-type="other"` (1Password v8)
|
||
|
||
## 2026-03-25 (8)
|
||
- Added Dropbox OAuth2 refresh token support (no redirect URI needed):
|
||
- New endpoints: `POST /api/backup/oauth/prepare` (generates auth URL) and `POST /api/backup/oauth/exchange` (exchanges code for refresh token)
|
||
- `_dbx()` now prefers refresh token mode (app key + secret + refresh token) with automatic renewal; falls back to legacy access token for backwards compatibility
|
||
- App key and secret stored encrypted in `credentials` table (`dropbox_app_key`, `dropbox_app_secret`)
|
||
- `DELETE /api/backup/credentials` now also cleans up `dropbox_app_key` and `dropbox_app_secret`
|
||
- Backup page updated with two-step OAuth flow UI (app key/secret → auth URL → paste code)
|
||
- Settings status shows `• refresh token` or `• legacy token` indicator
|
||
|
||
## 2026-03-25 (7)
|
||
- Added backup progress tracking: `GET /api/backup/progress` returns `{running, done, total, phase}`; sidebar shows live file count (e.g. `312 / 652 · uploading`) polling every 3 s while running
|
||
|
||
## 2026-03-25 (6)
|
||
- Fixed `GET /api/backup/health` missing `schedule_enabled` and `schedule_interval_hours` in response (values were read but not returned; Health card showed "disabled" even when schedule was on)
|
||
- Added backup status indicator in sidebar above Rescan library button: coloured dot + status text (OK/running/failed/never), links to `/backup`, auto-refreshes every 8 s while running, shows time-ago for last successful backup
|
||
|
||
## 2026-03-25 (5)
|
||
- One-time path migration: all 571 existing library files moved to correct format-prefixed structure (`epub/`, `pdf/`, `comics/`) and all DB references updated (`migrate_paths.py --execute`)
|
||
- Empty old publisher root directories pruned after migration
|
||
- Recovery script (`recover_decock049.py`) written; confirmed file was added after last Dropbox backup so not recoverable — to be re-imported manually
|
||
|
||
## 2026-03-25 (4)
|
||
- Added {publisher} directory to PDF and CBR/CBZ paths: `pdf/{publisher}/{author}/{title}.pdf`, `comics/{publisher}/{author}/{title}.cbr/cbz` — consistent with EPUB structure
|
||
|
||
## 2026-03-25 (3)
|
||
- Fixed CBZ extension in import: `common.make_rel_path` always generated `.cbr` for CBZ files; now accepts `ext` parameter; `library.py` passes actual suffix so CBZ files land at `comics/{author}/{title}.cbz`
|
||
- Added missing `GET /download/{filename}` endpoint (referenced in book.html but was 404)
|
||
- TECHNICAL.md fully rewritten: added File Storage Paths section, complete endpoint lists for all routers including settings.py, corrected path documentation
|
||
|
||
## 2026-03-25 (2)
|
||
- Fixed PDF metadata editing (PATCH /library/book):
|
||
- `_sync_epub_metadata` is now only called for `.epub` files; PDFs update DB only
|
||
- `_make_rel_path` now includes the format prefix matching import: EPUB → `epub/{publisher}/{author}/…`, PDF → `pdf/{author}/{title}.pdf`, CBR/CBZ → `comics/{author}/{title}{ext}`; previously files were moved outside their format directory on metadata save
|
||
- Fixed PDF reader (infinite loading screen):
|
||
- `reader_page` now passes `format` (epub/pdf/cbr/cbz) to `reader.html`
|
||
- Added `GET /api/pdf/info/{filename}` endpoint returning `{"page_count": N}`
|
||
- `reader.html` branches on `FORMAT`: PDFs render page images from `/library/pdf/{filename}?page=N`, EPUB flow unchanged
|
||
- PDF progress tracked per page; keyboard and button navigation work identically to EPUB
|
||
- `Edit EPUB` button in Book Detail hidden for non-EPUB files
|
||
|
||
## 2026-03-23
|
||
- Added `All books` Grid/List toggle in Library:
|
||
- same columns as `New` view (Publisher, Author, Series, Volume, Title, Has cover, Updated, Genres, Sub-genres, Tags, Status)
|
||
- column visibility filter in `List` mode
|
||
- no selection checkboxes or bulk actions
|
||
- view mode and column visibility persisted separately in `localStorage` (`novela.all.viewMode`, `novela.all.visibleColumns`)
|
||
- Added 1–5 star rating for books:
|
||
- stored in the database (`rating SMALLINT DEFAULT 0`)
|
||
- written to EPUB OPF as `<meta name="novela:rating" content="N"/>` on rating change
|
||
- written to CBZ `ComicInfo.xml` as `<NovelaRating>N</NovelaRating>` on rating change
|
||
- CBR and PDF are DB-only (file format constraints)
|
||
- rating is recovered from file metadata during rescan/DB rebuild (`upsert_book` preserves file rating over default 0)
|
||
- stars displayed under the cover (outside `.book-info`) in all grid views (Library, Home)
|
||
- stars in grid cards are display-only (no click) to prevent accidental taps while scrolling
|
||
- stars in Book Detail are interactive (larger, 1.1rem), clicking same star removes rating
|
||
- star color: amber (`#c8a03a`) for filled, `rgba(200, 160, 58, 0.25)` for unfilled — consistent across Library, Home, and Book Detail
|
||
- Added text colour setting to reader hamburger menu:
|
||
- 5 warm-tone presets from bright (`#e8e2d9`) to dim (`#938d86`)
|
||
- active preset shown with accent-coloured ring
|
||
- choice persisted in `localStorage` (`reader-text-colour`) and restored on next open
|
||
- Increased spacing between hamburger button and back link in reader header (`margin-left: 1rem`) to prevent accidental taps
|
||
- Removed `Cover Missing` auto-tag:
|
||
- tag is no longer added on import, rescan, or grabber download
|
||
- `ensure_cover_missing_tag()` removed from `common.py`, `library.py`, and `grabber.py`
|
||
- startup migration removes all existing `Cover Missing` tags from the database
|
||
- Fixed Tags/Genres/Sub-Genres not saving in book edit panel on desktop:
|
||
- `,` (comma) now acts as a confirmation key alongside Enter in `PillInput`
|
||
- `flush()` added to `PillInput`: any text still in the input field is auto-confirmed when Save is clicked
|
||
- Fixed tag/genre search and tag-pill navigation being broken:
|
||
- `renderGenreView` was filtering on `b.genres` (non-existent field); now uses `bookGenres()`, `bookSubgenres()`, `bookPlainTags()`
|
||
- `renderSearchResults` had the same bug; search now covers title, author, genres, sub-genres, and tags
|
||
- Made reading progress monotonic across devices:
|
||
- `POST /library/progress/{filename}` now rejects any save whose `(chapter_index, scrollFrac)` is not strictly ahead of the stored position (returns `{"ok": true, "skipped": true}`)
|
||
- prevents device A from overwriting further progress saved by device B when switching between devices without closing the book
|
||
- also prevents bookmark-based backward navigation (e.g. jumping back to correct an earlier chapter) from clobbering the furthest-read position
|
||
- progress reset remains via the explicit Read/Unread actions, which clear the `reading_progress` row
|