# Develop Changelog ## 2026-06-01 — Local pre-restore snapshots + token-free / upload restore ### Added - Every restore now writes its **pre-restore safety dump to a local store** on the config volume (`CONFIG_DIR/postgres_dumps/`, named `pre-restore-…`, with snapshot-equal retention), so the database can be restored or rolled back **without a Dropbox token**. Regular backups are intentionally left **Dropbox-only** — production backs up hourly, so writing every backup to disk would fill the volume. - `routers/backup.py`: `LOCAL_DUMP_DIR`, helpers `_save_local_dump`, `_list_local_dumps`, `_enforce_local_dump_retention`, `_resolve_local_dump`; `_run_pg_restore` persists the safety dump locally (and prunes by retention) before the destructive load. `_run_backup_internal` is unchanged (no local copy). - New **Local Database Restore** card on the Backup page with two token-free paths: - Restore from a locally stored dump — `GET /api/backup/local/dumps`, `POST /api/backup/local/restore`. - Upload a `.sql` dump (e.g. downloaded manually from dropbox.com) and restore it — `POST /api/backup/upload-restore`. The upload itself is not persisted; the automatic pre-restore safety snapshot covers the local copy. - Both reuse the safe restore path (pre-restore safety dump + automatic rollback) and require `psql`. - `templates/backup.html`: new card with a local-dump selector, an upload field, double-guarded restore buttons, and `fmtBytes` sizes. ## 2026-06-01 — Full Database Restore: safety dump + automatic rollback ### Fixed - Full Database Restore could leave the database **empty** if the dump failed to load: the restore dropped and recreated the public schema first and only then applied the dump, so any failure during the load (e.g. a header `transaction_timeout` error with `ON_ERROR_STOP`) wiped all data with nothing to fall back to. - `routers/backup.py`: split the load into `_apply_pg_dump`; `_run_pg_restore` now takes a safety `pg_dump` of the current database before the destructive load and, if applying the requested dump fails, automatically rolls back to that safety snapshot. A failed restore therefore no longer leaves an empty or broken database. ## 2026-06-01 — Full Database Restore: tolerate PostgreSQL version mismatch ### Fixed - Full Database Restore failed with `ERROR: unrecognized configuration parameter "transaction_timeout"` when the dump was produced by a newer `pg_dump` (PostgreSQL 17+, which emits `SET transaction_timeout = 0;` in the header) but restored into an older server (<17) that doesn't know that session parameter. The restore ran with `ON_ERROR_STOP=1` and aborted on that harmless header line. - `routers/backup.py` (`_run_pg_restore`): the dump is now applied without `ON_ERROR_STOP`; stderr is inspected afterwards via `_real_restore_errors`, which ignores benign "unrecognized configuration parameter" errors but still fails the restore on any other `ERROR:` line. The schema-reset step keeps `ON_ERROR_STOP=1` (it is our own controlled SQL). ## 2026-06-01 — Backup/Restore: database-stored books are now restorable ### Fixed - Database-stored books (`storage_type='db'`, synthetic `db/...` filenames) could not be restored through the Backup → Restore option. The restore UI only listed and restored files from the on-disk library object store, but db-books have no file on disk — their content lives entirely in PostgreSQL (`book_chapters` + `library` row + `book_tags` + `library_cover_cache`). They were captured only in the full `pg_dump`, which the UI offered no way to restore. As a result db-books never appeared in the restore list and were effectively unrecoverable per-book. ### Added - **Per-book db restore.** The backup writer now serializes each database book to JSON (library row, all chapters, tags, and the cached cover) and stores it in the same content-addressed Dropbox object store used for file books, referenced from the snapshot with a `"storage": "db"` marker. Such books now appear in the Restore table (format **DB**) and can be restored individually; restore re-inserts the library row, chapters, tags and cover into PostgreSQL. Inline chapter images are unaffected — they already live on disk under `library/images/` and are backed up as ordinary files. - `routers/backup.py`: new `_db_book_filenames`, `_serialize_db_book`, `_restore_db_book`, `_entry_storage` helpers; db-books serialized during `_run_backup_internal`; `_download_and_restore` branches on storage type; `/api/backup/snapshots/{name}/files` now reports `storage` and computes `exists_locally` for db-books from the `library` table. - Note: db-books only appear in snapshots created *after* this change. Older backups still contain them only in the `pg_dump` — recover those via Full Database Restore below. - **Full Database Restore.** New Backup-page card and endpoints to restore the entire PostgreSQL database from any Dropbox `pg_dump`. This recovers everything (all db-books, reading progress, tags, settings) from existing backups too. It is destructive: the public schema is dropped and recreated before the dump is applied, so any plain dump (with or without `--clean`) restores cleanly. Guarded behind a double confirmation in the UI. - `routers/backup.py`: `_psql_base_args`, `_run_pg_restore`, `_list_pg_dump_paths`; endpoints `GET /api/backup/postgres/dumps` and `POST /api/backup/postgres/restore`; health endpoint now also reports `psql_available`/`psql_path`. - `templates/backup.html`: db-books shown with **DB** format and "in library / not in library" status in the restore table; new "Full Database Restore" card with dump selector, double-confirm, and `psql` health row. ## 2026-06-01 — Reader: Sepia reading theme ### Added - The reader now offers a **Sepia** theme alongside the existing Dark theme, for easier long-form reading (warm paper background with dark brown text instead of light-on-black). New **Theme** toggle (Dark / Sepia) at the top of the reading settings drawer. - `static/theme.css`: added a `:root[data-theme="sepia"]` palette overriding `--bg`, `--surface`, `--surface2`, `--border`, `--accent`, `--accent2`, `--text`, `--text-dim`, `--text-faint`, `--success`, `--warning`, `--error`. Also added a `--shadow-ring` variable (light hairline in dark, dark hairline in sepia) so swatch rings read correctly in both themes. - `templates/reader.html`: theme toggle buttons + `.theme-btn` styling; an inline head script applies the saved theme before paint to avoid a flash; the Text colour row now has two theme-specific swatch sets (light tints for dark, dark brown tints for sepia), toggled by visibility. - JS: `applyTheme()`/`loadTheme()` set the `data-theme` attribute and persist to `localStorage` (`reader-theme`). Text colour is now stored per theme (`reader-text-colour-dark` / `reader-text-colour-sepia`); the old single `reader-text-colour` key is migrated into the dark slot. `init()` calls `loadTheme()` (which loads the active theme's text colour). - Replaced two hard-coded `rgba(255,255,255,...)` values in the reader CSS (swatch ring, series-button border) with theme variables so they don't show as white halos in sepia. ## 2026-05-31 — Build version in the sidebar ### Added - The sidebar now shows the running build version at the bottom (e.g. `v0.2.11` for releases, `v0.2.11.3` for dev builds), linking to the changelog page. - New `containers/novela/version.py` exposes `display_version()`. The semantic release version stays the single source in `changelog.py` (`CHANGELOG[0]["version"]`); a `BUILD` segment is appended for dev builds. - `shared_templates.py` registers `app_version` as a Jinja global; `_sidebar.html` renders it as a `.sidebar-version` link styled in `sidebar.css`. - `main.py` adds a `/api/version` endpoint returning `{"version": display_version()}`. ## 2026-05-31 — Find & Replace: scope option ### Added - Editor Find & Replace: new **Current chapter only** checkbox in the modal options. When checked, the search/replace runs against the currently open chapter instead of every chapter in the book. - `templates/editor.html`: added the `rp-current` checkbox next to Regex/Case sensitive; modal title changed from "Find & Replace — all chapters" to "Find & Replace" since scope is now selectable. - `static/editor.js` (`replaceInAllChapters()`): reads `rp-current`; builds a `targets` list of either just `currentCh()` or all `chapters`, and iterates that. Shows an error ("No chapter open.") if Current-chapter-only is selected with no open chapter. Result message reads "… in current chapter" for the single-chapter case. - Default is unchecked, so the existing all-chapters behaviour is preserved. --- *Released as v0.2.12 on 2026-06-01* ## 2026-05-10 - Reader: subheading/chat styling now also wins when the wrapper contains block elements with their own color rule (e.g. `
…
`/` `, but `.subheading` wrapping a ` ` lost its color
---
*Released as v0.2.11 on 2026-05-10*
---
*Released as v0.2.10 on 2026-05-10*
## 2026-05-09 (2)
- Editor / Reader: subheading (**S**) and chat (**C**) styling now also applies when the wrapper is a ` `, ` ` (or `
{title}
`; skips empty fields and separates description and source/updated blocks with `
`
- DB-storage info page: chapter title is `"Book Info"`; to preserve the leading `{book title}
` in its body, the leading-h-tag stripping in `reader.py` (`get_chapter_html` and the DB→EPUB export) is now skipped when `title == "Book Info"`
---
*Released as v0.2.5 on 2026-04-22*
## 2026-04-21 (2)
- Backup: separate "scanned" vs "uploaded" counters in backup_log and UI
- New migration `backup_log_scanned_files` adds `scanned_files INTEGER` column to `backup_log`
- `_run_backup_internal` now returns `(scanned_files, uploaded_count, uploaded_size)`
- `_finish_backup_log(...)` accepts `scanned_files=`; `_run_backup_job` passes the value through
- `/api/backup/status` and `/api/backup/history` return `uploaded_files` (old `files_count`) and `scanned_files`
- `backup.html`: Latest Status now shows separate "Scanned" and "Uploaded" rows; History table renamed "Files" column to two columns "Scanned" / "Uploaded" (colspan of loading/empty states updated to 8)
- Backup page: live phase indicator under the Run buttons while a backup is running
- New `#run-progress` status line is filled by `pollRunProgress()` which polls `/api/backup/progress` every 3 s
- Phase labels spelled out: `scanning library`, `uploading library objects`, `uploading snapshot`, `uploading pg_dump (may take minutes)` — so users understand why the counter sits at `N / N` during the final phases
- Polling starts on page load (via `refreshAll()`) and is re-kicked when Run Live/Dry is clicked; it auto-stops when progress reports `running=false`
---
*Released as v0.2.4 on 2026-04-21*
## 2026-04-21
- Backup: fix Dropbox `Read timed out. (read timeout=120)` error on large uploads
- `dropbox.Dropbox(...)` `timeout` raised from `120` to `300` in `routers/backup.py` (`_dbx()`, both refresh-token and legacy-token branches)
- `_DROPBOX_UPLOAD_CHUNK` reduced from `100 * 1024 * 1024` (100 MB) to `16 * 1024 * 1024` (16 MB)
- `_DROPBOX_UPLOAD_THRESHOLD` lowered to match (`16 * 1024 * 1024`) so the session upload path is used earlier
- Net effect: each chunk PUT finishes well within the socket timeout, and a stalled connection gets 5 minutes instead of 2 before erroring
---
*Released as v0.2.3 on 2026-04-21*
## 2026-04-16 (2)
- Editor: four inline formatting buttons added to the chapter editor toolbar
- **S** (subheading) — wraps selection in `` (red, bold in reader)
- **C** (chat) — wraps selection in `` (orange in reader)
- **→|** (indent) — wraps selection in `
` when the selection contains `
`, `
` also contains a "Back" button (``) and the author byline (``) - Fix: only direct `NavigableString` children of the h2 are used as the title, so link and span text is skipped ## 2026-04-08 (11) - New scraper: `TedLouisScraper` (`scrapers/tedlouis.py`) for `tedlouis.com` - Matches all `tedlouis.com` URLs; no login required - Entry point is the story index page (e.g. `?t=CWYSqpOryu2rQmT1`); raises an error when a chapter URL is used as entry point - Title from direct text nodes of `
` - Author from ` ` - Status from `` (strips "Status: " prefix) - Updated date from `` → `YYYY-MM-DD` - Chapters from all `
` elements (three columns); relative `?t=TOKEN` links resolved to absolute URLs - `fetch_chapter()`: content from ``; removes story title, chapter title and copyright blocks
## 2026-04-08 (10)
- Settings: break image upload added
- New "Break image" card on the settings page: upload a PNG/JPG/WebP as the scene break image
- Stored in the imagestore (sha256-addressed) and overwrites `static/break.png` so EPUB export uses the same image
- `app_settings` extended with `break_image_sha256` and `break_image_ext` columns (migration `migrate_app_settings_break_image`)
- New endpoints: `POST /api/app-settings/break-image`; `GET /api/app-settings` now also returns `break_image_url`
- Preview of the current break image visible on the settings page
## 2026-04-08 (9)
- Grabber: break image fix for DB-stored books
- Break images (`
`) contain a relative EPUB path that does not exist in the DB context
- Fix: `storage_mode` determined earlier in `_run_scrape`; for DB mode `/static/break.png` is passed as `break_img_path` to `element_to_xhtml`, for EPUB mode `../Images/break.png` is kept
- Additional fix: when a break image is detected in the image loop, the parent (``) is replaced with `
`; this allows `element_to_xhtml` to correctly detect the break after the image loop
## 2026-04-08 (8)
- New scraper: `IomfatsScraper` (`scrapers/iomfats.py`) for `iomfats.org`
- Matches all `iomfats.org` URLs; no login required
- Entry point is a chapter URL (e.g. `.../grasshopper/justhitsend-part1/justhitsend01.html`); automatically navigates to the author page to fetch all metadata and chapters
- Author page as entry point: raises a user-visible error message
- Detects two structures on the author page:
- Single story: outer `
` instead of decomposing the `
` = book title; chapters directly in `
` - Multi-part series: outer `
` = series name; nested `
` = book title per part; chapters in sub-`
` - Series index from folder name: `*-part{N}` or `*-{N}` → `series_index_hint` - Publication status from ``; removes `
[...]
` after book title - `fetch_chapter()`: content via ``/`
` headings, chapternav divs, footer elements ## 2026-04-08 (7) - New scraper: `CodeysWorldScraper` (`scrapers/codeysworld.py`) for `codeysworld.org` - Matches all `codeysworld.org` URLs; no login required - Title from `
`; author from `
` ("by …"); fallback to URL slug (`/{author}/{category}/filename`) - Category from URL path → tag (e.g. "Remembrances") - Single-file stories (no chapter links): the page itself is the only chapter - Multi-chapter: links to `.htm`/`.html` files in the same directory (audio/image links skipped) - `fetch_chapter()`: removes all `
`/`
` headings, navigation links ("Back to …"), audio links (`.mp3`), mailto links - Nifty scraper: category/subcategory moved from `genres`/`subgenres` to `tags` ## 2026-04-08 (6) - NiftyNewScraper: chapter extraction made more robust - `fetch_chapter()` now first tries the standard chapter HTML (`{url}`) and reads `
` directly - Fallback added for Next payloads with escaped paragraphs (`\u003cp...\u003c/p`) via `_extract_escaped_html_paragraphs()` - Last fallback remains `?_rsc=1`: first `_parse_rsc_paragraphs()`, then escaped-paragraph fallback again - Nifty (classic + new): standard boilerplate no longer visible in reader, but preserved in chapter - Lead/tail detection added for common blocks (e.g. `NOTICE This is a work of fiction…`, `If you enjoy this story…`, donate text) - Detected intro/closing boilerplate removed from visible paragraphs - Removed text stored as invisible HTML comment: - `` - `` - Detection also works when text contains inline HTML (e.g. `` or `