ivooskamp/novela
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Compare commits

base: ivooskamp:main
Branches Tags
ivooskamp:main
ivooskamp:v0.2.8
ivooskamp:v0.2.7
ivooskamp:v0.2.6
ivooskamp:v0.2.5
ivooskamp:v0.2.4
ivooskamp:v0.2.3
ivooskamp:v0.2.2
ivooskamp:v0.2.1
ivooskamp:v0.2.0
ivooskamp:v0.1.12
ivooskamp:v0.1.11
ivooskamp:v0.1.10
ivooskamp:v0.1.9
ivooskamp:v0.1.8
ivooskamp:v0.1.7
ivooskamp:v0.1.6
ivooskamp:v0.1.5
ivooskamp:v0.1.4
ivooskamp:v0.1.3
ivooskamp:v0.1.2
ivooskamp:v0.1.1
ivooskamp:v0.1.0
..
compare: ivooskamp:v0.1.8
Branches Tags
ivooskamp:main
ivooskamp:v0.2.8
ivooskamp:v0.2.7
ivooskamp:v0.2.6
ivooskamp:v0.2.5
ivooskamp:v0.2.4
ivooskamp:v0.2.3
ivooskamp:v0.2.2
ivooskamp:v0.2.1
ivooskamp:v0.2.0
ivooskamp:v0.1.12
ivooskamp:v0.1.11
ivooskamp:v0.1.10
ivooskamp:v0.1.9
ivooskamp:v0.1.8
ivooskamp:v0.1.7
ivooskamp:v0.1.6
ivooskamp:v0.1.5
ivooskamp:v0.1.4
ivooskamp:v0.1.3
ivooskamp:v0.1.2
ivooskamp:v0.1.1
ivooskamp:v0.1.0

No commits in common. "main" and "v0.1.8" have entirely different histories.
main ... v0.1.8

48 changed files with 224 additions and 2796 deletions
Show Stats Expand all files Collapse all files

173
containers/novela/changelog.py
View File

@ -3,179 +3,6 @@ Changelog data for Novela
""" """
CHANGELOG = [ CHANGELOG = [
{
"version": "v0.1.12",
"date": "2026-04-15",
"summary": "Font size slider in the reader settings drawer.",
"sections": [
{
"title": "New features",
"type": "feature",
"changes": [
"Reader: font size slider in the reading settings drawer — adjust text size from 80% to 150%; setting is saved per device so iPad and desktop each remember their own preference",
],
},
],
},
{
"version": "v0.1.11",
"date": "2026-04-13",
"summary": "Comma-separated values in genre, subgenre and tag inputs are now split into individual tags.",
"sections": [
{
"title": "Bug fixes",
"type": "bugfix",
"changes": [
"Edit metadata: pasting or typing a comma-separated list in the genre, subgenre or tag input now adds each value as a separate tag instead of one combined tag",
],
},
],
},
{
"version": "v0.1.10",
"date": "2026-04-12",
"summary": "Series navigation in the reader, series_volume support for annual comics, archive a series in one click, and a TedLouis scraper fix.",
"sections": [
{
"title": "New features",
"type": "feature",
"changes": [
"Reader: prev/next volume buttons in the header for books that are part of a series — buttons appear automatically when the book has adjacent volumes; tooltip shows the volume number and title; marking a book as read redirects directly to the next volume in the reader instead of the book detail page",
"Comics: series_volume field for annual series where issue numbers restart each year (e.g. Donald Duck (1982) [15]) — stored in the database and EPUB OPF; displayed as '(year)' after the series name on the book detail page; sorting respects series_volume before series_index; supported in Bulk Import via %series_volume% placeholder and 'Year/Vol.' shared field",
"Library: archive or unarchive an entire series in one click — 'Archive series' / 'Unarchive series' button in the series detail view; updates all books in the series via a single SQL UPDATE and recalculates sidebar counters without a page reload",
],
},
{
"title": "Bug fixes",
"type": "bugfix",
"changes": [
"TedLouis scraper: title extraction no longer includes the 'Back' button text or the author byline — only direct text nodes of the title heading are used",
],
},
],
},
{
"version": "v0.1.9",
"date": "2026-04-08",
"summary": "Five new scrapers (Nifty, codeysworld.org, iomfats.org, tedlouis.com), break image settings, and bug fixes.",
"sections": [
{
"title": "New features",
"type": "feature",
"changes": [
"New scraper: Nifty.org (classic) — scrapes plain-text email-format stories; email headers stripped, boilerplate paragraphs auto-detected and hidden, scene-break patterns converted to break images",
"New scraper: new.nifty.org — scrapes the Next.js version of Nifty; reads chapter content from RSC payload when the static HTML does not include it; boilerplate detection shared with classic Nifty",
"New scraper: codeysworld.org — single-file and multi-chapter stories; title and author extracted from heading elements; category from URL path stored as tag; navigation links and audio links stripped from chapter content",
"New scraper: iomfats.org — all stories are listed on a single author page; provide any chapter URL and the scraper finds the correct story automatically; supports single stories and multi-part series (series name, book title, and series index derived from the page structure)",
"New scraper: tedlouis.com — all pages use opaque token-based routing (?t=TOKEN); provide the story index URL and the scraper collects all chapter links from the three-column chapter list",
"Settings: break image upload — upload a custom PNG/JPG/WebP to use as the scene break image in all converted books; stored in the imagestore and applied to both DB-stored and EPUB-format books",
"Settings: develop mode toggle — shows a DEVELOP banner and updates the page title across all pages when enabled",
],
},
{
"title": "Bug fixes",
"type": "bugfix",
"changes": [
"Break images were not displayed in DB-stored books — the image path '../Images/break.png' is a relative EPUB path that does not exist for DB content; DB mode now uses '/static/break.png'",
"Break images were silently lost during import — the image was decomposed before element_to_xhtml ran, leaving an empty wrapper; the wrapper is now replaced with <hr> so the break is correctly rendered",
],
},
],
},
{
"version": "v0.1.8",
"date": "2026-04-06",
"summary": "Cover upload for DB-stored books, and rating moved to the Edit metadata panel.",
"sections": [
{
"title": "Bug fixes",
"type": "bugfix",
"changes": [
"Library: cover upload now works for DB-stored books — the upload endpoint previously returned 'File not found' because DB books have no file on disk; the cover is now stored directly in the cover cache",
],
},
{
"title": "Improvements",
"type": "improvement",
"changes": [
"Book detail: rating moved from clickable stars to a dropdown in the Edit metadata panel — avoids touch-input issues on iPad where hover state caused all stars to appear filled",
],
},
],
},
{
"version": "v0.1.7",
"date": "2026-04-06",
"summary": "Search filter for unread novels/shorts, Dropbox chunked upload fix, and underscores in new filenames.",
"sections": [
{
"title": "New feature",
"type": "feature",
"changes": [
"Search: filter on unread novels or unread shorts — a second toggle row (All / Unread novels / Unread shorts) restricts results to books with no reading history; filter is preserved in the URL",
],
},
{
"title": "Bug fixes",
"type": "bugfix",
"changes": [
"Backup: files larger than 148 MB now upload correctly — chunked upload session (100 MB per chunk) replaces the single-call upload that hit Dropbox's payload size limit",
],
},
{
"title": "Improvements",
"type": "improvement",
"changes": [
"File paths: spaces in new filenames are now replaced with underscores (publisher, author, title, series segments); series separator changed from ' - ' to '_-_'",
],
},
],
},
{
"version": "v0.1.6",
"date": "2026-04-05",
"summary": "Bug fixes: double chapter titles in exported EPUBs, and authors/publishers with only archived books now remain visible.",
"sections": [
{
"title": "Bug fixes",
"type": "bugfix",
"changes": [
"Export EPUB: double chapter titles fixed — same heading-stripping logic as the reader now applied before passing content to the chapter builder",
"Library: authors and publishers with only archived books now remain visible in the Authors and Publishers list views",
],
},
],
},
{
"version": "v0.1.5",
"date": "2026-04-04",
"summary": "Bug fixes: double chapter titles in pandoc-style EPUB content, and search now requires words in order (phrase match).",
"sections": [
{
"title": "Bug fixes",
"type": "bugfix",
"changes": [
"Reader: double chapter titles for pandoc-converted books — headings wrapped in a <section> element were not stripped by the previous regex; now also removes the first heading found directly inside an opening <section> or <div>",
"Search: multi-word queries no longer match chapters where the words appear far apart — switched to phraseto_tsquery so all words must appear in order",
],
},
],
},
{
"version": "v0.1.4",
"date": "2026-04-04",
"summary": "Bug fixes: double chapter titles in the reader for DB-stored books, and archived books now shown in author/publisher detail with an indicator badge.",
"sections": [
{
"title": "Bug fixes",
"type": "bugfix",
"changes": [
"Reader: double chapter titles in DB-stored books — the chapter endpoint now strips all leading headings from stored content before prepending its own chapter title; affects books scraped before front-matter stripping was added",
"Library: archived books were missing from author and publisher detail views — detail views now include all books (active and archived); archived books have a badge on their cover so they remain distinguishable",
],
},
],
},
{ {
"version": "v0.1.3", "version": "v0.1.3",
"date": "2026-04-03", "date": "2026-04-03",

30
containers/novela/migrations.py
View File

@ -349,33 +349,6 @@ def migrate_rebuild_chapter_tsv_with_title() -> None:
) )
def migrate_create_app_settings() -> None:
_exec(
"""
CREATE TABLE IF NOT EXISTS app_settings (
id INTEGER PRIMARY KEY DEFAULT 1,
develop_mode BOOLEAN NOT NULL DEFAULT FALSE,
CONSTRAINT single_row CHECK (id = 1)
)
"""
)
_exec("INSERT INTO app_settings (id, develop_mode) VALUES (1, FALSE) ON CONFLICT DO NOTHING")
def migrate_app_settings_break_image() -> None:
_exec("ALTER TABLE app_settings ADD COLUMN IF NOT EXISTS break_image_sha256 VARCHAR(64) DEFAULT NULL")
_exec("ALTER TABLE app_settings ADD COLUMN IF NOT EXISTS break_image_ext VARCHAR(10) DEFAULT NULL")
def migrate_series_volume() -> None:
_exec(
"""
ALTER TABLE library
ADD COLUMN IF NOT EXISTS series_volume VARCHAR(20) NOT NULL DEFAULT ''
"""
)
def run_migrations() -> None: def run_migrations() -> None:
migrate_create_library() migrate_create_library()
migrate_create_book_tags() migrate_create_book_tags()
@ -398,6 +371,3 @@ def run_migrations() -> None:
migrate_create_book_images() migrate_create_book_images()
migrate_create_book_chapters() migrate_create_book_chapters()
migrate_rebuild_chapter_tsv_with_title() migrate_rebuild_chapter_tsv_with_title()
migrate_create_app_settings()
migrate_app_settings_break_image()
migrate_series_volume()

29
containers/novela/routers/backup.py
View File

@ -14,12 +14,13 @@ import httpx
from dropbox.exceptions import ApiError, AuthError from dropbox.exceptions import ApiError, AuthError
from fastapi import APIRouter, Request from fastapi import APIRouter, Request
from fastapi.responses import HTMLResponse from fastapi.responses import HTMLResponse
from shared_templates import templates from fastapi.templating import Jinja2Templates
from db import get_db_conn from db import get_db_conn
from routers.common import scan_media, upsert_book from routers.common import scan_media, upsert_book
from security import decrypt_value, encrypt_value, is_encrypted_value from security import decrypt_value, encrypt_value, is_encrypted_value
templates = Jinja2Templates(directory="templates")
router = APIRouter() router = APIRouter()
LIBRARY_DIR = Path(os.environ.get("LIBRARY_DIR", "library")) LIBRARY_DIR = Path(os.environ.get("LIBRARY_DIR", "library"))
@ -434,36 +435,12 @@ def _ensure_dropbox_dir(client: dropbox.Dropbox, path: str) -> None:
pass pass
_DROPBOX_UPLOAD_CHUNK = 100 * 1024 * 1024 # 100 MB — below the 150 MB files_upload limit
_DROPBOX_UPLOAD_THRESHOLD = 148 * 1024 * 1024 # use session upload above this size
def _dropbox_upload_bytes(client: dropbox.Dropbox, target_path: str, data: bytes) -> int: def _dropbox_upload_bytes(client: dropbox.Dropbox, target_path: str, data: bytes) -> int:
parent = str(Path(target_path).parent).replace("\\", "/") parent = str(Path(target_path).parent).replace("\\", "/")
if not parent.startswith("/"): if not parent.startswith("/"):
parent = "/" + parent parent = "/" + parent
_ensure_dropbox_dir(client, parent) _ensure_dropbox_dir(client, parent)
mode = dropbox.files.WriteMode.overwrite client.files_upload(data, target_path, mode=dropbox.files.WriteMode.overwrite, mute=True)
if len(data) <= _DROPBOX_UPLOAD_THRESHOLD:
client.files_upload(data, target_path, mode=mode, mute=True)
else:
# Chunked upload session for large files
offset = 0
session_id = None
while offset < len(data):
chunk = data[offset : offset + _DROPBOX_UPLOAD_CHUNK]
if session_id is None:
res = client.files_upload_session_start(chunk)
session_id = res.session_id
else:
cursor = dropbox.files.UploadSessionCursor(session_id=session_id, offset=offset)
remaining = len(data) - offset - len(chunk)
if remaining == 0:
commit = dropbox.files.CommitInfo(path=target_path, mode=mode, mute=True)
client.files_upload_session_finish(chunk, cursor, commit)
else:
client.files_upload_session_append_v2(chunk, cursor)
offset += len(chunk)
return len(data) return len(data)

3
containers/novela/routers/builder.py
View File

@ -5,7 +5,7 @@ from pathlib import Path
from fastapi import APIRouter, Request from fastapi import APIRouter, Request
from fastapi.responses import HTMLResponse, JSONResponse, RedirectResponse from fastapi.responses import HTMLResponse, JSONResponse, RedirectResponse
from shared_templates import templates from fastapi.templating import Jinja2Templates
from db import get_db_conn from db import get_db_conn
from epub import build_epub from epub import build_epub
@ -13,6 +13,7 @@ from routers.common import LIBRARY_DIR, make_rel_path, upsert_book
from xhtml import normalize_wysiwyg_html from xhtml import normalize_wysiwyg_html
router = APIRouter() router = APIRouter()
templates = Jinja2Templates(directory="templates")
# ── Helpers ─────────────────────────────────────────────────────────────────── # ── Helpers ───────────────────────────────────────────────────────────────────

5
containers/novela/routers/bulk_import.py
View File

@ -4,7 +4,7 @@ from pathlib import Path
from fastapi import APIRouter, File, Form, Request, UploadFile from fastapi import APIRouter, File, Form, Request, UploadFile
from fastapi.responses import HTMLResponse, JSONResponse from fastapi.responses import HTMLResponse, JSONResponse
from shared_templates import templates from fastapi.templating import Jinja2Templates
from cbr import cbr_page_count from cbr import cbr_page_count
from db import get_db_conn from db import get_db_conn
@ -18,6 +18,7 @@ from routers.common import (
upsert_book, upsert_book,
) )
templates = Jinja2Templates(directory="templates")
router = APIRouter() router = APIRouter()
@ -72,7 +73,6 @@ async def library_bulk_import(
author = (row.get("author") or "").strip() or shared_author author = (row.get("author") or "").strip() or shared_author
publisher = (row.get("publisher") or "").strip() or shared_publisher publisher = (row.get("publisher") or "").strip() or shared_publisher
series = (row.get("series") or "").strip() or shared_data.get("series", "") series = (row.get("series") or "").strip() or shared_data.get("series", "")
series_volume = ((row.get("series_volume") or "").strip() or shared_data.get("series_volume", ""))[:20]
series_index, series_suffix = parse_volume_str(row.get("volume") or "") series_index, series_suffix = parse_volume_str(row.get("volume") or "")
status = (row.get("status") or "").strip() or shared_status status = (row.get("status") or "").strip() or shared_status
@ -121,7 +121,6 @@ async def library_bulk_import(
"series": series, "series": series,
"series_index": series_index, "series_index": series_index,
"series_suffix": series_suffix, "series_suffix": series_suffix,
"series_volume": series_volume if series else "",
"publication_status": status, "publication_status": status,
"publish_date": publish_date, "publish_date": publish_date,
"has_cover": has_cover, "has_cover": has_cover,

3
containers/novela/routers/changelog.py
View File

@ -1,10 +1,11 @@
from fastapi import APIRouter, Request from fastapi import APIRouter, Request
from fastapi.responses import HTMLResponse from fastapi.responses import HTMLResponse
from shared_templates import templates from fastapi.templating import Jinja2Templates
from changelog import CHANGELOG from changelog import CHANGELOG
router = APIRouter() router = APIRouter()
templates = Jinja2Templates(directory="templates")
@router.get("/changelog", response_class=HTMLResponse) @router.get("/changelog", response_class=HTMLResponse)

28
containers/novela/routers/common.py
View File

@ -30,9 +30,9 @@ def is_db_filename(filename: str) -> bool:
def clean_segment(value: str, fallback: str, max_len: int) -> str: def clean_segment(value: str, fallback: str, max_len: int) -> str:
txt = re.sub(r"\s+", "_", (value or "").strip()) txt = re.sub(r"\s+", " ", (value or "").strip())
txt = re.sub(r'[<>:"/\\|?*\x00-\x1f]', "", txt) txt = re.sub(r'[<>:"/\\|?*\x00-\x1f]', "", txt)
txt = re.sub(r"\.+$", "", txt).strip("_") txt = re.sub(r"\.+$", "", txt).strip()
return (txt or fallback)[:max_len] return (txt or fallback)[:max_len]
@ -93,7 +93,7 @@ def make_rel_path(*, media_type: str, publisher: str, author: str, title: str, s
if series_name: if series_name:
idx = coerce_series_index(series_index) idx = coerce_series_index(series_index)
sfx = re.sub(r"[^a-z]", "", (series_suffix or "").lower())[:5] sfx = re.sub(r"[^a-z]", "", (series_suffix or "").lower())[:5]
return Path("db") / pub / auth / "Series" / series_name / f"{idx:03d}{sfx}_-_{ttl}" return Path("db") / pub / auth / "Series" / series_name / f"{idx:03d}{sfx} - {ttl}"
return Path("db") / pub / auth / ttl return Path("db") / pub / auth / ttl
if media_type == "epub": if media_type == "epub":
@ -104,7 +104,7 @@ def make_rel_path(*, media_type: str, publisher: str, author: str, title: str, s
if series_name: if series_name:
idx = coerce_series_index(series_index) idx = coerce_series_index(series_index)
sfx = re.sub(r"[^a-z]", "", (series_suffix or "").lower())[:5] sfx = re.sub(r"[^a-z]", "", (series_suffix or "").lower())[:5]
return Path("epub") / pub / auth / "Series" / series_name / f"{idx:03d}{sfx}_-_{ttl}.epub" return Path("epub") / pub / auth / "Series" / series_name / f"{idx:03d}{sfx} - {ttl}.epub"
return Path("epub") / pub / auth / "Stories" / f"{ttl}.epub" return Path("epub") / pub / auth / "Stories" / f"{ttl}.epub"
if media_type == "pdf": if media_type == "pdf":
@ -122,7 +122,7 @@ def make_rel_path(*, media_type: str, publisher: str, author: str, title: str, s
if series_name: if series_name:
idx = coerce_series_index(series_index) idx = coerce_series_index(series_index)
sfx = re.sub(r"[^a-z]", "", (series_suffix or "").lower())[:5] sfx = re.sub(r"[^a-z]", "", (series_suffix or "").lower())[:5]
return Path("comics") / pub / auth / "Series" / series_name / f"{idx:03d}{sfx}_-_{ttl}{comics_ext}" return Path("comics") / pub / auth / "Series" / series_name / f"{idx:03d}{sfx} - {ttl}{comics_ext}"
return Path("comics") / pub / auth / f"{ttl}{comics_ext}" return Path("comics") / pub / auth / f"{ttl}{comics_ext}"
@ -239,7 +239,6 @@ def scan_epub(path: Path) -> dict:
"series": "", "series": "",
"series_index": 0, "series_index": 0,
"series_suffix": "", "series_suffix": "",
"series_volume": "",
"title": "", "title": "",
"publication_status": "", "publication_status": "",
"author": "", "author": "",
@ -281,9 +280,6 @@ def scan_epub(path: Path) -> dict:
m = re.search(r'<meta[^>]*name="novela:series_suffix"[^>]*content="([^"]+)"', opf, re.IGNORECASE) m = re.search(r'<meta[^>]*name="novela:series_suffix"[^>]*content="([^"]+)"', opf, re.IGNORECASE)
if m: if m:
out["series_suffix"] = re.sub(r"[^a-z]", "", m.group(1).lower())[:5] out["series_suffix"] = re.sub(r"[^a-z]", "", m.group(1).lower())[:5]
m = re.search(r'<meta[^>]*name="novela:series_volume"[^>]*content="([^"]+)"', opf, re.IGNORECASE)
if m:
out["series_volume"] = _html.unescape(m.group(1).strip())[:20]
m = re.search(r'<meta[^>]*name="publication_status"[^>]*content="([^"]+)"', opf, re.IGNORECASE) m = re.search(r'<meta[^>]*name="publication_status"[^>]*content="([^"]+)"', opf, re.IGNORECASE)
if m: if m:
out["publication_status"] = _html.unescape(m.group(1).strip()) out["publication_status"] = _html.unescape(m.group(1).strip())
@ -362,9 +358,9 @@ def upsert_book(conn, filename: str, meta: dict, tags: list[tuple[str, str]] | N
cur.execute( cur.execute(
""" """
INSERT INTO library (filename, media_type, storage_type, title, author, publisher, has_cover, INSERT INTO library (filename, media_type, storage_type, title, author, publisher, has_cover,
series, series_index, series_suffix, series_volume, publication_status, source_url, series, series_index, series_suffix, publication_status, source_url,
publish_date, description, needs_review, want_to_read, rating, updated_at) publish_date, description, needs_review, want_to_read, rating, updated_at)
VALUES (%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, FALSE, %s, NOW()) VALUES (%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, FALSE, %s, NOW())
ON CONFLICT (filename) DO UPDATE SET ON CONFLICT (filename) DO UPDATE SET
media_type = EXCLUDED.media_type, media_type = EXCLUDED.media_type,
storage_type = EXCLUDED.storage_type, storage_type = EXCLUDED.storage_type,
@ -375,7 +371,6 @@ def upsert_book(conn, filename: str, meta: dict, tags: list[tuple[str, str]] | N
series = COALESCE(NULLIF(EXCLUDED.series, ''), library.series), series = COALESCE(NULLIF(EXCLUDED.series, ''), library.series),
series_index = CASE WHEN COALESCE(EXCLUDED.series_index, 0) > 0 THEN EXCLUDED.series_index ELSE library.series_index END, series_index = CASE WHEN COALESCE(EXCLUDED.series_index, 0) > 0 THEN EXCLUDED.series_index ELSE library.series_index END,
series_suffix = COALESCE(NULLIF(EXCLUDED.series_suffix, ''), library.series_suffix), series_suffix = COALESCE(NULLIF(EXCLUDED.series_suffix, ''), library.series_suffix),
series_volume = COALESCE(NULLIF(EXCLUDED.series_volume, ''), library.series_volume),
publication_status = COALESCE(NULLIF(EXCLUDED.publication_status, ''), library.publication_status), publication_status = COALESCE(NULLIF(EXCLUDED.publication_status, ''), library.publication_status),
source_url = COALESCE(NULLIF(EXCLUDED.source_url, ''), library.source_url), source_url = COALESCE(NULLIF(EXCLUDED.source_url, ''), library.source_url),
publish_date = COALESCE(EXCLUDED.publish_date, library.publish_date), publish_date = COALESCE(EXCLUDED.publish_date, library.publish_date),
@ -394,7 +389,6 @@ def upsert_book(conn, filename: str, meta: dict, tags: list[tuple[str, str]] | N
meta.get("series", ""), meta.get("series", ""),
meta.get("series_index", 0), meta.get("series_index", 0),
meta.get("series_suffix", ""), meta.get("series_suffix", ""),
meta.get("series_volume", ""),
meta.get("publication_status", ""), meta.get("publication_status", ""),
meta.get("source_url", ""), meta.get("source_url", ""),
meta.get("publish_date") or None, meta.get("publish_date") or None,
@ -440,7 +434,6 @@ def list_library_json() -> list[dict]:
l.rating, l.rating,
COALESCE(l.series_suffix, '') AS series_suffix, COALESCE(l.series_suffix, '') AS series_suffix,
COALESCE(l.storage_type, 'file') AS storage_type, COALESCE(l.storage_type, 'file') AS storage_type,
COALESCE(l.series_volume, '') AS series_volume,
json_agg( json_agg(
json_build_object('tag', bt.tag, 'tag_type', bt.tag_type) json_build_object('tag', bt.tag, 'tag_type', bt.tag_type)
) FILTER (WHERE bt.tag IS NOT NULL) AS tags ) FILTER (WHERE bt.tag IS NOT NULL) AS tags
@ -458,8 +451,8 @@ def list_library_json() -> list[dict]:
l.archived, l.needs_review, l.updated_at, l.archived, l.needs_review, l.updated_at,
rp.progress, rp.cfi, rp.page, rp.progress, rp.cfi, rp.page,
rs.read_count, rs.last_read, rs.read_count, rs.last_read,
cc.filename, l.rating, l.series_suffix, l.storage_type, l.series_volume cc.filename, l.rating, l.series_suffix, l.storage_type
ORDER BY COALESCE(l.publisher, ''), COALESCE(l.author, ''), COALESCE(l.series, ''), COALESCE(l.series_volume, ''), l.series_index, COALESCE(l.title, '') ORDER BY COALESCE(l.publisher, ''), COALESCE(l.author, ''), COALESCE(l.series, ''), l.series_index, COALESCE(l.title, '')
""" """
) )
rows = cur.fetchall() rows = cur.fetchall()
@ -478,7 +471,6 @@ def list_library_json() -> list[dict]:
"series": r[6] or "", "series": r[6] or "",
"series_index": r[7] or 0, "series_index": r[7] or 0,
"series_suffix": r[20] or "", "series_suffix": r[20] or "",
"series_volume": r[22] or "",
"publication_status": r[8] or "", "publication_status": r[8] or "",
"want_to_read": bool(r[9]), "want_to_read": bool(r[9]),
"archived": bool(r[10]), "archived": bool(r[10]),
@ -490,7 +482,7 @@ def list_library_json() -> list[dict]:
"read_count": r[16] or 0, "read_count": r[16] or 0,
"last_read": r[17].isoformat() if r[17] else None, "last_read": r[17].isoformat() if r[17] else None,
"storage_type": r[21] or "file", "storage_type": r[21] or "file",
"tags": r[23] or [], "tags": r[22] or [],
"rating": r[19] or 0, "rating": r[19] or 0,
} }
) )

3
containers/novela/routers/editor.py
View File

@ -8,13 +8,14 @@ from pathlib import Path
from bs4 import BeautifulSoup from bs4 import BeautifulSoup
from fastapi import APIRouter, Request from fastapi import APIRouter, Request
from fastapi.responses import HTMLResponse, JSONResponse, Response from fastapi.responses import HTMLResponse, JSONResponse, Response
from shared_templates import templates from fastapi.templating import Jinja2Templates
from db import get_db_conn from db import get_db_conn
from epub import read_epub_file, write_epub_file from epub import read_epub_file, write_epub_file
from routers.common import LIBRARY_DIR, is_db_filename, resolve_library_path, upsert_chapter from routers.common import LIBRARY_DIR, is_db_filename, resolve_library_path, upsert_chapter
router = APIRouter() router = APIRouter()
templates = Jinja2Templates(directory="templates")
def _norm(base_dir: str, rel: str) -> str: def _norm(base_dir: str, rel: str) -> str:

3
containers/novela/routers/following.py
View File

@ -2,10 +2,11 @@ from urllib.parse import unquote
from fastapi import APIRouter, Request from fastapi import APIRouter, Request
from fastapi.responses import HTMLResponse from fastapi.responses import HTMLResponse
from shared_templates import templates from fastapi.templating import Jinja2Templates
from db import get_db_conn from db import get_db_conn
templates = Jinja2Templates(directory="templates")
router = APIRouter() router = APIRouter()

27
containers/novela/routers/grabber.py
View File

@ -8,10 +8,10 @@ from typing import AsyncGenerator
from urllib.parse import urljoin, urlparse from urllib.parse import urljoin, urlparse
import httpx import httpx
from bs4 import BeautifulSoup, NavigableString, Tag from bs4 import Tag
from fastapi import APIRouter, Request from fastapi import APIRouter, Request
from fastapi.responses import HTMLResponse, StreamingResponse from fastapi.responses import HTMLResponse, StreamingResponse
from shared_templates import templates from fastapi.templating import Jinja2Templates
from db import get_db_conn from db import get_db_conn
from epub import detect_image_format, make_chapter_xhtml, make_epub from epub import detect_image_format, make_chapter_xhtml, make_epub
@ -33,6 +33,7 @@ from scrapers.base import HEADERS
from security import decrypt_value, encrypt_value, is_encrypted_value from security import decrypt_value, encrypt_value, is_encrypted_value
from xhtml import configure_break_patterns, element_to_xhtml, is_break_element from xhtml import configure_break_patterns, element_to_xhtml, is_break_element
templates = Jinja2Templates(directory="templates")
router = APIRouter() router = APIRouter()
JOBS: dict[str, dict] = {} JOBS: dict[str, dict] = {}
@ -400,15 +401,6 @@ async def _run_scrape(job_id: str, url: str, username: str, password: str, send)
_load_break_patterns() _load_break_patterns()
storage_mode = job.get("storage_mode", "db")
# Break image path depends on storage mode:
# - EPUB: relative path inside the EPUB ZIP (break.png is embedded)
# - DB: absolute URL served by the static files handler
if storage_mode == "epub":
break_img_path = "../Images/break.png"
else:
break_img_path = "/static/break.png"
# Collect chapters as {title, content_html, images: [(sha256, ext, media_type, size, data)]} # Collect chapters as {title, content_html, images: [(sha256, ext, media_type, size, data)]}
chapters = [] chapters = []
for i, ch in enumerate(book["chapters"], 1): for i, ch in enumerate(book["chapters"], 1):
@ -421,16 +413,6 @@ async def _run_scrape(job_id: str, url: str, username: str, password: str, send)
if content_el: if content_el:
for img_tag in content_el.find_all("img"): for img_tag in content_el.find_all("img"):
if is_break_element(img_tag): if is_break_element(img_tag):
# Replace the parent with <hr> if it contains only
# this image, so element_to_xhtml can detect the break.
parent = img_tag.parent
meaningful = [
c for c in parent.children
if not (isinstance(c, NavigableString) and not c.strip())
]
if len(meaningful) == 1 and parent is not content_el:
parent.replace_with(BeautifulSoup("<hr/>", "html.parser").hr)
else:
img_tag.decompose() img_tag.decompose()
continue continue
src = img_tag.get("src", "") src = img_tag.get("src", "")
@ -467,7 +449,7 @@ async def _run_scrape(job_id: str, url: str, username: str, password: str, send)
filled_p = len(all_p) - empty_p filled_p = len(all_p) - empty_p
empty_p_is_spacer = filled_p > 0 and empty_p >= filled_p * 0.5 empty_p_is_spacer = filled_p > 0 and empty_p >= filled_p * 0.5
for child in content_el.children: for child in content_el.children:
part = element_to_xhtml(child, break_img_path=break_img_path, empty_p_is_spacer=empty_p_is_spacer) part = element_to_xhtml(child, empty_p_is_spacer=empty_p_is_spacer)
if part.strip(): if part.strip():
xhtml_parts.append(part) xhtml_parts.append(part)
@ -482,6 +464,7 @@ async def _run_scrape(job_id: str, url: str, username: str, password: str, send)
job["done"] = True job["done"] = True
return return
storage_mode = job.get("storage_mode", "db")
send("status", {"message": "Saving to library..."}) send("status", {"message": "Saving to library..."})
book_tags = ( book_tags = (

107
containers/novela/routers/library.py
View File

@ -6,7 +6,7 @@ from pathlib import Path
from fastapi import APIRouter, File, Request, UploadFile from fastapi import APIRouter, File, Request, UploadFile
from fastapi.responses import FileResponse, HTMLResponse, JSONResponse, Response from fastapi.responses import FileResponse, HTMLResponse, JSONResponse, Response
from shared_templates import templates from fastapi.templating import Jinja2Templates
from PIL import UnidentifiedImageError from PIL import UnidentifiedImageError
from db import get_db_conn from db import get_db_conn
@ -28,6 +28,7 @@ from routers.common import (
upsert_cover_cache, upsert_cover_cache,
) )
templates = Jinja2Templates(directory="templates")
router = APIRouter() router = APIRouter()
@ -342,6 +343,12 @@ async def library_cover(filename: str):
@router.post("/library/cover/{filename:path}") @router.post("/library/cover/{filename:path}")
async def library_add_cover(filename: str, request: Request): async def library_add_cover(filename: str, request: Request):
full = resolve_library_path(filename)
if full is None or not full.exists():
return {"error": "File not found"}
if media_type_from_suffix(full) != "epub":
return {"error": "Cover upload is only supported for EPUB"}
body = await request.json() body = await request.json()
cover_b64 = body.get("cover_b64", "") cover_b64 = body.get("cover_b64", "")
if not cover_b64: if not cover_b64:
@ -349,35 +356,6 @@ async def library_add_cover(filename: str, request: Request):
try: try:
cover_data = base64.b64decode(cover_b64) cover_data = base64.b64decode(cover_b64)
except Exception as e:
return {"error": str(e)}
if is_db_filename(filename):
with get_db_conn() as conn:
with conn.cursor() as cur:
cur.execute("SELECT 1 FROM library WHERE filename = %s", (filename,))
if not cur.fetchone():
return {"error": "Not found"}
with conn:
with conn.cursor() as cur:
cur.execute(
"UPDATE library SET has_cover = TRUE, updated_at = NOW() WHERE filename = %s",
(filename,),
)
try:
thumb = make_cover_thumb_webp(cover_data)
upsert_cover_cache(conn, filename, "image/webp", thumb)
except (UnidentifiedImageError, OSError, ValueError):
pass
return {"ok": True}
full = resolve_library_path(filename)
if full is None or not full.exists():
return {"error": "File not found"}
if media_type_from_suffix(full) != "epub":
return {"error": "Cover upload is only supported for EPUB"}
try:
add_cover_to_epub(full, cover_data) add_cover_to_epub(full, cover_data)
except Exception as e: except Exception as e:
return {"error": str(e)} return {"error": str(e)}
@ -441,24 +419,6 @@ async def library_archive(filename: str):
return {"ok": True, "archived": val} return {"ok": True, "archived": val}
@router.post("/library/archive-series")
async def library_archive_series(request: Request):
body = await request.json()
series = body.get("series", "")
archive = bool(body.get("archive", True))
if not series:
return {"error": "series is required"}
with get_db_conn() as conn:
with conn:
with conn.cursor() as cur:
cur.execute(
"UPDATE library SET archived = %s, updated_at = NOW() WHERE series = %s",
(archive, series),
)
count = cur.rowcount
return {"ok": True, "archived": archive, "count": count}
@router.post("/library/new/mark-reviewed") @router.post("/library/new/mark-reviewed")
async def library_mark_new_reviewed(request: Request): async def library_mark_new_reviewed(request: Request):
body = await request.json() body = await request.json()
@ -703,59 +663,46 @@ async def bulk_check_duplicates(request: Request):
for item in items: for item in items:
title = item.get("title", "").strip().lower() title = item.get("title", "").strip().lower()
author = item.get("author", "").strip().lower() author = item.get("author", "").strip().lower()
series = item.get("series", "").strip().lower()
vol_str = item.get("volume", "").strip() vol_str = item.get("volume", "").strip()
try: try:
vol_int = int(vol_str) if vol_str else None vol_int = int(vol_str) if vol_str else None
except ValueError: except ValueError:
vol_int = None vol_int = None
parsed.append((title, author, series, vol_int)) parsed.append((title, author, vol_int))
# Fetch all DB rows matching any (title, author) pair
title_author_pairs = list({(t, a) for t, a, _ in parsed if t})
if not title_author_pairs:
return {"duplicates": [False] * len(items)}
with get_db_conn() as conn:
with conn.cursor() as cur:
# Check by title+author
title_author_pairs = list({(t, a) for t, a, s, _ in parsed if t})
existing_with_vol: set = set()
existing_title_author: set = set()
if title_author_pairs:
conditions = " OR ".join( conditions = " OR ".join(
"(LOWER(TRIM(title)) = %s AND LOWER(TRIM(author)) = %s)" for _ in title_author_pairs "(LOWER(TRIM(title)) = %s AND LOWER(TRIM(author)) = %s)" for _ in title_author_pairs
) )
params = [v for pair in title_author_pairs for v in pair] params = [v for pair in title_author_pairs for v in pair]
with get_db_conn() as conn:
with conn.cursor() as cur:
cur.execute( cur.execute(
f"SELECT LOWER(TRIM(title)), LOWER(TRIM(author)), series_index" f"SELECT LOWER(TRIM(title)), LOWER(TRIM(author)), series_index"
f" FROM library WHERE {conditions}", f" FROM library WHERE {conditions}",
params, params,
) )
rows = cur.fetchall() rows = cur.fetchall()
# (title, author, series_index) for volume-aware lookup
existing_with_vol = {(r[0] or "", r[1] or "", r[2]) for r in rows} existing_with_vol = {(r[0] or "", r[1] or "", r[2]) for r in rows}
# (title, author) for volume-less lookup
existing_title_author = {(r[0] or "", r[1] or "") for r in rows} existing_title_author = {(r[0] or "", r[1] or "") for r in rows}
# Check by series+author+series_index (catches title-format changes)
series_author_pairs = list({(s, a) for _, a, s, _ in parsed if s and a})
existing_series_vol: set = set()
if series_author_pairs:
conditions2 = " OR ".join(
"(LOWER(TRIM(series)) = %s AND LOWER(TRIM(author)) = %s)" for _ in series_author_pairs
)
params2 = [v for pair in series_author_pairs for v in pair]
cur.execute(
f"SELECT LOWER(TRIM(series)), LOWER(TRIM(author)), series_index"
f" FROM library WHERE {conditions2}",
params2,
)
existing_series_vol = {(r[0] or "", r[1] or "", r[2]) for r in cur.fetchall()}
duplicates = [] duplicates = []
for title, author, series, vol_int in parsed: for title, author, vol_int in parsed:
if vol_int is not None: if not title:
by_title = (title, author, vol_int) in existing_with_vol if title else False
by_series = (series, author, vol_int) in existing_series_vol if series else False
duplicates.append(by_title or by_series)
elif title:
duplicates.append((title, author) in existing_title_author)
else:
duplicates.append(False) duplicates.append(False)
elif vol_int is not None:
# Volume known: only a duplicate when title+author+volume all match
duplicates.append((title, author, vol_int) in existing_with_vol)
else:
# No volume: duplicate if any title+author match exists
duplicates.append((title, author) in existing_title_author)
return {"duplicates": duplicates} return {"duplicates": duplicates}

92
containers/novela/routers/reader.py
View File

@ -12,7 +12,7 @@ from pathlib import Path
from bs4 import BeautifulSoup from bs4 import BeautifulSoup
from fastapi import APIRouter, Request from fastapi import APIRouter, Request
from fastapi.responses import FileResponse, HTMLResponse, JSONResponse, Response from fastapi.responses import FileResponse, HTMLResponse, JSONResponse, Response
from shared_templates import templates from fastapi.templating import Jinja2Templates
from cbr import cbr_get_page, cbr_page_count from cbr import cbr_get_page, cbr_page_count
from db import get_db_conn from db import get_db_conn
@ -34,6 +34,7 @@ from routers.common import (
) )
router = APIRouter() router = APIRouter()
templates = Jinja2Templates(directory="templates")
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
# EPUB helpers # EPUB helpers
@ -280,7 +281,6 @@ def _sync_epub_metadata(
series: str, series: str,
series_index: int | str | None, series_index: int | str | None,
series_suffix: str = "", series_suffix: str = "",
series_volume: str = "",
subjects: list[str], subjects: list[str],
) -> None: ) -> None:
"""Write edited metadata back into OPF so DB and EPUB stay aligned.""" """Write edited metadata back into OPF so DB and EPUB stay aligned."""
@ -360,11 +360,9 @@ def _sync_epub_metadata(
set_named_meta('calibre:series_index', str(_coerce_series_index(series_index))) set_named_meta('calibre:series_index', str(_coerce_series_index(series_index)))
sfx = re.sub(r"[^a-z]", "", (series_suffix or "").lower())[:5] sfx = re.sub(r"[^a-z]", "", (series_suffix or "").lower())[:5]
set_named_meta('novela:series_suffix', sfx) set_named_meta('novela:series_suffix', sfx)
set_named_meta('novela:series_volume', (series_volume or '').strip()[:20])
else: else:
set_named_meta('calibre:series_index', '') set_named_meta('calibre:series_index', '')
set_named_meta('novela:series_suffix', '') set_named_meta('novela:series_suffix', '')
set_named_meta('novela:series_volume', '')
_rewrite_epub_entries(epub_path, {opf_path: str(opf).encode('utf-8')}) _rewrite_epub_entries(epub_path, {opf_path: str(opf).encode('utf-8')})
@ -400,9 +398,9 @@ def _rewrite_epub_entries(epub_path: Path, updates: dict[str, bytes], remove_pat
def _clean_segment(value: str, fallback: str, max_len: int = 100) -> str: def _clean_segment(value: str, fallback: str, max_len: int = 100) -> str:
txt = re.sub(r"\s+", "_", (value or "").strip()) txt = re.sub(r"\s+", " ", (value or "").strip())
txt = re.sub(r'[<>:"/\\|?*\x00-\x1f]', "", txt) txt = re.sub(r'[<>:"/\\|?*\x00-\x1f]', "", txt)
txt = re.sub(r"\.+$", "", txt).strip("_") txt = re.sub(r"\.+$", "", txt).strip()
if not txt: if not txt:
txt = fallback txt = fallback
return txt[:max_len] return txt[:max_len]
@ -434,7 +432,7 @@ def _make_rel_path(
if series_name: if series_name:
idx = _coerce_series_index(series_index) idx = _coerce_series_index(series_index)
sfx = re.sub(r"[^a-z]", "", (series_suffix or "").lower())[:5] sfx = re.sub(r"[^a-z]", "", (series_suffix or "").lower())[:5]
return Path("epub") / pub / auth / "Series" / series_name / f"{idx:03d}{sfx}_-_{ttl}.epub" return Path("epub") / pub / auth / "Series" / series_name / f"{idx:03d}{sfx} - {ttl}.epub"
return Path("epub") / pub / auth / "Stories" / f"{ttl}.epub" return Path("epub") / pub / auth / "Stories" / f"{ttl}.epub"
if ext == ".pdf": if ext == ".pdf":
@ -447,7 +445,7 @@ def _make_rel_path(
if series_name: if series_name:
idx = _coerce_series_index(series_index) idx = _coerce_series_index(series_index)
sfx = re.sub(r"[^a-z]", "", (series_suffix or "").lower())[:5] sfx = re.sub(r"[^a-z]", "", (series_suffix or "").lower())[:5]
return Path("comics") / pub / auth / "Series" / series_name / f"{idx:03d}{sfx}_-_{ttl}{ext}" return Path("comics") / pub / auth / "Series" / series_name / f"{idx:03d}{sfx} - {ttl}{ext}"
return Path("comics") / pub / auth / f"{ttl}{ext}" return Path("comics") / pub / auth / f"{ttl}{ext}"
@ -537,14 +535,6 @@ async def get_chapter_html(filename: str, index: int):
return Response(status_code=404) return Response(status_code=404)
title, content = row title, content = row
safe_title = _html.escape(title or "") safe_title = _html.escape(title or "")
# Strip leading h-tags from stored content — the endpoint always
# prepends its own <h2 class="chapter-title">, so content scraped
# before front-matter stripping was added would show the title twice.
# Handles two layouts:
# 1. <h1>…</h1> at the very start of content
# 2. <section …>\n<h1>…</h1> (pandoc-style wrapping)
content = re.sub(r'(?si)^(\s*<h[1-4](?:\s[^>]*)?>.*?</h[1-4]>)+\s*', '', content)
content = re.sub(r'(?si)(<(?:section|div)[^>]*>\s*)<h[1-4][^>]*>.*?</h[1-4]>\s*', r'\1', content, count=1)
return Response( return Response(
f'<body><h2 class="chapter-title">{safe_title}</h2>\n{content}\n</body>', f'<body><h2 class="chapter-title">{safe_title}</h2>\n{content}\n</body>',
media_type="text/html", media_type="text/html",
@ -723,8 +713,7 @@ async def book_detail_page(filename: str, request: Request):
SELECT title, author, publisher, has_cover, series, series_index, SELECT title, author, publisher, has_cover, series, series_index,
publication_status, want_to_read, source_url, archived, publish_date, description, publication_status, want_to_read, source_url, archived, publish_date, description,
rating, COALESCE(series_suffix, '') AS series_suffix, rating, COALESCE(series_suffix, '') AS series_suffix,
COALESCE(storage_type, 'file') AS storage_type, COALESCE(storage_type, 'file') AS storage_type
COALESCE(series_volume, '') AS series_volume
FROM library WHERE filename = %s FROM library WHERE filename = %s
""", """,
(filename,), (filename,),
@ -739,7 +728,6 @@ async def book_detail_page(filename: str, request: Request):
"series": lib_row[4] or "", "series": lib_row[4] or "",
"series_index": lib_row[5] or 0, "series_index": lib_row[5] or 0,
"series_suffix": lib_row[13] or "", "series_suffix": lib_row[13] or "",
"series_volume": lib_row[15] or "",
"publication_status": lib_row[6] or "", "publication_status": lib_row[6] or "",
"want_to_read": lib_row[7] or False, "want_to_read": lib_row[7] or False,
"source_url": lib_row[8] or "", "source_url": lib_row[8] or "",
@ -826,7 +814,6 @@ async def book_detail_page(filename: str, request: Request):
"series": entry["series"], "series": entry["series"],
"series_index": entry["series_index"], "series_index": entry["series_index"],
"series_suffix": entry["series_suffix"], "series_suffix": entry["series_suffix"],
"series_volume": entry.get("series_volume", ""),
"genres": genres, "genres": genres,
"subgenres": subgenres, "subgenres": subgenres,
"tags": tags_list, "tags": tags_list,
@ -886,53 +873,6 @@ async def api_suggestions(type: str | None = None):
return JSONResponse([r[0] for r in cur.fetchall()]) return JSONResponse([r[0] for r in cur.fetchall()])
@router.get("/api/series-nav/{filename:path}")
async def api_series_nav(filename: str):
"""Return the previous and next book in the same series, ordered by series_index."""
with get_db_conn() as conn:
with conn.cursor() as cur:
cur.execute(
"SELECT series, series_index, COALESCE(series_suffix, '') FROM library WHERE filename = %s",
(filename,),
)
row = cur.fetchone()
if not row or not row[0]:
return JSONResponse({"prev": None, "next": None})
series, current_index, current_suffix = row
with get_db_conn() as conn:
with conn.cursor() as cur:
cur.execute(
"""
SELECT filename, title, series_index, COALESCE(series_suffix, '')
FROM library
WHERE series = %s AND series IS NOT NULL AND series <> ''
ORDER BY series_index ASC, series_suffix ASC
""",
(series,),
)
siblings = cur.fetchall()
# Find position of current book in ordered list
pos = None
for i, (fn, _title, idx, sfx) in enumerate(siblings):
if fn == filename:
pos = i
break
if pos is None:
return JSONResponse({"prev": None, "next": None})
def entry(row):
return {"filename": row[0], "title": row[1], "index": row[2], "suffix": row[3]}
return JSONResponse({
"prev": entry(siblings[pos - 1]) if pos > 0 else None,
"next": entry(siblings[pos + 1]) if pos < len(siblings) - 1 else None,
})
@router.patch("/library/book/{filename:path}") @router.patch("/library/book/{filename:path}")
async def book_update(filename: str, request: Request): async def book_update(filename: str, request: Request):
"""Update book metadata and tags, and rename/move the file when needed.""" """Update book metadata and tags, and rename/move the file when needed."""
@ -941,7 +881,6 @@ async def book_update(filename: str, request: Request):
author = body.get("author", "") author = body.get("author", "")
publisher = body.get("publisher", "") publisher = body.get("publisher", "")
series = body.get("series", "") series = body.get("series", "")
series_volume = (body.get("series_volume", "") or "").strip()[:20]
from routers.common import parse_volume_str from routers.common import parse_volume_str
series_index, series_suffix = parse_volume_str(body.get("series_index", "")) series_index, series_suffix = parse_volume_str(body.get("series_index", ""))
@ -971,11 +910,11 @@ async def book_update(filename: str, request: Request):
""" """
INSERT INTO library ( INSERT INTO library (
filename, title, author, publisher, has_cover, filename, title, author, publisher, has_cover,
series, series_index, series_suffix, series_volume, publication_status, series, series_index, series_suffix, publication_status,
source_url, publish_date, description, source_url, publish_date, description,
archived, needs_review, storage_type, updated_at archived, needs_review, storage_type, updated_at
) )
VALUES (%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, FALSE, FALSE, 'db', NOW()) VALUES (%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, FALSE, FALSE, 'db', NOW())
ON CONFLICT (filename) DO UPDATE SET ON CONFLICT (filename) DO UPDATE SET
title = EXCLUDED.title, title = EXCLUDED.title,
author = EXCLUDED.author, author = EXCLUDED.author,
@ -983,7 +922,6 @@ async def book_update(filename: str, request: Request):
series = EXCLUDED.series, series = EXCLUDED.series,
series_index = EXCLUDED.series_index, series_index = EXCLUDED.series_index,
series_suffix = EXCLUDED.series_suffix, series_suffix = EXCLUDED.series_suffix,
series_volume = EXCLUDED.series_volume,
publication_status = EXCLUDED.publication_status, publication_status = EXCLUDED.publication_status,
source_url = EXCLUDED.source_url, source_url = EXCLUDED.source_url,
publish_date = EXCLUDED.publish_date, publish_date = EXCLUDED.publish_date,
@ -995,7 +933,6 @@ async def book_update(filename: str, request: Request):
new_filename, title, author, publisher, has_cover, new_filename, title, author, publisher, has_cover,
series, series_index if series else 0, series, series_index if series else 0,
series_suffix if series else "", series_suffix if series else "",
series_volume if series else "",
body.get("publication_status", ""), body.get("publication_status", ""),
body.get("source_url", ""), body.get("source_url", ""),
body.get("publish_date") or None, body.get("publish_date") or None,
@ -1064,7 +1001,6 @@ async def book_update(filename: str, request: Request):
series=series, series=series,
series_index=series_index if series else 0, series_index=series_index if series else 0,
series_suffix=series_suffix if series else "", series_suffix=series_suffix if series else "",
series_volume=series_volume if series else "",
subjects=(body.get("genres", []) + body.get("subgenres", []) + body.get("tags", [])), subjects=(body.get("genres", []) + body.get("subgenres", []) + body.get("tags", [])),
) )
@ -1079,11 +1015,11 @@ async def book_update(filename: str, request: Request):
""" """
INSERT INTO library ( INSERT INTO library (
filename, title, author, publisher, has_cover, filename, title, author, publisher, has_cover,
series, series_index, series_suffix, series_volume, publication_status, series, series_index, series_suffix, publication_status,
source_url, publish_date, description, source_url, publish_date, description,
archived, needs_review, updated_at archived, needs_review, updated_at
) )
VALUES (%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, FALSE, FALSE, NOW()) VALUES (%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, FALSE, FALSE, NOW())
ON CONFLICT (filename) DO UPDATE SET ON CONFLICT (filename) DO UPDATE SET
title = EXCLUDED.title, title = EXCLUDED.title,
author = EXCLUDED.author, author = EXCLUDED.author,
@ -1091,7 +1027,6 @@ async def book_update(filename: str, request: Request):
series = EXCLUDED.series, series = EXCLUDED.series,
series_index = EXCLUDED.series_index, series_index = EXCLUDED.series_index,
series_suffix = EXCLUDED.series_suffix, series_suffix = EXCLUDED.series_suffix,
series_volume = EXCLUDED.series_volume,
publication_status = EXCLUDED.publication_status, publication_status = EXCLUDED.publication_status,
source_url = EXCLUDED.source_url, source_url = EXCLUDED.source_url,
publish_date = EXCLUDED.publish_date, publish_date = EXCLUDED.publish_date,
@ -1108,7 +1043,6 @@ async def book_update(filename: str, request: Request):
series, series,
series_index if series else 0, series_index if series else 0,
series_suffix if series else "", series_suffix if series else "",
series_volume if series else "",
body.get("publication_status", ""), body.get("publication_status", ""),
body.get("source_url", ""), body.get("source_url", ""),
body.get("publish_date") or None, body.get("publish_date") or None,
@ -1444,10 +1378,6 @@ async def export_epub(filename: str):
seen_images: dict[str, str] = {} seen_images: dict[str, str] = {}
chapters = [] chapters = []
for ch_idx, ch_title, ch_content in ch_rows: for ch_idx, ch_title, ch_content in ch_rows:
# Strip leading h-tags from stored content (same logic as chapter endpoint)
# to prevent double titles when make_chapter_xhtml prepends its own heading.
ch_content = re.sub(r'(?si)^(\s*<h[1-4](?:\s[^>]*)?>.*?</h[1-4]>)+\s*', '', ch_content)
ch_content = re.sub(r'(?si)(<(?:section|div)[^>]*>\s*)<h[1-4][^>]*>.*?</h[1-4]>\s*', r'\1', ch_content, count=1)
modified_html, new_imgs = _rewrite_db_images_for_epub(ch_content, seen_images) modified_html, new_imgs = _rewrite_db_images_for_epub(ch_content, seen_images)
chapter_xhtml = make_chapter_xhtml(ch_title or f"Chapter {ch_idx + 1}", modified_html, ch_idx + 1) chapter_xhtml = make_chapter_xhtml(ch_title or f"Chapter {ch_idx + 1}", modified_html, ch_idx + 1)
chapters.append({"title": ch_title or f"Chapter {ch_idx + 1}", "xhtml": chapter_xhtml, "images": new_imgs}) chapters.append({"title": ch_title or f"Chapter {ch_idx + 1}", "xhtml": chapter_xhtml, "images": new_imgs})

37
containers/novela/routers/search.py
View File

@ -2,11 +2,12 @@
from fastapi import APIRouter, Request from fastapi import APIRouter, Request
from fastapi.responses import HTMLResponse, JSONResponse from fastapi.responses import HTMLResponse, JSONResponse
from shared_templates import templates from fastapi.templating import Jinja2Templates
from db import get_db_conn from db import get_db_conn
router = APIRouter() router = APIRouter()
templates = Jinja2Templates(directory="templates")
@router.get("/search", response_class=HTMLResponse) @router.get("/search", response_class=HTMLResponse)
@ -15,38 +16,14 @@ async def search_page(request: Request):
@router.get("/api/search") @router.get("/api/search")
async def api_search(q: str = "", mode: str = "phrase", filter: str = "all"): async def api_search(q: str = ""):
q = q.strip() q = q.strip()
if not q or len(q) > 500: if not q or len(q) > 500:
return JSONResponse([]) return JSONResponse([])
tsquery_fn = "phraseto_tsquery" if mode == "phrase" else "plainto_tsquery"
extra_joins = ""
extra_where = ""
if filter in ("unread_novels", "unread_shorts"):
extra_joins = """
LEFT JOIN reading_sessions rs ON rs.filename = l.filename
LEFT JOIN reading_progress rp ON rp.filename = l.filename"""
extra_where = " AND rs.id IS NULL AND COALESCE(rp.progress, 0) = 0"
if filter == "unread_novels":
extra_where += """
AND NOT EXISTS (
SELECT 1 FROM book_tags bt
WHERE bt.filename = l.filename AND bt.tag = 'Shorts'
AND bt.tag_type IN ('tag', 'subject')
)"""
elif filter == "unread_shorts":
extra_where += """
AND EXISTS (
SELECT 1 FROM book_tags bt
WHERE bt.filename = l.filename AND bt.tag = 'Shorts'
AND bt.tag_type IN ('tag', 'subject')
)"""
with get_db_conn() as conn: with get_db_conn() as conn:
with conn.cursor() as cur: with conn.cursor() as cur:
cur.execute( cur.execute(
f""" """
SELECT SELECT
l.filename, l.filename,
l.title, l.title,
@ -61,13 +38,11 @@ async def api_search(q: str = "", mode: str = "phrase", filter: str = "all"):
ts_rank(bc.content_tsv, plainto_tsquery('simple', %s)) AS rank ts_rank(bc.content_tsv, plainto_tsquery('simple', %s)) AS rank
FROM book_chapters bc FROM book_chapters bc
JOIN library l ON l.filename = bc.filename JOIN library l ON l.filename = bc.filename
{extra_joins} WHERE (bc.content_tsv @@ plainto_tsquery('simple', %s)
WHERE (bc.content_tsv @@ {tsquery_fn}('simple', %s)
OR LOWER(bc.title) LIKE LOWER('%%' || %s || '%%')) OR LOWER(bc.title) LIKE LOWER('%%' || %s || '%%'))
AND NOT l.archived AND NOT l.archived
{extra_where}
GROUP BY l.filename, l.title, l.author, bc.chapter_index, bc.title, bc.content, bc.content_tsv
ORDER BY rank DESC, bc.chapter_index ASC ORDER BY rank DESC, bc.chapter_index ASC
LIMIT 30
""", """,
(q, q, q, q), (q, q, q, q),
) )

58
containers/novela/routers/settings.py
View File

@ -1,13 +1,12 @@
import re import re
from fastapi import APIRouter, Request, UploadFile, File from fastapi import APIRouter, Request
from fastapi.responses import HTMLResponse from fastapi.responses import HTMLResponse
from shared_templates import templates from fastapi.templating import Jinja2Templates
from db import get_db_conn from db import get_db_conn
from epub import detect_image_format
from routers.common import write_image_file
templates = Jinja2Templates(directory="templates")
router = APIRouter() router = APIRouter()
@ -96,57 +95,6 @@ async def delete_break_pattern(pid: int):
return {"ok": True} return {"ok": True}
@router.get("/api/app-settings")
async def get_app_settings():
with get_db_conn() as conn:
with conn.cursor() as cur:
cur.execute("SELECT develop_mode, break_image_sha256, break_image_ext FROM app_settings WHERE id = 1")
row = cur.fetchone()
if not row:
return {"develop_mode": False, "break_image_url": None}
sha, ext = row[1], row[2]
break_image_url = f"/library/db-images/{sha[:2]}/{sha}{ext}" if sha and ext else None
return {"develop_mode": bool(row[0]), "break_image_url": break_image_url}
@router.patch("/api/app-settings")
async def update_app_settings(request: Request):
body = await request.json()
with get_db_conn() as conn:
with conn:
with conn.cursor() as cur:
if "develop_mode" in body:
cur.execute(
"UPDATE app_settings SET develop_mode = %s WHERE id = 1",
(bool(body["develop_mode"]),),
)
return {"ok": True}
@router.post("/api/app-settings/break-image")
async def upload_break_image(file: UploadFile = File(...)):
data = await file.read()
if not data:
return {"error": "Empty file"}
_, media_type = detect_image_format(data, file.filename or "break")
sha, ext, _ = write_image_file(data, media_type)
# Also write to static/break.png so EPUB embeds the same image
try:
with open("static/break.png", "wb") as f:
f.write(data)
except Exception:
pass
with get_db_conn() as conn:
with conn:
with conn.cursor() as cur:
cur.execute(
"UPDATE app_settings SET break_image_sha256 = %s, break_image_ext = %s WHERE id = 1",
(sha, ext),
)
url = f"/library/db-images/{sha[:2]}/{sha}{ext}"
return {"ok": True, "url": url}
@router.delete("/api/reading-history") @router.delete("/api/reading-history")
async def reset_reading_history(): async def reset_reading_history():
with get_db_conn() as conn: with get_db_conn() as conn:

10
containers/novela/scrapers/__init__.py
View File

@ -1,23 +1,13 @@
from .base import BaseScraper from .base import BaseScraper
from .archiveofourown import ArchiveOfOurOwnScraper from .archiveofourown import ArchiveOfOurOwnScraper
from .awesomedude import AwesomeDudeScraper from .awesomedude import AwesomeDudeScraper
from .codeysworld import CodeysWorldScraper
from .gayauthors import GayAuthorsScraper from .gayauthors import GayAuthorsScraper
from .iomfats import IomfatsScraper
from .nifty import NiftyScraper
from .nifty_new import NiftyNewScraper
from .tedlouis import TedLouisScraper
# Register scrapers in priority order (first match wins) # Register scrapers in priority order (first match wins)
_SCRAPERS: list[type[BaseScraper]] = [ _SCRAPERS: list[type[BaseScraper]] = [
ArchiveOfOurOwnScraper, ArchiveOfOurOwnScraper,
AwesomeDudeScraper, AwesomeDudeScraper,
CodeysWorldScraper,
GayAuthorsScraper, GayAuthorsScraper,
IomfatsScraper,
NiftyNewScraper,
NiftyScraper,
TedLouisScraper,
] ]

208
containers/novela/scrapers/codeysworld.py
View File

@ -1,208 +0,0 @@
import re
from urllib.parse import urljoin, urlparse
import httpx
from bs4 import BeautifulSoup
from .base import BaseScraper
CW_BASE = "https://www.codeysworld.org"
LAYOUT_RE = re.compile(
r"nav|menu|sidebar|header|footer|breadcrumb|pagination|"
r"comment|widget|aside|banner|ad|rating|follow|share",
re.I,
)
class CodeysWorldScraper(BaseScraper):
@classmethod
def matches(cls, url: str) -> bool:
return "codeysworld.org" in url
async def login(self, client: httpx.AsyncClient, username: str, password: str) -> bool:
return True # no login required
async def fetch_book_info(self, client: httpx.AsyncClient, url: str) -> dict:
r = await client.get(url)
soup = BeautifulSoup(r.text, "html.parser")
actual_url = str(r.url)
# Title: <h1>
h1 = soup.find("h1")
book_title = h1.get_text(strip=True) if h1 else "Unknown title"
# Author: <h2> "by Author Name"
author = "Unknown author"
h2 = soup.find("h2")
if h2:
text = h2.get_text(strip=True)
m = re.match(r"^by\s+(.+)$", text, re.I)
if m:
author = m.group(1).strip()
# URL path: /{author_slug}/{category}/filename.htm
tags: list[str] = []
path_parts = urlparse(actual_url).path.strip("/").split("/")
if len(path_parts) >= 3:
author_slug = path_parts[-3]
category = path_parts[-2]
elif len(path_parts) >= 2:
author_slug = path_parts[-2]
category = ""
else:
author_slug = ""
category = ""
# Fallback: derive author from URL slug if not found in page
if author == "Unknown author" and author_slug:
author = author_slug.replace("_", " ").title()
# Category → tag
if category and category.lower() not in ("codey", author_slug.lower()):
tags = [category.replace("_", " ").title()]
# Chapter discovery: links to .htm/.html in the same directory,
# excluding the index page itself and audio/image files.
base_dir = actual_url.rsplit("/", 1)[0] + "/"
chapter_links: list[dict] = []
seen: set[str] = set()
for a in soup.find_all("a", href=True):
href = a["href"]
if re.search(r"\.(mp3|mp4|ogg|wav|jpg|jpeg|png|gif)$", href, re.I):
continue
full = urljoin(actual_url, href)
if (
full.startswith(base_dir)
and re.search(r"\.html?(\?.*)?$", full, re.I)
and full.rstrip("/") != actual_url.rstrip("/")
and full not in seen
):
seen.add(full)
text = re.sub(r"\s+", " ", a.get_text(separator=" ")).strip()
chapter_links.append({"url": full, "title": text, "book_title": book_title, "author": author})
if not chapter_links:
# Single-file story
chapter_links = [{"url": actual_url, "title": book_title, "book_title": book_title, "author": author}]
chapter_method = "single_page"
else:
chapter_method = "html_scan"
for i, c in enumerate(chapter_links, 1):
t = c["title"]
if not t or t.lower() == book_title.lower():
c["title"] = f"Chapter {i}"
elif re.match(r"^\d+$", t):
c["title"] = f"Chapter {t}"
return {
"title": book_title,
"author": author,
"publisher": "codeysworld.org",
"series": "",
"series_index_hint": 0,
"genres": [],
"subgenres": [],
"tags": tags,
"description": "",
"updated_date": "",
"publication_status": "",
"source_url": url,
"chapters": chapter_links,
"chapter_method": chapter_method,
"index_image_url": None,
}
async def fetch_chapter(self, client: httpx.AsyncClient, ch: dict) -> dict:
cr = await client.get(ch["url"])
csoup = BeautifulSoup(cr.text, "html.parser")
title = ch["title"]
book_title_lc = ch.get("book_title", "").lower()
author_lc = ch.get("author", "").lower()
# Refine chapter title from an in-page heading,
# skipping the book title and "by Author" headings.
for tag in csoup.find_all(["h1", "h2", "h3"]):
text = re.sub(r"\s+", " ", tag.get_text(separator=" ")).strip()
if not text or len(text) >= 120:
continue
text_lc = text.lower()
if re.search(r"\s+by\s+", text, re.I):
continue
if book_title_lc and book_title_lc in text_lc:
continue
if author_lc and author_lc in text_lc:
continue
title = text
break
# Content extraction: prefer a content-like wrapper; fall back to body.
content_el = (
csoup.find(id=re.compile(r"^(chapter|story|content|text)[_-]?", re.I))
or csoup.find(class_=re.compile(r"story.?text|chapter.?text|post.?content|entry.?content", re.I))
or csoup.find("article")
)
if not content_el:
candidates = [
el for el in csoup.find_all(["div", "article", "section"])
if not re.search(LAYOUT_RE, " ".join(el.get("class", [])))
and not re.search(LAYOUT_RE, el.get("id", ""))
]
if candidates:
content_el = max(candidates, key=lambda el: len(el.get_text(" ", strip=True)))
body = csoup.find("body")
if body:
body_text_len = len(body.get_text(" ", strip=True))
selected_p_count = len(content_el.find_all("p")) if content_el else 0
selected_text_len = len(content_el.get_text(" ", strip=True)) if content_el else 0
# Codeysworld stores story text as direct <p> children of body.
if not content_el or selected_p_count < 3 or selected_text_len < int(body_text_len * 0.35):
content_el = body
if not content_el:
content_el = body
# Strip site boilerplate: headings (title/byline), navigation links,
# audio links and empty nodes — anywhere in the content element.
if content_el:
# Remove all h1/h2 headings (title and "by Author")
for el in content_el.find_all(["h1", "h2"]):
el.decompose()
# Remove navigation links ("Back to …", "Home", etc.)
for el in content_el.find_all("a", href=True):
text = el.get_text(strip=True)
if re.search(r"back\s+to|<{0,2}\s*back|home", text, re.I):
parent = el.parent
el.decompose()
# Remove the parent too if it's now empty
if parent and not parent.get_text(strip=True):
parent.decompose()
# Remove audio links (links to .mp3 files or containing "listen"/"audio")
for el in content_el.find_all("a", href=True):
href = el.get("href", "")
text = el.get_text(strip=True)
if re.search(r"\.mp3$", href, re.I) or re.search(r"listen|audio", text, re.I):
parent = el.parent
el.decompose()
if parent and not parent.get_text(strip=True):
parent.decompose()
# Remove email links ("Email Author")
for el in content_el.find_all("a", href=re.compile(r"^mailto:", re.I)):
parent = el.parent
el.decompose()
if parent and not parent.get_text(strip=True):
parent.decompose()
return {
"title": title,
"content_el": content_el,
"selector_id": content_el.get("id") if content_el else None,
"selector_class": " ".join(content_el.get("class", [])) if content_el else None,
}

267
containers/novela/scrapers/iomfats.py
View File

@ -1,267 +0,0 @@
import re
from urllib.parse import urljoin, urlparse
import httpx
from bs4 import BeautifulSoup, NavigableString
from .base import BaseScraper
IOMFATS_BASE = "https://iomfats.org"
class IomfatsScraper(BaseScraper):
@classmethod
def matches(cls, url: str) -> bool:
return "iomfats.org" in url
async def login(self, client: httpx.AsyncClient, username: str, password: str) -> bool:
return True # no login required
def _author_page_url(self, url: str) -> str:
"""Derive the author index page URL from any iomfats.org URL."""
parsed = urlparse(url)
parts = parsed.path.strip("/").split("/")
# Path: storyshelf/hosted/{author}/...
# Author page is the first 3 segments.
if len(parts) >= 3 and parts[0] == "storyshelf" and parts[1] == "hosted":
author_path = "/" + "/".join(parts[:3]) + "/"
return f"{parsed.scheme}://{parsed.netloc}{author_path}"
return url
def _is_author_page(self, url: str) -> bool:
parts = urlparse(url).path.strip("/").split("/")
return (
len(parts) <= 3
and len(parts) >= 2
and parts[0] == "storyshelf"
and parts[1] == "hosted"
)
def _story_folder(self, url: str) -> str | None:
"""Return the story folder segment from a chapter URL, or None."""
parts = urlparse(url).path.strip("/").split("/")
# storyshelf/hosted/{author}/{story-folder}/{chapter}.html
if len(parts) >= 5:
return parts[3]
return None
async def fetch_book_info(self, client: httpx.AsyncClient, url: str) -> dict:
if self._is_author_page(url):
raise ValueError(
"Voer een chapter-URL in, geen author-pagina. "
"Kopieer de URL van het eerste hoofdstuk van het gewenste verhaal."
)
story_folder = self._story_folder(url)
if not story_folder:
raise ValueError(
"Onverwacht URL-formaat voor iomfats.org. "
"Gebruik de URL van een hoofdstuk, bijv. …/grasshopper/dreamchasers/01.html"
)
author_url = self._author_page_url(url)
r = await client.get(author_url)
soup = BeautifulSoup(r.text, "html.parser")
content = soup.find("div", id="content")
if not content:
raise ValueError("Kan de author-pagina niet verwerken (geen #content element).")
# Author name from "by <a>Name</a>" heading
author = "Unknown author"
for el in content.find_all(["h2", "h3"]):
text = el.get_text(strip=True)
m = re.match(r"^by\s+(.+)$", text, re.I)
if m:
author = m.group(1).strip()
break
# Fallback: author slug from URL
if author == "Unknown author":
parts = urlparse(author_url).path.strip("/").split("/")
if len(parts) >= 3:
author = parts[2].replace("_", " ").title()
# Walk the content to find the story matching story_folder.
#
# Two structures on the author page:
#
# Single story:
# <h3>Book Title</h3>
# <ul><li><a href="folder/ch01.html">Chapter 1</a></li>…</ul>
#
# Multi-part series:
# <h3>Series Name</h3>
# <ul>
# <li><h3>Book Title (part 1)</h3>
# <p><small>[status]</small></p>
# <ul><li><a href="folder-part1/ch01.html">…</a></li>…</ul>
# </li>
# <li><h3>Book Title (part 2)</h3>…</li>
# </ul>
book_title = ""
series = ""
series_index_hint = 0
publication_status = ""
chapter_links: list[dict] = []
nodes = list(content.children)
i = 0
while i < len(nodes):
node = nodes[i]
if not hasattr(node, "name"):
i += 1
continue
if node.name == "h3":
outer_title = node.get_text(strip=True)
if re.match(r"^by\s+", outer_title, re.I):
i += 1
continue
# Find the following <ul>
j = i + 1
outer_ul = None
outer_status = ""
while j < len(nodes):
n = nodes[j]
if not hasattr(n, "name"):
j += 1
continue
if n.name == "h3":
break
if n.name == "p":
small = n.find("small")
if small:
outer_status = small.get_text(strip=True).strip("[]")
j += 1
continue
if n.name == "ul":
outer_ul = n
break
j += 1
if not outer_ul:
i += 1
continue
# Check if this <ul> has sub-section <li><h3> entries (multi-part series)
sub_sections = [
li for li in outer_ul.find_all("li", recursive=False)
if li.find("h3")
]
if sub_sections:
# Multi-part series: outer_title = series name
for li in sub_sections:
sub_h3 = li.find("h3")
sub_ul = li.find("ul")
if not sub_h3 or not sub_ul:
continue
chapters = self._collect_chapters(sub_ul, story_folder, author_url)
if chapters:
book_title = sub_h3.get_text(strip=True)
series = outer_title
chapter_links = chapters
# Status from <p><small> inside this <li>
small = li.find("small")
if small:
publication_status = small.get_text(strip=True).strip("[]")
break
else:
# Single story: outer_title = book title
chapters = self._collect_chapters(outer_ul, story_folder, author_url)
if chapters:
book_title = outer_title
publication_status = outer_status
chapter_links = chapters
if chapter_links:
break
i += 1
if not chapter_links:
raise ValueError(
f"Geen hoofdstukken gevonden voor folder '{story_folder}'. "
"Controleer de URL."
)
# Series index from folder name suffix: *-part{N} or *-{N}
m = re.search(r"-part(\d+)$", story_folder, re.I)
if not m:
m = re.search(r"-(\d+)$", story_folder)
if m:
series_index_hint = int(m.group(1))
return {
"title": book_title or story_folder.replace("-", " ").title(),
"author": author,
"publisher": "iomfats.org",
"series": series,
"series_index_hint": series_index_hint,
"genres": [],
"subgenres": [],
"tags": [],
"description": "",
"updated_date": "",
"publication_status": publication_status,
"source_url": url,
"chapters": chapter_links,
"chapter_method": "html_scan",
"index_image_url": None,
}
def _collect_chapters(self, ul, story_folder: str, base_url: str) -> list[dict]:
"""Collect chapter links from a flat <ul>, filtered by story_folder."""
out: list[dict] = []
for li in ul.find_all("li", recursive=False):
a = li.find("a", href=True)
if not a:
continue
full_url = urljoin(base_url, a["href"])
if story_folder not in urlparse(full_url).path:
continue
raw_title = a.get_text(strip=True)
title = f"Chapter {raw_title}" if re.match(r"^\d+$", raw_title) else raw_title
out.append({"url": full_url, "title": title})
return out
async def fetch_chapter(self, client: httpx.AsyncClient, ch: dict) -> dict:
cr = await client.get(ch["url"])
csoup = BeautifulSoup(cr.text, "html.parser")
title = ch["title"]
content_el = csoup.find("div", id="content")
if not content_el:
content_el = csoup.find("body")
if content_el:
# Remove headings (story title, author, chapter number)
for el in content_el.find_all(["h2", "h3"]):
el.decompose()
# Remove chapter navigation divs
for el in content_el.find_all("div", class_=re.compile(r"chapternav", re.I)):
el.decompose()
# Remove footer elements (author note, forum button)
for el in content_el.find_all("div", class_="important"):
el.decompose()
for el in content_el.find_all("a", class_="styled-button"):
parent = el.parent
el.decompose()
if parent and not parent.get_text(strip=True):
parent.decompose()
# Remove anchor tags used as page anchors (<a name="content">)
for el in content_el.find_all("a", attrs={"name": True}):
if not el.get("href"):
el.unwrap()
return {
"title": title,
"content_el": content_el,
"selector_id": "content",
"selector_class": None,
}

358
containers/novela/scrapers/nifty.py
View File

@ -1,358 +0,0 @@
import re
from email.utils import parsedate
from html import escape as he
from time import mktime
from datetime import date as _date
from urllib.parse import urljoin, urlparse
import httpx
from bs4 import BeautifulSoup
from .base import BaseScraper
# Email header field names that appear at the top of Nifty classic chapters.
_HEADER_RE = re.compile(
r"^(Date|From|Subject|Reply-To|Message-ID|MIME-Version|Content-Type|X-[\w-]+):",
re.I,
)
# Scene-break patterns in plain text (subset of xhtml.BREAK_PATTERNS for text matching).
_BREAK_RE = re.compile(
r"^("
r"[\*\-]{3,}"
r"|[~=]{3,}"
r"|#{3,}"
r"|[·•◦‣⁃]\s*[·•◦‣⁃]\s*[·•◦‣⁃]"
r"|[-–—]\s*[oO0]\s*[-–—]"
r")$"
)
class NiftyScraper(BaseScraper):
_LEAD_MARKERS = (
"notice this is a work of fiction",
"if it is illegal to read stories",
"if you enjoy this story",
"for my other stories",
"nifty archive",
"code of conduct",
"author note",
"author's note",
"disclaimer",
"this story contains",
"this story includes",
"all characters are",
"all characters depicted",
)
_TAIL_MARKERS = (
"please remember to donate",
"donate",
"support nifty",
"support the archive",
"nifty archive alliance",
"donate.nifty.org",
"nifty.org/donate",
"nifty.org/support",
"patreon",
"buy me a coffee",
"tip jar",
"become a supporter",
)
@classmethod
def matches(cls, url: str) -> bool:
return "nifty.org" in url and "new.nifty.org" not in url
async def login(self, client: httpx.AsyncClient, username: str, password: str) -> bool:
return True # no login required
# ── Helpers ───────────────────────────────────────────────────────────────
def _to_index_url(self, url: str) -> str:
"""Return the story index URL for any Nifty URL (index or chapter).
Nifty path structure:
/nifty/{category}/{subcategory}/{story}/ ← index (4 segments)
/nifty/{category}/{subcategory}/{story}/{chapter} ← chapter (5 segments)
"""
parsed = urlparse(url)
parts = [p for p in parsed.path.split("/") if p]
if len(parts) >= 5:
path = "/" + "/".join(parts[:4]) + "/"
else:
path = parsed.path.rstrip("/") + "/"
return f"{parsed.scheme}://{parsed.netloc}{path}"
def _slug_to_title(self, slug: str) -> str:
return slug.replace("-", " ").title()
def _parse_date_header(self, text: str) -> str:
"""Return YYYY-MM-DD from a 'Date: …' line, or ''."""
m = re.search(r"^Date:\s+(.+)$", text, re.M)
if not m:
return ""
try:
parsed = parsedate(m.group(1).strip())
if parsed:
return _date.fromtimestamp(mktime(parsed)).isoformat()
except Exception:
pass
return ""
def _parse_author_header(self, text: str) -> str:
"""Return author name from 'From: Name <email>' line, or ''."""
m = re.search(r"^From:\s+([^<\n]+?)(?:\s*<[^>]+>)?\s*$", text, re.M)
return m.group(1).strip() if m else ""
def _parse_subject_header(self, text: str) -> str:
"""Return the Subject header value, or ''."""
m = re.search(r"^Subject:\s+(.+)$", text, re.M)
return m.group(1).strip() if m else ""
def _normalize(self, text: str) -> str:
"""Normalise text for boilerplate comparison (lowercase, collapsed whitespace)."""
return re.sub(r"\s+", " ", text.lower()).strip()
async def _get_text(self, client: httpx.AsyncClient, url: str) -> tuple[BeautifulSoup, str]:
"""Fetch *url* and return (soup, raw_text).
Nifty classic pages wrap the story content in a <pre> element.
Falls back to the full body text if no <pre> is found.
"""
r = await client.get(url)
soup = BeautifulSoup(r.text, "html.parser")
pre = soup.find("pre")
if pre:
raw = pre.get_text()
else:
body = soup.find("body")
raw = body.get_text("\n") if body else soup.get_text("\n")
return soup, raw
def _strip_email_headers(self, text: str) -> str:
"""Remove the leading email header block (Date/From/Subject/…) from chapter text.
Tolerates blank lines between header fields — some Nifty pages place the
Subject on a separate line after a blank line:
Date: …
From: …
Subject: …
"""
lines = text.splitlines()
i = 0
# Skip leading blank lines.
while i < len(lines) and not lines[i].strip():
i += 1
# Only strip if this actually looks like an email header block.
if not any(_HEADER_RE.match(lines[j]) for j in range(i, min(i + 12, len(lines)))):
return text
# Skip header lines, tolerating blank lines between them.
# A blank line ends the block only when no further header line follows.
while i < len(lines):
stripped = lines[i].strip()
if _HEADER_RE.match(stripped):
i += 1
elif not stripped:
# Peek ahead past any blank lines.
j = i + 1
while j < len(lines) and not lines[j].strip():
j += 1
if j < len(lines) and _HEADER_RE.match(lines[j].strip()):
i = j # more headers follow — jump over the blank line(s)
else:
i += 1
break # no more headers — end of block
else:
break # non-header, non-blank line — end of block
# Skip blank lines immediately after the header block.
while i < len(lines) and not lines[i].strip():
i += 1
return "\n".join(lines[i:])
def _text_to_paragraphs(self, text: str) -> list[str]:
"""Split plain text into paragraphs; join hard-wrapped lines within each paragraph.
Nifty classic stories are stored as email submissions: paragraphs are
separated by blank lines, and each line is wrapped at ~70 characters.
This function merges those wrapped lines back into a single line per
paragraph.
"""
text = text.replace("\r\n", "\n").replace("\r", "\n")
blocks = re.split(r"\n{2,}", text)
result = []
for block in blocks:
lines = [l.strip() for l in block.splitlines() if l.strip()]
if lines:
result.append(" ".join(lines))
return result
def _comment_safe(self, text: str) -> str:
return text.replace("--", "- -")
def _plain_text(self, text: str) -> str:
if "<" in text and ">" in text:
return BeautifulSoup(text, "html.parser").get_text(" ", strip=True)
return text
def _looks_like_lead_boilerplate(self, text: str) -> bool:
t = re.sub(r"\s+", " ", self._plain_text(text).lower()).strip()
if not t or len(t) > 4000:
return False
return any(m in t for m in self._LEAD_MARKERS)
def _looks_like_tail_boilerplate(self, text: str) -> bool:
t = re.sub(r"\s+", " ", self._plain_text(text).lower()).strip()
if not t or len(t) > 4000:
return False
return any(m in t for m in self._TAIL_MARKERS)
def _extract_hidden_boilerplate(self, paragraphs: list[str]) -> tuple[list[str], list[str], list[str]]:
visible = list(paragraphs)
leading: list[str] = []
trailing: list[str] = []
while visible and len(leading) < 6 and self._looks_like_lead_boilerplate(visible[0]):
leading.append(visible.pop(0))
while visible and len(trailing) < 6 and self._looks_like_tail_boilerplate(visible[-1]):
trailing.insert(0, visible.pop())
if not visible:
return list(paragraphs), [], []
return visible, leading, trailing
# ── BaseScraper interface ─────────────────────────────────────────────────
async def fetch_book_info(self, client: httpx.AsyncClient, url: str) -> dict:
index_url = self._to_index_url(url)
r = await client.get(index_url)
soup = BeautifulSoup(r.text, "html.parser")
# Title from URL slug.
slug = urlparse(index_url).path.rstrip("/").rsplit("/", 1)[-1]
book_title = self._slug_to_title(slug)
# Genres from URL path: /nifty/{category}/{subcategory}/{story}/
path_parts = [p for p in urlparse(index_url).path.split("/") if p]
category = self._slug_to_title(path_parts[1]) if len(path_parts) > 1 else ""
subcategory = self._slug_to_title(path_parts[2]) if len(path_parts) > 2 else ""
# Chapter links: all <a> tags pointing one level deeper than the index.
chapter_links: list[dict] = []
seen: set[str] = set()
for a in soup.find_all("a", href=True):
full = urljoin(index_url, a["href"])
if (
full.startswith(index_url)
and full.rstrip("/") != index_url.rstrip("/")
and full not in seen
):
seen.add(full)
chapter_links.append({"url": full, "title": a.get_text(strip=True)})
# Sort by trailing chapter number.
def _num(ch: dict) -> int:
m = re.search(r"-(\d+)$", ch["url"].rstrip("/"))
return int(m.group(1)) if m else 0
chapter_links.sort(key=_num)
for i, ch in enumerate(chapter_links, 1):
ch["title"] = f"Chapter {i}"
# Author and dates: extract from email headers in first and last chapters.
author = "Unknown author"
updated_date = ""
preamble_count = 0
if chapter_links:
_, first_text = await self._get_text(client, chapter_links[0]["url"])
author = self._parse_author_header(first_text) or author
pub_date = self._parse_date_header(first_text)
if len(chapter_links) > 1:
_, last_text = await self._get_text(client, chapter_links[-1]["url"])
updated_date = self._parse_date_header(last_text) or pub_date
else:
updated_date = pub_date
# Boilerplate detection: compare leading paragraphs of chapters 1 and 2.
# Paragraphs present in both (after header strip) are repeated preamble.
if len(chapter_links) >= 2:
_, ch2_text = await self._get_text(client, chapter_links[1]["url"])
paras1 = self._text_to_paragraphs(self._strip_email_headers(first_text))
paras2 = self._text_to_paragraphs(self._strip_email_headers(ch2_text))
for p1, p2 in zip(paras1, paras2):
if self._normalize(p1) == self._normalize(p2):
preamble_count += 1
else:
break
for ch in chapter_links:
ch["preamble_count"] = preamble_count
return {
"title": book_title,
"author": author,
"publisher": "nifty.org",
"series": "",
"series_index_hint": 0,
"genres": [],
"subgenres": [],
"tags": [t for t in [category, subcategory] if t],
"description": "",
"updated_date": updated_date,
"publication_status": "",
"source_url": index_url,
"chapters": chapter_links,
"chapter_method": "html_scan",
"index_image_url": None,
}
async def fetch_chapter(self, client: httpx.AsyncClient, ch: dict) -> dict:
_, raw_text = await self._get_text(client, ch["url"])
# Extract Subject before stripping headers; store as invisible comment.
subject = self._parse_subject_header(raw_text)
# Remove email header block.
story_text = self._strip_email_headers(raw_text)
# Convert hard-wrapped plain text to paragraphs.
paragraphs = self._text_to_paragraphs(story_text)
# Skip repeated boilerplate paragraphs at the top of each chapter.
preamble_count = ch.get("preamble_count", 0)
if preamble_count:
paragraphs = paragraphs[preamble_count:]
paragraphs, hidden_lead, hidden_tail = self._extract_hidden_boilerplate(paragraphs)
# Build an HTML fragment: subject as comment, scene-breaks as <hr/>, rest as <p>.
html_parts: list[str] = []
if subject:
html_parts.append(f"<!-- Subject: {self._comment_safe(subject)} -->")
if hidden_lead:
lead_text = " || ".join(re.sub(r"\s+", " ", p).strip() for p in hidden_lead if p.strip())
if lead_text:
html_parts.append(f"<!-- NIFTY_HIDDEN_LEAD: {self._comment_safe(lead_text)} -->")
for para in paragraphs:
if _BREAK_RE.match(para.strip()):
html_parts.append("<hr/>")
else:
html_parts.append(f"<p>{he(para)}</p>")
if hidden_tail:
tail_text = " || ".join(re.sub(r"\s+", " ", p).strip() for p in hidden_tail if p.strip())
if tail_text:
html_parts.append(f"<!-- NIFTY_HIDDEN_TAIL: {self._comment_safe(tail_text)} -->")
wrapper = BeautifulSoup(
"<div>" + "".join(html_parts) + "</div>",
"html.parser",
)
content_el = wrapper.find("div")
return {
"title": ch["title"],
"content_el": content_el,
"selector_id": None,
"selector_class": None,
}

310
containers/novela/scrapers/nifty_new.py
View File

@ -1,310 +0,0 @@
import json
import re
from html import unescape as html_unescape
from urllib.parse import urlparse
import httpx
from bs4 import BeautifulSoup, Comment
from .base import BaseScraper
class NiftyNewScraper(BaseScraper):
_LEAD_MARKERS = (
"notice this is a work of fiction",
"if it is illegal to read stories",
"if you enjoy this story",
"for my other stories",
"nifty archive",
"code of conduct",
"author note",
"author's note",
"disclaimer",
"this story contains",
"this story includes",
"all characters are",
"all characters depicted",
)
_TAIL_MARKERS = (
"please remember to donate",
"donate",
"support nifty",
"support the archive",
"nifty archive alliance",
"donate.nifty.org",
"nifty.org/donate",
"nifty.org/support",
"patreon",
"buy me a coffee",
"tip jar",
"become a supporter",
)
@classmethod
def matches(cls, url: str) -> bool:
return "new.nifty.org" in url
async def login(self, client: httpx.AsyncClient, username: str, password: str) -> bool:
return True # no login required
# ── Helpers ───────────────────────────────────────────────────────────────
def _to_index_url(self, url: str) -> str:
"""Strip trailing chapter number, return story index URL.
/stories/some-slug-83036/3 → /stories/some-slug-83036
/stories/some-slug-83036 → /stories/some-slug-83036
"""
parsed = urlparse(url)
path = re.sub(r"/\d+$", "", parsed.path.rstrip("/"))
return f"{parsed.scheme}://{parsed.netloc}{path}"
def _parse_date(self, iso: str) -> str:
"""Return YYYY-MM-DD from an ISO datetime string, or ''."""
if not iso:
return ""
return iso[:10]
# ── BaseScraper interface ─────────────────────────────────────────────────
async def fetch_book_info(self, client: httpx.AsyncClient, url: str) -> dict:
index_url = self._to_index_url(url)
r = await client.get(index_url)
soup = BeautifulSoup(r.text, "html.parser")
# Title: <h1>, fallback to <title> (strip "- … - Nifty Archive …" suffix)
h1 = soup.find("h1")
if h1:
title = h1.get_text(strip=True)
else:
title_el = soup.find("title")
raw = title_el.get_text(strip=True) if title_el else ""
title = re.split(r"\s+[-–]\s+", raw)[0].strip() if raw else ""
# Author: <strong itemprop="name"> inside /authors/ link
author = "Unknown author"
author_link = soup.find("a", href=re.compile(r"^/authors/\d+"))
if author_link:
name_el = author_link.find("strong", itemprop="name")
if name_el:
author = name_el.get_text(strip=True)
# Dates: <time itemprop="datePublished/dateModified">
pub_el = soup.find("time", itemprop="datePublished")
mod_el = soup.find("time", itemprop="dateModified")
pub_date = self._parse_date(pub_el.get("datetime", "") if pub_el else "")
updated_date = self._parse_date(mod_el.get("datetime", "") if mod_el else "") or pub_date
# Tags: from all <ul aria-label="Tags"> containers (category links + generated tags)
tags: list[str] = []
seen: set[str] = set()
for ul in soup.find_all("ul", attrs={"aria-label": "Tags"}):
for a in ul.find_all("a", href=True):
label = a.get_text(strip=True)
if label and label.lower() not in seen:
seen.add(label.lower())
tags.append(label)
# Description: <meta name="description">
desc = ""
meta_desc = soup.find("meta", attrs={"name": "description"})
if meta_desc and meta_desc.get("content"):
desc = meta_desc["content"].strip()
# Chapters: find /stories/{slug}/N links in the page HTML
slug_path = urlparse(index_url).path # e.g. /stories/some-slug-83036
chapter_pattern = re.compile(r"^" + re.escape(slug_path) + r"/(\d+)$")
chapter_nums: set[int] = set()
for a in soup.find_all("a", href=True):
m = chapter_pattern.match(a["href"])
if m:
chapter_nums.add(int(m.group(1)))
# Fallback: scan RSC stream for chapter index values
if not chapter_nums:
for m in re.finditer(r'"index"\s*:\s*(\d+)', r.text):
chapter_nums.add(int(m.group(1)))
if not chapter_nums:
chapter_nums = {1}
chapters = [
{"url": f"{index_url}/{i}", "title": f"Chapter {i}"}
for i in range(1, max(chapter_nums) + 1)
]
return {
"title": title,
"author": author,
"publisher": "nifty.org",
"series": "",
"series_index_hint": 0,
"genres": [],
"subgenres": [],
"tags": tags,
"description": desc,
"updated_date": updated_date,
"publication_status": "",
"source_url": index_url,
"chapters": chapters,
"chapter_method": "html_scan",
"index_image_url": None,
}
# ── RSC parser ───────────────────────────────────────────────────────────
def _parse_rsc_paragraphs(self, rsc_text: str) -> list[str]:
"""Extract story paragraph text from a Next.js RSC stream.
The RSC format is a series of lines: ``{hex_id}:{json_value}``.
Each line that represents a <p> element looks like:
2c:["$","p",null,{"children":"Paragraph text."}]
"""
paragraphs: list[str] = []
for line in rsc_text.splitlines():
colon = line.find(":")
if colon < 0:
continue
try:
node = json.loads(line[colon + 1:])
except Exception:
continue
paragraphs.extend(self._rsc_find_paragraphs(node))
return paragraphs
def _rsc_find_paragraphs(self, node) -> list[str]:
"""Recursively find <p> text in an RSC component tree node."""
if not isinstance(node, list):
return []
# React element: ["$", tagname, key, props]
if len(node) >= 4 and node[0] == "$" and isinstance(node[1], str):
tag = node[1]
props = node[3] if isinstance(node[3], dict) else {}
if tag == "p":
text = self._rsc_text(props.get("children", ""))
return [text] if text.strip() else []
children = props.get("children")
if children is not None:
return self._rsc_find_paragraphs(children)
return []
# Plain list of child nodes
result: list[str] = []
for item in node:
result.extend(self._rsc_find_paragraphs(item))
return result
def _rsc_text(self, children) -> str:
"""Flatten RSC children (string or nested array) into plain text."""
if isinstance(children, str):
return children if not children.startswith("$") else ""
if isinstance(children, list):
parts: list[str] = []
for item in children:
if isinstance(item, str) and not item.startswith("$"):
parts.append(item)
elif isinstance(item, list) and len(item) >= 4 and item[0] == "$":
inner = item[3] if isinstance(item[3], dict) else {}
parts.append(self._rsc_text(inner.get("children", "")))
return "".join(parts)
return ""
def _extract_escaped_html_paragraphs(self, text: str) -> list[str]:
"""Extract \\u003cp\\u003e...\\u003c/p\\u003e paragraphs from Next payload text."""
paragraphs: list[str] = []
for raw in re.findall(r"\\u003cp\\u003e(.*?)\\u003c/p\\u003e", text, flags=re.S):
try:
decoded = bytes(raw, "utf-8").decode("unicode_escape")
except Exception:
decoded = raw
decoded = html_unescape(decoded)
decoded = re.sub(r"\s+", " ", decoded).strip()
if decoded:
paragraphs.append(decoded)
return paragraphs
def _comment_safe(self, text: str) -> str:
return text.replace("--", "- -")
def _plain_text(self, text: str) -> str:
# Some payload variants contain inline HTML inside paragraph text.
# Convert to plain text before marker matching.
if "<" in text and ">" in text:
return BeautifulSoup(text, "html.parser").get_text(" ", strip=True)
return text
def _looks_like_lead_boilerplate(self, text: str) -> bool:
t = re.sub(r"\s+", " ", self._plain_text(text).lower()).strip()
if not t or len(t) > 4000:
return False
return any(m in t for m in self._LEAD_MARKERS)
def _looks_like_tail_boilerplate(self, text: str) -> bool:
t = re.sub(r"\s+", " ", self._plain_text(text).lower()).strip()
if not t or len(t) > 4000:
return False
return any(m in t for m in self._TAIL_MARKERS)
def _extract_hidden_boilerplate(self, paragraphs: list[str]) -> tuple[list[str], list[str], list[str]]:
visible = list(paragraphs)
leading: list[str] = []
trailing: list[str] = []
while visible and len(leading) < 6 and self._looks_like_lead_boilerplate(visible[0]):
leading.append(visible.pop(0))
while visible and len(trailing) < 6 and self._looks_like_tail_boilerplate(visible[-1]):
trailing.insert(0, visible.pop())
# Never return an empty chapter due to over-eager filtering.
if not visible:
return list(paragraphs), [], []
return visible, leading, trailing
async def fetch_chapter(self, client: httpx.AsyncClient, ch: dict) -> dict:
# Primary path: fetch chapter HTML and read the rendered <article> content.
r = await client.get(ch["url"])
soup = BeautifulSoup(r.text, "html.parser")
paragraphs: list[str] = []
article = soup.find("article")
if article:
for p in article.find_all("p"):
text = p.get_text(" ", strip=True)
if text:
paragraphs.append(text)
# Fallback: paragraph HTML may only appear escaped in Next payload scripts.
if not paragraphs:
paragraphs = self._extract_escaped_html_paragraphs(r.text)
# Last fallback: request ?_rsc=1 and parse both RSC line format + escaped chunks.
if not paragraphs:
r_rsc = await client.get(ch["url"] + "?_rsc=1")
paragraphs = self._parse_rsc_paragraphs(r_rsc.text)
if not paragraphs:
paragraphs = self._extract_escaped_html_paragraphs(r_rsc.text)
paragraphs, hidden_lead, hidden_tail = self._extract_hidden_boilerplate(paragraphs)
# Build a BeautifulSoup <div> with <p> elements.
wrapper = BeautifulSoup("", "html.parser")
div = wrapper.new_tag("div")
if hidden_lead:
lead_text = " || ".join(re.sub(r"\s+", " ", p).strip() for p in hidden_lead if p.strip())
if lead_text:
div.append(Comment(self._comment_safe(f"NIFTY_HIDDEN_LEAD: {lead_text}")))
for text in paragraphs:
p = wrapper.new_tag("p")
p.string = text
div.append(p)
if hidden_tail:
tail_text = " || ".join(re.sub(r"\s+", " ", p).strip() for p in hidden_tail if p.strip())
if tail_text:
div.append(Comment(self._comment_safe(f"NIFTY_HIDDEN_TAIL: {tail_text}")))
return {
"title": ch["title"],
"content_el": div,
"selector_id": None,
"selector_class": None,
}

139
containers/novela/scrapers/tedlouis.py
View File

@ -1,139 +0,0 @@
import re
from urllib.parse import urljoin
import httpx
from bs4 import BeautifulSoup
from .base import BaseScraper
TED_BASE = "https://tedlouis.com/"
class TedLouisScraper(BaseScraper):
@classmethod
def matches(cls, url: str) -> bool:
return "tedlouis.com" in url
async def login(self, client: httpx.AsyncClient, username: str, password: str) -> bool:
return True # no login required
async def fetch_book_info(self, client: httpx.AsyncClient, url: str) -> dict:
r = await client.get(url)
soup = BeautifulSoup(r.text, "html.parser")
# Detect chapter page (wrong entry point)
if soup.find("h1", class_="story-title") and not soup.find("h2", class_="story-page-title"):
raise ValueError(
"Voer de story index-URL in, geen chapter-URL. "
"Kopieer de URL van de verhaal-indexpagina (de pagina met de hoofdstukkenlijst)."
)
# Title: extract only direct NavigableString children from the h2,
# ignoring nested elements like the "Back" link and author byline.
book_title = "Unknown title"
title_el = soup.find("h2", class_="story-page-title")
if title_el:
from bs4 import NavigableString
parts = [
str(c).strip()
for c in title_el.children
if isinstance(c, NavigableString) and str(c).strip()
]
book_title = " ".join(parts) or title_el.get_text(strip=True)
# Author: from byline span (may be inside the h2 or elsewhere)
author = "Unknown author"
byline = soup.find("span", class_="story-author-by-line")
if byline:
a = byline.find("a")
if a:
author = a.get_text(strip=True)
# Publication status
status_el = soup.find("span", class_="story-status-text")
publication_status = ""
if status_el:
raw = status_el.get_text(strip=True)
publication_status = re.sub(r"^Status:\s*", "", raw, flags=re.I).strip()
# Updated date: "Last Updated: Month D, YYYY" → "YYYY-MM-DD"
updated_date = ""
updated_el = soup.find("span", class_="story-last-updated")
if updated_el:
raw = re.sub(r"^Last\s+Updated:\s*", "", updated_el.get_text(strip=True), flags=re.I).strip()
try:
from datetime import datetime
updated_date = datetime.strptime(raw, "%B %d, %Y").strftime("%Y-%m-%d")
except ValueError:
try:
updated_date = datetime.strptime(raw, "%B %Y").strftime("%Y-%m-01")
except ValueError:
pass
# Chapter links from all story-index-list columns
actual_url = str(r.url)
chapter_links: list[dict] = []
seen: set[str] = set()
for ul in soup.find_all("ul", class_="story-index-list"):
for li in ul.find_all("li"):
a = li.find("a", href=True)
if not a:
continue
href = a["href"]
full_url = urljoin(actual_url, href)
if full_url in seen:
continue
seen.add(full_url)
chapter_links.append({"url": full_url, "title": a.get_text(strip=True)})
return {
"title": book_title,
"author": author,
"publisher": "tedlouis.com",
"series": "",
"series_index_hint": 0,
"genres": [],
"subgenres": [],
"tags": [],
"description": "",
"updated_date": updated_date,
"publication_status": publication_status,
"source_url": url,
"chapters": chapter_links,
"chapter_method": "html_scan",
"index_image_url": None,
}
async def fetch_chapter(self, client: httpx.AsyncClient, ch: dict) -> dict:
cr = await client.get(ch["url"])
csoup = BeautifulSoup(cr.text, "html.parser")
title = ch["title"]
# Refine chapter title from <h2 class="chapter-title"><span>…</span></h2>
chapter_h2 = csoup.find("h2", class_="chapter-title")
if chapter_h2:
span = chapter_h2.find("span")
refined = (span or chapter_h2).get_text(strip=True)
if refined:
title = refined
content_el = csoup.find("div", id="chapter")
if content_el:
# Remove story title, chapter title, copyright blocks
for el in content_el.find_all("h1", class_="story-title"):
el.decompose()
for el in content_el.find_all("h2", class_="chapter-title"):
el.decompose()
for el in content_el.find_all("div", class_="chapter-copyright-line"):
el.decompose()
for el in content_el.find_all("div", class_=re.compile(r"chapter-copyright-notice", re.I)):
el.decompose()
return {
"title": title,
"content_el": content_el,
"selector_id": "chapter",
"selector_class": None,
}

18
containers/novela/shared_templates.py
View File

@ -1,18 +0,0 @@
from fastapi.templating import Jinja2Templates
from db import get_db_conn
def _develop_mode() -> bool:
try:
with get_db_conn() as conn:
with conn.cursor() as cur:
cur.execute("SELECT develop_mode FROM app_settings WHERE id = 1")
row = cur.fetchone()
return bool(row[0]) if row else False
except Exception:
return False
templates = Jinja2Templates(directory="templates")
templates.env.globals["develop_mode"] = _develop_mode

53
containers/novela/static/book.js
View File

@ -19,6 +19,23 @@ if (BOOK.has_cover) {
let currentRating = BOOK.rating || 0; let currentRating = BOOK.rating || 0;
async function rateBook(rating) {
const newRating = currentRating === rating ? 0 : rating;
try {
const resp = await fetch(`/library/rating/${encodeURIComponent(filename)}`, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ rating: newRating }),
});
const result = await resp.json();
if (!resp.ok || result.error) return;
currentRating = result.rating;
document.querySelectorAll('#book-stars .star').forEach((el, idx) => {
el.classList.toggle('filled', idx + 1 <= currentRating);
});
} catch {}
}
// ── Want to Read toggle ──────────────────────────────────────────────────── // ── Want to Read toggle ────────────────────────────────────────────────────
async function toggleWtr() { async function toggleWtr() {
@ -148,8 +165,8 @@ class PillInput {
} }
_add(v) { _add(v) {
const parts = v.split(',').map(p => p.trim()).filter(p => p && !this.values.includes(p)); v = v.trim();
if (parts.length) { this.values.push(...parts); this._render(); } if (v && !this.values.includes(v)) { this.values.push(v); this._render(); }
this.input.value = ''; this.input.value = '';
this._hideDropdown(); this._hideDropdown();
} }
@ -288,10 +305,8 @@ async function openEdit() {
document.getElementById('ed-author').value = BOOK.author; document.getElementById('ed-author').value = BOOK.author;
document.getElementById('ed-publisher').value = BOOK.publisher; document.getElementById('ed-publisher').value = BOOK.publisher;
document.getElementById('ed-series').value = BOOK.series; document.getElementById('ed-series').value = BOOK.series;
document.getElementById('ed-series-volume').value = BOOK.series_volume || '';
document.getElementById('ed-series-index').value = BOOK.series_index + (BOOK.series_suffix || ''); document.getElementById('ed-series-index').value = BOOK.series_index + (BOOK.series_suffix || '');
document.getElementById('ed-status').value = BOOK.publication_status || 'Complete'; document.getElementById('ed-status').value = BOOK.publication_status || 'Complete';
document.getElementById('ed-rating').value = String(currentRating);
document.getElementById('ed-url').value = BOOK.source_url; document.getElementById('ed-url').value = BOOK.source_url;
document.getElementById('ed-publish-date').value = BOOK.publish_date; document.getElementById('ed-publish-date').value = BOOK.publish_date;
document.getElementById('ed-description').value = BOOK.description; document.getElementById('ed-description').value = BOOK.description;
@ -304,17 +319,6 @@ async function openEdit() {
document.getElementById('edit-panel').classList.add('open'); document.getElementById('edit-panel').classList.add('open');
} }
function generateTitle() {
const series = document.getElementById('ed-series').value.trim();
if (!series) return;
const volume = document.getElementById('ed-series-volume').value.trim();
const index = document.getElementById('ed-series-index').value.trim();
let title = series;
if (volume) title += ` (${volume})`;
if (index) title += ` #${index}`;
document.getElementById('ed-title').value = title;
}
function closeEdit() { function closeEdit() {
document.getElementById('edit-backdrop').classList.remove('open'); document.getElementById('edit-backdrop').classList.remove('open');
document.getElementById('edit-panel').classList.remove('open'); document.getElementById('edit-panel').classList.remove('open');
@ -332,7 +336,6 @@ async function saveEdit() {
author: document.getElementById('ed-author').value, author: document.getElementById('ed-author').value,
publisher: document.getElementById('ed-publisher').value, publisher: document.getElementById('ed-publisher').value,
series: document.getElementById('ed-series').value, series: document.getElementById('ed-series').value,
series_volume: document.getElementById('ed-series-volume').value,
series_index: document.getElementById('ed-series-index').value, series_index: document.getElementById('ed-series-index').value,
publication_status: document.getElementById('ed-status').value, publication_status: document.getElementById('ed-status').value,
source_url: document.getElementById('ed-url').value, source_url: document.getElementById('ed-url').value,
@ -342,27 +345,17 @@ async function saveEdit() {
subgenres: subgenreInput.getValues(), subgenres: subgenreInput.getValues(),
tags: tagInput.getValues(), tags: tagInput.getValues(),
}; };
const newRating = parseInt(document.getElementById('ed-rating').value, 10) || 0;
const resp = await fetch(`/library/book/${encodeURIComponent(filename)}`, { const resp = await fetch(`/library/book/${encodeURIComponent(filename)}`, {
method: 'PATCH', method: 'PATCH',
headers: { 'Content-Type': 'application/json' }, headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(body), body: JSON.stringify(body),
}); });
const result = await resp.json(); const result = await resp.json();
if (!resp.ok) { if (resp.ok && result.filename) {
alert(result.error || 'Save failed.');
return;
}
if (newRating !== currentRating) {
await fetch(`/library/rating/${encodeURIComponent(result.filename || filename)}`, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ rating: newRating }),
});
}
if (result.filename) {
window.location.href = `/library/book/${encodeURIComponent(result.filename)}`; window.location.href = `/library/book/${encodeURIComponent(result.filename)}`;
} else { } else if (resp.ok) {
window.location.reload(); window.location.reload();
} else {
alert(result.error || 'Save failed.');
} }
} }

21
containers/novela/static/library.css
View File

@ -140,24 +140,6 @@ html, body {
.badge-temporary-hold { color: #c8a03a; } .badge-temporary-hold { color: #c8a03a; }
.badge-long-term-hold { color: #ffa20e; } .badge-long-term-hold { color: #ffa20e; }
/* Archived badge: bottom-left of cover */
.badge-archived {
position: absolute;
bottom: 0.35rem;
left: 0.35rem;
width: 22px;
height: 22px;
border-radius: 50%;
display: flex;
align-items: center;
justify-content: center;
z-index: 2;
background: rgba(15,14,12,0.82);
box-shadow: 0 0 0 2px #0f0e0c;
color: #888;
pointer-events: none;
}
/* Star: want-to-read top-left */ /* Star: want-to-read top-left */
.btn-star { .btn-star {
position: absolute; position: absolute;
@ -376,9 +358,6 @@ html, body {
.slot-missing-inner svg { opacity: 0.5; } .slot-missing-inner svg { opacity: 0.5; }
.slot-missing-inner span { font-family: var(--mono); font-size: 0.65rem; } .slot-missing-inner span { font-family: var(--mono); font-size: 0.65rem; }
/* ── Series detail header ─────────────────────────────────────────────────── */
.series-detail-header { display: flex; justify-content: flex-end; margin-bottom: 1rem; }
/* ── Authors list ─────────────────────────────────────────────────────────── */ /* ── Authors list ─────────────────────────────────────────────────────────── */
.author-list { display: flex; flex-direction: column; gap: 0.3rem; } .author-list { display: flex; flex-direction: column; gap: 0.3rem; }

91
containers/novela/static/library.js
View File

@ -938,7 +938,6 @@ function renderBooksGrid(books) {
</svg> </svg>
</button> </button>
${statusBadge} ${statusBadge}
${b.archived ? `<div class="badge-archived" title="Archived"><svg width="10" height="10" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2.5"><polyline points="21 8 21 21 3 21 3 8"/><rect x="1" y="3" width="22" height="5"/><line x1="10" y1="12" x2="14" y2="12"/></svg></div>` : ''}
${b.read_count > 0 ? `<div class="read-pill">${b.read_count}\u00d7</div>` : ''} ${b.read_count > 0 ? `<div class="read-pill">${b.read_count}\u00d7</div>` : ''}
${b.progress > 0 ? `<div class="progress-mini"><div class="progress-mini-fill" style="width:${b.progress}%"></div></div>` : ''} ${b.progress > 0 ? `<div class="progress-mini"><div class="progress-mini-fill" style="width:${b.progress}%"></div></div>` : ''}
</div> </div>
@ -989,8 +988,6 @@ function groupBySeries() {
map[b.series].push(b); map[b.series].push(b);
} }
for (const s of Object.values(map)) s.sort((a, b) => { for (const s of Object.values(map)) s.sort((a, b) => {
const va = a.series_volume || '', vb = b.series_volume || '';
if (va !== vb) return va.localeCompare(vb);
if (a.series_index !== b.series_index) return a.series_index - b.series_index; if (a.series_index !== b.series_index) return a.series_index - b.series_index;
return (a.series_suffix || '').localeCompare(b.series_suffix || ''); return (a.series_suffix || '').localeCompare(b.series_suffix || '');
}); });
@ -1085,26 +1082,6 @@ function renderSeriesGrid() {
// ── Series detail ────────────────────────────────────────────────────────── // ── Series detail ──────────────────────────────────────────────────────────
function _slotsForBooks(books, seriesVolume) {
const sorted = [...books].sort((a, b) => {
if (a.series_index !== b.series_index) return a.series_index - b.series_index;
return (a.series_suffix || '').localeCompare(b.series_suffix || '');
});
const byIndex = {};
for (const b of sorted) {
if (!byIndex[b.series_index]) byIndex[b.series_index] = [];
byIndex[b.series_index].push(b);
}
const min = Math.min(...sorted.map(b => b.series_index));
const max = Math.max(...sorted.map(b => b.series_index));
const slots = [];
for (let i = min; i <= max; i++) {
if (byIndex[i]) for (const b of byIndex[i]) slots.push(b);
else slots.push({ missing: true, series_index: i, series_volume: seriesVolume || '' });
}
return slots;
}
function getSeriesSlots(books) { function getSeriesSlots(books) {
// Treat books as indexed (including index 0) only when at least one book // Treat books as indexed (including index 0) only when at least one book
// has series_index > 0 — this preserves the "unindexed flat list" behaviour // has series_index > 0 — this preserves the "unindexed flat list" behaviour
@ -1112,24 +1089,29 @@ function getSeriesSlots(books) {
const hasPositiveIndex = books.some(b => b.series_index > 0); const hasPositiveIndex = books.some(b => b.series_index > 0);
if (!hasPositiveIndex) return books; if (!hasPositiveIndex) return books;
// When series_volume is used, do gap-detection per volume (year) separately. // Sort indexed books by (series_index, series_suffix) so 21 < 21a < 21b < 22.
if (books.some(b => b.series_volume)) { const indexed = [...books].sort((a, b) => {
const byVolume = {}; if (a.series_index !== b.series_index) return a.series_index - b.series_index;
for (const b of books) { return (a.series_suffix || '').localeCompare(b.series_suffix || '');
const vol = b.series_volume || ''; });
if (!byVolume[vol]) byVolume[vol] = [];
byVolume[vol].push(b); // Build slot map keyed by numeric index only (for gap detection).
const byIndex = {};
for (const b of indexed) {
if (!byIndex[b.series_index]) byIndex[b.series_index] = [];
byIndex[b.series_index].push(b);
} }
const min = Math.min(...indexed.map(b => b.series_index));
const max = Math.max(...indexed.map(b => b.series_index));
const slots = []; const slots = [];
for (const vol of Object.keys(byVolume).sort()) { for (let i = min; i <= max; i++) {
for (const slot of _slotsForBooks(byVolume[vol], vol)) slots.push(slot); if (byIndex[i]) for (const b of byIndex[i]) slots.push(b);
else slots.push({ missing: true, series_index: i });
} }
return slots; return slots;
} }
return _slotsForBooks(books, '');
}
function renderSeriesDetail(seriesName) { function renderSeriesDetail(seriesName) {
const map = groupBySeries(); const map = groupBySeries();
const books = map[seriesName] || []; const books = map[seriesName] || [];
@ -1152,9 +1134,7 @@ function renderSeriesDetail(seriesName) {
if (hasPositiveIndex || slot.series_index > 0 || slot.series_suffix) { if (hasPositiveIndex || slot.series_index > 0 || slot.series_suffix) {
const lbl = document.createElement('div'); const lbl = document.createElement('div');
lbl.className = 'slot-index-label'; lbl.className = 'slot-index-label';
lbl.textContent = slot.series_volume lbl.textContent = `#${slot.series_index}${slot.series_suffix || ''}`;
? `(${slot.series_volume}) #${slot.series_index}${slot.series_suffix || ''}`
: `#${slot.series_index}${slot.series_suffix || ''}`;
wrapper.appendChild(lbl); wrapper.appendChild(lbl);
} }
@ -1189,7 +1169,6 @@ function renderSeriesDetail(seriesName) {
<div class="cover-wrap"> <div class="cover-wrap">
<canvas class="cover-canvas"></canvas> <canvas class="cover-canvas"></canvas>
${statusBadge} ${statusBadge}
${b.archived ? `<div class="badge-archived" title="Archived"><svg width="10" height="10" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2.5"><polyline points="21 8 21 21 3 21 3 8"/><rect x="1" y="3" width="22" height="5"/><line x1="10" y1="12" x2="14" y2="12"/></svg></div>` : ''}
${b.read_count > 0 ? `<div class="read-pill">${b.read_count}\u00d7</div>` : ''} ${b.read_count > 0 ? `<div class="read-pill">${b.read_count}\u00d7</div>` : ''}
${b.progress > 0 ? `<div class="progress-mini"><div class="progress-mini-fill" style="width:${b.progress}%"></div></div>` : ''} ${b.progress > 0 ? `<div class="progress-mini"><div class="progress-mini-fill" style="width:${b.progress}%"></div></div>` : ''}
</div> </div>
@ -1219,32 +1198,7 @@ function renderSeriesDetail(seriesName) {
grid.appendChild(wrapper); grid.appendChild(wrapper);
}); });
const allArchived = books.every(b => b.archived);
const header = document.createElement('div');
header.className = 'series-detail-header';
const archiveBtn = document.createElement('button');
archiveBtn.className = 'btn btn-sm';
archiveBtn.textContent = allArchived ? 'Unarchive series' : 'Archive series';
archiveBtn.onclick = async () => {
archiveBtn.disabled = true;
const res = await fetch('/library/archive-series', {
method: 'POST',
headers: {'Content-Type': 'application/json'},
body: JSON.stringify({series: seriesName, archive: !allArchived}),
});
const data = await res.json();
if (data.ok) {
allBooks.forEach(b => { if (b.series === seriesName) b.archived = !allArchived; });
updateCounts();
renderSeriesDetail(seriesName);
} else {
archiveBtn.disabled = false;
}
};
header.appendChild(archiveBtn);
container.innerHTML = ''; container.innerHTML = '';
container.appendChild(header);
container.appendChild(grid); container.appendChild(grid);
} }
@ -1254,7 +1208,7 @@ function renderAuthorsView() {
const container = document.getElementById('grid-container'); const container = document.getElementById('grid-container');
const authorMap = {}; const authorMap = {};
for (const b of allBooks) { for (const b of activeBooks()) {
const a = bookAuthor(b); const a = bookAuthor(b);
if (!a) continue; if (!a) continue;
if (!authorMap[a]) authorMap[a] = []; if (!authorMap[a]) authorMap[a] = [];
@ -1297,7 +1251,7 @@ function renderPublishersView() {
const container = document.getElementById('grid-container'); const container = document.getElementById('grid-container');
const publisherMap = {}; const publisherMap = {};
for (const b of allBooks) { for (const b of activeBooks()) {
const key = bookPublisherKey(b); const key = bookPublisherKey(b);
if (!publisherMap[key]) publisherMap[key] = []; if (!publisherMap[key]) publisherMap[key] = [];
publisherMap[key].push(b); publisherMap[key].push(b);
@ -1543,7 +1497,6 @@ function renderDuplicatesView() {
</svg> </svg>
</button> </button>
${statusBadge} ${statusBadge}
${b.archived ? `<div class="badge-archived" title="Archived"><svg width="10" height="10" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2.5"><polyline points="21 8 21 21 3 21 3 8"/><rect x="1" y="3" width="22" height="5"/><line x1="10" y1="12" x2="14" y2="12"/></svg></div>` : ''}
${b.read_count > 0 ? `<div class="read-pill">${b.read_count}\u00d7</div>` : ''} ${b.read_count > 0 ? `<div class="read-pill">${b.read_count}\u00d7</div>` : ''}
${b.progress > 0 ? `<div class="progress-mini"><div class="progress-mini-fill" style="width:${b.progress}%"></div></div>` : ''} ${b.progress > 0 ? `<div class="progress-mini"><div class="progress-mini-fill" style="width:${b.progress}%"></div></div>` : ''}
</div> </div>
@ -1591,8 +1544,6 @@ function renderAuthorDetail(authorName) {
const sa = a.series || '\uffff'; const sa = a.series || '\uffff';
const sb = b.series || '\uffff'; const sb = b.series || '\uffff';
if (sa !== sb) return sa.localeCompare(sb); if (sa !== sb) return sa.localeCompare(sb);
const va = a.series_volume || '', vb = b.series_volume || '';
if (va !== vb) return va.localeCompare(vb);
if (a.series_index !== b.series_index) return a.series_index - b.series_index; if (a.series_index !== b.series_index) return a.series_index - b.series_index;
if ((a.series_suffix || '') !== (b.series_suffix || '')) return (a.series_suffix || '').localeCompare(b.series_suffix || ''); if ((a.series_suffix || '') !== (b.series_suffix || '')) return (a.series_suffix || '').localeCompare(b.series_suffix || '');
return bookTitle(a).localeCompare(bookTitle(b)); return bookTitle(a).localeCompare(bookTitle(b));
@ -1608,8 +1559,6 @@ function renderPublisherDetail(publisherName) {
const sa = a.series || '\uffff'; const sa = a.series || '\uffff';
const sb = b.series || '\uffff'; const sb = b.series || '\uffff';
if (sa !== sb) return sa.localeCompare(sb); if (sa !== sb) return sa.localeCompare(sb);
const va = a.series_volume || '', vb = b.series_volume || '';
if (va !== vb) return va.localeCompare(vb);
if (a.series_index !== b.series_index) return a.series_index - b.series_index; if (a.series_index !== b.series_index) return a.series_index - b.series_index;
if ((a.series_suffix || '') !== (b.series_suffix || '')) return (a.series_suffix || '').localeCompare(b.series_suffix || ''); if ((a.series_suffix || '') !== (b.series_suffix || '')) return (a.series_suffix || '').localeCompare(b.series_suffix || '');
return bookTitle(a).localeCompare(bookTitle(b)); return bookTitle(a).localeCompare(bookTitle(b));

24
containers/novela/static/sidebar.css
View File

@ -1,27 +1,3 @@
/* ── Develop banner ── */
.develop-banner {
position: fixed;
top: 26px;
left: -52px;
width: 200px;
background: rgba(194, 120, 20, 0.5);
color: rgba(255, 255, 255, 0.95);
text-align: center;
transform: rotate(-45deg);
z-index: 9999;
pointer-events: none;
}
.develop-banner-text {
display: block;
padding: 6px 0;
font-family: var(--mono);
font-size: 0.62rem;
font-weight: 600;
letter-spacing: 0.2em;
}
/* ── Sidebar ── */ /* ── Sidebar ── */
html { html {

4
containers/novela/templates/_sidebar.html
View File

@ -1,7 +1,3 @@
{% if develop_mode() %}
<div class="develop-banner"><span class="develop-banner-text">DEVELOP</span></div>
{% endif %}
<button class="sidebar-toggle" id="sidebar-toggle" onclick="toggleSidebar()" aria-label="Menu"> <button class="sidebar-toggle" id="sidebar-toggle" onclick="toggleSidebar()" aria-label="Menu">
<svg width="16" height="16" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2.5"> <svg width="16" height="16" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2.5">
<line x1="3" y1="6" x2="21" y2="6"/> <line x1="3" y1="6" x2="21" y2="6"/>

2
containers/novela/templates/backup.html
View File

@ -3,7 +3,7 @@
<head> <head>
<meta charset="UTF-8"/> <meta charset="UTF-8"/>
<meta name="viewport" content="width=device-width, initial-scale=1.0"/> <meta name="viewport" content="width=device-width, initial-scale=1.0"/>
<title>Novela{% if develop_mode() %} Develop{% endif %} — Backup</title> <title>Novela - Backup</title>
<link rel="icon" href="/static/favicon.ico" sizes="16x16"/> <link rel="icon" href="/static/favicon.ico" sizes="16x16"/>
<link rel="icon" type="image/png" sizes="32x32" href="/static/favicon-32.png"/> <link rel="icon" type="image/png" sizes="32x32" href="/static/favicon-32.png"/>
<link rel="icon" type="image/png" sizes="256x256" href="/static/favicon-256.png"/> <link rel="icon" type="image/png" sizes="256x256" href="/static/favicon-256.png"/>

31
containers/novela/templates/book.html
View File

@ -3,7 +3,7 @@
<head> <head>
<meta charset="UTF-8"/> <meta charset="UTF-8"/>
<meta name="viewport" content="width=device-width, initial-scale=1.0"/> <meta name="viewport" content="width=device-width, initial-scale=1.0"/>
<title>Novela{% if develop_mode() %} Develop{% endif %} — {{ title or filename }}</title> <title>Novela — {{ title or filename }}</title>
<link rel="icon" href="/static/favicon.ico" sizes="16x16"/> <link rel="icon" href="/static/favicon.ico" sizes="16x16"/>
<link rel="icon" type="image/png" sizes="32x32" href="/static/favicon-32.png"/> <link rel="icon" type="image/png" sizes="32x32" href="/static/favicon-32.png"/>
<link rel="icon" type="image/png" sizes="256x256" href="/static/favicon-256.png"/> <link rel="icon" type="image/png" sizes="256x256" href="/static/favicon-256.png"/>
@ -31,9 +31,9 @@
</div> </div>
{% set r = (rating | default(0)) | int %} {% set r = (rating | default(0)) | int %}
<div class="star-row" id="book-stars"> <div class="star-row interactive" id="book-stars">
{% for i in range(1, 6) %} {% for i in range(1, 6) %}
<span class="star {% if i <= r %}filled{% endif %}">★</span> <span class="star {% if i <= r %}filled{% endif %}" onclick="rateBook({{ i }})">★</span>
{% endfor %} {% endfor %}
</div> </div>
@ -54,7 +54,7 @@
{% if series %} {% if series %}
<div class="meta-row"> <div class="meta-row">
<span class="meta-label">Series</span> <span class="meta-label">Series</span>
<span class="meta-value">{{ series }}{% if series_volume %} ({{ series_volume }}){% endif %}{% if series_index is defined and (series_index or series_suffix or series_is_indexed) %} [{{ series_index }}{{ series_suffix }}]{% endif %}</span> <span class="meta-value">{{ series }}{% if series_index is defined and (series_index or series_suffix or series_is_indexed) %} [{{ series_index }}{{ series_suffix }}]{% endif %}</span>
</div> </div>
{% endif %} {% endif %}
<div class="meta-row"> <div class="meta-row">
@ -254,13 +254,7 @@
</svg> </svg>
</button> </button>
</div> </div>
<div class="edit-field"> <div class="edit-field"><label class="edit-label">Title</label><input class="edit-input" id="ed-title" type="text"/></div>
<label class="edit-label">Title</label>
<div style="display:flex;gap:0.5rem;align-items:center">
<input class="edit-input" id="ed-title" type="text" style="flex:1"/>
<button type="button" class="btn-secondary" style="white-space:nowrap;padding:0.35rem 0.6rem;font-size:0.75rem" onclick="generateTitle()" title="Generate title from series info">Auto</button>
</div>
</div>
<div class="edit-field"> <div class="edit-field">
<label class="edit-label">Author</label> <label class="edit-label">Author</label>
<div class="genre-wrap"> <div class="genre-wrap">
@ -283,8 +277,7 @@
<div class="genre-dropdown" id="series-dropdown" style="display:none"></div> <div class="genre-dropdown" id="series-dropdown" style="display:none"></div>
</div> </div>
</div> </div>
<div class="edit-field"><label class="edit-label">Year/Volume</label><input class="edit-input" id="ed-series-volume" type="text" placeholder="e.g. 2024"/></div> <div class="edit-field"><label class="edit-label">Volume</label><input class="edit-input" id="ed-series-index" type="text" placeholder="e.g. 1 or 21a"/></div>
<div class="edit-field"><label class="edit-label">Number</label><input class="edit-input" id="ed-series-index" type="text" placeholder="e.g. 1 or 21a"/></div>
</div> </div>
<div class="edit-field"> <div class="edit-field">
<label class="edit-label">Status</label> <label class="edit-label">Status</label>
@ -296,17 +289,6 @@
<option value="Long-Term Hold">Long-Term Hold</option> <option value="Long-Term Hold">Long-Term Hold</option>
</select> </select>
</div> </div>
<div class="edit-field">
<label class="edit-label">Rating</label>
<select class="edit-select" id="ed-rating">
<option value="0">— No rating</option>
<option value="1">★ 1</option>
<option value="2">★★ 2</option>
<option value="3">★★★ 3</option>
<option value="4">★★★★ 4</option>
<option value="5">★★★★★ 5</option>
</select>
</div>
<div class="edit-field"><label class="edit-label">Source URL</label><input class="edit-input" id="ed-url" type="url" placeholder="https://…"/></div> <div class="edit-field"><label class="edit-label">Source URL</label><input class="edit-input" id="ed-url" type="url" placeholder="https://…"/></div>
<div class="edit-field"><label class="edit-label">Description</label><textarea class="edit-input edit-textarea" id="ed-description" rows="5" placeholder="Story description…"></textarea></div> <div class="edit-field"><label class="edit-label">Description</label><textarea class="edit-input edit-textarea" id="ed-description" rows="5" placeholder="Story description…"></textarea></div>
<div class="edit-field"><label class="edit-label">Updated</label><input class="edit-input" id="ed-publish-date" type="date" style="color-scheme:dark"/></div> <div class="edit-field"><label class="edit-label">Updated</label><input class="edit-input" id="ed-publish-date" type="date" style="color-scheme:dark"/></div>
@ -386,7 +368,6 @@
series: {{ (series or '') | tojson }}, series: {{ (series or '') | tojson }},
series_index: {{ series_index or 0 }}, series_index: {{ series_index or 0 }},
series_suffix: {{ (series_suffix or '') | tojson }}, series_suffix: {{ (series_suffix or '') | tojson }},
series_volume: {{ (series_volume or '') | tojson }},
publication_status: {{ (publication_status or '') | tojson }}, publication_status: {{ (publication_status or '') | tojson }},
source_url: {{ (source_url or '') | tojson }}, source_url: {{ (source_url or '') | tojson }},
publish_date: {{ (publish_date or '') | tojson }}, publish_date: {{ (publish_date or '') | tojson }},

2
containers/novela/templates/builder.html
View File

@ -3,7 +3,7 @@
<head> <head>
<meta charset="UTF-8"/> <meta charset="UTF-8"/>
<meta name="viewport" content="width=device-width, initial-scale=1.0"/> <meta name="viewport" content="width=device-width, initial-scale=1.0"/>
<title>Novela{% if develop_mode() %} Develop{% endif %} — Book Builder{% if view == 'editor' %}: {{ draft.title }}{% endif %}</title> <title>Novela — Book Builder{% if view == 'editor' %}: {{ draft.title }}{% endif %}</title>
<link rel="icon" href="/static/favicon.ico" sizes="16x16"/> <link rel="icon" href="/static/favicon.ico" sizes="16x16"/>
<link rel="icon" type="image/png" sizes="32x32" href="/static/favicon-32.png"/> <link rel="icon" type="image/png" sizes="32x32" href="/static/favicon-32.png"/>
<link rel="icon" type="image/png" sizes="256x256" href="/static/favicon-256.png"/> <link rel="icon" type="image/png" sizes="256x256" href="/static/favicon-256.png"/>

66
containers/novela/templates/bulk_import.html
View File

@ -3,7 +3,7 @@
<head> <head>
<meta charset="UTF-8"/> <meta charset="UTF-8"/>
<meta name="viewport" content="width=device-width, initial-scale=1.0"/> <meta name="viewport" content="width=device-width, initial-scale=1.0"/>
<title>Novela{% if develop_mode() %} Develop{% endif %} — Bulk Import</title> <title>Novela – Bulk Import</title>
<link rel="icon" href="/static/favicon.ico" sizes="16x16"/> <link rel="icon" href="/static/favicon.ico" sizes="16x16"/>
<link rel="icon" type="image/png" sizes="32x32" href="/static/favicon-32.png"/> <link rel="icon" type="image/png" sizes="32x32" href="/static/favicon-32.png"/>
<link rel="icon" type="image/png" sizes="256x256" href="/static/favicon-256.png"/> <link rel="icon" type="image/png" sizes="256x256" href="/static/favicon-256.png"/>
@ -189,7 +189,7 @@
td.td-skip input[type="checkbox"] { cursor: pointer; accent-color: var(--error); } td.td-skip input[type="checkbox"] { cursor: pointer; accent-color: var(--error); }
tbody tr.row-dup { background: rgba(200,90,58,0.06); } tbody tr.row-dup { background: rgba(200,90,58,0.06); }
tbody tr.row-dup:hover { background: rgba(200,90,58,0.10); } tbody tr.row-dup:hover { background: rgba(200,90,58,0.10); }
tbody tr.row-skipped { opacity: 0.38; } tbody tr.row-dup.row-skipped { opacity: 0.38; }
.cnt-dup { color: var(--error); } .cnt-dup { color: var(--error); }
.dup-actions { .dup-actions {
display: flex; gap: 0.5rem; align-items: center; display: flex; gap: 0.5rem; align-items: center;
@ -322,10 +322,6 @@
<div class="suggest-dropdown" id="series-dropdown" style="display:none"></div> <div class="suggest-dropdown" id="series-dropdown" style="display:none"></div>
</div> </div>
</div> </div>
<div>
<label>Year/Vol. <span style="color:var(--text-dim)">(series volume)</span></label>
<input type="text" id="shared-series-volume" autocomplete="off" oninput="updatePreview()" placeholder="e.g. 2024"/>
</div>
<div> <div>
<label>Status</label> <label>Status</label>
<select id="shared-status" oninput="updatePreview()"> <select id="shared-status" oninput="updatePreview()">
@ -344,14 +340,6 @@
<label>Tags <span style="color:var(--text-dim)">(comma-separated)</span></label> <label>Tags <span style="color:var(--text-dim)">(comma-separated)</span></label>
<input type="text" id="shared-tags"/> <input type="text" id="shared-tags"/>
<label style="display:flex;align-items:center;gap:0.5rem;margin-top:0.5rem;cursor:pointer">
<input type="checkbox" id="auto-title" oninput="updatePreview()"/>
Auto-generate titles from series info &mdash; <span style="color:var(--text-dim)">for comics without individual titles</span>
</label>
<p class="hint" id="auto-title-hint" style="display:none">
Format: <code>Series (Year/Vol) #Number</code>. Only fills in rows where title is empty or matches the filename stem.
</p>
</div> </div>
<!-- Card 3: Files --> <!-- Card 3: Files -->
@ -385,13 +373,12 @@
<th style="width:2.5rem">#</th> <th style="width:2.5rem">#</th>
<th>Filename</th> <th>Filename</th>
<th>Series</th> <th>Series</th>
<th>Yr/Vol</th>
<th>Vol</th> <th>Vol</th>
<th>Title</th> <th>Title</th>
<th>Author</th> <th>Author</th>
<th>Publisher</th> <th>Publisher</th>
<th>Year</th> <th>Year</th>
<th style="width:2.5rem" title="Skip this file during import">Skip</th> <th id="th-skip" style="width:2.5rem;display:none" title="Skip this file during import">Skip</th>
<th style="width:1.5rem"></th> <th style="width:1.5rem"></th>
</tr> </tr>
</thead> </thead>
@ -446,7 +433,6 @@
// ── State ────────────────────────────────────────────────────────────────── // ── State ──────────────────────────────────────────────────────────────────
const PLACEHOLDER_META = [ const PLACEHOLDER_META = [
{ key: 'series', label: '%series%', color: 'var(--accent)' }, { key: 'series', label: '%series%', color: 'var(--accent)' },
{ key: 'series_volume', label: '%series_volume%', color: '#d07840' },
{ key: 'volume', label: '%volume%', color: '#4a90b8' }, { key: 'volume', label: '%volume%', color: '#4a90b8' },
{ key: 'title', label: '%title%', color: 'var(--success)' }, { key: 'title', label: '%title%', color: 'var(--success)' },
{ key: 'year', label: '%year%', color: 'var(--warning)' }, { key: 'year', label: '%year%', color: 'var(--warning)' },
@ -458,7 +444,7 @@
]; ];
let selectedFiles = []; let selectedFiles = [];
let parsedRows = []; // [{original_filename, series, series_volume, volume, title, year, author, publisher, status, genres, tags, _warn}] let parsedRows = []; // [{original_filename, series, volume, title, year, author, publisher, status, genres, tags, _warn}]
const BATCH_SIZE = 5; const BATCH_SIZE = 5;
@ -602,30 +588,14 @@
const pattern = document.getElementById('pattern-input').value; const pattern = document.getElementById('pattern-input').value;
const sharedSeries = document.getElementById('shared-series').value.trim(); const sharedSeries = document.getElementById('shared-series').value.trim();
const sharedSeriesVolume = document.getElementById('shared-series-volume').value.trim();
const autoTitle = document.getElementById('auto-title').checked;
document.getElementById('auto-title-hint').style.display = autoTitle ? '' : 'none';
parsedRows = selectedFiles.map(f => { parsedRows = selectedFiles.map(f => {
const stem = f.name.replace(/\.[^.]+$/, ''); const stem = f.name.replace(/\.[^.]+$/, '');
const parsed = parseFilename(stem, pattern); const parsed = parseFilename(stem, pattern);
const series = sharedSeries || parsed.series || '';
const series_volume = sharedSeriesVolume || parsed.series_volume || '';
const volume = parsed.volume || '';
let title = parsed.title || '';
if (autoTitle && series && !title) {
title = series;
if (series_volume) title += ` (${series_volume})`;
if (volume) title += ` #${volume}`;
}
return { return {
original_filename: f.name, original_filename: f.name,
series, series: sharedSeries || parsed.series || '',
series_volume, volume: parsed.volume || '',
volume, title: parsed.title || stem,
title: title || stem,
year: parsed.year || '', year: parsed.year || '',
author: parsed.author || '', author: parsed.author || '',
publisher: parsed.publisher || '', publisher: parsed.publisher || '',
@ -648,7 +618,6 @@
const items = parsedRows.map(r => ({ const items = parsedRows.map(r => ({
title: r.title, title: r.title,
author: r.author || sharedAuthor, author: r.author || sharedAuthor,
series: r.series,
volume: r.volume, volume: r.volume,
})); }));
try { try {
@ -677,6 +646,8 @@
tbody.innerHTML = ''; tbody.innerHTML = '';
const hasDups = parsedRows.some(r => r._duplicate); const hasDups = parsedRows.some(r => r._duplicate);
const thSkip = document.getElementById('th-skip');
if (thSkip) thSkip.style.display = hasDups ? '' : 'none';
let warnCount = 0; let warnCount = 0;
let dupCount = 0; let dupCount = 0;
@ -691,7 +662,7 @@
const classes = []; const classes = [];
if (row._warn) classes.push('row-warn'); if (row._warn) classes.push('row-warn');
if (row._duplicate) classes.push('row-dup'); if (row._duplicate) classes.push('row-dup');
if (row._skip) classes.push('row-skipped'); if (row._duplicate && row._skip) classes.push('row-skipped');
if (classes.length) tr.className = classes.join(' '); if (classes.length) tr.className = classes.join(' ');
// # // #
@ -708,10 +679,8 @@
tr.appendChild(tdFn); tr.appendChild(tdFn);
// Editable fields // Editable fields
const sharedSeriesVolume = document.getElementById('shared-series-volume').value.trim();
const fields = [ const fields = [
{ key: 'series', placeholder: '—' }, { key: 'series', placeholder: '—' },
{ key: 'series_volume', placeholder: sharedSeriesVolume || '—' },
{ key: 'volume', placeholder: '—' }, { key: 'volume', placeholder: '—' },
{ key: 'title', placeholder: 'Title' }, { key: 'title', placeholder: 'Title' },
{ key: 'author', placeholder: sharedAuthor || '—' }, { key: 'author', placeholder: sharedAuthor || '—' },
@ -740,9 +709,11 @@
tr.appendChild(td); tr.appendChild(td);
}); });
// Skip checkbox (always visible) // Skip checkbox (only shown when duplicates exist)
const tdSkip = document.createElement('td'); const tdSkip = document.createElement('td');
tdSkip.className = 'td-skip'; tdSkip.className = 'td-skip';
tdSkip.style.display = hasDups ? '' : 'none';
if (row._duplicate) {
const cb = document.createElement('input'); const cb = document.createElement('input');
cb.type = 'checkbox'; cb.type = 'checkbox';
cb.checked = row._skip; cb.checked = row._skip;
@ -753,6 +724,7 @@
renderPreviewStats(); renderPreviewStats();
}); });
tdSkip.appendChild(cb); tdSkip.appendChild(cb);
}
tr.appendChild(tdSkip); tr.appendChild(tdSkip);
// Warning indicator // Warning indicator
@ -778,9 +750,14 @@
const statsEl = document.getElementById('preview-stats'); const statsEl = document.getElementById('preview-stats');
let stats = `<span class="cnt-ok">${importCount} to import</span>`; let stats = `<span class="cnt-ok">${importCount} to import</span>`;
if (skipCount) stats += ` &nbsp; <span class="cnt-warn">${skipCount} skipped</span>`;
if (warnCount) stats += ` &nbsp; <span class="cnt-warn">${warnCount} to check</span>`; if (warnCount) stats += ` &nbsp; <span class="cnt-warn">${warnCount} to check</span>`;
if (dupCount) stats += ` &nbsp; <span class="cnt-dup">${dupCount} duplicate${dupCount !== 1 ? 's' : ''}</span>`; if (dupCount) {
stats += ` &nbsp; <span class="cnt-dup">${dupCount} duplicate${dupCount !== 1 ? 's' : ''}</span>`;
stats += ` &nbsp; <span class="dup-actions">`;
stats += `<button onclick="setAllDuplicatesSkip(true)">Skip all</button>`;
stats += `<button onclick="setAllDuplicatesSkip(false)">Import all</button>`;
stats += `</span>`;
}
statsEl.innerHTML = stats; statsEl.innerHTML = stats;
document.getElementById('import-btn-label').textContent = document.getElementById('import-btn-label').textContent =
@ -800,7 +777,6 @@
author: document.getElementById('shared-author').value.trim(), author: document.getElementById('shared-author').value.trim(),
publisher: document.getElementById('shared-publisher').value.trim(), publisher: document.getElementById('shared-publisher').value.trim(),
series: document.getElementById('shared-series').value.trim(), series: document.getElementById('shared-series').value.trim(),
series_volume: document.getElementById('shared-series-volume').value.trim(),
status: document.getElementById('shared-status').value, status: document.getElementById('shared-status').value,
genres: document.getElementById('shared-genres').value.trim(), genres: document.getElementById('shared-genres').value.trim(),
tags: document.getElementById('shared-tags').value.trim(), tags: document.getElementById('shared-tags').value.trim(),

2
containers/novela/templates/changelog.html
View File

@ -3,7 +3,7 @@
<head> <head>
<meta charset="UTF-8"/> <meta charset="UTF-8"/>
<meta name="viewport" content="width=device-width, initial-scale=1.0"/> <meta name="viewport" content="width=device-width, initial-scale=1.0"/>
<title>Novela{% if develop_mode() %} Develop{% endif %} — Changelog</title> <title>Novela — Changelog</title>
<link rel="icon" href="/static/favicon.ico" sizes="16x16"/> <link rel="icon" href="/static/favicon.ico" sizes="16x16"/>
<link rel="icon" type="image/png" sizes="32x32" href="/static/favicon-32.png"/> <link rel="icon" type="image/png" sizes="32x32" href="/static/favicon-32.png"/>
<link rel="icon" type="image/png" sizes="256x256" href="/static/favicon-256.png"/> <link rel="icon" type="image/png" sizes="256x256" href="/static/favicon-256.png"/>

2
containers/novela/templates/credentials.html
View File

@ -3,7 +3,7 @@
<head> <head>
<meta charset="UTF-8"/> <meta charset="UTF-8"/>
<meta name="viewport" content="width=device-width, initial-scale=1.0"/> <meta name="viewport" content="width=device-width, initial-scale=1.0"/>
<title>Novela{% if develop_mode() %} Develop{% endif %} — Credentials</title> <title>Novela — Credentials</title>
<link rel="icon" href="/static/favicon.ico" sizes="16x16"/> <link rel="icon" href="/static/favicon.ico" sizes="16x16"/>
<link rel="icon" type="image/png" sizes="32x32" href="/static/favicon-32.png"/> <link rel="icon" type="image/png" sizes="32x32" href="/static/favicon-32.png"/>
<link rel="icon" type="image/png" sizes="256x256" href="/static/favicon-256.png"/> <link rel="icon" type="image/png" sizes="256x256" href="/static/favicon-256.png"/>

2
containers/novela/templates/debug.html
View File

@ -3,7 +3,7 @@
<head> <head>
<meta charset="UTF-8"/> <meta charset="UTF-8"/>
<meta name="viewport" content="width=device-width, initial-scale=1.0"/> <meta name="viewport" content="width=device-width, initial-scale=1.0"/>
<title>Novela{% if develop_mode() %} Develop{% endif %} — Debug</title> <title>Novela — Debug</title>
<link rel="icon" href="/static/favicon.ico" sizes="16x16"/> <link rel="icon" href="/static/favicon.ico" sizes="16x16"/>
<link rel="icon" type="image/png" sizes="32x32" href="/static/favicon-32.png"/> <link rel="icon" type="image/png" sizes="32x32" href="/static/favicon-32.png"/>
<link rel="icon" type="image/png" sizes="256x256" href="/static/favicon-256.png"/> <link rel="icon" type="image/png" sizes="256x256" href="/static/favicon-256.png"/>

2
containers/novela/templates/editor.html
View File

@ -3,7 +3,7 @@
<head> <head>
<meta charset="UTF-8"/> <meta charset="UTF-8"/>
<meta name="viewport" content="width=device-width, initial-scale=1.0"/> <meta name="viewport" content="width=device-width, initial-scale=1.0"/>
<title>Novela{% if develop_mode() %} Develop{% endif %} — Edit {{ title or filename }}</title> <title>Novela — Edit {{ title or filename }}</title>
<link rel="icon" href="/static/favicon.ico" sizes="16x16"/> <link rel="icon" href="/static/favicon.ico" sizes="16x16"/>
<link rel="icon" type="image/png" sizes="32x32" href="/static/favicon-32.png"/> <link rel="icon" type="image/png" sizes="32x32" href="/static/favicon-32.png"/>
<link rel="icon" type="image/png" sizes="256x256" href="/static/favicon-256.png"/> <link rel="icon" type="image/png" sizes="256x256" href="/static/favicon-256.png"/>

2
containers/novela/templates/following.html
View File

@ -3,7 +3,7 @@
<head> <head>
<meta charset="UTF-8"/> <meta charset="UTF-8"/>
<meta name="viewport" content="width=device-width, initial-scale=1.0"/> <meta name="viewport" content="width=device-width, initial-scale=1.0"/>
<title>Novela{% if develop_mode() %} Develop{% endif %} — Following</title> <title>Novela — Following</title>
<link rel="icon" href="/static/favicon.ico" sizes="16x16"/> <link rel="icon" href="/static/favicon.ico" sizes="16x16"/>
<link rel="icon" type="image/png" sizes="32x32" href="/static/favicon-32.png"/> <link rel="icon" type="image/png" sizes="32x32" href="/static/favicon-32.png"/>
<link rel="icon" type="image/png" sizes="256x256" href="/static/favicon-256.png"/> <link rel="icon" type="image/png" sizes="256x256" href="/static/favicon-256.png"/>

2
containers/novela/templates/grabber.html
View File

@ -3,7 +3,7 @@
<head> <head>
<meta charset="UTF-8"/> <meta charset="UTF-8"/>
<meta name="viewport" content="width=device-width, initial-scale=1.0"/> <meta name="viewport" content="width=device-width, initial-scale=1.0"/>
<title>Novela{% if develop_mode() %} Develop{% endif %}</title> <title>Novela</title>
<link rel="icon" href="/static/favicon.ico" sizes="16x16"/> <link rel="icon" href="/static/favicon.ico" sizes="16x16"/>
<link rel="icon" type="image/png" sizes="32x32" href="/static/favicon-32.png"/> <link rel="icon" type="image/png" sizes="32x32" href="/static/favicon-32.png"/>
<link rel="icon" type="image/png" sizes="256x256" href="/static/favicon-256.png"/> <link rel="icon" type="image/png" sizes="256x256" href="/static/favicon-256.png"/>

2
containers/novela/templates/home.html
View File

@ -3,7 +3,7 @@
<head> <head>
<meta charset="UTF-8"/> <meta charset="UTF-8"/>
<meta name="viewport" content="width=device-width, initial-scale=1.0"/> <meta name="viewport" content="width=device-width, initial-scale=1.0"/>
<title>Novela{% if develop_mode() %} Develop{% endif %} — Home</title> <title>Novela — Home</title>
<link rel="icon" href="/static/favicon.ico" sizes="16x16"/> <link rel="icon" href="/static/favicon.ico" sizes="16x16"/>
<link rel="icon" type="image/png" sizes="32x32" href="/static/favicon-32.png"/> <link rel="icon" type="image/png" sizes="32x32" href="/static/favicon-32.png"/>
<link rel="icon" type="image/png" sizes="256x256" href="/static/favicon-256.png"/> <link rel="icon" type="image/png" sizes="256x256" href="/static/favicon-256.png"/>

2
containers/novela/templates/index.html
View File

@ -3,7 +3,7 @@
<head> <head>
<meta charset="UTF-8"/> <meta charset="UTF-8"/>
<meta name="viewport" content="width=device-width, initial-scale=1.0"/> <meta name="viewport" content="width=device-width, initial-scale=1.0"/>
<title>Novela{% if develop_mode() %} Develop{% endif %}</title> <title>Novela</title>
<link rel="icon" href="/static/favicon.ico" sizes="16x16"/> <link rel="icon" href="/static/favicon.ico" sizes="16x16"/>
<link rel="icon" type="image/png" sizes="32x32" href="/static/favicon-32.png"/> <link rel="icon" type="image/png" sizes="32x32" href="/static/favicon-32.png"/>
<link rel="icon" type="image/png" sizes="256x256" href="/static/favicon-256.png"/> <link rel="icon" type="image/png" sizes="256x256" href="/static/favicon-256.png"/>

2
containers/novela/templates/library.html
View File

@ -3,7 +3,7 @@
<head> <head>
<meta charset="UTF-8"/> <meta charset="UTF-8"/>
<meta name="viewport" content="width=device-width, initial-scale=1.0"/> <meta name="viewport" content="width=device-width, initial-scale=1.0"/>
<title>Novela{% if develop_mode() %} Develop{% endif %} — Library</title> <title>Novela — Library</title>
<link rel="icon" href="/static/favicon.ico" sizes="16x16"/> <link rel="icon" href="/static/favicon.ico" sizes="16x16"/>
<link rel="icon" type="image/png" sizes="32x32" href="/static/favicon-32.png"/> <link rel="icon" type="image/png" sizes="32x32" href="/static/favicon-32.png"/>
<link rel="icon" type="image/png" sizes="256x256" href="/static/favicon-256.png"/> <link rel="icon" type="image/png" sizes="256x256" href="/static/favicon-256.png"/>

85
containers/novela/templates/reader.html
View File

@ -3,7 +3,7 @@
<head> <head>
<meta charset="UTF-8"/> <meta charset="UTF-8"/>
<meta name="viewport" content="width=device-width, initial-scale=1.0"/> <meta name="viewport" content="width=device-width, initial-scale=1.0"/>
<title>Novela{% if develop_mode() %} Develop{% endif %} — {{ title }}</title> <title>Novela — {{ title }}</title>
<link rel="icon" href="/static/favicon.ico" sizes="16x16"/> <link rel="icon" href="/static/favicon.ico" sizes="16x16"/>
<link rel="icon" type="image/png" sizes="32x32" href="/static/favicon-32.png"/> <link rel="icon" type="image/png" sizes="32x32" href="/static/favicon-32.png"/>
<link rel="icon" type="image/png" sizes="256x256" href="/static/favicon-256.png"/> <link rel="icon" type="image/png" sizes="256x256" href="/static/favicon-256.png"/>
@ -69,16 +69,6 @@
.btn-header-read:hover { background: rgba(107,170,107,0.08); border-color: var(--success); } .btn-header-read:hover { background: rgba(107,170,107,0.08); border-color: var(--success); }
.btn-header-bm { color: var(--accent); border-color: rgba(255,162,14,0.3); } .btn-header-bm { color: var(--accent); border-color: rgba(255,162,14,0.3); }
.btn-header-bm:hover { background: rgba(255,162,14,0.08); border-color: var(--accent); } .btn-header-bm:hover { background: rgba(255,162,14,0.08); border-color: var(--accent); }
.btn-header-series {
display: none;
color: var(--text-faint); border-color: rgba(255,255,255,0.08);
padding: 0.3rem 0.5rem;
}
.btn-header-series.active {
display: flex;
color: var(--text-dim); border-color: var(--border);
}
.btn-header-series.active:hover { color: var(--text); border-color: var(--text-faint); }
/* ── Bookmark modal ── */ /* ── Bookmark modal ── */
.bm-overlay { .bm-overlay {
@ -197,7 +187,7 @@
/* Chapter content */ /* Chapter content */
#chapter-content { #chapter-content {
font-family: var(--serif); font-family: var(--serif);
font-size: var(--reader-font-size, 1.05rem); font-size: 1.05rem;
line-height: 1.85; line-height: 1.85;
color: var(--text); color: var(--text);
} }
@ -320,14 +310,6 @@
<input type="range" id="width-slider" min="30" max="100" step="1" <input type="range" id="width-slider" min="30" max="100" step="1"
value="65" oninput="applyWidth(this.value)"/> value="65" oninput="applyWidth(this.value)"/>
</div> </div>
<div class="settings-row">
<div class="settings-label">
Font size
<span id="fontsize-value">105%</span>
</div>
<input type="range" id="fontsize-slider" min="80" max="150" step="1"
value="105" oninput="applyFontSize(this.value)"/>
</div>
<div class="settings-row"> <div class="settings-row">
<div class="settings-label">Text colour</div> <div class="settings-label">Text colour</div>
<div class="colour-swatches"> <div class="colour-swatches">
@ -357,18 +339,6 @@
</a> </a>
<div class="header-title" id="header-title"></div> <div class="header-title" id="header-title"></div>
<div class="header-actions"> <div class="header-actions">
<button class="btn-header btn-header-series" id="btn-series-prev" title="" onclick="goSeriesPrev()">
<svg width="13" height="13" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2.5">
<polyline points="19 20 9 12 19 4"/>
<line x1="5" y1="4" x2="5" y2="20"/>
</svg>
</button>
<button class="btn-header btn-header-series" id="btn-series-next" title="" onclick="goSeriesNext()">
<svg width="13" height="13" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2.5">
<polyline points="5 4 15 12 5 20"/>
<line x1="19" y1="4" x2="19" y2="20"/>
</svg>
</button>
<button class="btn-header btn-header-bm" onclick="openBookmarkModal()" title="Add bookmark at current position"> <button class="btn-header btn-header-bm" onclick="openBookmarkModal()" title="Add bookmark at current position">
<svg width="12" height="12" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2"> <svg width="12" height="12" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2">
<path d="M19 21l-7-5-7 5V5a2 2 0 0 1 2-2h10a2 2 0 0 1 2 2z"/> <path d="M19 21l-7-5-7 5V5a2 2 0 0 1 2-2h10a2 2 0 0 1 2 2z"/>
@ -421,7 +391,6 @@
let currentIndex = 0; let currentIndex = 0;
let saveTimer = null; let saveTimer = null;
let scrollTimer = null; let scrollTimer = null;
let seriesNav = { prev: null, next: null };
// ── Width setting ────────────────────────────────────────────── // ── Width setting ──────────────────────────────────────────────
function applyWidth(pct) { function applyWidth(pct) {
@ -451,20 +420,6 @@
applyTextColour(saved); applyTextColour(saved);
} }
// ── Font size ──────────────────────────────────────────────────
function applyFontSize(pct) {
const val = parseInt(pct, 10);
document.documentElement.style.setProperty('--reader-font-size', (val / 100) + 'rem');
document.getElementById('fontsize-value').textContent = val + '%';
document.getElementById('fontsize-slider').value = val;
localStorage.setItem('reader-font-size', val);
}
function loadFontSize() {
const saved = parseInt(localStorage.getItem('reader-font-size') || '105', 10);
applyFontSize(saved);
}
// ── Settings drawer ──────────────────────────────────────────── // ── Settings drawer ────────────────────────────────────────────
function toggleSettings() { function toggleSettings() {
const open = document.getElementById('settings-drawer').classList.toggle('open'); const open = document.getElementById('settings-drawer').classList.toggle('open');
@ -625,8 +580,6 @@
async function init() { async function init() {
loadWidth(); loadWidth();
loadTextColour(); loadTextColour();
loadFontSize();
loadSeriesNav();
const progResp = await fetch(`/library/progress/${encodeURIComponent(filename)}`); const progResp = await fetch(`/library/progress/${encodeURIComponent(filename)}`);
const prog = await progResp.json(); const prog = await progResp.json();
@ -673,45 +626,11 @@
document.getElementById('loading').style.display = 'none'; document.getElementById('loading').style.display = 'none';
} }
// ── Series navigation ──────────────────────────────────────────
async function loadSeriesNav() {
try {
const resp = await fetch(`/api/series-nav/${encodeURIComponent(filename)}`);
seriesNav = await resp.json();
} catch { return; }
const btnPrev = document.getElementById('btn-series-prev');
const btnNext = document.getElementById('btn-series-next');
if (seriesNav.prev) {
const label = seriesNav.prev.index ? `#${seriesNav.prev.index}${seriesNav.prev.suffix} ${seriesNav.prev.title}` : seriesNav.prev.title;
btnPrev.title = label;
btnPrev.classList.add('active');
}
if (seriesNav.next) {
const label = seriesNav.next.index ? `#${seriesNav.next.index}${seriesNav.next.suffix} ${seriesNav.next.title}` : seriesNav.next.title;
btnNext.title = label;
btnNext.classList.add('active');
}
}
function goSeriesPrev() {
if (seriesNav.prev) window.location.href = `/library/read/${encodeURIComponent(seriesNav.prev.filename)}`;
}
function goSeriesNext() {
if (seriesNav.next) window.location.href = `/library/read/${encodeURIComponent(seriesNav.next.filename)}`;
}
async function markRead() { async function markRead() {
clearTimeout(saveTimer); clearTimeout(saveTimer);
await fetch(`/library/mark-read/${encodeURIComponent(filename)}`, { method: 'POST' }); await fetch(`/library/mark-read/${encodeURIComponent(filename)}`, { method: 'POST' });
if (seriesNav.next) {
window.location.href = `/library/read/${encodeURIComponent(seriesNav.next.filename)}`;
} else {
window.location.href = `/library/book/${encodeURIComponent(filename)}`; window.location.href = `/library/book/${encodeURIComponent(filename)}`;
} }
}
// ── Bookmarks ────────────────────────────────────────────────── // ── Bookmarks ──────────────────────────────────────────────────
function openBookmarkModal() { function openBookmarkModal() {

50
containers/novela/templates/search.html
View File

@ -3,7 +3,7 @@
<head> <head>
<meta charset="UTF-8"/> <meta charset="UTF-8"/>
<meta name="viewport" content="width=device-width, initial-scale=1.0"/> <meta name="viewport" content="width=device-width, initial-scale=1.0"/>
<title>Novela{% if develop_mode() %} Develop{% endif %} — Search</title> <title>Novela — Search</title>
<link rel="icon" href="/static/favicon.ico" sizes="16x16"/> <link rel="icon" href="/static/favicon.ico" sizes="16x16"/>
<link rel="icon" type="image/png" sizes="32x32" href="/static/favicon-32.png"/> <link rel="icon" type="image/png" sizes="32x32" href="/static/favicon-32.png"/>
<link rel="icon" type="image/png" sizes="256x256" href="/static/favicon-256.png"/> <link rel="icon" type="image/png" sizes="256x256" href="/static/favicon-256.png"/>
@ -19,18 +19,7 @@
.page-title { font-size: 1.4rem; font-weight: 700; margin-bottom: 1.5rem; } .page-title { font-size: 1.4rem; font-weight: 700; margin-bottom: 1.5rem; }
.search-bar { display: flex; gap: 0.5rem; margin-bottom: 0.75rem; } .search-bar { display: flex; gap: 0.5rem; margin-bottom: 2rem; }
.search-mode { display: flex; gap: 0; margin-bottom: 0.5rem; }
.search-filter { display: flex; gap: 0; margin-bottom: 2rem; }
.mode-btn {
padding: 0.35rem 0.85rem; font-size: 0.8rem; cursor: pointer;
border: 1px solid var(--border); background: var(--surface); color: var(--text-dim);
font-family: var(--mono);
}
.mode-btn:first-child { border-radius: var(--radius) 0 0 var(--radius); }
.mode-btn:last-child { border-radius: 0 var(--radius) var(--radius) 0; border-left: none; }
.mode-btn + .mode-btn { border-left: none; }
.mode-btn.active { background: var(--surface2); color: var(--text); border-color: var(--accent); z-index: 1; }
.search-input { .search-input {
flex: 1; padding: 0.55rem 0.8rem; border-radius: var(--radius); flex: 1; padding: 0.55rem 0.8rem; border-radius: var(--radius);
border: 1px solid var(--border); background: var(--surface); color: var(--text); border: 1px solid var(--border); background: var(--surface); color: var(--text);
@ -95,15 +84,6 @@
placeholder="Search inside books…" autocomplete="off"/> placeholder="Search inside books…" autocomplete="off"/>
<button class="search-btn" onclick="doSearch()">Search</button> <button class="search-btn" onclick="doSearch()">Search</button>
</div> </div>
<div class="search-mode">
<button class="mode-btn active" id="mode-phrase" onclick="setMode('phrase')">Phrase</button>
<button class="mode-btn" id="mode-words" onclick="setMode('words')">All words</button>
</div>
<div class="search-filter">
<button class="mode-btn active" id="filter-all" onclick="setFilter('all')">All</button>
<button class="mode-btn" id="filter-unread_novels" onclick="setFilter('unread_novels')">Unread novels</button>
<button class="mode-btn" id="filter-unread_shorts" onclick="setFilter('unread_shorts')">Unread shorts</button>
</div>
<div class="search-status" id="search-status"></div> <div class="search-status" id="search-status"></div>
<div class="result-list" id="result-list"></div> <div class="result-list" id="result-list"></div>
@ -113,41 +93,21 @@
const input = document.getElementById('search-input'); const input = document.getElementById('search-input');
const statusEl = document.getElementById('search-status'); const statusEl = document.getElementById('search-status');
const listEl = document.getElementById('result-list'); const listEl = document.getElementById('result-list');
let searchMode = 'phrase';
let searchFilter = 'all';
function setMode(mode) {
searchMode = mode;
document.getElementById('mode-phrase').classList.toggle('active', mode === 'phrase');
document.getElementById('mode-words').classList.toggle('active', mode === 'words');
}
function setFilter(filter) {
searchFilter = filter;
document.getElementById('filter-all').classList.toggle('active', filter === 'all');
document.getElementById('filter-unread_novels').classList.toggle('active', filter === 'unread_novels');
document.getElementById('filter-unread_shorts').classList.toggle('active', filter === 'unread_shorts');
}
input.addEventListener('keydown', e => { if (e.key === 'Enter') doSearch(); }); input.addEventListener('keydown', e => { if (e.key === 'Enter') doSearch(); });
// Auto-run if ?q= param provided // Auto-run if ?q= param provided
const urlParams = new URLSearchParams(location.search); const urlQ = new URLSearchParams(location.search).get('q');
const urlQ = urlParams.get('q');
const urlMode = urlParams.get('mode');
const urlFilter = urlParams.get('filter');
if (urlMode === 'words') setMode('words');
if (urlFilter === 'unread_novels' || urlFilter === 'unread_shorts') setFilter(urlFilter);
if (urlQ) { input.value = urlQ; doSearch(); } if (urlQ) { input.value = urlQ; doSearch(); }
async function doSearch() { async function doSearch() {
const q = input.value.trim(); const q = input.value.trim();
if (!q) { listEl.innerHTML = ''; statusEl.textContent = ''; return; } if (!q) { listEl.innerHTML = ''; statusEl.textContent = ''; return; }
history.replaceState(null, '', '/search?q=' + encodeURIComponent(q) + '&mode=' + searchMode + '&filter=' + searchFilter); history.replaceState(null, '', '/search?q=' + encodeURIComponent(q));
statusEl.textContent = 'Searching…'; statusEl.textContent = 'Searching…';
listEl.innerHTML = ''; listEl.innerHTML = '';
try { try {
const resp = await fetch('/api/search?q=' + encodeURIComponent(q) + '&mode=' + searchMode + '&filter=' + searchFilter); const resp = await fetch('/api/search?q=' + encodeURIComponent(q));
if (!resp.ok) throw new Error('Search failed'); if (!resp.ok) throw new Error('Search failed');
const results = await resp.json(); const results = await resp.json();
render(results, q); render(results, q);

95
containers/novela/templates/settings.html
View File

@ -3,7 +3,7 @@
<head> <head>
<meta charset="UTF-8"/> <meta charset="UTF-8"/>
<meta name="viewport" content="width=device-width, initial-scale=1.0"/> <meta name="viewport" content="width=device-width, initial-scale=1.0"/>
<title>Novela{% if develop_mode() %} Develop{% endif %} — Settings</title> <title>Novela — Settings</title>
<link rel="icon" href="/static/favicon.ico" sizes="16x16"/> <link rel="icon" href="/static/favicon.ico" sizes="16x16"/>
<link rel="icon" type="image/png" sizes="32x32" href="/static/favicon-32.png"/> <link rel="icon" type="image/png" sizes="32x32" href="/static/favicon-32.png"/>
<link rel="icon" type="image/png" sizes="256x256" href="/static/favicon-256.png"/> <link rel="icon" type="image/png" sizes="256x256" href="/static/favicon-256.png"/>
@ -169,22 +169,6 @@
<main class="main"> <main class="main">
<div class="main-title">Settings</div> <div class="main-title">Settings</div>
<!-- Develop mode -->
<div class="card">
<div class="card-title">Develop mode</div>
<div class="card-desc">
Mark this as a development instance. Adds a <strong>DEVELOP</strong> banner to every page
and shows <strong>Novela Develop</strong> in the browser tab title.
</div>
<label style="display:flex;align-items:center;gap:0.75rem;cursor:pointer">
<input type="checkbox" id="develop-mode-toggle" style="width:16px;height:16px;accent-color:var(--accent);cursor:pointer"
{% if develop_mode() %}checked{% endif %}
onchange="toggleDevelopMode(this.checked)"/>
<span style="font-family:var(--mono);font-size:0.8rem">Enable develop mode</span>
</label>
<div class="feedback" id="develop-feedback"></div>
</div>
<!-- Reading history reset --> <!-- Reading history reset -->
<div class="card"> <div class="card">
<div class="card-title">Reading history</div> <div class="card-title">Reading history</div>
@ -204,30 +188,6 @@
</button> </button>
<div class="feedback" id="reset-feedback"></div> <div class="feedback" id="reset-feedback"></div>
</div> </div>
<!-- Break image -->
<div class="card">
<div class="card-title">Break image</div>
<div class="card-desc">
The image used as a scene break in converted books.
Applies to all newly imported books. Upload a PNG, JPG or WebP image.
</div>
<div id="break-image-preview" style="margin-bottom:1rem;display:none">
<img id="break-image-img" src="" alt="break image"
style="max-height:40px;border:1px solid var(--border);border-radius:4px;padding:4px;background:var(--surface)"/>
</div>
<label class="btn" style="background:var(--surface);border:1px solid var(--border);color:var(--text);cursor:pointer">
<svg width="13" height="13" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2.5">
<path d="M21 15v4a2 2 0 0 1-2 2H5a2 2 0 0 1-2-2v-4"/>
<polyline points="17 8 12 3 7 8"/>
<line x1="12" y1="3" x2="12" y2="15"/>
</svg>
Upload break image
<input type="file" id="break-image-file" accept="image/png,image/jpeg,image/webp"
style="display:none" onchange="uploadBreakImage(this)"/>
</label>
<div class="feedback" id="break-image-feedback"></div>
</div>
<!-- Break detection patterns --> <!-- Break detection patterns -->
<div class="card"> <div class="card">
<div class="card-title">Break detection</div> <div class="card-title">Break detection</div>
@ -296,26 +256,6 @@
<script src="/static/books.js"></script> <script src="/static/books.js"></script>
<script> <script>
// ── Develop mode ───────────────────────────────────────────────────────────
async function toggleDevelopMode(enabled) {
const fb = document.getElementById('develop-feedback');
fb.className = 'feedback';
fb.textContent = '';
const resp = await fetch('/api/app-settings', {
method: 'PATCH',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ develop_mode: enabled }),
});
const data = await resp.json();
if (data.ok) {
location.reload();
} else {
fb.className = 'feedback err';
fb.textContent = 'Could not save setting.';
document.getElementById('develop-mode-toggle').checked = !enabled;
}
}
// ── Break patterns ───────────────────────────────────────────────────────── // ── Break patterns ─────────────────────────────────────────────────────────
let bpPatterns = []; let bpPatterns = [];
@ -443,45 +383,12 @@
// Enter key in add inputs // Enter key in add inputs
// ── Break image ────────────────────────────────────────────────────────────
async function loadBreakImagePreview() {
const resp = await fetch('/api/app-settings');
const data = await resp.json();
if (data.break_image_url) {
document.getElementById('break-image-img').src = data.break_image_url + '?t=' + Date.now();
document.getElementById('break-image-preview').style.display = 'block';
}
}
async function uploadBreakImage(input) {
const file = input.files[0];
if (!file) return;
const fb = document.getElementById('break-image-feedback');
fb.className = 'feedback';
fb.textContent = 'Uploading…';
const form = new FormData();
form.append('file', file);
const resp = await fetch('/api/app-settings/break-image', { method: 'POST', body: form });
const data = await resp.json();
input.value = '';
if (data.ok) {
fb.className = 'feedback ok';
fb.textContent = 'Break image updated.';
document.getElementById('break-image-img').src = data.url + '?t=' + Date.now();
document.getElementById('break-image-preview').style.display = 'block';
} else {
fb.className = 'feedback err';
fb.textContent = data.error || 'Upload failed.';
}
}
document.addEventListener('DOMContentLoaded', () => { document.addEventListener('DOMContentLoaded', () => {
document.getElementById('bp-text-input').addEventListener('keydown', e => { if (e.key === 'Enter') addPatternFromText(); }); document.getElementById('bp-text-input').addEventListener('keydown', e => { if (e.key === 'Enter') addPatternFromText(); });
document.getElementById('bp-regex-input').addEventListener('keydown', e => { if (e.key === 'Enter') addPattern('regex'); }); document.getElementById('bp-regex-input').addEventListener('keydown', e => { if (e.key === 'Enter') addPattern('regex'); });
document.getElementById('bp-css-input').addEventListener('keydown', e => { if (e.key === 'Enter') addPattern('css_class'); }); document.getElementById('bp-css-input').addEventListener('keydown', e => { if (e.key === 'Enter') addPattern('css_class'); });
document.getElementById('bp-test-input').addEventListener('keydown', e => { if (e.key === 'Enter') testBreak(); }); document.getElementById('bp-test-input').addEventListener('keydown', e => { if (e.key === 'Enter') testBreak(); });
loadBreakPatterns(); loadBreakPatterns();
loadBreakImagePreview();
}); });
// ── Reading history ──────────────────────────────────────────────────────── // ── Reading history ────────────────────────────────────────────────────────

2
containers/novela/templates/stats.html
View File

@ -3,7 +3,7 @@
<head> <head>
<meta charset="UTF-8"/> <meta charset="UTF-8"/>
<meta name="viewport" content="width=device-width, initial-scale=1.0"/> <meta name="viewport" content="width=device-width, initial-scale=1.0"/>
<title>Novela{% if develop_mode() %} Develop{% endif %} — Statistics</title> <title>Novela — Statistics</title>
<link rel="icon" href="/static/favicon.ico" sizes="16x16"/> <link rel="icon" href="/static/favicon.ico" sizes="16x16"/>
<link rel="icon" type="image/png" sizes="32x32" href="/static/favicon-32.png"/> <link rel="icon" type="image/png" sizes="32x32" href="/static/favicon-32.png"/>
<link rel="icon" type="image/png" sizes="256x256" href="/static/favicon-256.png"/> <link rel="icon" type="image/png" sizes="256x256" href="/static/favicon-256.png"/>

8
containers/novela/xhtml.py
View File

@ -1,7 +1,7 @@
import re import re
from html import escape as he from html import escape as he
from bs4 import Comment, NavigableString, Tag from bs4 import NavigableString, Tag
BREAK_PATTERNS = [ BREAK_PATTERNS = [
re.compile(r"^\s*[\*\-]{3,}\s*$"), # *** of --- re.compile(r"^\s*[\*\-]{3,}\s*$"), # *** of ---
@ -111,12 +111,6 @@ def element_to_xhtml(el, break_img_path: str = "../Images/break.png", empty_p_is
result += "\n" + trailer result += "\n" + trailer
return result return result
if isinstance(el, Comment):
# Preserve HTML comments as XML comments (e.g. scraper metadata).
# Sanitise "--" sequences which are illegal inside XML comments.
body = str(el).replace("--", "- -")
return f"<!-- {body} -->"
if isinstance(el, NavigableString): if isinstance(el, NavigableString):
text = str(el) text = str(el)
if text.strip(): if text.strip():

153
docs/TECHNICAL.md
View File

@ -6,7 +6,6 @@ It is the primary technical reference for the current implementation.
## Architecture ## Architecture
- Stack: FastAPI, Jinja2 templates, plain JavaScript, PostgreSQL 16, Docker. - Stack: FastAPI, Jinja2 templates, plain JavaScript, PostgreSQL 16, Docker.
- All routers import `templates` from `shared_templates.py` (a single `Jinja2Templates` instance). This module registers a `develop_mode()` callable as a Jinja2 global, making it available in every template without passing it explicitly per route.
- Startup lifecycle (`main.py`): - Startup lifecycle (`main.py`):
1. `init_pool()` 1. `init_pool()`
2. `run_migrations()` 2. `run_migrations()`
@ -27,23 +26,22 @@ All files are stored under `library/` (relative to the app working directory, ma
| Format | Path pattern | | Format | Path pattern |
|--------|-------------| |--------|-------------|
| EPUB (no series) | `library/epub/{publisher}/{author}/Stories/{title}.epub` | | EPUB (no series) | `library/epub/{publisher}/{author}/Stories/{title}.epub` |
| EPUB (series) | `library/epub/{publisher}/{author}/Series/{series}/{idx:03d}_-_{title}.epub` | | EPUB (series) | `library/epub/{publisher}/{author}/Series/{series}/{idx:03d} - {title}.epub` |
| PDF | `library/pdf/{publisher}/{author}/{title}.pdf` | | PDF | `library/pdf/{publisher}/{author}/{title}.pdf` |
| CBR (no series) | `library/comics/{publisher}/{author}/{title}.cbr` | | CBR (no series) | `library/comics/{publisher}/{author}/{title}.cbr` |
| CBR (series) | `library/comics/{publisher}/{author}/Series/{series}/{idx:03d}_-_{title}.cbr` | | CBR (series) | `library/comics/{publisher}/{author}/Series/{series}/{idx:03d} - {title}.cbr` |
| CBZ (no series) | `library/comics/{publisher}/{author}/{title}.cbz` | | CBZ (no series) | `library/comics/{publisher}/{author}/{title}.cbz` |
| CBZ (series) | `library/comics/{publisher}/{author}/Series/{series}/{idx:03d}_-_{title}.cbz` | | CBZ (series) | `library/comics/{publisher}/{author}/Series/{series}/{idx:03d} - {title}.cbz` |
- Segments are sanitised: special chars stripped, spaces replaced with `_`, max lengths applied (publisher/author 80, title 140, series 80). - Segments are sanitised: special chars stripped, max lengths applied (publisher/author 80, title 140, series 80).
- Series index is zero-padded to 3 digits (`001`, `002`, …), clamped to 1–999. - Series index is zero-padded to 3 digits (`001`, `002`, …), clamped to 1–999.
- Duplicate filenames get a `(2)`, `(3)`, … suffix. - Duplicate filenames get a `(2)`, `(3)`, … suffix.
- After any file move, empty parent directories are pruned up to `LIBRARY_ROOT`. - After any file move, empty parent directories are pruned up to `LIBRARY_ROOT`.
### Path logic ### Path logic
- `common.make_rel_path(media_type, publisher, author, title, series, series_index, series_suffix, ext)` — used by import and grabber. - `common.make_rel_path(media_type, publisher, author, title, series, series_index, ext)` — used by import and grabber.
- `reader.py _make_rel_path(publisher, author, title, series, series_index, series_suffix, ext)` — used by metadata PATCH; same logic, uses actual file extension. - `reader.py _make_rel_path(publisher, author, title, series, series_index, ext)` — used by metadata PATCH; same logic, uses actual file extension.
- `series_volume` is not part of the file path; it is stored in DB and OPF only.
- Both functions produce identical paths for all formats. - Both functions produce identical paths for all formats.
### Metadata save behaviour per format ### Metadata save behaviour per format
@ -68,10 +66,9 @@ All files are stored under `library/` (relative to the app working directory, ma
- `GET /download/{filename}` — download file with `Content-Disposition: attachment` - `GET /download/{filename}` — download file with `Content-Disposition: attachment`
- `GET /library/cover/{filename}` — serve cover (EPUB from file; PDF/CBR from cache) - `GET /library/cover/{filename}` — serve cover (EPUB from file; PDF/CBR from cache)
- `GET /library/cover-cached/{filename}` — serve cover from DB cache only - `GET /library/cover-cached/{filename}` — serve cover from DB cache only
- `POST /library/cover/{filename}` — upload/replace cover; for EPUB files: embeds cover in the EPUB and updates cache; for DB-stored books: stores cover directly in `library_cover_cache` and sets `has_cover = TRUE` - `POST /library/cover/{filename}` — upload/replace cover (EPUB only)
- `POST /library/want-to-read/{filename}` — toggle want-to-read flag - `POST /library/want-to-read/{filename}` — toggle want-to-read flag
- `POST /library/archive/{filename}` — toggle archived flag - `POST /library/archive/{filename}` — toggle archived flag
- `POST /library/archive-series` — set `archived` for all books in a series; body: `{"series": "…", "archive": true|false}`; returns `{ok, archived, count}`
- `POST /library/new/mark-reviewed` — bulk set `needs_review=false` - `POST /library/new/mark-reviewed` — bulk set `needs_review=false`
- `POST /library/bulk-delete` — delete multiple files; accepts `{"filenames": [...]}`, removes files from disk and DB in one query per batch; returns `{ok, deleted, skipped}` - `POST /library/bulk-delete` — delete multiple files; accepts `{"filenames": [...]}`, removes files from disk and DB in one query per batch; returns `{ok, deleted, skipped}`
- `POST /library/rating/{filename}` — set/clear star rating `{"rating": 0-5}` - `POST /library/rating/{filename}` — set/clear star rating `{"rating": 0-5}`
@ -80,7 +77,7 @@ All files are stored under `library/` (relative to the app working directory, ma
- `GET /stats` — statistics page - `GET /stats` — statistics page
- `GET /api/stats` — statistics data JSON - `GET /api/stats` — statistics data JSON
- `GET /api/disk` — partition usage for the library directory: `{total, used, free, pct_used}` - `GET /api/disk` — partition usage for the library directory: `{total, used, free, pct_used}`
- `POST /api/bulk-check-duplicates` — accepts `{"items": [{title, author, series, volume}, ...]}`, returns `{"duplicates": [bool, ...]}` — checks by title+author+series_index; also checks by series+author+series_index as fallback (catches duplicate detection when title format changed); when volume is absent, matches on title+author only - `POST /api/bulk-check-duplicates` — accepts `{"items": [{title, author, volume}, ...]}`, returns `{"duplicates": [bool, ...]}` — when `volume` is a number, requires title+author+series_index to all match; when volume is absent, matches on title+author only
- `GET /library/list` — compat alias - `GET /library/list` — compat alias
`GET /api/library` runs in fast-path mode by default (DB-only, no full disk rescan). `GET /api/library` runs in fast-path mode by default (DB-only, no full disk rescan).
@ -128,7 +125,6 @@ Home read sections are ordered oldest-first:
- `PATCH /library/book/{filename}` — update metadata + tags; moves file if path fields change; DB-only for non-EPUB; for `storage_type='db'` books: recomputes synthetic `db/…` filename, FK-safe rename (INSERT→UPDATE children→DELETE old), updates `book_chapters` + `bookmarks` as well - `PATCH /library/book/{filename}` — update metadata + tags; moves file if path fields change; DB-only for non-EPUB; for `storage_type='db'` books: recomputes synthetic `db/…` filename, FK-safe rename (INSERT→UPDATE children→DELETE old), updates `book_chapters` + `bookmarks` as well
- `POST /library/rating/{filename}` — set/clear 1–5 star rating; writes to EPUB OPF / CBZ ComicInfo.xml; DB-only for CBR/PDF - `POST /library/rating/{filename}` — set/clear 1–5 star rating; writes to EPUB OPF / CBZ ComicInfo.xml; DB-only for CBR/PDF
- `GET /library/read/{filename}` — reader page (EPUB or PDF); supports `?bm_ch=N&bm_scroll=F` to jump to bookmark position - `GET /library/read/{filename}` — reader page (EPUB or PDF); supports `?bm_ch=N&bm_scroll=F` to jump to bookmark position
- `GET /api/series-nav/{filename}` — returns `{prev, next}` (`{filename, title, index, suffix}` or `null`) for the adjacent books in the same series ordered by `series_index ASC, series_suffix ASC`; used by the reader for series navigation buttons and `markRead()` redirect
- `GET /library/bookmarks/{filename}` — list bookmarks for a book - `GET /library/bookmarks/{filename}` — list bookmarks for a book
- `POST /library/bookmarks/{filename}` — add bookmark `{chapter_index, scroll_frac, chapter_title, note}` - `POST /library/bookmarks/{filename}` — add bookmark `{chapter_index, scroll_frac, chapter_title, note}`
- `PATCH /library/bookmarks/{id}` — update bookmark note - `PATCH /library/bookmarks/{id}` — update bookmark note
@ -139,7 +135,7 @@ Home read sections are ordered oldest-first:
- `GET /bulk-import` — Bulk Import page - `GET /bulk-import` — Bulk Import page
- `POST /library/bulk-import` — import files with pre-parsed metadata; accepts multipart `files[]`, `rows` (JSON array of per-file metadata), `shared` (JSON with author/publisher/status/genres/tags applied to all files) - `POST /library/bulk-import` — import files with pre-parsed metadata; accepts multipart `files[]`, `rows` (JSON array of per-file metadata), `shared` (JSON with author/publisher/status/genres/tags applied to all files)
Filename parsing is done client-side in `bulk_import.html`. The page uses a free-text `%placeholder%` pattern (e.g. `%series% - %series_volume% - %volume% - %title% - %year%`). Available placeholders: `%series%` `%series_volume%` `%volume%` `%title%` `%year%` `%month%` `%day%` `%author%` `%publisher%` `%ignore%`. Colored chips can be clicked (insert at cursor) or dragged onto the input. Pattern is converted to a regex at parse time. Shared metadata fields (including "Year/Vol." for `series_volume`) override filename-parsed values. "Auto-generate titles" checkbox fills empty title cells as `Series (Year/Vol) #Number`. Skip checkbox is always visible for every row; skipped rows are excluded from import. Files are uploaded in batches of 5 with a progress bar. Filename parsing is done client-side in `bulk_import.html`. The page uses a free-text `%placeholder%` pattern (e.g. `%series% - %volume% - %title% - %year%`). Available placeholders: `%series%` `%volume%` `%title%` `%year%` `%month%` `%day%` `%author%` `%publisher%` `%ignore%`. Colored chips can be clicked (insert at cursor) or dragged onto the input. Pattern is converted to a regex at parse time. Shared metadata fields override filename-parsed values. Files are uploaded in batches of 5 with a progress bar.
### `routers/editor.py` ### `routers/editor.py`
- `GET /library/editor/{filename}` — chapter editor page; supports both EPUB files and DB-stored books (`db/…` filenames); passes `is_db` flag to template; DB branch queries `library` table directly (no file check) - `GET /library/editor/{filename}` — chapter editor page; supports both EPUB files and DB-stored books (`db/…` filenames); passes `is_db` flag to template; DB branch queries `library` table directly (no file check)
@ -163,130 +159,27 @@ Filename parsing is done client-side in `bulk_import.html`. The page uses a free
Scrape/convert flow (DB storage — default): Scrape/convert flow (DB storage — default):
1. Fetch book info + chapters via scraper 1. Fetch book info + chapters via scraper
2. Per chapter: download images → write to `library/images/{sha2}/{sha256}{ext}` (content-addressed) → rewrite `img[src]` to `/library/db-images/...`; break images replaced with `<hr>` before `element_to_xhtml` runs → build `content_html` via `element_to_xhtml` with `break_img_path="/static/break.png"` 2. Per chapter: download images → write to `library/images/{sha2}/{sha256}{ext}` (content-addressed) → rewrite `img[src]` to `/library/db-images/...` → build `content_html` via `element_to_xhtml`
3. One DB transaction: `ensure_unique_db_filename` → `upsert_book` (storage_type='db') → `upsert_chapter` for each chapter → `upsert_cover_cache` if cover provided 3. One DB transaction: `ensure_unique_db_filename` → `upsert_book` (storage_type='db') → `upsert_chapter` for each chapter → `upsert_cover_cache` if cover provided
4. Synthetic filename: `db/{publisher}/{author}/{title}` (or `db/{pub}/{auth}/Series/{series}/{idx} - {title}` for series) 4. Synthetic filename: `db/{publisher}/{author}/{title}` (or `db/{pub}/{auth}/Series/{series}/{idx} - {title}` for series)
Scrape/convert flow (EPUB file — `storage_mode: "epub"`): Scrape/convert flow (EPUB file — `storage_mode: "epub"`):
1–2. Same as DB flow; `break_img_path="../Images/break.png"` passed to `element_to_xhtml` 1–2. Same as DB flow (images downloaded, HTML built)
3. Chapters converted to XHTML via `make_chapter_xhtml`; EPUB file built via `make_epub` (embeds `static/break.png` as `OEBPS/Images/break.png`) and written to `library/epub/…` 3. Chapters converted to XHTML via `make_chapter_xhtml`; EPUB file built via `make_epub` and written to `library/epub/…`
4. `upsert_book` called with `storage_type='file'` 4. `upsert_book` called with `storage_type='file'`
### Scrapers (`scrapers/`)
All scrapers inherit `BaseScraper` and implement `matches(url)`, `login()`, `fetch_book_info()`, `fetch_chapter()`. Registration order in `scrapers/__init__.py` determines priority (first match wins).
| Scraper | Domain | Login | Notes |
|---|---|---|---|
| `ArchiveOfOurOwnScraper` | archiveofourown.org | Optional | Uses authenticity token; adult content gate via `?view_adult=true` |
| `AwesomeDudeScraper` | awesomedude.org | No | Chapter discovery via `.htm/.html` links in same directory; content extracted from largest non-layout block |
| `CodeysWorldScraper` | codeysworld.org | No | See below |
| `GayAuthorsScraper` | gayauthors.org | Optional | Genres + subgenres from `itemprop="genre"` links; tags from `ipsTags` list |
| `IomfatsScraper` | iomfats.org | No | See below; requires chapter URL as entry point |
| `NiftyNewScraper` | new.nifty.org | No | See below; registered before NiftyScraper |
| `NiftyScraper` | nifty.org (classic) | No | See below; excludes new.nifty.org; category/subcategory stored as tags |
| `TedLouisScraper` | tedlouis.com | No | Story index URL required as entry point; all pages use `?t=TOKEN` routing; chapter links in `<ul class="story-index-list">` |
#### NiftyNewScraper
`new.nifty.org` is a Next.js RSC application. Pages render proper HTML with semantic markup — no plain-text email format.
- URL normalisation: `_to_index_url()` strips a trailing `/N` (chapter index) so any URL (index or chapter) can be passed as entry point. Story URL pattern: `/stories/{slug}-{id}`.
- `fetch_book_info()`:
- Title from `<h1>`; fallback: `<title>` with ` - … - Nifty Archive …` suffix stripped.
- Author from `<strong itemprop="name">` inside `<a href="/authors/{id}">`.
- Publication date from `<time itemprop="datePublished" datetime="…">`, updated date from `<time itemprop="dateModified" datetime="…">`; both truncated to `YYYY-MM-DD`.
- Tags from all `<ul aria-label="Tags">` containers on the page — covers both the story category links (`/collections/…`) and the AI-generated content tags (`/search?query=tags%3A…`); deduplicated; `genres` and `subgenres` are always empty.
- Description from `<meta name="description">`.
- Chapter list: `<a>` links matching `/stories/{slug}/N` collected from page HTML; fallback: regex scan of RSC stream for `"index": N` values. URLs generated as `{index_url}/1` … `{index_url}/max`.
- `fetch_chapter()`:
- Content extraction order:
1. Chapter HTML (`{url}`): read `<article>` and collect `<p>` text
2. Fallback on same HTML: extract escaped Next payload paragraphs (`\u003cp...\u003c/p`)
3. Last fallback (`{url}?_rsc=1`): parse RSC line format (`{hex_id}:{json}`) for `["$","p",…]` nodes, then escaped paragraph fallback
- Chapter title uses the precomputed chapter dict title (`Chapter N`).
- Lead/tail boilerplate detection for common Nifty intro/donate text. Removed boilerplate is preserved as invisible HTML comments in chapter content:
- `<!-- NIFTY_HIDDEN_LEAD: ... -->`
- `<!-- NIFTY_HIDDEN_TAIL: ... -->`
- No email-header stripping and no plain-text line-joining (those are specific to Nifty classic).
#### NiftyScraper
Nifty classic pages are plain-text email submissions wrapped in a `<pre>` element.
- URL normalisation: `_to_index_url()` strips the chapter segment so any URL (index or chapter) can be passed as the entry point. Path structure: `/nifty/{category}/{subcategory}/{story}/` (index, 4 segments) vs `/nifty/{category}/{subcategory}/{story}/{chapter}` (chapter, 5 segments).
- `fetch_book_info()` performs up to 3 extra HTTP requests: chapter 1 (author + publication date), last chapter (`updated_date`), chapter 2 (boilerplate detection). Author and dates are extracted from the email headers (`From:`, `Date:`) embedded at the top of each chapter file. Date is parsed via `email.utils.parsedate` → `YYYY-MM-DD`.
- Boilerplate detection: leading paragraphs of chapters 1 and 2 (after email-header strip) are compared using normalised text (lowercase, whitespace collapsed). Consecutive matching paragraphs are recorded as `preamble_count` and stored in each chapter dict; `fetch_chapter()` skips them.
- `fetch_chapter()` pipeline:
1. Extract `<pre>` text (fallback: full body text)
2. Parse `Subject:` header → store as `<!-- Subject: … -->` comment in chapter content (invisible in reader, extractable later)
3. Strip email header block (up to first blank line after `Date:`/`From:`/`Subject:` lines)
4. Skip first `preamble_count` paragraphs
5. Split on blank lines → paragraphs; join hard-wrapped lines within each paragraph with a space
6. Detect and remove lead/tail boilerplate (common notice/disclaimer/author promo/donate blocks)
7. Persist removed boilerplate as invisible comments:
- `<!-- NIFTY_HIDDEN_LEAD: ... -->`
- `<!-- NIFTY_HIDDEN_TAIL: ... -->`
8. Scene-break patterns (`***`, `---`, `~~~`, `• • •`, etc.) → `<hr/>`
9. Build `content_el` as a BeautifulSoup `<div>` of comments + `<p>` + `<hr/>` nodes
- Genres/subgenres from URL path: `category` (e.g. `gay` → `Gay`) and `subcategory` (e.g. `young-friends` → `Young Friends`).
#### CodeysWorldScraper
- Entry point: any `codeysworld.org` URL.
- Title from `<h1>`; author from `<h2>` matching `"by …"` pattern; fallback: URL path segment `/{author}/{category}/filename`.
- Category from URL path (second-to-last segment, e.g. `remembrances` → tag `"Remembrances"`).
- Chapter discovery: `.htm/.html` links in the same directory as the entry URL; audio/image links skipped. No chapter links → single-file story (entry URL is the only chapter).
- `fetch_chapter()`: removes all `<h1>`/`<h2>` headings, back-navigation links, audio links (`.mp3`), mailto links; falls back to `<body>` when no content wrapper is found.
#### IomfatsScraper
All stories by an author are listed on a single author page (`/storyshelf/hosted/{author}/`). Individual story pages do not exist.
- Entry point must be a **chapter URL** (`/storyshelf/hosted/{author}/{story-folder}/{chapter}.html`). Passing the author page URL raises a `ValueError` with a user-visible message.
- On load: navigates to the author page and scans `<div id="content">` for the matching story.
- Two page structures detected:
- **Single story**: outer `<h3>` = book title; chapters are direct `<li><a>` children of the following `<ul>`.
- **Multi-part series**: outer `<h3>` = series name; nested `<li><h3>` = book title per part; chapters in the sub-`<ul>` matching `story_folder`.
- Series index extracted from folder name suffix: `*-part{N}` or `*-{N}`.
- Publication status from `<p><small>[…]</small></p>` after the book title heading.
- `fetch_chapter()`: content from `<div id="content">`; removes `<h2>`/`<h3>` headings, `.chapternav` divs, `div.important` footer blocks, anchor-name elements.
#### TedLouisScraper
All pages on `tedlouis.com` use opaque token-based routing: `https://tedlouis.com/?t=<TOKEN>`. There are no predictable URL patterns — tokens must be followed from the story index page.
- Entry point must be a **story index URL** (the page listing all chapters). Passing a chapter URL raises a `ValueError` with a user-visible message. Detection: story index has `<h2 class="story-page-title">`, chapter page has `<h1 class="story-title">`.
- `fetch_book_info()`:
- Title from direct `NavigableString` children of `<h2 class="story-page-title">` — the element also contains a "Back" button (`<a class="btn">`) and the author byline (`<span class="story-author-by-line">`), which are skipped.
- Author from `<span class="story-author-by-line"> <a>`.
- Publication status from `<span class="story-status-text">` with "Status: " prefix stripped.
- Updated date from `<span class="story-last-updated">` ("Last Updated: Month D, YYYY") → `YYYY-MM-DD`.
- Chapter list from all `<ul class="story-index-list">` elements (three columns on the page); relative `?t=TOKEN` hrefs resolved to absolute URLs. Order preserved; duplicates deduplicated.
- No genres, subgenres, tags or description available on the page.
- `fetch_chapter()`: content from `<div id="chapter">`; strips `<h1 class="story-title">`, `<h2 class="chapter-title">`, `div.chapter-copyright-line`, and `div.chapter-copyright-notice-text` blocks. Chapter title refined from `<h2 class="chapter-title"> <span>`.
#### `xhtml.element_to_xhtml()` — Comment handling
`bs4.Comment` objects (a `NavigableString` subclass) are now emitted as XML comments: `<!-- … -->`. The `--` sequence (illegal inside XML comments) is sanitised to `- -`. This allows scrapers to embed invisible metadata (e.g. the Nifty `Subject:` header) in chapter content without it appearing in the rendered reader.
### `routers/search.py` ### `routers/search.py`
- `GET /search` — full-text search page (`search.html`); Enter-to-search, `?q=` param auto-runs on load - `GET /search` — full-text search page (`search.html`); Enter-to-search, `?q=` param auto-runs on load
- `GET /api/search?q=…&mode=phrase|words&filter=all|unread_novels|unread_shorts` — FTS over `book_chapters.content_tsv`; `mode=phrase` (default) uses `phraseto_tsquery` (words in order); `mode=words` uses `plainto_tsquery` (all words present, any order); `ts_rank` and `ts_headline` always use `plainto_tsquery`; also matches chapters whose `title` contains the query (case-insensitive LIKE fallback); no result limit; excludes archived books; `filter=unread_novels` restricts to books with no reading sessions/progress and no `Shorts` tag; `filter=unread_shorts` restricts to books with no reading sessions/progress and a `Shorts` tag; results include `filename`, `title`, `author`, `chapter_index`, `chapter_title`, `snippet`, `rank` - `GET /api/search?q=…` — FTS over `book_chapters.content_tsv`; uses `plainto_tsquery('simple', q)` with `ts_rank` ordering and `ts_headline` for highlighted snippets; also matches chapters whose `title` contains the query (case-insensitive `ILIKE` fallback); LIMIT 30; excludes archived books; results include `filename`, `title`, `author`, `chapter_index`, `chapter_title`, `snippet`, `rank`
### `routers/settings.py` ### `routers/settings.py`
- `GET /settings` — settings page - `GET /settings` — settings page
- `GET /api/app-settings` — returns `{"develop_mode": bool, "break_image_url": str|null}`
- `PATCH /api/app-settings` — accepts `{"develop_mode": bool}`, persists to `app_settings` table
- `POST /api/app-settings/break-image` — multipart file upload (PNG/JPG/WebP); stores image in imagestore + overwrites `static/break.png`; saves `break_image_sha256` + `break_image_ext` to `app_settings`; returns `{"ok": true, "url": "/library/db-images/…"}`
- `GET /api/break-patterns` — list chapter-break patterns - `GET /api/break-patterns` — list chapter-break patterns
- `POST /api/break-patterns` — add break pattern (type: `regex` or `css_class`) - `POST /api/break-patterns` — add break pattern (type: `regex` or `css_class`)
- `PATCH /api/break-patterns/{id}` — update pattern (enable/disable or change value) - `PATCH /api/break-patterns/{id}` — update pattern (enable/disable or change value)
- `DELETE /api/break-patterns/{id}` — delete pattern - `DELETE /api/break-patterns/{id}` — delete pattern
- `DELETE /api/reading-history` — wipe all reading sessions - `DELETE /api/reading-history` — wipe all reading sessions
`app_settings` table (single row, `id = 1`): `develop_mode BOOLEAN`, `break_image_sha256 VARCHAR(64)`, `break_image_ext VARCHAR(10)`.
### `routers/builder.py` ### `routers/builder.py`
- `GET /builder` — Book Builder index (draft list + new draft form) - `GET /builder` — Book Builder index (draft list + new draft form)
- `POST /builder` — create new draft; redirects to `/builder/{id}` - `POST /builder` — create new draft; redirects to `/builder/{id}`
@ -483,21 +376,16 @@ Loaded by `index.html` (Convert page) and `grabber.html` (Grabber page). Require
- Amber: filled `#c8a03a`, unfilled `rgba(200, 160, 58, 0.25)`. - Amber: filled `#c8a03a`, unfilled `rgba(200, 160, 58, 0.25)`.
- Reader settings (hamburger menu): - Reader settings (hamburger menu):
- Content width slider (30–100 vw), persisted as `reader-content-width-pct`. - Content width slider (30–100 vw), persisted as `reader-content-width-pct`.
- Font size slider (80–150%, default 105%), persisted as `reader-font-size`; applied via `--reader-font-size` CSS custom property on `#chapter-content`.
- Text colour: 5 warm-tone presets `#e8e2d9` → `#938d86`, persisted as `reader-text-colour`. - Text colour: 5 warm-tone presets `#e8e2d9` → `#938d86`, persisted as `reader-text-colour`.
- Hamburger and back-link separated with `margin-left: 1rem` on `.header-back`. - Hamburger and back-link separated with `margin-left: 1rem` on `.header-back`.
- Reader supports EPUB, PDF, and CBR/CBZ: - Reader supports EPUB and PDF:
- EPUB: chapter-text rendering; progress = `{chapterIndex}:{scrollFrac}`; progress % = `(chapterIndex + scrollFrac) / total * 100`. - EPUB: chapter-text rendering; progress = `{chapterIndex}:{scrollFrac}`; progress % = `(chapterIndex + scrollFrac) / total * 100`.
- PDF: page-image rendering via `/library/pdf/{filename}?page=N`; page count from `/api/pdf/info/{filename}`; progress = `{pageIndex}:0`; keyboard/button navigation identical. - PDF: page-image rendering via `/library/pdf/{filename}?page=N`; page count from `/api/pdf/info/{filename}`; progress = `{pageIndex}:0`; keyboard/button navigation identical.
- `reader.html` branches on `FORMAT` variable injected by the server. - `reader.html` branches on `FORMAT` variable injected by the server.
- Series navigation: on load, `loadSeriesNav()` fetches `/api/series-nav/{filename}` and activates prev/next volume buttons in the header (hidden when no series); `markRead()` redirects to `/library/read/{next.filename}` when a next volume exists, otherwise to the book detail page.
- `Edit EPUB` button in Book Detail is only shown for `.epub` files. - `Edit EPUB` button in Book Detail is only shown for `.epub` files.
- Backup page supports: manual run, dry-run, Dropbox root, retention count, schedule (on/off + hours), status + history. - Backup page supports: manual run, dry-run, Dropbox root, retention count, schedule (on/off + hours), status + history.
- Bookmarks: saved per book via `POST /library/bookmarks/{filename}`; shown in Library sidebar section; navigated via `?bm_ch=N&bm_scroll=F` URL params on reader page. - Bookmarks: saved per book via `POST /library/bookmarks/{filename}`; shown in Library sidebar section; navigated via `?bm_ch=N&bm_scroll=F` URL params on reader page.
- Convert page: after loading metadata, if a book with the same title+author already exists in the library, a warning banner is shown (with a link to the existing book); user can still proceed with conversion. Check is done server-side in `/preload` response (`already_exists`, `existing_books`). - Convert page: after loading metadata, if a book with the same title+author already exists in the library, a warning banner is shown (with a link to the existing book); user can still proceed with conversion. Check is done server-side in `/preload` response (`already_exists`, `existing_books`).
- Authors view (`#authors`): lists all authors across `allBooks` (active + archived); authors whose books are all archived still appear. Sidebar counter (`count-authors`) counts only active-book authors. Author detail view (`#authors/{name}`) also uses `allBooks`; archived books show the `.badge-archived` overlay on their cover.
- Publishers view (`#publishers`): same rule — `allBooks` (active + archived); publishers with only archived books still appear. Sidebar counter uses active books only. Publisher detail also uses `allBooks`.
- Series detail view (`#series/{name}`): shows all books in a series as a cover grid. Header contains an "Archive series" / "Unarchive series" button — calls `POST /library/archive-series` to set `archived` for every book in the series at once; the button label reflects whether any book is still active.
- Duplicates view (`#duplicates`): groups non-archived books by `(title, author)` (case-insensitive); shows only groups with ≥ 2 copies; counter in sidebar shows total number of duplicate books. Detection is entirely client-side from the existing library data. - Duplicates view (`#duplicates`): groups non-archived books by `(title, author)` (case-insensitive); shows only groups with ≥ 2 copies; counter in sidebar shows total number of duplicate books. Detection is entirely client-side from the existing library data.
- Incomplete view (`#incomplete`): shows all non-archived books where `publication_status` is not `Complete` (Ongoing, Temporary Hold, Long-Term Hold, or blank); sidebar counter included. - Incomplete view (`#incomplete`): shows all non-archived books where `publication_status` is not `Complete` (Ongoing, Temporary Hold, Long-Term Hold, or blank); sidebar counter included.
- Following page (`/following`): dedicated page in its own sidebar section between Library and Tools; shows all library authors with their external URL; two tabs — Following (authors with URL set) and All Authors; inline URL editing with keyboard support (Enter = save, Escape = cancel); clicking Visit opens the external URL in a new tab. Author URLs are stored in the `authors` table. Sidebar counter shows number of followed authors. - Following page (`/following`): dedicated page in its own sidebar section between Library and Tools; shows all library authors with their external URL; two tabs — Following (authors with URL set) and All Authors; inline URL editing with keyboard support (Enter = save, Escape = cancel); clicking Visit opens the external URL in a new tab. Author URLs are stored in the `authors` table. Sidebar counter shows number of followed authors.
@ -505,17 +393,6 @@ Loaded by `index.html` (Convert page) and `grabber.html` (Grabber page). Require
--- ---
## Develop Mode
When enabled, every page shows a diagonal **DEVELOP** ribbon in the top-left corner and the browser tab title becomes **Novela Develop — …** instead of **Novela — …**.
- Persisted in `app_settings` table (single row, `id = 1`); created by `migrate_create_app_settings()`.
- `shared_templates._develop_mode()` reads this value from DB on every template render and is registered as a Jinja2 global (`develop_mode`), so all templates can use `{% if develop_mode() %}` without explicit context injection.
- Banner CSS lives in `static/sidebar.css` (`.develop-banner` / `.develop-banner-text`); rendered at the top of `templates/_sidebar.html`.
- Toggled via the **Develop mode** card on the Settings page (`/settings`); saving reloads the page so the banner and title take effect immediately.
---
## Known Conventions ## Known Conventions
- Book deletion flow: `unlink` file → `prune_empty_dirs(parent)` → `DELETE FROM library` (cascade removes child rows). - Book deletion flow: `unlink` file → `prune_empty_dirs(parent)` → `DELETE FROM library` (cascade removes child rows).
- Empty dir pruning: `prune_empty_dirs(start)` walks up from `start` to `LIBRARY_ROOT`, removing each dir if empty; stops at first non-empty dir. - Empty dir pruning: `prune_empty_dirs(start)` walks up from `start` to `LIBRARY_ROOT`, removing each dir if empty; stops at first non-empty dir.
@ -529,7 +406,6 @@ When enabled, every page shows a diagonal **DEVELOP** ribbon in the top-left cor
- CBR/PDF: DB only - CBR/PDF: DB only
- `upsert_book` uses `CASE WHEN EXCLUDED.rating > 0 THEN EXCLUDED.rating ELSE library.rating END` to restore rating from file without overwriting existing DB value. - `upsert_book` uses `CASE WHEN EXCLUDED.rating > 0 THEN EXCLUDED.rating ELSE library.rating END` to restore rating from file without overwriting existing DB value.
- Tag types in `book_tags`: `genre`, `subgenre`, `tag`, `subject`. No direct `genres`/`subgenres` fields on book objects; always use helpers `bookGenres()`, `bookSubgenres()`, `bookPlainTags()`. - Tag types in `book_tags`: `genre`, `subgenre`, `tag`, `subject`. No direct `genres`/`subgenres` fields on book objects; always use helpers `bookGenres()`, `bookSubgenres()`, `bookPlainTags()`.
- `series_volume` (e.g. `"1982"`) is used for annual comic series where issue numbers restart each year. It is separate from `series_index` (issue number within the year) and `series_suffix` (letter variant like `"a"`). Stored in DB and EPUB OPF (`novela:series_volume`); not reflected in the file path. Sort order: `series → series_volume → series_index → series_suffix`. In `getSeriesSlots`, gap-detection runs per volume independently when any book has `series_volume` set; slot labels show as `(year) #index`.
--- ---
@ -605,7 +481,6 @@ Images embedded in chapter HTML are stored content-addressed at `library/images/
## Known Bugs Fixed ## Known Bugs Fixed
- `renderGenreView` and `renderSearchResults` in `library.js` referenced `b.genres` (non-existent). Fixed: use `bookGenres()`, `bookSubgenres()`, `bookPlainTags()`. - `renderGenreView` and `renderSearchResults` in `library.js` referenced `b.genres` (non-existent). Fixed: use `bookGenres()`, `bookSubgenres()`, `bookPlainTags()`.
- `PillInput` in `book.js` did not handle comma as delimiter and did not flush on save. Fixed: comma keydown + `flush()` in `saveEdit()`. - `PillInput` in `book.js` did not handle comma as delimiter and did not flush on save. Fixed: comma keydown + `flush()` in `saveEdit()`.
- `PillInput._add` in `book.js` added a pasted comma-separated list as one tag instead of splitting it. Fixed: `_add` now splits the value on commas and pushes each trimmed, non-empty, non-duplicate part individually.
- `PATCH /library/book` failed for PDFs: `_sync_epub_metadata` tried to open PDF as ZIP. Fixed: only called for `.epub`. - `PATCH /library/book` failed for PDFs: `_sync_epub_metadata` tried to open PDF as ZIP. Fixed: only called for `.epub`.
- `_make_rel_path` in `reader.py` lacked format prefix (`epub/`, `pdf/`, `comics/`). Fixed: aligned with `common.make_rel_path`. - `_make_rel_path` in `reader.py` lacked format prefix (`epub/`, `pdf/`, `comics/`). Fixed: aligned with `common.make_rel_path`.
- `common.make_rel_path` always generated `.cbr` extension for CBZ files (both map to `media_type="cbr"`). Fixed: accepts optional `ext` parameter; `library.py` import now passes actual suffix. - `common.make_rel_path` always generated `.cbr` extension for CBZ files (both map to `media_type="cbr"`). Fixed: accepts optional `ext` parameter; `library.py` import now passes actual suffix.

215
docs/changelog-develop.md
View File

@ -1,220 +1,5 @@
# Develop Changelog # Develop Changelog
## 2026-04-15 (1)
- Reader: font size control in reading settings
- New "Font size" slider (80–150%, default 105%) in the settings drawer, between "Content width" and "Text colour"
- Applies via CSS custom property `--reader-font-size` on `#chapter-content`
- Persisted per-device in `localStorage` as `reader-font-size` — iPad and desktop each remember their own preference
## 2026-04-13 (1)
- Edit metadata: comma-separated tag input fix
- `PillInput._add` in `book.js` now splits the incoming value on commas before adding — each trimmed, non-empty, non-duplicate part is pushed individually
- Applies to genres, subgenres and plain tags; pasting e.g. `Fiction, Thriller, Adventure` adds 3 separate pills instead of 1
---
*Released as v0.1.11 on 2026-04-13*
## 2026-04-12 (2)
- Series navigation in the reader
- New `GET /api/series-nav/{filename}` endpoint: returns `{prev, next}` with `{filename, title, index, suffix}` for adjacent books in the same series, ordered by `series_index ASC, series_suffix ASC`; returns `{prev: null, next: null}` for books without a series
- Reader header now shows prev/next series buttons (skip-to-start/skip-to-end icons); hidden for books without a series, visible once `loadSeriesNav()` resolves
- Hovering a series button shows a tooltip: `#<index><suffix> <title>` (e.g. `#4 Batman: Year One`)
- `markRead()` in the reader redirects to `/library/read/{next.filename}` when a next volume exists, so reading continues without leaving the reader; falls back to the book detail page when the series is complete or the book has no series
## 2026-04-12 (1)
- Comics: series_volume support for annual series (issue numbers restart each year)
- New `series_volume VARCHAR(20)` column on `library` (migration `migrate_series_volume`); default `''`
- Stored in EPUB OPF as `<meta name="novela:series_volume" content="…"/>`; read back by `scan_epub`
- `upsert_book` inserts/preserves `series_volume` with the same COALESCE strategy as `series_suffix`
- `list_library_json` ORDER BY now: `publisher → author → series → series_volume → series_index → title`
- `PATCH /library/book/{filename}` reads `series_volume` from request body; persists to DB and OPF for both file-based and DB-stored books
- Book detail page: displays `(year)` after series name when `series_volume` is set (e.g. `Donald Duck (1982) [15]`); edit panel has a new "Year/Volume" input field
- `book.js`: "Auto" button next to the Title field generates `Series (Year/Vol) #Number` from the current series fields
- Bulk importer: `series_volume` support
- New `%series_volume%` placeholder token (orange) for filename pattern parsing
- New "Year/Vol." shared metadata field (applies to all files; overridden by per-row value or pattern)
- Preview table has a new "Yr/Vol" column
- New "Auto-generate titles from series info" checkbox: when enabled, rows without a parsed title get title `Series (Year/Vol) #Number`
- Skip checkbox now always visible for every row (previously only shown when duplicates were detected); any file can be manually excluded before import; skipped rows are dimmed; stats bar shows "X skipped"
- `POST /api/bulk-check-duplicates` extended: now also checks `(series, series_index, author)` as fallback — detects duplicates even when the title format has changed; items must include `series` field
- Library front-end: sorting within a series now respects `series_volume` before `series_index`
- `groupBySeries`, `renderAuthorDetail`, `renderPublisherDetail` all sort by `series_volume → series_index → series_suffix`
- `getSeriesSlots` refactored: when any book in the series has `series_volume` set, gap-detection runs per volume (year) independently — prevents `#5 (1982)` and `#5 (1983)` from colliding in the same slot
- Slot index label shows `(year) #index` for annual series (e.g. `(1982) #5`); unchanged for regular series
---
*Released as v0.1.10 on 2026-04-12*
## 2026-04-08 (13)
- Library: archive series in one action
- "Archive series" / "Unarchive series" button in the series detail view (`#series/{name}`)
- New endpoint `POST /library/archive-series` — sets `archived` for all books in the series via a single SQL UPDATE; body: `{"series": "…", "archive": true|false}`; returns `{ok, archived, count}`
- Button label reflects current state: "Archive series" when any book is active, "Unarchive series" when all are archived
- After the call, `allBooks` is updated in place and sidebar counters are recalculated without a full page reload
## 2026-04-08 (12)
- TedLouisScraper: title extraction fix
- `<h2 class="story-page-title">` also contains a "Back" button (`<a class="btn">`) and the author byline (`<span class="story-author-by-line">`)
- Fix: only direct `NavigableString` children of the h2 are used as the title, so link and span text is skipped
## 2026-04-08 (11)
- New scraper: `TedLouisScraper` (`scrapers/tedlouis.py`) for `tedlouis.com`
- Matches all `tedlouis.com` URLs; no login required
- Entry point is the story index page (e.g. `?t=CWYSqpOryu2rQmT1`); raises an error when a chapter URL is used as entry point
- Title from direct text nodes of `<h2 class="story-page-title">`
- Author from `<span class="story-author-by-line"> <a>`
- Status from `<span class="story-status-text">` (strips "Status: " prefix)
- Updated date from `<span class="story-last-updated">` → `YYYY-MM-DD`
- Chapters from all `<ul class="story-index-list">` elements (three columns); relative `?t=TOKEN` links resolved to absolute URLs
- `fetch_chapter()`: content from `<div id="chapter">`; removes story title, chapter title and copyright blocks
## 2026-04-08 (10)
- Settings: break image upload added
- New "Break image" card on the settings page: upload a PNG/JPG/WebP as the scene break image
- Stored in the imagestore (sha256-addressed) and overwrites `static/break.png` so EPUB export uses the same image
- `app_settings` extended with `break_image_sha256` and `break_image_ext` columns (migration `migrate_app_settings_break_image`)
- New endpoints: `POST /api/app-settings/break-image`; `GET /api/app-settings` now also returns `break_image_url`
- Preview of the current break image visible on the settings page
## 2026-04-08 (9)
- Grabber: break image fix for DB-stored books
- Break images (`<center><img src="../Images/break.png">`) contain a relative EPUB path that does not exist in the DB context
- Fix: `storage_mode` determined earlier in `_run_scrape`; for DB mode `/static/break.png` is passed as `break_img_path` to `element_to_xhtml`, for EPUB mode `../Images/break.png` is kept
- Additional fix: when a break image is detected in the image loop, the parent (`<center>`) is replaced with `<hr>` instead of decomposing the `<img>`; this allows `element_to_xhtml` to correctly detect the break after the image loop
## 2026-04-08 (8)
- New scraper: `IomfatsScraper` (`scrapers/iomfats.py`) for `iomfats.org`
- Matches all `iomfats.org` URLs; no login required
- Entry point is a chapter URL (e.g. `.../grasshopper/justhitsend-part1/justhitsend01.html`); automatically navigates to the author page to fetch all metadata and chapters
- Author page as entry point: raises a user-visible error message
- Detects two structures on the author page:
- Single story: outer `<h3>` = book title; chapters directly in `<ul>`
- Multi-part series: outer `<h3>` = series name; nested `<li><h3>` = book title per part; chapters in sub-`<ul>`
- Series index from folder name: `*-part{N}` or `*-{N}` → `series_index_hint`
- Publication status from `<p><small>[...]</small></p>` after book title
- `fetch_chapter()`: content via `<div id="content">`; removes `<h2>`/`<h3>` headings, chapternav divs, footer elements
## 2026-04-08 (7)
- New scraper: `CodeysWorldScraper` (`scrapers/codeysworld.py`) for `codeysworld.org`
- Matches all `codeysworld.org` URLs; no login required
- Title from `<h1>`; author from `<h2>` ("by …"); fallback to URL slug (`/{author}/{category}/filename`)
- Category from URL path → tag (e.g. "Remembrances")
- Single-file stories (no chapter links): the page itself is the only chapter
- Multi-chapter: links to `.htm`/`.html` files in the same directory (audio/image links skipped)
- `fetch_chapter()`: removes all `<h1>`/`<h2>` headings, navigation links ("Back to …"), audio links (`.mp3`), mailto links
- Nifty scraper: category/subcategory moved from `genres`/`subgenres` to `tags`
## 2026-04-08 (6)
- NiftyNewScraper: chapter extraction made more robust
- `fetch_chapter()` now first tries the standard chapter HTML (`{url}`) and reads `<article><p>` directly
- Fallback added for Next payloads with escaped paragraphs (`\u003cp...\u003c/p`) via `_extract_escaped_html_paragraphs()`
- Last fallback remains `?_rsc=1`: first `_parse_rsc_paragraphs()`, then escaped-paragraph fallback again
- Nifty (classic + new): standard boilerplate no longer visible in reader, but preserved in chapter
- Lead/tail detection added for common blocks (e.g. `NOTICE This is a work of fiction…`, `If you enjoy this story…`, donate text)
- Detected intro/closing boilerplate removed from visible paragraphs
- Removed text stored as invisible HTML comment:
- `<!-- NIFTY_HIDDEN_LEAD: ... -->`
- `<!-- NIFTY_HIDDEN_TAIL: ... -->`
- Detection also works when text contains inline HTML (e.g. `<a>` or `<svg>` in donate links)
## 2026-04-08 (5)
- NiftyNewScraper: chapter content fix
- `new.nifty.org` uses Next.js RSC — chapter content is not present in the static HTML but in the RSC payload (React component tree)
- `fetch_chapter()` now fetches `{url}?_rsc=1` instead of the regular HTML page
- RSC parser added: `_parse_rsc_paragraphs()`, `_rsc_find_paragraphs()`, `_rsc_text()` — parse the RSC stream line by line (format: `{hex_id}:{json}`), recursively search for `["$","p",null,{...}]` nodes and extract text from `children`
## 2026-04-08 (4)
- NiftyNewScraper added (`scrapers/nifty_new.py`) for `new.nifty.org`
- Matches all `new.nifty.org` URLs; no login required
- `_to_index_url()`: strips trailing `/N` (chapter number) so both index and chapter URLs can be used as entry point
- `fetch_book_info()`: title from `<h1>`, author from `<strong itemprop="name">` in author link, dates from `<time itemprop="datePublished/dateModified">`, tags from all `<ul aria-label="Tags">` containers (category links and generated tags, deduplicated), description from `<meta name="description">`, chapter list via `<a>` links matching `/stories/{slug}/N` (RSC stream regex as fallback)
- `fetch_chapter()`: title from JSON-LD `@type: "Chapter"`, content from `<article>`; no email header stripping, no line joining, no boilerplate detection
- `NiftyScraper.matches()` updated: excludes `new.nifty.org` (`"nifty.org" in url and "new.nifty.org" not in url`)
- `NiftyNewScraper` registered before `NiftyScraper` in `scrapers/__init__.py`
## 2026-04-08 (3)
- Settings: develop mode added
- New `app_settings` table (single row, `id = 1`) with `develop_mode` boolean; created via `migrate_create_app_settings()`
- `shared_templates.py`: shared `Jinja2Templates` instance for all routers; `develop_mode()` registered as Jinja2 global so all templates can access it without explicit context injection
- All 11 routers now import `templates` from `shared_templates` instead of each creating their own instance
- New endpoints in `routers/settings.py`: `GET /api/app-settings` and `PATCH /api/app-settings`
- Diagonal **DEVELOP** banner in the top-left of every page (CSS in `static/sidebar.css`, HTML in `templates/_sidebar.html`); only visible when develop mode is enabled
- All 17 HTML templates: `<title>` shows **Novela Develop — …** when develop mode is active
- Settings page: new card with checkbox toggle; page reloads after saving so banner and title apply immediately
## 2026-04-08 (2)
- Nifty scraper: fix `_strip_email_headers` now tolerates blank lines between header fields
- Some Nifty pages place `Subject:` after a blank line (`Date:\nFrom:\n\nSubject:\n`) — the previous implementation stopped at the first blank line causing `Subject:` to appear as a paragraph in chapter text
- Fix: on a blank line, look ahead to check whether a header field still follows; if so, skip the blank line(s) and continue stripping
## 2026-04-08 (1)
- Nifty scraper added (`scrapers/nifty.py`)
- Matches all `nifty.org` URLs; no login required
- `fetch_book_info`: accepts index or chapter URL; normalises to index; title from URL slug, author and publication date from email headers of chapter 1, `updated_date` from email headers of the last chapter; genres/subgenres from URL path (`/nifty/{category}/{subcategory}/…`)
- Boilerplate detection: compares first paragraphs of chapters 1 and 2 after header stripping; matching paragraphs are skipped in all chapters (`preamble_count` per chapter dict)
- `fetch_chapter`: retrieves `<pre>` content (fallback: body); subject header stored as invisible HTML comment `<!-- Subject: … -->` at the top of chapter content; email headers stripped; hard line breaks within paragraphs joined into a single line; scene breaks (`***`, `---`, etc.) → `<hr/>`
- Date parsing via `email.utils.parsedate` → `YYYY-MM-DD`
- `xhtml.py`: `element_to_xhtml` now handles `bs4.Comment` objects as XML comments (`<!-- … -->`); `--` in the body is sanitised to `- -` (illegal in XML comments)
## 2026-04-06 (3)
- Book detail: rating moved from clickable stars to dropdown in the Edit metadata panel
- Stars on the detail page are now purely visual (no longer clickable)
- New `<select id="ed-rating">` field in the edit panel, directly below Status
- `openEdit()` populates the select with the current rating; `saveEdit()` calls `POST /library/rating/…` if the value has changed
- `rateBook()` function removed from `book.js`
- Overview pages unchanged
## 2026-04-06 (2)
- Library: cover upload now also supported for DB-stored books
- `POST /library/cover/{filename}` previously returned an error for DB books (`"File not found"`) because no physical file exists
- Fix: DB books are now detected via `is_db_filename`; cover is stored directly in `library_cover_cache` and `has_cover = TRUE` set in the `library` table — `add_cover_to_epub` is not called
## 2026-04-06 (1)
- Search: filter for unread novels / unread shorts
- New `filter` parameter on `GET /api/search?q=…&filter=all|unread_novels|unread_shorts` (default: `all`)
- `unread_novels`: restricts to books with no reading sessions/progress and no `Shorts` tag
- `unread_shorts`: restricts to books with no reading sessions/progress and with a `Shorts` tag
- UI: second toggle row (All / Unread novels / Unread shorts) below the Phrase/All words toggle
- Filter persisted in URL (`?filter=…`) and restored on page load
## 2026-04-05 (4)
- Backup: large files (> 148 MB) now uploaded via Dropbox upload session in 100 MB chunks
- `_dropbox_upload_bytes`: files ≤ 148 MB go via `files_upload` (unchanged); larger files via `upload_session_start` → `upload_session_append_v2` → `upload_session_finish`
- Fixes `ApiError: UploadError('payload_too_large')`
## 2026-04-05 (3)
- Filenames: spaces now replaced with underscores when saving new files
- `clean_segment` (common.py) and `_clean_segment` (reader.py): `\s+` → `_` instead of space
- Series separator changed from ` - ` to `_-_` (applies to EPUB, CBR, CBZ and DB filenames)
- Existing files are not renamed
## 2026-04-05 (2)
- Search: removed result limit; added Phrase / All words mode toggle
- `LIMIT 30` removed — all matching chapters are returned
- New `mode` parameter on `GET /api/search?q=…&mode=phrase|words`: `phrase` (default) requires words in order (`phraseto_tsquery`); `words` requires all words present in any order (`plainto_tsquery`)
- Toggle in the UI (Phrase / All words) above the results; mode is included in the URL
## 2026-04-05 (1)
- Export EPUB: double chapter titles fixed — same heading-stripping logic as the reader now applied before passing content to `make_chapter_xhtml`
- Library: authors and publishers with only archived books now remain visible in the Authors and Publishers list views
- `renderAuthorsView` and `renderPublishersView` switched from `activeBooks()` to `allBooks` — consistent with `renderAuthorDetail` and `renderPublisherDetail`
## 2026-04-04 (2)
- Reader: fixed double chapter titles for books where the heading is wrapped in a `<section>` element (pandoc-style)
- Previous regex only stripped `<h1>`–`<h4>` at the very start of content; pandoc-converted EPUBs wrap headings as `<section …><h1>…</h1>` — heading was not at position 0 so the regex didn't fire
- Added a second `re.sub` pass that removes the first `<h1>`–`<h4>` found directly after an opening `<section>` or `<div>` tag, preserving the wrapper element
- Same stripping applied in the DB→EPUB export (`export_epub`) before passing content to `make_chapter_xhtml`
- Search: switched from `plainto_tsquery` to `phraseto_tsquery` in the FTS WHERE clause
- `plainto_tsquery` ANDs all words but treats them as independent terms (any order, any distance) — multi-word queries like "4 years later" matched chapters where the words appeared far apart
- `phraseto_tsquery` requires all words to appear in sequence; `ts_rank` and `ts_headline` still use `plainto_tsquery` for correct scoring and highlighting
## 2026-04-04 (1)
- Reader: fixed double chapter titles in DB-stored books
- Chapter endpoint (`GET /library/chapter/{index}/{filename}`) now strips all leading `<h1>`–`<h4>` tags from stored content before prepending its own `<h2 class="chapter-title">` — books scraped before front-matter stripping was added to the scraper showed the title (and chapter heading) twice
- Library: archived books now visible in author and publisher detail views, with an indicator badge on the cover
- `renderAuthorDetail` and `renderPublisherDetail` reverted to use `allBooks` (including archived) so archived books remain accessible from the author/publisher pages
- New `.badge-archived` overlay (bottom-left of cover): dark circle with archive icon, consistent with existing `badge-status` and `read-pill` overlays; added to all three card-rendering code paths
## 2026-04-03 (3) ## 2026-04-03 (3)
- DB chapter editor: Monaco-based editor now supports DB-stored books - DB chapter editor: Monaco-based editor now supports DB-stored books
- `GET /library/editor/{filename}` handles `db/…` filenames; `is_db` flag passed to template - `GET /library/editor/{filename}` handles `db/…` filenames; `is_db` flag passed to template

104
docs/changelog.md
View File

@ -1,109 +1,5 @@
# Changelog # Changelog
## v0.1.12 — 2026-04-15
### New features
- Reader: **font size slider** in the reading settings drawer — adjust text size from 80% to 150%; setting is saved per device so iPad and desktop each remember their own preference
---
## v0.1.11 — 2026-04-13
### Bug fixes
- Edit metadata: pasting or typing a comma-separated list in the genre, subgenre or tag input now adds each value as a separate tag instead of one combined tag
---
## v0.1.10 — 2026-04-12
### New features
- Reader: **prev/next volume buttons** in the header for books that are part of a series — buttons appear automatically when the book has adjacent volumes; tooltip shows the volume number and title; marking a book as read redirects directly to the next volume in the reader instead of the book detail page
- Comics: **series_volume field** for annual series where issue numbers restart each year (e.g. Donald Duck (1982) [15]) — stored in the database and EPUB OPF; displayed as `(year)` after the series name on the book detail page; sorting respects `series_volume` before `series_index`; supported in Bulk Import via `%series_volume%` placeholder and a "Year/Vol." shared field
- Library: **archive a series in one click** — "Archive series" / "Unarchive series" button in the series detail view; updates all books in the series via a single SQL UPDATE and recalculates sidebar counters without a page reload
### Bug fixes
- TedLouis scraper: title extraction no longer includes the "Back" button text or the author byline — only direct text nodes of the title heading are used
---
## v0.1.9 — 2026-04-08
### New features
- New scraper: **Nifty.org (classic)** — scrapes plain-text email-format stories; email headers stripped, boilerplate paragraphs auto-detected and hidden, scene-break patterns converted to break images
- New scraper: **new.nifty.org** — scrapes the Next.js version of Nifty; reads chapter content from the RSC payload when the static HTML does not include it; boilerplate detection shared with classic Nifty
- New scraper: **codeysworld.org** — single-file and multi-chapter stories; title and author extracted from heading elements; category from URL path stored as tag; navigation links and audio links stripped from chapter content
- New scraper: **iomfats.org** — all stories are listed on a single author page; provide any chapter URL and the scraper finds the correct story automatically; supports single stories and multi-part series (series name, book title, and series index derived from the page structure)
- New scraper: **tedlouis.com** — all pages use opaque token-based routing (`?t=TOKEN`); provide the story index URL and the scraper collects all chapter links from the three-column chapter list
- Settings: break image upload — upload a custom PNG/JPG/WebP to use as the scene break image in all converted books; stored in the imagestore and applied to both DB-stored and EPUB-format books
- Settings: develop mode toggle — shows a DEVELOP banner and updates the page title across all pages when enabled
### Bug fixes
- Break images were not displayed in DB-stored books — the image path `../Images/break.png` is a relative EPUB path that does not exist for DB content; DB mode now uses `/static/break.png`
- Break images were silently lost during import — the image was decomposed before `element_to_xhtml` ran, leaving an empty wrapper; the wrapper is now replaced with `<hr>` so the break is correctly rendered
---
## v0.1.8 — 2026-04-06
### Bug fixes
- Library: cover upload now works for DB-stored books — the upload endpoint previously returned "File not found" because DB books have no file on disk; the cover is now stored directly in the cover cache
### Improvements
- Book detail: rating moved from clickable stars to a dropdown in the Edit metadata panel — avoids touch-input issues on iPad where hover state caused all stars to appear filled
---
## v0.1.7 — 2026-04-06
### New feature
- Search: filter on unread novels or unread shorts — a second toggle row (All / Unread novels / Unread shorts) restricts results to books with no reading history; filter is preserved in the URL
### Bug fixes
- Backup: files larger than 148 MB now upload correctly — chunked upload session (100 MB per chunk) replaces the single-call upload that hit Dropbox's payload size limit
### Improvements
- File paths: spaces in new filenames are now replaced with underscores (publisher, author, title, series segments); series separator changed from ` - ` to `_-_`
---
## v0.1.6 — 2026-04-05
### Bug fixes
- Export EPUB: double chapter titles fixed — same heading-stripping logic as the reader now applied before passing content to `make_chapter_xhtml`
- Library: authors and publishers with only archived books now remain visible in the Authors and Publishers list views
---
## v0.1.5 — 2026-04-04
### Bug fixes
- Reader: double chapter titles for pandoc-converted books — headings wrapped in a `<section>` element were not stripped by the previous regex; now also removes the first heading found directly inside an opening `<section>` or `<div>`
- Search: multi-word queries no longer match chapters where the words appear far apart — switched to `phraseto_tsquery` so all words must appear in order
---
## v0.1.4 — 2026-04-04
### Bug fixes
- Reader: double chapter titles in DB-stored books — the chapter endpoint now strips all leading headings from stored content before prepending its own chapter title; affects books scraped before front-matter stripping was added
- Library: archived books were missing from author and publisher detail views — detail views now include all books (active and archived); archived books have a badge on their cover so they remain distinguishable
---
## v0.1.3 — 2026-04-03 ## v0.1.3 — 2026-04-03
### New feature ### New feature

2
version.txt
View File

@ -1 +1 @@
v0.2.8 v0.1.8