Compare commits

..

No commits in common. "main" and "v0.1.2" have entirely different histories.
main ... v0.1.2

55 changed files with 349 additions and 4844 deletions

View File

@ -235,11 +235,10 @@ for svc_path in "${services[@]}"; do
echo "============================================================" echo "============================================================"
echo "[INFO] Building ${svc} -> tags: ${NEW_VERSION}, latest" echo "[INFO] Building ${svc} -> tags: ${NEW_VERSION}, latest"
echo "============================================================" echo "============================================================"
docker build -t "${IMAGE_BASE}:${NEW_VERSION}" -t "${IMAGE_BASE}:latest" -t "${IMAGE_BASE}:dev" "$svc_path" docker build -t "${IMAGE_BASE}:${NEW_VERSION}" -t "${IMAGE_BASE}:dev" "$svc_path"
docker push "${IMAGE_BASE}:${NEW_VERSION}" docker push "${IMAGE_BASE}:${NEW_VERSION}"
docker push "${IMAGE_BASE}:latest"
docker push "${IMAGE_BASE}:dev" docker push "${IMAGE_BASE}:dev"
BUILT_IMAGES+=("${IMAGE_BASE}:${NEW_VERSION}" "${IMAGE_BASE}:latest" "${IMAGE_BASE}:dev") BUILT_IMAGES+=("${IMAGE_BASE}:${NEW_VERSION}" "${IMAGE_BASE}:dev")
else else
echo "============================================================" echo "============================================================"
echo "[INFO] Test build ${svc} -> tag: latest" echo "[INFO] Test build ${svc} -> tag: latest"

View File

@ -3,213 +3,6 @@ Changelog data for Novela
""" """
CHANGELOG = [ CHANGELOG = [
{
"version": "v0.1.12",
"date": "2026-04-15",
"summary": "Font size slider in the reader settings drawer.",
"sections": [
{
"title": "New features",
"type": "feature",
"changes": [
"Reader: font size slider in the reading settings drawer — adjust text size from 80% to 150%; setting is saved per device so iPad and desktop each remember their own preference",
],
},
],
},
{
"version": "v0.1.11",
"date": "2026-04-13",
"summary": "Comma-separated values in genre, subgenre and tag inputs are now split into individual tags.",
"sections": [
{
"title": "Bug fixes",
"type": "bugfix",
"changes": [
"Edit metadata: pasting or typing a comma-separated list in the genre, subgenre or tag input now adds each value as a separate tag instead of one combined tag",
],
},
],
},
{
"version": "v0.1.10",
"date": "2026-04-12",
"summary": "Series navigation in the reader, series_volume support for annual comics, archive a series in one click, and a TedLouis scraper fix.",
"sections": [
{
"title": "New features",
"type": "feature",
"changes": [
"Reader: prev/next volume buttons in the header for books that are part of a series — buttons appear automatically when the book has adjacent volumes; tooltip shows the volume number and title; marking a book as read redirects directly to the next volume in the reader instead of the book detail page",
"Comics: series_volume field for annual series where issue numbers restart each year (e.g. Donald Duck (1982) [15]) — stored in the database and EPUB OPF; displayed as '(year)' after the series name on the book detail page; sorting respects series_volume before series_index; supported in Bulk Import via %series_volume% placeholder and 'Year/Vol.' shared field",
"Library: archive or unarchive an entire series in one click — 'Archive series' / 'Unarchive series' button in the series detail view; updates all books in the series via a single SQL UPDATE and recalculates sidebar counters without a page reload",
],
},
{
"title": "Bug fixes",
"type": "bugfix",
"changes": [
"TedLouis scraper: title extraction no longer includes the 'Back' button text or the author byline — only direct text nodes of the title heading are used",
],
},
],
},
{
"version": "v0.1.9",
"date": "2026-04-08",
"summary": "Five new scrapers (Nifty, codeysworld.org, iomfats.org, tedlouis.com), break image settings, and bug fixes.",
"sections": [
{
"title": "New features",
"type": "feature",
"changes": [
"New scraper: Nifty.org (classic) — scrapes plain-text email-format stories; email headers stripped, boilerplate paragraphs auto-detected and hidden, scene-break patterns converted to break images",
"New scraper: new.nifty.org — scrapes the Next.js version of Nifty; reads chapter content from RSC payload when the static HTML does not include it; boilerplate detection shared with classic Nifty",
"New scraper: codeysworld.org — single-file and multi-chapter stories; title and author extracted from heading elements; category from URL path stored as tag; navigation links and audio links stripped from chapter content",
"New scraper: iomfats.org — all stories are listed on a single author page; provide any chapter URL and the scraper finds the correct story automatically; supports single stories and multi-part series (series name, book title, and series index derived from the page structure)",
"New scraper: tedlouis.com — all pages use opaque token-based routing (?t=TOKEN); provide the story index URL and the scraper collects all chapter links from the three-column chapter list",
"Settings: break image upload — upload a custom PNG/JPG/WebP to use as the scene break image in all converted books; stored in the imagestore and applied to both DB-stored and EPUB-format books",
"Settings: develop mode toggle — shows a DEVELOP banner and updates the page title across all pages when enabled",
],
},
{
"title": "Bug fixes",
"type": "bugfix",
"changes": [
"Break images were not displayed in DB-stored books — the image path '../Images/break.png' is a relative EPUB path that does not exist for DB content; DB mode now uses '/static/break.png'",
"Break images were silently lost during import — the image was decomposed before element_to_xhtml ran, leaving an empty wrapper; the wrapper is now replaced with <hr> so the break is correctly rendered",
],
},
],
},
{
"version": "v0.1.8",
"date": "2026-04-06",
"summary": "Cover upload for DB-stored books, and rating moved to the Edit metadata panel.",
"sections": [
{
"title": "Bug fixes",
"type": "bugfix",
"changes": [
"Library: cover upload now works for DB-stored books — the upload endpoint previously returned 'File not found' because DB books have no file on disk; the cover is now stored directly in the cover cache",
],
},
{
"title": "Improvements",
"type": "improvement",
"changes": [
"Book detail: rating moved from clickable stars to a dropdown in the Edit metadata panel — avoids touch-input issues on iPad where hover state caused all stars to appear filled",
],
},
],
},
{
"version": "v0.1.7",
"date": "2026-04-06",
"summary": "Search filter for unread novels/shorts, Dropbox chunked upload fix, and underscores in new filenames.",
"sections": [
{
"title": "New feature",
"type": "feature",
"changes": [
"Search: filter on unread novels or unread shorts — a second toggle row (All / Unread novels / Unread shorts) restricts results to books with no reading history; filter is preserved in the URL",
],
},
{
"title": "Bug fixes",
"type": "bugfix",
"changes": [
"Backup: files larger than 148 MB now upload correctly — chunked upload session (100 MB per chunk) replaces the single-call upload that hit Dropbox's payload size limit",
],
},
{
"title": "Improvements",
"type": "improvement",
"changes": [
"File paths: spaces in new filenames are now replaced with underscores (publisher, author, title, series segments); series separator changed from ' - ' to '_-_'",
],
},
],
},
{
"version": "v0.1.6",
"date": "2026-04-05",
"summary": "Bug fixes: double chapter titles in exported EPUBs, and authors/publishers with only archived books now remain visible.",
"sections": [
{
"title": "Bug fixes",
"type": "bugfix",
"changes": [
"Export EPUB: double chapter titles fixed — same heading-stripping logic as the reader now applied before passing content to the chapter builder",
"Library: authors and publishers with only archived books now remain visible in the Authors and Publishers list views",
],
},
],
},
{
"version": "v0.1.5",
"date": "2026-04-04",
"summary": "Bug fixes: double chapter titles in pandoc-style EPUB content, and search now requires words in order (phrase match).",
"sections": [
{
"title": "Bug fixes",
"type": "bugfix",
"changes": [
"Reader: double chapter titles for pandoc-converted books — headings wrapped in a <section> element were not stripped by the previous regex; now also removes the first heading found directly inside an opening <section> or <div>",
"Search: multi-word queries no longer match chapters where the words appear far apart — switched to phraseto_tsquery so all words must appear in order",
],
},
],
},
{
"version": "v0.1.4",
"date": "2026-04-04",
"summary": "Bug fixes: double chapter titles in the reader for DB-stored books, and archived books now shown in author/publisher detail with an indicator badge.",
"sections": [
{
"title": "Bug fixes",
"type": "bugfix",
"changes": [
"Reader: double chapter titles in DB-stored books — the chapter endpoint now strips all leading headings from stored content before prepending its own chapter title; affects books scraped before front-matter stripping was added",
"Library: archived books were missing from author and publisher detail views — detail views now include all books (active and archived); archived books have a badge on their cover so they remain distinguishable",
],
},
],
},
{
"version": "v0.1.3",
"date": "2026-04-03",
"summary": "DB-stored books: chapters stored in PostgreSQL with full-text search, EPUB conversion, export, and a storage toggle in the grabber.",
"sections": [
{
"title": "New feature",
"type": "feature",
"changes": [
"DB-stored books: scraped books are now stored as chapters in PostgreSQL instead of EPUB files on disk — full-text search, content deduplication, and backup coverage are all handled automatically",
"Grabber stores chapters in book_chapters and images in a content-addressed imagestore (sha256-based, automatic deduplication across all books)",
"EPUB-to-DB conversion: Convert to DB button on any EPUB book detail page — extracts chapters, migrates all metadata and child rows (tags, progress, bookmarks, cover), removes the EPUB file",
"DB-to-EPUB export: Export EPUB button on DB-stored books — builds and streams a standards-compliant EPUB without writing a file to disk",
"Full-text search (/search): searches across all DB-stored chapter content via PostgreSQL FTS (tsvector / plainto_tsquery), returns highlighted snippets with direct links to the chapter position in the reader",
"Chapter editor supports DB-stored books: Monaco-based editor reads and writes book_chapters directly; chapter titles editable inline; title-only changes correctly included in Save All",
"Grabber: storage toggle on the Convert page — choose between DB storage and EPUB file before converting",
],
},
],
},
{
"version": "v0.1.2",
"date": "2026-04-02",
"summary": "Restore functionality on the Backup page.",
"sections": [
{
"title": "New feature",
"type": "feature",
"changes": [
"Restore functionality on the Backup page: browse any available Dropbox snapshot, see which files are currently missing from disk, and restore individual books or a selection back to the library — file is written to disk and immediately re-indexed",
],
},
],
},
{ {
"version": "v0.1.1", "version": "v0.1.1",
"date": "2026-03-31", "date": "2026-03-31",

View File

@ -17,7 +17,6 @@ from routers import (
grabber_router, grabber_router,
library_router, library_router,
reader_router, reader_router,
search_router,
settings_router, settings_router,
) )
@ -47,7 +46,6 @@ app.include_router(builder_router)
app.include_router(bulk_import_router) app.include_router(bulk_import_router)
app.include_router(following_router) app.include_router(following_router)
app.include_router(changelog_router) app.include_router(changelog_router)
app.include_router(search_router)
@app.get("/") @app.get("/")

View File

@ -296,86 +296,6 @@ def migrate_rename_hiatus() -> None:
_exec("UPDATE library SET publication_status = 'Long-Term Hold' WHERE publication_status = 'Hiatus'") _exec("UPDATE library SET publication_status = 'Long-Term Hold' WHERE publication_status = 'Hiatus'")
def migrate_add_storage_type() -> None:
_exec(
"ALTER TABLE library ADD COLUMN IF NOT EXISTS storage_type VARCHAR(10) NOT NULL DEFAULT 'file'"
)
def migrate_create_book_images() -> None:
_exec(
"""
CREATE TABLE IF NOT EXISTS book_images (
sha256 CHAR(64) PRIMARY KEY,
ext VARCHAR(10) NOT NULL,
media_type VARCHAR(100) NOT NULL,
size_bytes INTEGER NOT NULL DEFAULT 0
)
"""
)
def migrate_create_book_chapters() -> None:
_exec(
"""
CREATE TABLE IF NOT EXISTS book_chapters (
id SERIAL PRIMARY KEY,
filename VARCHAR(600) NOT NULL REFERENCES library(filename) ON DELETE CASCADE,
chapter_index INTEGER NOT NULL,
title VARCHAR(500) NOT NULL DEFAULT '',
content TEXT NOT NULL DEFAULT '',
content_tsv TSVECTOR,
UNIQUE (filename, chapter_index)
)
"""
)
_exec(
"CREATE INDEX IF NOT EXISTS idx_book_chapters_filename ON book_chapters (filename, chapter_index)"
)
_exec(
"CREATE INDEX IF NOT EXISTS idx_book_chapters_tsv ON book_chapters USING GIN (content_tsv)"
)
def migrate_rebuild_chapter_tsv_with_title() -> None:
"""Rebuild content_tsv to include chapter title (safe to run repeatedly)."""
_exec(
"""
UPDATE book_chapters
SET content_tsv = to_tsvector('simple',
COALESCE(title, '') || ' ' ||
regexp_replace(COALESCE(content, ''), '<[^>]*>', ' ', 'g'))
"""
)
def migrate_create_app_settings() -> None:
_exec(
"""
CREATE TABLE IF NOT EXISTS app_settings (
id INTEGER PRIMARY KEY DEFAULT 1,
develop_mode BOOLEAN NOT NULL DEFAULT FALSE,
CONSTRAINT single_row CHECK (id = 1)
)
"""
)
_exec("INSERT INTO app_settings (id, develop_mode) VALUES (1, FALSE) ON CONFLICT DO NOTHING")
def migrate_app_settings_break_image() -> None:
_exec("ALTER TABLE app_settings ADD COLUMN IF NOT EXISTS break_image_sha256 VARCHAR(64) DEFAULT NULL")
_exec("ALTER TABLE app_settings ADD COLUMN IF NOT EXISTS break_image_ext VARCHAR(10) DEFAULT NULL")
def migrate_series_volume() -> None:
_exec(
"""
ALTER TABLE library
ADD COLUMN IF NOT EXISTS series_volume VARCHAR(20) NOT NULL DEFAULT ''
"""
)
def run_migrations() -> None: def run_migrations() -> None:
migrate_create_library() migrate_create_library()
migrate_create_book_tags() migrate_create_book_tags()
@ -394,10 +314,3 @@ def run_migrations() -> None:
migrate_create_builder_drafts() migrate_create_builder_drafts()
migrate_create_authors() migrate_create_authors()
migrate_rename_hiatus() migrate_rename_hiatus()
migrate_add_storage_type()
migrate_create_book_images()
migrate_create_book_chapters()
migrate_rebuild_chapter_tsv_with_title()
migrate_create_app_settings()
migrate_app_settings_break_image()
migrate_series_volume()

View File

@ -7,7 +7,6 @@ from routers.following import router as following_router
from routers.grabber import router as grabber_router from routers.grabber import router as grabber_router
from routers.library import router as library_router from routers.library import router as library_router
from routers.reader import router as reader_router from routers.reader import router as reader_router
from routers.search import router as search_router
from routers.settings import router as settings_router from routers.settings import router as settings_router
__all__ = [ __all__ = [
@ -21,5 +20,4 @@ __all__ = [
"bulk_import_router", "bulk_import_router",
"following_router", "following_router",
"changelog_router", "changelog_router",
"search_router",
] ]

View File

@ -14,12 +14,12 @@ import httpx
from dropbox.exceptions import ApiError, AuthError from dropbox.exceptions import ApiError, AuthError
from fastapi import APIRouter, Request from fastapi import APIRouter, Request
from fastapi.responses import HTMLResponse from fastapi.responses import HTMLResponse
from shared_templates import templates from fastapi.templating import Jinja2Templates
from db import get_db_conn from db import get_db_conn
from routers.common import scan_media, upsert_book
from security import decrypt_value, encrypt_value, is_encrypted_value from security import decrypt_value, encrypt_value, is_encrypted_value
templates = Jinja2Templates(directory="templates")
router = APIRouter() router = APIRouter()
LIBRARY_DIR = Path(os.environ.get("LIBRARY_DIR", "library")) LIBRARY_DIR = Path(os.environ.get("LIBRARY_DIR", "library"))
@ -434,36 +434,12 @@ def _ensure_dropbox_dir(client: dropbox.Dropbox, path: str) -> None:
pass pass
_DROPBOX_UPLOAD_CHUNK = 100 * 1024 * 1024 # 100 MB — below the 150 MB files_upload limit
_DROPBOX_UPLOAD_THRESHOLD = 148 * 1024 * 1024 # use session upload above this size
def _dropbox_upload_bytes(client: dropbox.Dropbox, target_path: str, data: bytes) -> int: def _dropbox_upload_bytes(client: dropbox.Dropbox, target_path: str, data: bytes) -> int:
parent = str(Path(target_path).parent).replace("\\", "/") parent = str(Path(target_path).parent).replace("\\", "/")
if not parent.startswith("/"): if not parent.startswith("/"):
parent = "/" + parent parent = "/" + parent
_ensure_dropbox_dir(client, parent) _ensure_dropbox_dir(client, parent)
mode = dropbox.files.WriteMode.overwrite client.files_upload(data, target_path, mode=dropbox.files.WriteMode.overwrite, mute=True)
if len(data) <= _DROPBOX_UPLOAD_THRESHOLD:
client.files_upload(data, target_path, mode=mode, mute=True)
else:
# Chunked upload session for large files
offset = 0
session_id = None
while offset < len(data):
chunk = data[offset : offset + _DROPBOX_UPLOAD_CHUNK]
if session_id is None:
res = client.files_upload_session_start(chunk)
session_id = res.session_id
else:
cursor = dropbox.files.UploadSessionCursor(session_id=session_id, offset=offset)
remaining = len(data) - offset - len(chunk)
if remaining == 0:
commit = dropbox.files.CommitInfo(path=target_path, mode=mode, mute=True)
client.files_upload_session_finish(chunk, cursor, commit)
else:
client.files_upload_session_append_v2(chunk, cursor)
offset += len(chunk)
return len(data) return len(data)
@ -1220,131 +1196,3 @@ async def run_backup(request: Request):
"message": "Backup started in background.", "message": "Backup started in background.",
"started_at": _now_iso(), "started_at": _now_iso(),
} }
def _parse_snapshot_date(name: str) -> str:
"""Parse 'snapshot-20260329-123456.json''2026-03-29T12:34:56Z'."""
stem = Path(name).stem # snapshot-20260329-123456
parts = stem.split("-")
if len(parts) >= 3:
d, t = parts[1], parts[2]
if len(d) == 8 and len(t) == 6:
return f"{d[:4]}-{d[4:6]}-{d[6:]}T{t[:2]}:{t[2:4]}:{t[4:]}Z"
return ""
def _download_and_restore(client: dropbox.Dropbox, objects_root: str, rel: str, info: dict) -> None:
sha256 = str(info.get("sha256") or "")
if not sha256:
raise ValueError("No sha256 in snapshot entry")
obj_path = _object_path(objects_root, sha256)
_meta, res = client.files_download(obj_path)
data = res.content
dest = LIBRARY_DIR / rel
dest.parent.mkdir(parents=True, exist_ok=True)
dest.write_bytes(data)
meta = scan_media(dest)
tags = [(s, "subject") for s in meta.get("subjects", [])]
with get_db_conn() as conn:
with conn:
upsert_book(conn, rel, meta, tags)
@router.get("/api/backup/snapshots")
async def list_snapshots():
try:
client = await asyncio.to_thread(_dbx)
except Exception as e:
return {"ok": False, "error": str(e), "snapshots": []}
dropbox_root = _load_dropbox_root()
snapshots_root = _dropbox_join(dropbox_root, "library_snapshots")
try:
paths = await asyncio.to_thread(_list_snapshot_paths, client, snapshots_root)
except Exception as e:
return {"ok": False, "error": str(e), "snapshots": []}
snapshots = [
{"name": Path(p).name, "created_at": _parse_snapshot_date(Path(p).name)}
for p in paths
]
return {"ok": True, "snapshots": snapshots}
@router.get("/api/backup/snapshots/{snapshot_name}/files")
async def snapshot_files(snapshot_name: str):
try:
client = await asyncio.to_thread(_dbx)
except Exception as e:
return {"ok": False, "error": str(e), "files": []}
dropbox_root = _load_dropbox_root()
snapshots_root = _dropbox_join(dropbox_root, "library_snapshots")
snapshot_path = _dropbox_join(snapshots_root, snapshot_name)
try:
snap = await asyncio.to_thread(_load_snapshot_data, client, snapshot_path)
except Exception as e:
return {"ok": False, "error": str(e), "files": []}
files_data = snap.get("files", {})
result = [
{
"path": rel,
"size": info.get("size", 0),
"sha256": info.get("sha256", ""),
"exists_locally": (LIBRARY_DIR / rel).exists(),
}
for rel, info in sorted(files_data.items())
if isinstance(info, dict)
]
return {"ok": True, "snapshot": snapshot_name, "files": result}
@router.post("/api/backup/restore")
async def restore_files(request: Request):
body = {}
try:
body = await request.json()
except Exception:
pass
snapshot_name = (body.get("snapshot_name") or "").strip()
files_to_restore: list[str] = body.get("files", [])
if not snapshot_name:
return {"ok": False, "error": "snapshot_name is required"}
if not files_to_restore:
return {"ok": False, "error": "No files specified"}
try:
client = await asyncio.to_thread(_dbx)
except Exception as e:
return {"ok": False, "error": str(e)}
dropbox_root = _load_dropbox_root()
snapshots_root = _dropbox_join(dropbox_root, "library_snapshots")
objects_root = _dropbox_join(dropbox_root, "library_objects")
snapshot_path = _dropbox_join(snapshots_root, snapshot_name)
try:
snap = await asyncio.to_thread(_load_snapshot_data, client, snapshot_path)
except Exception as e:
return {"ok": False, "error": f"Failed to load snapshot: {e}"}
files_data = snap.get("files", {})
results = []
for rel in files_to_restore:
if rel not in files_data:
results.append({"path": rel, "ok": False, "error": "Not found in snapshot"})
continue
try:
await asyncio.to_thread(_download_and_restore, client, objects_root, rel, files_data[rel])
results.append({"path": rel, "ok": True})
except Exception as e:
results.append({"path": rel, "ok": False, "error": str(e)})
ok_count = sum(1 for r in results if r["ok"])
return {"ok": True, "restored": ok_count, "total": len(results), "results": results}

View File

@ -5,7 +5,7 @@ from pathlib import Path
from fastapi import APIRouter, Request from fastapi import APIRouter, Request
from fastapi.responses import HTMLResponse, JSONResponse, RedirectResponse from fastapi.responses import HTMLResponse, JSONResponse, RedirectResponse
from shared_templates import templates from fastapi.templating import Jinja2Templates
from db import get_db_conn from db import get_db_conn
from epub import build_epub from epub import build_epub
@ -13,6 +13,7 @@ from routers.common import LIBRARY_DIR, make_rel_path, upsert_book
from xhtml import normalize_wysiwyg_html from xhtml import normalize_wysiwyg_html
router = APIRouter() router = APIRouter()
templates = Jinja2Templates(directory="templates")
# ── Helpers ─────────────────────────────────────────────────────────────────── # ── Helpers ───────────────────────────────────────────────────────────────────

View File

@ -4,7 +4,7 @@ from pathlib import Path
from fastapi import APIRouter, File, Form, Request, UploadFile from fastapi import APIRouter, File, Form, Request, UploadFile
from fastapi.responses import HTMLResponse, JSONResponse from fastapi.responses import HTMLResponse, JSONResponse
from shared_templates import templates from fastapi.templating import Jinja2Templates
from cbr import cbr_page_count from cbr import cbr_page_count
from db import get_db_conn from db import get_db_conn
@ -18,6 +18,7 @@ from routers.common import (
upsert_book, upsert_book,
) )
templates = Jinja2Templates(directory="templates")
router = APIRouter() router = APIRouter()
@ -72,7 +73,6 @@ async def library_bulk_import(
author = (row.get("author") or "").strip() or shared_author author = (row.get("author") or "").strip() or shared_author
publisher = (row.get("publisher") or "").strip() or shared_publisher publisher = (row.get("publisher") or "").strip() or shared_publisher
series = (row.get("series") or "").strip() or shared_data.get("series", "") series = (row.get("series") or "").strip() or shared_data.get("series", "")
series_volume = ((row.get("series_volume") or "").strip() or shared_data.get("series_volume", ""))[:20]
series_index, series_suffix = parse_volume_str(row.get("volume") or "") series_index, series_suffix = parse_volume_str(row.get("volume") or "")
status = (row.get("status") or "").strip() or shared_status status = (row.get("status") or "").strip() or shared_status
@ -121,7 +121,6 @@ async def library_bulk_import(
"series": series, "series": series,
"series_index": series_index, "series_index": series_index,
"series_suffix": series_suffix, "series_suffix": series_suffix,
"series_volume": series_volume if series else "",
"publication_status": status, "publication_status": status,
"publish_date": publish_date, "publish_date": publish_date,
"has_cover": has_cover, "has_cover": has_cover,

View File

@ -1,10 +1,11 @@
from fastapi import APIRouter, Request from fastapi import APIRouter, Request
from fastapi.responses import HTMLResponse from fastapi.responses import HTMLResponse
from shared_templates import templates from fastapi.templating import Jinja2Templates
from changelog import CHANGELOG from changelog import CHANGELOG
router = APIRouter() router = APIRouter()
templates = Jinja2Templates(directory="templates")
@router.get("/changelog", response_class=HTMLResponse) @router.get("/changelog", response_class=HTMLResponse)

View File

@ -1,5 +1,4 @@
import base64 import base64
import hashlib
import html as _html import html as _html
import io import io
import posixpath import posixpath
@ -19,20 +18,14 @@ from pdf import pdf_cover_thumb, pdf_page_count, pdf_scan_metadata
LIBRARY_DIR = Path("library") LIBRARY_DIR = Path("library")
LIBRARY_DIR.mkdir(exist_ok=True) LIBRARY_DIR.mkdir(exist_ok=True)
LIBRARY_ROOT = LIBRARY_DIR.resolve() LIBRARY_ROOT = LIBRARY_DIR.resolve()
IMAGES_DIR = LIBRARY_DIR / "images"
COVER_W = 300 COVER_W = 300
COVER_H = 450 COVER_H = 450
def is_db_filename(filename: str) -> bool:
"""True if the filename is a synthetic DB-stored book path (no file on disk)."""
return (filename or "").startswith("db/")
def clean_segment(value: str, fallback: str, max_len: int) -> str: def clean_segment(value: str, fallback: str, max_len: int) -> str:
txt = re.sub(r"\s+", "_", (value or "").strip()) txt = re.sub(r"\s+", " ", (value or "").strip())
txt = re.sub(r'[<>:"/\\|?*\x00-\x1f]', "", txt) txt = re.sub(r'[<>:"/\\|?*\x00-\x1f]', "", txt)
txt = re.sub(r"\.+$", "", txt).strip("_") txt = re.sub(r"\.+$", "", txt).strip()
return (txt or fallback)[:max_len] return (txt or fallback)[:max_len]
@ -85,17 +78,6 @@ def coerce_series_index(value: int | str | None) -> int:
def make_rel_path(*, media_type: str, publisher: str, author: str, title: str, series: str, series_index: int | str | None, series_suffix: str = "", ext: str = "") -> Path: def make_rel_path(*, media_type: str, publisher: str, author: str, title: str, series: str, series_index: int | str | None, series_suffix: str = "", ext: str = "") -> Path:
if media_type == "db":
pub = clean_segment(publisher, "Unknown Publisher", 80)
auth = clean_segment(author, "Unknown Author", 80)
ttl = clean_segment(title, "Untitled", 140)
series_name = clean_segment(series, "", 80)
if series_name:
idx = coerce_series_index(series_index)
sfx = re.sub(r"[^a-z]", "", (series_suffix or "").lower())[:5]
return Path("db") / pub / auth / "Series" / series_name / f"{idx:03d}{sfx}_-_{ttl}"
return Path("db") / pub / auth / ttl
if media_type == "epub": if media_type == "epub":
pub = clean_segment(publisher, "Unknown Publisher", 80) pub = clean_segment(publisher, "Unknown Publisher", 80)
auth = clean_segment(author, "Unknown Author", 80) auth = clean_segment(author, "Unknown Author", 80)
@ -104,7 +86,7 @@ def make_rel_path(*, media_type: str, publisher: str, author: str, title: str, s
if series_name: if series_name:
idx = coerce_series_index(series_index) idx = coerce_series_index(series_index)
sfx = re.sub(r"[^a-z]", "", (series_suffix or "").lower())[:5] sfx = re.sub(r"[^a-z]", "", (series_suffix or "").lower())[:5]
return Path("epub") / pub / auth / "Series" / series_name / f"{idx:03d}{sfx}_-_{ttl}.epub" return Path("epub") / pub / auth / "Series" / series_name / f"{idx:03d}{sfx} - {ttl}.epub"
return Path("epub") / pub / auth / "Stories" / f"{ttl}.epub" return Path("epub") / pub / auth / "Stories" / f"{ttl}.epub"
if media_type == "pdf": if media_type == "pdf":
@ -122,7 +104,7 @@ def make_rel_path(*, media_type: str, publisher: str, author: str, title: str, s
if series_name: if series_name:
idx = coerce_series_index(series_index) idx = coerce_series_index(series_index)
sfx = re.sub(r"[^a-z]", "", (series_suffix or "").lower())[:5] sfx = re.sub(r"[^a-z]", "", (series_suffix or "").lower())[:5]
return Path("comics") / pub / auth / "Series" / series_name / f"{idx:03d}{sfx}_-_{ttl}{comics_ext}" return Path("comics") / pub / auth / "Series" / series_name / f"{idx:03d}{sfx} - {ttl}{comics_ext}"
return Path("comics") / pub / auth / f"{ttl}{comics_ext}" return Path("comics") / pub / auth / f"{ttl}{comics_ext}"
@ -239,7 +221,6 @@ def scan_epub(path: Path) -> dict:
"series": "", "series": "",
"series_index": 0, "series_index": 0,
"series_suffix": "", "series_suffix": "",
"series_volume": "",
"title": "", "title": "",
"publication_status": "", "publication_status": "",
"author": "", "author": "",
@ -281,9 +262,6 @@ def scan_epub(path: Path) -> dict:
m = re.search(r'<meta[^>]*name="novela:series_suffix"[^>]*content="([^"]+)"', opf, re.IGNORECASE) m = re.search(r'<meta[^>]*name="novela:series_suffix"[^>]*content="([^"]+)"', opf, re.IGNORECASE)
if m: if m:
out["series_suffix"] = re.sub(r"[^a-z]", "", m.group(1).lower())[:5] out["series_suffix"] = re.sub(r"[^a-z]", "", m.group(1).lower())[:5]
m = re.search(r'<meta[^>]*name="novela:series_volume"[^>]*content="([^"]+)"', opf, re.IGNORECASE)
if m:
out["series_volume"] = _html.unescape(m.group(1).strip())[:20]
m = re.search(r'<meta[^>]*name="publication_status"[^>]*content="([^"]+)"', opf, re.IGNORECASE) m = re.search(r'<meta[^>]*name="publication_status"[^>]*content="([^"]+)"', opf, re.IGNORECASE)
if m: if m:
out["publication_status"] = _html.unescape(m.group(1).strip()) out["publication_status"] = _html.unescape(m.group(1).strip())
@ -361,13 +339,12 @@ def upsert_book(conn, filename: str, meta: dict, tags: list[tuple[str, str]] | N
with conn.cursor() as cur: with conn.cursor() as cur:
cur.execute( cur.execute(
""" """
INSERT INTO library (filename, media_type, storage_type, title, author, publisher, has_cover, INSERT INTO library (filename, media_type, title, author, publisher, has_cover,
series, series_index, series_suffix, series_volume, publication_status, source_url, series, series_index, series_suffix, publication_status, source_url,
publish_date, description, needs_review, want_to_read, rating, updated_at) publish_date, description, needs_review, want_to_read, rating, updated_at)
VALUES (%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, FALSE, %s, NOW()) VALUES (%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, FALSE, %s, NOW())
ON CONFLICT (filename) DO UPDATE SET ON CONFLICT (filename) DO UPDATE SET
media_type = EXCLUDED.media_type, media_type = EXCLUDED.media_type,
storage_type = EXCLUDED.storage_type,
title = COALESCE(NULLIF(EXCLUDED.title, ''), library.title), title = COALESCE(NULLIF(EXCLUDED.title, ''), library.title),
author = COALESCE(NULLIF(EXCLUDED.author, ''), library.author), author = COALESCE(NULLIF(EXCLUDED.author, ''), library.author),
publisher = COALESCE(NULLIF(EXCLUDED.publisher, ''), library.publisher), publisher = COALESCE(NULLIF(EXCLUDED.publisher, ''), library.publisher),
@ -375,7 +352,6 @@ def upsert_book(conn, filename: str, meta: dict, tags: list[tuple[str, str]] | N
series = COALESCE(NULLIF(EXCLUDED.series, ''), library.series), series = COALESCE(NULLIF(EXCLUDED.series, ''), library.series),
series_index = CASE WHEN COALESCE(EXCLUDED.series_index, 0) > 0 THEN EXCLUDED.series_index ELSE library.series_index END, series_index = CASE WHEN COALESCE(EXCLUDED.series_index, 0) > 0 THEN EXCLUDED.series_index ELSE library.series_index END,
series_suffix = COALESCE(NULLIF(EXCLUDED.series_suffix, ''), library.series_suffix), series_suffix = COALESCE(NULLIF(EXCLUDED.series_suffix, ''), library.series_suffix),
series_volume = COALESCE(NULLIF(EXCLUDED.series_volume, ''), library.series_volume),
publication_status = COALESCE(NULLIF(EXCLUDED.publication_status, ''), library.publication_status), publication_status = COALESCE(NULLIF(EXCLUDED.publication_status, ''), library.publication_status),
source_url = COALESCE(NULLIF(EXCLUDED.source_url, ''), library.source_url), source_url = COALESCE(NULLIF(EXCLUDED.source_url, ''), library.source_url),
publish_date = COALESCE(EXCLUDED.publish_date, library.publish_date), publish_date = COALESCE(EXCLUDED.publish_date, library.publish_date),
@ -386,7 +362,6 @@ def upsert_book(conn, filename: str, meta: dict, tags: list[tuple[str, str]] | N
( (
filename, filename,
meta.get("media_type", "epub"), meta.get("media_type", "epub"),
meta.get("storage_type", "file"),
meta.get("title", ""), meta.get("title", ""),
meta.get("author", ""), meta.get("author", ""),
meta.get("publisher", ""), meta.get("publisher", ""),
@ -394,7 +369,6 @@ def upsert_book(conn, filename: str, meta: dict, tags: list[tuple[str, str]] | N
meta.get("series", ""), meta.get("series", ""),
meta.get("series_index", 0), meta.get("series_index", 0),
meta.get("series_suffix", ""), meta.get("series_suffix", ""),
meta.get("series_volume", ""),
meta.get("publication_status", ""), meta.get("publication_status", ""),
meta.get("source_url", ""), meta.get("source_url", ""),
meta.get("publish_date") or None, meta.get("publish_date") or None,
@ -439,8 +413,6 @@ def list_library_json() -> list[dict]:
(cc.filename IS NOT NULL) AS has_cached_cover, (cc.filename IS NOT NULL) AS has_cached_cover,
l.rating, l.rating,
COALESCE(l.series_suffix, '') AS series_suffix, COALESCE(l.series_suffix, '') AS series_suffix,
COALESCE(l.storage_type, 'file') AS storage_type,
COALESCE(l.series_volume, '') AS series_volume,
json_agg( json_agg(
json_build_object('tag', bt.tag, 'tag_type', bt.tag_type) json_build_object('tag', bt.tag, 'tag_type', bt.tag_type)
) FILTER (WHERE bt.tag IS NOT NULL) AS tags ) FILTER (WHERE bt.tag IS NOT NULL) AS tags
@ -458,8 +430,8 @@ def list_library_json() -> list[dict]:
l.archived, l.needs_review, l.updated_at, l.archived, l.needs_review, l.updated_at,
rp.progress, rp.cfi, rp.page, rp.progress, rp.cfi, rp.page,
rs.read_count, rs.last_read, rs.read_count, rs.last_read,
cc.filename, l.rating, l.series_suffix, l.storage_type, l.series_volume cc.filename, l.rating, l.series_suffix
ORDER BY COALESCE(l.publisher, ''), COALESCE(l.author, ''), COALESCE(l.series, ''), COALESCE(l.series_volume, ''), l.series_index, COALESCE(l.title, '') ORDER BY COALESCE(l.publisher, ''), COALESCE(l.author, ''), COALESCE(l.series, ''), l.series_index, COALESCE(l.title, '')
""" """
) )
rows = cur.fetchall() rows = cur.fetchall()
@ -478,7 +450,6 @@ def list_library_json() -> list[dict]:
"series": r[6] or "", "series": r[6] or "",
"series_index": r[7] or 0, "series_index": r[7] or 0,
"series_suffix": r[20] or "", "series_suffix": r[20] or "",
"series_volume": r[22] or "",
"publication_status": r[8] or "", "publication_status": r[8] or "",
"want_to_read": bool(r[9]), "want_to_read": bool(r[9]),
"archived": bool(r[10]), "archived": bool(r[10]),
@ -489,92 +460,13 @@ def list_library_json() -> list[dict]:
"page": r[15], "page": r[15],
"read_count": r[16] or 0, "read_count": r[16] or 0,
"last_read": r[17].isoformat() if r[17] else None, "last_read": r[17].isoformat() if r[17] else None,
"storage_type": r[21] or "file", "tags": r[21] or [],
"tags": r[23] or [],
"rating": r[19] or 0, "rating": r[19] or 0,
} }
) )
return out return out
_IMAGE_EXT_MAP = {
"image/jpeg": ".jpg",
"image/png": ".png",
"image/webp": ".webp",
"image/gif": ".gif",
}
def write_image_file(data: bytes, media_type: str) -> tuple[str, str, str]:
"""Write image bytes to the content-addressed imagestore (no DB).
Returns (sha256, ext, url).
"""
sha256 = hashlib.sha256(data).hexdigest()
ext = _IMAGE_EXT_MAP.get(media_type, ".jpg")
img_path = IMAGES_DIR / sha256[:2] / f"{sha256}{ext}"
if not img_path.exists():
img_path.parent.mkdir(parents=True, exist_ok=True)
img_path.write_bytes(data)
url = f"/library/db-images/{sha256[:2]}/{sha256}{ext}"
return sha256, ext, url
def store_db_image(conn, data: bytes, media_type: str) -> tuple[str, str, str]:
"""Write image to imagestore and register in book_images table.
Returns (sha256, ext, url).
"""
sha256, ext, url = write_image_file(data, media_type)
with conn.cursor() as cur:
cur.execute(
"""
INSERT INTO book_images (sha256, ext, media_type, size_bytes)
VALUES (%s, %s, %s, %s)
ON CONFLICT (sha256) DO NOTHING
""",
(sha256, ext, media_type, len(data)),
)
return sha256, ext, url
def html_to_plain(html: str) -> str:
"""Strip HTML tags for tsvector input."""
from bs4 import BeautifulSoup
return BeautifulSoup(html, "html.parser").get_text(" ", strip=True)
def upsert_chapter(conn, filename: str, chapter_index: int, title: str, content_html: str) -> None:
"""Insert or replace a chapter in book_chapters and update its tsvector."""
plain = html_to_plain(content_html)
tsv_input = (title or "") + " " + plain
with conn.cursor() as cur:
cur.execute(
"""
INSERT INTO book_chapters (filename, chapter_index, title, content, content_tsv)
VALUES (%s, %s, %s, %s, to_tsvector('simple', %s))
ON CONFLICT (filename, chapter_index) DO UPDATE SET
title = EXCLUDED.title,
content = EXCLUDED.content,
content_tsv = EXCLUDED.content_tsv
""",
(filename, chapter_index, title, content_html, tsv_input),
)
def ensure_unique_db_filename(conn, base_filename: str) -> str:
"""Return a filename that doesn't yet exist in the library table."""
candidate = base_filename
counter = 2
while True:
with conn.cursor() as cur:
cur.execute("SELECT 1 FROM library WHERE filename = %s", (candidate,))
if not cur.fetchone():
return candidate
candidate = f"{base_filename} ({counter})"
counter += 1
def normalize_site(raw: str) -> str: def normalize_site(raw: str) -> str:
raw = (raw or "").strip() raw = (raw or "").strip()
if "://" in raw: if "://" in raw:

View File

@ -8,13 +8,14 @@ from pathlib import Path
from bs4 import BeautifulSoup from bs4 import BeautifulSoup
from fastapi import APIRouter, Request from fastapi import APIRouter, Request
from fastapi.responses import HTMLResponse, JSONResponse, Response from fastapi.responses import HTMLResponse, JSONResponse, Response
from shared_templates import templates from fastapi.templating import Jinja2Templates
from db import get_db_conn from db import get_db_conn
from epub import read_epub_file, write_epub_file from epub import read_epub_file, write_epub_file
from routers.common import LIBRARY_DIR, is_db_filename, resolve_library_path, upsert_chapter from routers.common import LIBRARY_DIR, resolve_library_path
router = APIRouter() router = APIRouter()
templates = Jinja2Templates(directory="templates")
def _norm(base_dir: str, rel: str) -> str: def _norm(base_dir: str, rel: str) -> str:
@ -157,7 +158,6 @@ def _rewrite_epub_entries(epub_path: Path, updates: dict[str, bytes], remove_pat
@router.get("/library/editor/{filename:path}", response_class=HTMLResponse) @router.get("/library/editor/{filename:path}", response_class=HTMLResponse)
async def editor_page(filename: str, request: Request): async def editor_page(filename: str, request: Request):
if not is_db_filename(filename):
path = resolve_library_path(filename) path = resolve_library_path(filename)
if path is None or not path.exists(): if path is None or not path.exists():
return HTMLResponse("Not found", status_code=404) return HTMLResponse("Not found", status_code=404)
@ -166,31 +166,13 @@ async def editor_page(filename: str, request: Request):
with conn.cursor() as cur: with conn.cursor() as cur:
cur.execute("SELECT title FROM library WHERE filename = %s", (filename,)) cur.execute("SELECT title FROM library WHERE filename = %s", (filename,))
row = cur.fetchone() row = cur.fetchone()
if not row: title = row[0] if row and row[0] else filename
return HTMLResponse("Not found", status_code=404)
title = row[0] if row[0] else filename
return templates.TemplateResponse(request, "editor.html", { return templates.TemplateResponse(request, "editor.html", {"filename": filename, "title": title})
"filename": filename,
"title": title,
"is_db": is_db_filename(filename),
})
@router.get("/api/edit/chapter/{index:int}/{filename:path}") @router.get("/api/edit/chapter/{index:int}/{filename:path}")
async def get_edit_chapter(filename: str, index: int): async def get_edit_chapter(filename: str, index: int):
if is_db_filename(filename):
with get_db_conn() as conn:
with conn.cursor() as cur:
cur.execute(
"SELECT title, content FROM book_chapters WHERE filename = %s AND chapter_index = %s",
(filename, index),
)
row = cur.fetchone()
if not row:
return Response(status_code=404)
return JSONResponse({"index": index, "href": f"db:{index}", "title": row[0], "content": row[1]})
path = resolve_library_path(filename) path = resolve_library_path(filename)
if path is None or not path.exists(): if path is None or not path.exists():
return Response(status_code=404) return Response(status_code=404)
@ -204,29 +186,13 @@ async def get_edit_chapter(filename: str, index: int):
@router.post("/api/edit/chapter/{index:int}/{filename:path}") @router.post("/api/edit/chapter/{index:int}/{filename:path}")
async def save_edit_chapter(filename: str, index: int, request: Request): async def save_edit_chapter(filename: str, index: int, request: Request):
body = await request.json()
content = body.get("content", "")
if is_db_filename(filename):
with get_db_conn() as conn:
with conn.cursor() as cur:
cur.execute(
"SELECT title FROM book_chapters WHERE filename = %s AND chapter_index = %s",
(filename, index),
)
row = cur.fetchone()
if not row:
return JSONResponse({"error": "Chapter not found"}, status_code=404)
new_title = (body.get("title") or "").strip() or row[0]
with conn:
upsert_chapter(conn, filename, index, new_title, content)
return JSONResponse({"ok": True})
path = resolve_library_path(filename) path = resolve_library_path(filename)
if path is None: if path is None:
return JSONResponse({"error": "not found"}, status_code=404) return JSONResponse({"error": "not found"}, status_code=404)
if not path.exists(): if not path.exists():
return JSONResponse({"error": "File not found"}, status_code=404) return JSONResponse({"error": "File not found"}, status_code=404)
body = await request.json()
content = body.get("content", "")
if not content: if not content:
return JSONResponse({"error": "No content"}, status_code=400) return JSONResponse({"error": "No content"}, status_code=400)
spine = _epub_spine(path) spine = _epub_spine(path)
@ -242,42 +208,15 @@ async def save_edit_chapter(filename: str, index: int, request: Request):
@router.post("/api/edit/chapter/add/{filename:path}") @router.post("/api/edit/chapter/add/{filename:path}")
async def add_edit_chapter(filename: str, request: Request): async def add_edit_chapter(filename: str, request: Request):
body = await request.json()
title = (body.get("title") or "New chapter").strip() or "New chapter"
after_index = body.get("after_index", -1)
if is_db_filename(filename):
try:
after_index = int(after_index)
except Exception:
after_index = -1
with get_db_conn() as conn:
with conn.cursor() as cur:
cur.execute("SELECT COUNT(*) FROM book_chapters WHERE filename = %s", (filename,))
total = cur.fetchone()[0]
cur.execute("SELECT 1 FROM library WHERE filename = %s", (filename,))
if not cur.fetchone():
return JSONResponse({"error": "not found"}, status_code=404)
insert_idx = total if after_index < 0 or after_index >= total else after_index + 1
with conn:
with conn.cursor() as cur:
cur.execute(
"UPDATE book_chapters SET chapter_index = chapter_index + 1 WHERE filename = %s AND chapter_index >= %s",
(filename, insert_idx),
)
upsert_chapter(conn, filename, insert_idx, title, "")
return JSONResponse({"ok": True, "index": insert_idx, "count": total + 1})
path = resolve_library_path(filename) path = resolve_library_path(filename)
if path is None: if path is None:
return JSONResponse({"error": "not found"}, status_code=404) return JSONResponse({"error": "not found"}, status_code=404)
if not path.exists(): if not path.exists():
return JSONResponse({"error": "File not found"}, status_code=404) return JSONResponse({"error": "File not found"}, status_code=404)
try: body = await request.json()
after_index = int(after_index) title = (body.get("title") or "New chapter").strip() or "New chapter"
except Exception: after_index = body.get("after_index", -1)
after_index = -1
try: try:
after_index = int(after_index) after_index = int(after_index)
except Exception: except Exception:
@ -400,26 +339,6 @@ async def add_edit_chapter(filename: str, request: Request):
@router.delete("/api/edit/chapter/{index:int}/{filename:path}") @router.delete("/api/edit/chapter/{index:int}/{filename:path}")
async def delete_edit_chapter(filename: str, index: int): async def delete_edit_chapter(filename: str, index: int):
if is_db_filename(filename):
with get_db_conn() as conn:
with conn.cursor() as cur:
cur.execute("SELECT COUNT(*) FROM book_chapters WHERE filename = %s", (filename,))
total = cur.fetchone()[0]
if total <= 1:
return JSONResponse({"error": "Cannot delete the last chapter"}, status_code=400)
with conn:
with conn.cursor() as cur:
cur.execute(
"DELETE FROM book_chapters WHERE filename = %s AND chapter_index = %s",
(filename, index),
)
cur.execute(
"UPDATE book_chapters SET chapter_index = chapter_index - 1 WHERE filename = %s AND chapter_index > %s",
(filename, index),
)
new_total = total - 1
return JSONResponse({"ok": True, "index": min(index, new_total - 1), "count": new_total})
path = resolve_library_path(filename) path = resolve_library_path(filename)
if path is None: if path is None:
return JSONResponse({"error": "not found"}, status_code=404) return JSONResponse({"error": "not found"}, status_code=404)

View File

@ -2,10 +2,11 @@ from urllib.parse import unquote
from fastapi import APIRouter, Request from fastapi import APIRouter, Request
from fastapi.responses import HTMLResponse from fastapi.responses import HTMLResponse
from shared_templates import templates from fastapi.templating import Jinja2Templates
from db import get_db_conn from db import get_db_conn
templates = Jinja2Templates(directory="templates")
router = APIRouter() router = APIRouter()

View File

@ -8,31 +8,27 @@ from typing import AsyncGenerator
from urllib.parse import urljoin, urlparse from urllib.parse import urljoin, urlparse
import httpx import httpx
from bs4 import BeautifulSoup, NavigableString, Tag from bs4 import Tag
from fastapi import APIRouter, Request from fastapi import APIRouter, Request
from fastapi.responses import HTMLResponse, StreamingResponse from fastapi.responses import HTMLResponse, StreamingResponse
from shared_templates import templates from fastapi.templating import Jinja2Templates
from db import get_db_conn from db import get_db_conn
from epub import detect_image_format, make_chapter_xhtml, make_epub from epub import detect_image_format, make_chapter_xhtml, make_epub
from routers.common import ( from routers.common import (
LIBRARY_DIR, LIBRARY_DIR,
ensure_unique_db_filename, ensure_cover_cache_for_book,
ensure_unique_rel_path, ensure_unique_rel_path,
make_cover_thumb_webp,
make_rel_path, make_rel_path,
normalize_site, normalize_site,
store_db_image,
upsert_book, upsert_book,
upsert_chapter,
upsert_cover_cache,
write_image_file,
) )
from scrapers import get_scraper from scrapers import get_scraper
from scrapers.base import HEADERS from scrapers.base import HEADERS
from security import decrypt_value, encrypt_value, is_encrypted_value from security import decrypt_value, encrypt_value, is_encrypted_value
from xhtml import configure_break_patterns, element_to_xhtml, is_break_element from xhtml import configure_break_patterns, element_to_xhtml, is_break_element
templates = Jinja2Templates(directory="templates")
router = APIRouter() router = APIRouter()
JOBS: dict[str, dict] = {} JOBS: dict[str, dict] = {}
@ -139,87 +135,22 @@ async def debug_run(request: Request):
result: dict = {} result: dict = {}
try: try:
async with httpx.AsyncClient(headers=HEADERS, follow_redirects=True, timeout=30) as client: async with httpx.AsyncClient(headers=HEADERS, follow_redirects=True, timeout=30) as client:
# Login
login_success = False
if username: if username:
login_success = await scraper.login(client, username, password) await scraper.login(client, username, password)
result["login"] = {
"attempted": bool(username),
"success": login_success,
"username": username,
}
book = await scraper.fetch_book_info(client, url) book = await scraper.fetch_book_info(client, url)
chapters = book.get("chapters", []) result = {
# Compute output filename
series = book.get("series", "")
series_index = int(book.get("series_index_hint", 1) or 1)
filename = make_rel_path(
media_type="epub",
publisher=book.get("publisher", ""),
author=book.get("author", ""),
title=book.get("title", ""),
series=series,
series_index=series_index,
).as_posix()
result["meta"] = {
"title": book.get("title", ""), "title": book.get("title", ""),
"author": book.get("author", ""), "author": book.get("author", ""),
"publisher": book.get("publisher", ""), "publisher": book.get("publisher", ""),
"series": book.get("series", ""), "series": book.get("series", ""),
"chapter_count": len(book.get("chapters", [])),
"chapter_method": book.get("chapter_method", ""),
"genres": book.get("genres", []), "genres": book.get("genres", []),
"subgenres": book.get("subgenres", []), "subgenres": book.get("subgenres", []),
"tags": book.get("tags", []), "tags": book.get("tags", []),
"description": book.get("description", ""), "description": book.get("description", ""),
"updated_date": book.get("updated_date", ""),
"publication_status": book.get("publication_status", ""), "publication_status": book.get("publication_status", ""),
"filename": filename,
} }
result["chapters"] = {
"count": len(chapters),
"method": book.get("chapter_method", ""),
"list": chapters,
}
# Fetch first chapter
if chapters:
ch = chapters[0]
try:
_load_break_patterns()
ch_data = await scraper.fetch_chapter(client, ch)
content_el = ch_data.get("content_el")
raw_html = content_el.decode_contents() if content_el else ""
xhtml_parts = []
if content_el:
from bs4 import Tag
all_p = content_el.find_all("p")
empty_p = sum(
1 for p in all_p
if not [c for c in p.children if isinstance(c, Tag)]
and not p.get_text().replace("\xa0", "").strip()
)
filled_p = len(all_p) - empty_p
empty_p_is_spacer = filled_p > 0 and empty_p >= filled_p * 0.5
for child in content_el.children:
part = element_to_xhtml(child, empty_p_is_spacer=empty_p_is_spacer)
if part.strip():
xhtml_parts.append(part)
result["first_chapter"] = {
"title": ch_data.get("title", ch["title"]),
"url": ch["url"],
"selector_id": ch_data.get("selector_id"),
"selector_class": ch_data.get("selector_class"),
"raw_html": raw_html[:8000],
"converted_xhtml": "\n".join(xhtml_parts)[:8000],
}
except Exception as e:
result["first_chapter"] = {"title": ch["title"], "url": ch["url"], "error": str(e)}
except Exception: except Exception:
result["error"] = traceback.format_exc() result["error"] = traceback.format_exc()
return result return result
@ -399,17 +330,8 @@ async def _run_scrape(job_id: str, url: str, username: str, password: str, send)
} }
_load_break_patterns() _load_break_patterns()
break_img_data = open("static/break.png", "rb").read()
storage_mode = job.get("storage_mode", "db")
# Break image path depends on storage mode:
# - EPUB: relative path inside the EPUB ZIP (break.png is embedded)
# - DB: absolute URL served by the static files handler
if storage_mode == "epub":
break_img_path = "../Images/break.png"
else:
break_img_path = "/static/break.png"
# Collect chapters as {title, content_html, images: [(sha256, ext, media_type, size, data)]}
chapters = [] chapters = []
for i, ch in enumerate(book["chapters"], 1): for i, ch in enumerate(book["chapters"], 1):
send("progress", {"current": i, "total": len(book["chapters"]), "title": ch["title"]}) send("progress", {"current": i, "total": len(book["chapters"]), "title": ch["title"]})
@ -417,21 +339,11 @@ async def _run_scrape(job_id: str, url: str, username: str, password: str, send)
ch_data = await scraper.fetch_chapter(client, ch) ch_data = await scraper.fetch_chapter(client, ch)
content_el = ch_data["content_el"] content_el = ch_data["content_el"]
# Download images and store to disk (no DB yet); rewrite src to absolute URL chapter_images = []
if content_el: if content_el:
img_counter = 1
for img_tag in content_el.find_all("img"): for img_tag in content_el.find_all("img"):
if is_break_element(img_tag): if is_break_element(img_tag):
# Replace the parent with <hr> if it contains only
# this image, so element_to_xhtml can detect the break.
parent = img_tag.parent
meaningful = [
c for c in parent.children
if not (isinstance(c, NavigableString) and not c.strip())
]
if len(meaningful) == 1 and parent is not content_el:
parent.replace_with(BeautifulSoup("<hr/>", "html.parser").hr)
else:
img_tag.decompose()
continue continue
src = img_tag.get("src", "") src = img_tag.get("src", "")
if not src or src.startswith("data:"): if not src or src.startswith("data:"):
@ -440,16 +352,19 @@ async def _run_scrape(job_id: str, url: str, username: str, password: str, send)
try: try:
img_resp = await client.get(urljoin(ch["url"], src)) img_resp = await client.get(urljoin(ch["url"], src))
if img_resp.status_code == 200: if img_resp.status_code == 200:
_, img_mime = detect_image_format( img_name, img_mime = detect_image_format(
img_resp.content, f"ch{i:03d}_img" img_resp.content, f"ch{i:03d}_img{img_counter:03d}"
) )
sha, ext_i, url = write_image_file(img_resp.content, img_mime) img_tag["src"] = f"../Images/{img_name}"
img_tag["src"] = url
img_tag["alt"] = img_tag.get("alt", "") img_tag["alt"] = img_tag.get("alt", "")
img_tag.attrs = { chapter_images.append(
k: v for k, v in img_tag.attrs.items() {
if k in ("src", "alt", "width", "height") "epub_path": f"OEBPS/Images/{img_name}",
"data": img_resp.content,
"media_type": img_mime,
} }
)
img_counter += 1
else: else:
img_tag.decompose() img_tag.decompose()
except Exception: except Exception:
@ -467,12 +382,13 @@ async def _run_scrape(job_id: str, url: str, username: str, password: str, send)
filled_p = len(all_p) - empty_p filled_p = len(all_p) - empty_p
empty_p_is_spacer = filled_p > 0 and empty_p >= filled_p * 0.5 empty_p_is_spacer = filled_p > 0 and empty_p >= filled_p * 0.5
for child in content_el.children: for child in content_el.children:
part = element_to_xhtml(child, break_img_path=break_img_path, empty_p_is_spacer=empty_p_is_spacer) part = element_to_xhtml(child, empty_p_is_spacer=empty_p_is_spacer)
if part.strip(): if part.strip():
xhtml_parts.append(part) xhtml_parts.append(part)
content_html = "\n".join(xhtml_parts) content_xhtml = "\n".join(xhtml_parts)
chapters.append({"title": ch_data["title"], "content_html": content_html}) chapter_xhtml = make_chapter_xhtml(ch_data["title"], content_xhtml, i)
chapters.append({"title": ch_data["title"], "xhtml": chapter_xhtml, "images": chapter_images})
await asyncio.sleep(0.2) await asyncio.sleep(0.2)
except Exception as e: except Exception as e:
send("warning", {"message": f"Chapter {i} skipped: {e}"}) send("warning", {"message": f"Chapter {i} skipped: {e}"})
@ -482,29 +398,12 @@ async def _run_scrape(job_id: str, url: str, username: str, password: str, send)
job["done"] = True job["done"] = True
return return
send("status", {"message": "Saving to library..."}) send("status", {"message": "Building EPUB..."})
book_id = str(uuid.uuid4())
epub_bytes = make_epub(book_title, author, chapters, cover_data, break_img_data, book_id, book_info)
book_tags = ( rel = ensure_unique_rel_path(
[(g, "genre") for g in book_info.get("genres", [])] make_rel_path(
+ [(g, "subgenre") for g in book_info.get("subgenres", [])]
+ [(g, "tag") for g in book_info.get("tags", [])]
)
if storage_mode == "epub":
# ── EPUB file on disk ──────────────────────────────────────────
epub_chapters = [
{"title": ch["title"], "xhtml": make_chapter_xhtml(ch["title"], ch["content_html"], i + 1), "images": []}
for i, ch in enumerate(chapters)
]
try:
break_img_data = open("static/break.png", "rb").read()
except Exception:
break_img_data = b""
epub_bytes = make_epub(
book_title, author, epub_chapters, cover_data, break_img_data,
str(uuid.uuid4()), book_info,
)
rel_path = make_rel_path(
media_type="epub", media_type="epub",
publisher=book_info.get("publisher", ""), publisher=book_info.get("publisher", ""),
author=author, author=author,
@ -512,51 +411,16 @@ async def _run_scrape(job_id: str, url: str, username: str, password: str, send)
series=series, series=series,
series_index=series_index, series_index=series_index,
) )
rel_path = ensure_unique_rel_path(rel_path) )
out_path = LIBRARY_DIR / rel_path out_path = LIBRARY_DIR / rel
out_path.parent.mkdir(parents=True, exist_ok=True) out_path.parent.mkdir(parents=True, exist_ok=True)
out_path.write_bytes(epub_bytes) out_path.write_bytes(epub_bytes)
rel_filename = rel_path.as_posix()
rel_filename = rel.as_posix()
job["filename"] = rel_filename
book_meta = { book_meta = {
"media_type": "epub", "media_type": "epub",
"storage_type": "file",
"has_cover": cover_data is not None,
"series": series,
"series_index": series_index if series else 0,
"title": book_title,
"publication_status": book_info.get("publication_status", ""),
"author": author,
"publisher": book_info.get("publisher", ""),
"source_url": book_info.get("source_url", ""),
"description": book_info.get("description", ""),
"publish_date": final_updated_date,
"needs_review": False,
}
with get_db_conn() as conn:
with conn:
upsert_book(conn, rel_filename, book_meta, book_tags)
if cover_data:
try:
thumb = make_cover_thumb_webp(cover_data)
upsert_cover_cache(conn, rel_filename, "image/webp", thumb)
except Exception:
pass
else:
# ── DB storage (default) ───────────────────────────────────────
base_filename = make_rel_path(
media_type="db",
publisher=book_info.get("publisher", ""),
author=author,
title=book_title,
series=series,
series_index=series_index,
).as_posix()
book_meta = {
"media_type": "epub",
"storage_type": "db",
"has_cover": cover_data is not None, "has_cover": cover_data is not None,
"series": book_info.get("series", ""), "series": book_info.get("series", ""),
"series_index": series_index if book_info.get("series") else 0, "series_index": series_index if book_info.get("series") else 0,
@ -569,21 +433,18 @@ async def _run_scrape(job_id: str, url: str, username: str, password: str, send)
"publish_date": final_updated_date, "publish_date": final_updated_date,
"needs_review": False, "needs_review": False,
} }
book_tags = (
[(g, "genre") for g in book_info.get("genres", [])]
+ [(g, "subgenre") for g in book_info.get("subgenres", [])]
+ [(g, "tag") for g in book_info.get("tags", [])]
)
with get_db_conn() as conn: with get_db_conn() as conn:
with conn: with conn:
rel_filename = ensure_unique_db_filename(conn, base_filename)
upsert_book(conn, rel_filename, book_meta, book_tags) upsert_book(conn, rel_filename, book_meta, book_tags)
for idx, ch in enumerate(chapters): ensure_cover_cache_for_book(conn, rel_filename, out_path, "epub")
upsert_chapter(conn, rel_filename, idx, ch["title"], ch["content_html"])
if cover_data:
try:
thumb = make_cover_thumb_webp(cover_data)
upsert_cover_cache(conn, rel_filename, "image/webp", thumb)
except Exception:
pass
job["filename"] = rel_filename send("done", {"filename": rel_filename, "title": book_title, "chapters": len(chapters)})
send("done", {"filename": rel_filename, "title": book_title, "chapters": len(chapters), "storage_type": storage_mode})
job["done"] = True job["done"] = True
@ -610,7 +471,6 @@ async def convert(request: Request):
job["series_index"] = int(body.get("series_index", 1) or 1) job["series_index"] = int(body.get("series_index", 1) or 1)
job["updated_date_override"] = (body.get("updated_date") or "").strip() job["updated_date_override"] = (body.get("updated_date") or "").strip()
job["storage_mode"] = "epub" if body.get("storage_mode") == "epub" else "db"
JOBS[job_id] = job JOBS[job_id] = job
asyncio.create_task(scrape_book(job_id, url, username, password)) asyncio.create_task(scrape_book(job_id, url, username, password))

View File

@ -6,7 +6,7 @@ from pathlib import Path
from fastapi import APIRouter, File, Request, UploadFile from fastapi import APIRouter, File, Request, UploadFile
from fastapi.responses import FileResponse, HTMLResponse, JSONResponse, Response from fastapi.responses import FileResponse, HTMLResponse, JSONResponse, Response
from shared_templates import templates from fastapi.templating import Jinja2Templates
from PIL import UnidentifiedImageError from PIL import UnidentifiedImageError
from db import get_db_conn from db import get_db_conn
@ -15,7 +15,6 @@ from routers.common import (
LIBRARY_DIR, LIBRARY_DIR,
ensure_cover_cache_for_book, ensure_cover_cache_for_book,
ensure_unique_rel_path, ensure_unique_rel_path,
is_db_filename,
list_library_json, list_library_json,
make_cover_thumb_webp, make_cover_thumb_webp,
make_rel_path, make_rel_path,
@ -28,6 +27,7 @@ from routers.common import (
upsert_cover_cache, upsert_cover_cache,
) )
templates = Jinja2Templates(directory="templates")
router = APIRouter() router = APIRouter()
@ -175,17 +175,6 @@ async def library_download(filename: str):
@router.delete("/library/file/{filename:path}") @router.delete("/library/file/{filename:path}")
async def library_delete(filename: str): async def library_delete(filename: str):
if is_db_filename(filename):
with get_db_conn() as conn:
with conn.cursor() as cur:
cur.execute("SELECT 1 FROM library WHERE filename = %s", (filename,))
if not cur.fetchone():
return {"error": "Not found"}
with conn:
with conn.cursor() as cur:
cur.execute("DELETE FROM library WHERE filename = %s", (filename,))
return {"ok": True}
full = resolve_library_path(filename) full = resolve_library_path(filename)
if full is None: if full is None:
return {"error": "Invalid filename"} return {"error": "Invalid filename"}
@ -244,7 +233,6 @@ async def library_bulk_delete(request: Request):
@router.get("/library/cover-cached/{filename:path}") @router.get("/library/cover-cached/{filename:path}")
async def library_cover_cached(filename: str): async def library_cover_cached(filename: str):
if not is_db_filename(filename):
full = resolve_library_path(filename) full = resolve_library_path(filename)
if full is None or not full.exists(): if full is None or not full.exists():
return Response(status_code=404) return Response(status_code=404)
@ -278,19 +266,6 @@ async def library_cover_cached(filename: str):
@router.get("/library/cover/{filename:path}") @router.get("/library/cover/{filename:path}")
async def library_cover(filename: str): async def library_cover(filename: str):
if is_db_filename(filename):
# DB books: cover is always served from the cache
with get_db_conn() as conn:
with conn.cursor() as cur:
cur.execute(
"SELECT thumb_webp FROM library_cover_cache WHERE filename = %s",
(filename,),
)
row = cur.fetchone()
if row and row[0]:
return Response(content=bytes(row[0]), media_type="image/webp")
return Response(status_code=404)
full = resolve_library_path(filename) full = resolve_library_path(filename)
if full is None or not full.exists(): if full is None or not full.exists():
return Response(status_code=404) return Response(status_code=404)
@ -342,6 +317,12 @@ async def library_cover(filename: str):
@router.post("/library/cover/{filename:path}") @router.post("/library/cover/{filename:path}")
async def library_add_cover(filename: str, request: Request): async def library_add_cover(filename: str, request: Request):
full = resolve_library_path(filename)
if full is None or not full.exists():
return {"error": "File not found"}
if media_type_from_suffix(full) != "epub":
return {"error": "Cover upload is only supported for EPUB"}
body = await request.json() body = await request.json()
cover_b64 = body.get("cover_b64", "") cover_b64 = body.get("cover_b64", "")
if not cover_b64: if not cover_b64:
@ -349,35 +330,6 @@ async def library_add_cover(filename: str, request: Request):
try: try:
cover_data = base64.b64decode(cover_b64) cover_data = base64.b64decode(cover_b64)
except Exception as e:
return {"error": str(e)}
if is_db_filename(filename):
with get_db_conn() as conn:
with conn.cursor() as cur:
cur.execute("SELECT 1 FROM library WHERE filename = %s", (filename,))
if not cur.fetchone():
return {"error": "Not found"}
with conn:
with conn.cursor() as cur:
cur.execute(
"UPDATE library SET has_cover = TRUE, updated_at = NOW() WHERE filename = %s",
(filename,),
)
try:
thumb = make_cover_thumb_webp(cover_data)
upsert_cover_cache(conn, filename, "image/webp", thumb)
except (UnidentifiedImageError, OSError, ValueError):
pass
return {"ok": True}
full = resolve_library_path(filename)
if full is None or not full.exists():
return {"error": "File not found"}
if media_type_from_suffix(full) != "epub":
return {"error": "Cover upload is only supported for EPUB"}
try:
add_cover_to_epub(full, cover_data) add_cover_to_epub(full, cover_data)
except Exception as e: except Exception as e:
return {"error": str(e)} return {"error": str(e)}
@ -441,24 +393,6 @@ async def library_archive(filename: str):
return {"ok": True, "archived": val} return {"ok": True, "archived": val}
@router.post("/library/archive-series")
async def library_archive_series(request: Request):
body = await request.json()
series = body.get("series", "")
archive = bool(body.get("archive", True))
if not series:
return {"error": "series is required"}
with get_db_conn() as conn:
with conn:
with conn.cursor() as cur:
cur.execute(
"UPDATE library SET archived = %s, updated_at = NOW() WHERE series = %s",
(archive, series),
)
count = cur.rowcount
return {"ok": True, "archived": archive, "count": count}
@router.post("/library/new/mark-reviewed") @router.post("/library/new/mark-reviewed")
async def library_mark_new_reviewed(request: Request): async def library_mark_new_reviewed(request: Request):
body = await request.json() body = await request.json()
@ -703,59 +637,46 @@ async def bulk_check_duplicates(request: Request):
for item in items: for item in items:
title = item.get("title", "").strip().lower() title = item.get("title", "").strip().lower()
author = item.get("author", "").strip().lower() author = item.get("author", "").strip().lower()
series = item.get("series", "").strip().lower()
vol_str = item.get("volume", "").strip() vol_str = item.get("volume", "").strip()
try: try:
vol_int = int(vol_str) if vol_str else None vol_int = int(vol_str) if vol_str else None
except ValueError: except ValueError:
vol_int = None vol_int = None
parsed.append((title, author, series, vol_int)) parsed.append((title, author, vol_int))
# Fetch all DB rows matching any (title, author) pair
title_author_pairs = list({(t, a) for t, a, _ in parsed if t})
if not title_author_pairs:
return {"duplicates": [False] * len(items)}
with get_db_conn() as conn:
with conn.cursor() as cur:
# Check by title+author
title_author_pairs = list({(t, a) for t, a, s, _ in parsed if t})
existing_with_vol: set = set()
existing_title_author: set = set()
if title_author_pairs:
conditions = " OR ".join( conditions = " OR ".join(
"(LOWER(TRIM(title)) = %s AND LOWER(TRIM(author)) = %s)" for _ in title_author_pairs "(LOWER(TRIM(title)) = %s AND LOWER(TRIM(author)) = %s)" for _ in title_author_pairs
) )
params = [v for pair in title_author_pairs for v in pair] params = [v for pair in title_author_pairs for v in pair]
with get_db_conn() as conn:
with conn.cursor() as cur:
cur.execute( cur.execute(
f"SELECT LOWER(TRIM(title)), LOWER(TRIM(author)), series_index" f"SELECT LOWER(TRIM(title)), LOWER(TRIM(author)), series_index"
f" FROM library WHERE {conditions}", f" FROM library WHERE {conditions}",
params, params,
) )
rows = cur.fetchall() rows = cur.fetchall()
# (title, author, series_index) for volume-aware lookup
existing_with_vol = {(r[0] or "", r[1] or "", r[2]) for r in rows} existing_with_vol = {(r[0] or "", r[1] or "", r[2]) for r in rows}
# (title, author) for volume-less lookup
existing_title_author = {(r[0] or "", r[1] or "") for r in rows} existing_title_author = {(r[0] or "", r[1] or "") for r in rows}
# Check by series+author+series_index (catches title-format changes)
series_author_pairs = list({(s, a) for _, a, s, _ in parsed if s and a})
existing_series_vol: set = set()
if series_author_pairs:
conditions2 = " OR ".join(
"(LOWER(TRIM(series)) = %s AND LOWER(TRIM(author)) = %s)" for _ in series_author_pairs
)
params2 = [v for pair in series_author_pairs for v in pair]
cur.execute(
f"SELECT LOWER(TRIM(series)), LOWER(TRIM(author)), series_index"
f" FROM library WHERE {conditions2}",
params2,
)
existing_series_vol = {(r[0] or "", r[1] or "", r[2]) for r in cur.fetchall()}
duplicates = [] duplicates = []
for title, author, series, vol_int in parsed: for title, author, vol_int in parsed:
if vol_int is not None: if not title:
by_title = (title, author, vol_int) in existing_with_vol if title else False
by_series = (series, author, vol_int) in existing_series_vol if series else False
duplicates.append(by_title or by_series)
elif title:
duplicates.append((title, author) in existing_title_author)
else:
duplicates.append(False) duplicates.append(False)
elif vol_int is not None:
# Volume known: only a duplicate when title+author+volume all match
duplicates.append((title, author, vol_int) in existing_with_vol)
else:
# No volume: duplicate if any title+author match exists
duplicates.append((title, author) in existing_title_author)
return {"duplicates": duplicates} return {"duplicates": duplicates}

View File

@ -12,28 +12,16 @@ from pathlib import Path
from bs4 import BeautifulSoup from bs4 import BeautifulSoup
from fastapi import APIRouter, Request from fastapi import APIRouter, Request
from fastapi.responses import FileResponse, HTMLResponse, JSONResponse, Response from fastapi.responses import FileResponse, HTMLResponse, JSONResponse, Response
from shared_templates import templates from fastapi.templating import Jinja2Templates
from cbr import cbr_get_page, cbr_page_count from cbr import cbr_get_page, cbr_page_count
from db import get_db_conn from db import get_db_conn
from epub import make_chapter_xhtml, make_epub, read_epub_file, write_epub_file from epub import read_epub_file, write_epub_file
from pdf import pdf_page_count, pdf_render_page from pdf import pdf_page_count, pdf_render_page
from routers.common import ( from routers.common import LIBRARY_DIR, prune_empty_dirs, resolve_library_path, scan_epub
IMAGES_DIR,
LIBRARY_DIR,
ensure_unique_db_filename,
is_db_filename,
make_cover_thumb_webp,
make_rel_path,
prune_empty_dirs,
resolve_library_path,
scan_epub,
upsert_chapter,
upsert_cover_cache,
write_image_file,
)
router = APIRouter() router = APIRouter()
templates = Jinja2Templates(directory="templates")
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
# EPUB helpers # EPUB helpers
@ -280,7 +268,6 @@ def _sync_epub_metadata(
series: str, series: str,
series_index: int | str | None, series_index: int | str | None,
series_suffix: str = "", series_suffix: str = "",
series_volume: str = "",
subjects: list[str], subjects: list[str],
) -> None: ) -> None:
"""Write edited metadata back into OPF so DB and EPUB stay aligned.""" """Write edited metadata back into OPF so DB and EPUB stay aligned."""
@ -360,11 +347,9 @@ def _sync_epub_metadata(
set_named_meta('calibre:series_index', str(_coerce_series_index(series_index))) set_named_meta('calibre:series_index', str(_coerce_series_index(series_index)))
sfx = re.sub(r"[^a-z]", "", (series_suffix or "").lower())[:5] sfx = re.sub(r"[^a-z]", "", (series_suffix or "").lower())[:5]
set_named_meta('novela:series_suffix', sfx) set_named_meta('novela:series_suffix', sfx)
set_named_meta('novela:series_volume', (series_volume or '').strip()[:20])
else: else:
set_named_meta('calibre:series_index', '') set_named_meta('calibre:series_index', '')
set_named_meta('novela:series_suffix', '') set_named_meta('novela:series_suffix', '')
set_named_meta('novela:series_volume', '')
_rewrite_epub_entries(epub_path, {opf_path: str(opf).encode('utf-8')}) _rewrite_epub_entries(epub_path, {opf_path: str(opf).encode('utf-8')})
@ -400,9 +385,9 @@ def _rewrite_epub_entries(epub_path: Path, updates: dict[str, bytes], remove_pat
def _clean_segment(value: str, fallback: str, max_len: int = 100) -> str: def _clean_segment(value: str, fallback: str, max_len: int = 100) -> str:
txt = re.sub(r"\s+", "_", (value or "").strip()) txt = re.sub(r"\s+", " ", (value or "").strip())
txt = re.sub(r'[<>:"/\\|?*\x00-\x1f]', "", txt) txt = re.sub(r'[<>:"/\\|?*\x00-\x1f]', "", txt)
txt = re.sub(r"\.+$", "", txt).strip("_") txt = re.sub(r"\.+$", "", txt).strip()
if not txt: if not txt:
txt = fallback txt = fallback
return txt[:max_len] return txt[:max_len]
@ -434,7 +419,7 @@ def _make_rel_path(
if series_name: if series_name:
idx = _coerce_series_index(series_index) idx = _coerce_series_index(series_index)
sfx = re.sub(r"[^a-z]", "", (series_suffix or "").lower())[:5] sfx = re.sub(r"[^a-z]", "", (series_suffix or "").lower())[:5]
return Path("epub") / pub / auth / "Series" / series_name / f"{idx:03d}{sfx}_-_{ttl}.epub" return Path("epub") / pub / auth / "Series" / series_name / f"{idx:03d}{sfx} - {ttl}.epub"
return Path("epub") / pub / auth / "Stories" / f"{ttl}.epub" return Path("epub") / pub / auth / "Stories" / f"{ttl}.epub"
if ext == ".pdf": if ext == ".pdf":
@ -447,7 +432,7 @@ def _make_rel_path(
if series_name: if series_name:
idx = _coerce_series_index(series_index) idx = _coerce_series_index(series_index)
sfx = re.sub(r"[^a-z]", "", (series_suffix or "").lower())[:5] sfx = re.sub(r"[^a-z]", "", (series_suffix or "").lower())[:5]
return Path("comics") / pub / auth / "Series" / series_name / f"{idx:03d}{sfx}_-_{ttl}{ext}" return Path("comics") / pub / auth / "Series" / series_name / f"{idx:03d}{sfx} - {ttl}{ext}"
return Path("comics") / pub / auth / f"{ttl}{ext}" return Path("comics") / pub / auth / f"{ttl}{ext}"
@ -474,21 +459,6 @@ def _guard(filename: str) -> bool:
# Routes # Routes
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
@router.get("/library/db-images/{path:path}")
async def serve_db_image(path: str):
"""Serve an image from the content-addressed imagestore."""
img_path = (IMAGES_DIR / path).resolve()
try:
img_path.relative_to(IMAGES_DIR.resolve())
except ValueError:
return Response(status_code=404)
if not img_path.exists():
return Response(status_code=404)
ext = img_path.suffix.lower()
mt = {".jpg": "image/jpeg", ".png": "image/png", ".webp": "image/webp", ".gif": "image/gif"}.get(ext, "application/octet-stream")
return FileResponse(img_path, media_type=mt)
@router.get("/library/epub/{filename:path}") @router.get("/library/epub/{filename:path}")
async def library_epub(filename: str): async def library_epub(filename: str):
"""Serve EPUB inline (no Content-Disposition: attachment) for the reader.""" """Serve EPUB inline (no Content-Disposition: attachment) for the reader."""
@ -502,18 +472,6 @@ async def library_epub(filename: str):
@router.get("/library/chapters/{filename:path}") @router.get("/library/chapters/{filename:path}")
async def get_chapter_list(filename: str): async def get_chapter_list(filename: str):
if is_db_filename(filename):
with get_db_conn() as conn:
with conn.cursor() as cur:
cur.execute(
"SELECT chapter_index, title FROM book_chapters WHERE filename = %s ORDER BY chapter_index",
(filename,),
)
rows = cur.fetchall()
if not rows:
return Response(status_code=404)
return [{"index": r[0], "title": r[1], "href": f"db:{r[0]}"} for r in rows]
path = resolve_library_path(filename) path = resolve_library_path(filename)
if path is None: if path is None:
return Response(status_code=404) return Response(status_code=404)
@ -524,32 +482,7 @@ async def get_chapter_list(filename: str):
@router.get("/library/chapter/{index}/{filename:path}") @router.get("/library/chapter/{index}/{filename:path}")
async def get_chapter_html(filename: str, index: int): async def get_chapter_html(filename: str, index: int):
"""Extract a single chapter from the EPUB (or DB) and return it as an HTML fragment.""" """Extract a single chapter from the EPUB and return it as an HTML fragment."""
if is_db_filename(filename):
with get_db_conn() as conn:
with conn.cursor() as cur:
cur.execute(
"SELECT title, content FROM book_chapters WHERE filename = %s AND chapter_index = %s",
(filename, index),
)
row = cur.fetchone()
if not row:
return Response(status_code=404)
title, content = row
safe_title = _html.escape(title or "")
# Strip leading h-tags from stored content — the endpoint always
# prepends its own <h2 class="chapter-title">, so content scraped
# before front-matter stripping was added would show the title twice.
# Handles two layouts:
# 1. <h1>…</h1> at the very start of content
# 2. <section …>\n<h1>…</h1> (pandoc-style wrapping)
content = re.sub(r'(?si)^(\s*<h[1-4](?:\s[^>]*)?>.*?</h[1-4]>)+\s*', '', content)
content = re.sub(r'(?si)(<(?:section|div)[^>]*>\s*)<h[1-4][^>]*>.*?</h[1-4]>\s*', r'\1', content, count=1)
return Response(
f'<body><h2 class="chapter-title">{safe_title}</h2>\n{content}\n</body>',
media_type="text/html",
)
path = resolve_library_path(filename) path = resolve_library_path(filename)
if path is None: if path is None:
return Response(status_code=404) return Response(status_code=404)
@ -672,16 +605,11 @@ async def save_progress(filename: str, request: Request):
@router.post("/library/mark-read/{filename:path}") @router.post("/library/mark-read/{filename:path}")
async def library_mark_read(filename: str, request: Request): async def library_mark_read(filename: str, request: Request):
if not is_db_filename(filename): if resolve_library_path(filename) is None:
return {"error": "Invalid filename"}
path = resolve_library_path(filename) path = resolve_library_path(filename)
if path is None or not path.exists(): if path is None or not path.exists():
return {"error": "File not found"} return {"error": "File not found"}
else:
with get_db_conn() as conn:
with conn.cursor() as cur:
cur.execute("SELECT 1 FROM library WHERE filename = %s", (filename,))
if not cur.fetchone():
return {"error": "Not found"}
body = {} body = {}
try: try:
body = await request.json() body = await request.json()
@ -707,24 +635,18 @@ async def library_mark_read(filename: str, request: Request):
@router.get("/library/book/{filename:path}", response_class=HTMLResponse) @router.get("/library/book/{filename:path}", response_class=HTMLResponse)
async def book_detail_page(filename: str, request: Request): async def book_detail_page(filename: str, request: Request):
db_book = is_db_filename(filename)
if not db_book:
path = resolve_library_path(filename) path = resolve_library_path(filename)
if path is None: if path is None:
return HTMLResponse("Not found", status_code=404) return HTMLResponse("Not found", status_code=404)
if not path.exists(): if not path.exists():
return HTMLResponse("Not found", status_code=404) return HTMLResponse("Not found", status_code=404)
else:
path = None
with get_db_conn() as conn: with get_db_conn() as conn:
with conn.cursor() as cur: with conn.cursor() as cur:
cur.execute( cur.execute(
""" """
SELECT title, author, publisher, has_cover, series, series_index, SELECT title, author, publisher, has_cover, series, series_index,
publication_status, want_to_read, source_url, archived, publish_date, description, publication_status, want_to_read, source_url, archived, publish_date, description,
rating, COALESCE(series_suffix, '') AS series_suffix, rating, COALESCE(series_suffix, '') AS series_suffix
COALESCE(storage_type, 'file') AS storage_type,
COALESCE(series_volume, '') AS series_volume
FROM library WHERE filename = %s FROM library WHERE filename = %s
""", """,
(filename,), (filename,),
@ -739,7 +661,6 @@ async def book_detail_page(filename: str, request: Request):
"series": lib_row[4] or "", "series": lib_row[4] or "",
"series_index": lib_row[5] or 0, "series_index": lib_row[5] or 0,
"series_suffix": lib_row[13] or "", "series_suffix": lib_row[13] or "",
"series_volume": lib_row[15] or "",
"publication_status": lib_row[6] or "", "publication_status": lib_row[6] or "",
"want_to_read": lib_row[7] or False, "want_to_read": lib_row[7] or False,
"source_url": lib_row[8] or "", "source_url": lib_row[8] or "",
@ -747,12 +668,9 @@ async def book_detail_page(filename: str, request: Request):
"publish_date": lib_row[10].isoformat() if lib_row[10] else "", "publish_date": lib_row[10].isoformat() if lib_row[10] else "",
"description": lib_row[11] or "", "description": lib_row[11] or "",
"rating": lib_row[12] or 0, "rating": lib_row[12] or 0,
"storage_type": lib_row[14] or "file",
} }
# Supplement empty fields from EPUB metadata (file-based books only) # Supplement empty fields from EPUB metadata
if not db_book and path and ( if not entry["source_url"] or not entry["publish_date"] or not entry["description"]:
not entry["source_url"] or not entry["publish_date"] or not entry["description"]
):
epub_meta = scan_epub(path) epub_meta = scan_epub(path)
if not entry["source_url"]: if not entry["source_url"]:
entry["source_url"] = epub_meta.get("source_url", "") entry["source_url"] = epub_meta.get("source_url", "")
@ -761,15 +679,12 @@ async def book_detail_page(filename: str, request: Request):
if not entry["description"]: if not entry["description"]:
entry["description"] = epub_meta.get("description", "") entry["description"] = epub_meta.get("description", "")
else: else:
if db_book:
return HTMLResponse("Not found", status_code=404)
entry = scan_epub(path) entry = scan_epub(path)
entry.setdefault("want_to_read", False) entry.setdefault("want_to_read", False)
entry.setdefault("archived", False) entry.setdefault("archived", False)
entry.setdefault("publish_date", "") entry.setdefault("publish_date", "")
entry.setdefault("description", "") entry.setdefault("description", "")
entry.setdefault("rating", 0) entry.setdefault("rating", 0)
entry.setdefault("storage_type", "file")
cur.execute( cur.execute(
"SELECT tag, tag_type FROM book_tags WHERE filename = %s ORDER BY tag_type, tag", "SELECT tag, tag_type FROM book_tags WHERE filename = %s ORDER BY tag_type, tag",
@ -787,7 +702,7 @@ async def book_detail_page(filename: str, request: Request):
else: else:
tags_list.append(tag) tags_list.append(tag)
if not rows and not db_book and path: if not rows:
# Fallback for books where tags only exist in OPF after DB loss/rebuild. # Fallback for books where tags only exist in OPF after DB loss/rebuild.
epub_meta = scan_epub(path) epub_meta = scan_epub(path)
for subject in epub_meta.get("subjects", []): for subject in epub_meta.get("subjects", []):
@ -826,7 +741,6 @@ async def book_detail_page(filename: str, request: Request):
"series": entry["series"], "series": entry["series"],
"series_index": entry["series_index"], "series_index": entry["series_index"],
"series_suffix": entry["series_suffix"], "series_suffix": entry["series_suffix"],
"series_volume": entry.get("series_volume", ""),
"genres": genres, "genres": genres,
"subgenres": subgenres, "subgenres": subgenres,
"tags": tags_list, "tags": tags_list,
@ -844,7 +758,6 @@ async def book_detail_page(filename: str, request: Request):
"cfi": cfi, "cfi": cfi,
"rating": entry.get("rating", 0), "rating": entry.get("rating", 0),
"series_is_indexed": series_is_indexed, "series_is_indexed": series_is_indexed,
"storage_type": entry.get("storage_type", "file"),
}) })
@ -886,149 +799,20 @@ async def api_suggestions(type: str | None = None):
return JSONResponse([r[0] for r in cur.fetchall()]) return JSONResponse([r[0] for r in cur.fetchall()])
@router.get("/api/series-nav/{filename:path}")
async def api_series_nav(filename: str):
"""Return the previous and next book in the same series, ordered by series_index."""
with get_db_conn() as conn:
with conn.cursor() as cur:
cur.execute(
"SELECT series, series_index, COALESCE(series_suffix, '') FROM library WHERE filename = %s",
(filename,),
)
row = cur.fetchone()
if not row or not row[0]:
return JSONResponse({"prev": None, "next": None})
series, current_index, current_suffix = row
with get_db_conn() as conn:
with conn.cursor() as cur:
cur.execute(
"""
SELECT filename, title, series_index, COALESCE(series_suffix, '')
FROM library
WHERE series = %s AND series IS NOT NULL AND series <> ''
ORDER BY series_index ASC, series_suffix ASC
""",
(series,),
)
siblings = cur.fetchall()
# Find position of current book in ordered list
pos = None
for i, (fn, _title, idx, sfx) in enumerate(siblings):
if fn == filename:
pos = i
break
if pos is None:
return JSONResponse({"prev": None, "next": None})
def entry(row):
return {"filename": row[0], "title": row[1], "index": row[2], "suffix": row[3]}
return JSONResponse({
"prev": entry(siblings[pos - 1]) if pos > 0 else None,
"next": entry(siblings[pos + 1]) if pos < len(siblings) - 1 else None,
})
@router.patch("/library/book/{filename:path}") @router.patch("/library/book/{filename:path}")
async def book_update(filename: str, request: Request): async def book_update(filename: str, request: Request):
"""Update book metadata and tags, and rename/move the file when needed.""" """Update book metadata and tags, and rename/move the file when needed."""
old_path = resolve_library_path(filename)
if old_path is None or not old_path.exists():
return JSONResponse({"error": "not found"}, status_code=404)
body = await request.json() body = await request.json()
title = body.get("title", "") title = body.get("title", "")
author = body.get("author", "") author = body.get("author", "")
publisher = body.get("publisher", "") publisher = body.get("publisher", "")
series = body.get("series", "") series = body.get("series", "")
series_volume = (body.get("series_volume", "") or "").strip()[:20]
from routers.common import parse_volume_str from routers.common import parse_volume_str
series_index, series_suffix = parse_volume_str(body.get("series_index", "")) series_index, series_suffix = parse_volume_str(body.get("series_index", ""))
# --- DB-stored book branch (no file on disk) ---
if is_db_filename(filename):
base_new = make_rel_path(
media_type="db",
publisher=publisher,
author=author,
title=title,
series=series,
series_index=series_index,
series_suffix=series_suffix,
).as_posix()
with get_db_conn() as conn:
with conn.cursor() as cur:
cur.execute("SELECT 1 FROM library WHERE filename = %s", (filename,))
if not cur.fetchone():
return JSONResponse({"error": "not found"}, status_code=404)
new_filename = ensure_unique_db_filename(conn, base_new) if base_new != filename else filename
with conn:
with conn.cursor() as cur:
cur.execute("SELECT has_cover FROM library WHERE filename = %s", (filename,))
row = cur.fetchone()
has_cover = bool(row[0]) if row else False
cur.execute(
"""
INSERT INTO library (
filename, title, author, publisher, has_cover,
series, series_index, series_suffix, series_volume, publication_status,
source_url, publish_date, description,
archived, needs_review, storage_type, updated_at
)
VALUES (%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, FALSE, FALSE, 'db', NOW())
ON CONFLICT (filename) DO UPDATE SET
title = EXCLUDED.title,
author = EXCLUDED.author,
publisher = EXCLUDED.publisher,
series = EXCLUDED.series,
series_index = EXCLUDED.series_index,
series_suffix = EXCLUDED.series_suffix,
series_volume = EXCLUDED.series_volume,
publication_status = EXCLUDED.publication_status,
source_url = EXCLUDED.source_url,
publish_date = EXCLUDED.publish_date,
description = EXCLUDED.description,
needs_review = FALSE,
updated_at = NOW()
""",
(
new_filename, title, author, publisher, has_cover,
series, series_index if series else 0,
series_suffix if series else "",
series_volume if series else "",
body.get("publication_status", ""),
body.get("source_url", ""),
body.get("publish_date") or None,
body.get("description", ""),
),
)
if new_filename != filename:
cur.execute("UPDATE book_tags SET filename = %s WHERE filename = %s", (new_filename, filename))
cur.execute("UPDATE reading_progress SET filename = %s WHERE filename = %s", (new_filename, filename))
cur.execute("UPDATE reading_sessions SET filename = %s WHERE filename = %s", (new_filename, filename))
cur.execute("UPDATE library_cover_cache SET filename = %s WHERE filename = %s", (new_filename, filename))
cur.execute("UPDATE book_chapters SET filename = %s WHERE filename = %s", (new_filename, filename))
cur.execute("UPDATE bookmarks SET filename = %s WHERE filename = %s", (new_filename, filename))
cur.execute("DELETE FROM library WHERE filename = %s", (filename,))
cur.execute("DELETE FROM book_tags WHERE filename = %s", (new_filename,))
rows = (
[(new_filename, g, "genre") for g in body.get("genres", []) if g]
+ [(new_filename, g, "subgenre") for g in body.get("subgenres", []) if g]
+ [(new_filename, g, "tag") for g in body.get("tags", []) if g]
)
if rows:
cur.executemany(
"INSERT INTO book_tags (filename, tag, tag_type) VALUES (%s, %s, %s)"
" ON CONFLICT (filename, tag, tag_type) DO NOTHING",
rows,
)
return JSONResponse({"ok": True, "filename": new_filename, "renamed": new_filename != filename})
# --- File-based book branch ---
old_path = resolve_library_path(filename)
if old_path is None or not old_path.exists():
return JSONResponse({"error": "not found"}, status_code=404)
ext = old_path.suffix.lower() ext = old_path.suffix.lower()
target_rel = _make_rel_path( target_rel = _make_rel_path(
@ -1064,7 +848,6 @@ async def book_update(filename: str, request: Request):
series=series, series=series,
series_index=series_index if series else 0, series_index=series_index if series else 0,
series_suffix=series_suffix if series else "", series_suffix=series_suffix if series else "",
series_volume=series_volume if series else "",
subjects=(body.get("genres", []) + body.get("subgenres", []) + body.get("tags", [])), subjects=(body.get("genres", []) + body.get("subgenres", []) + body.get("tags", [])),
) )
@ -1079,11 +862,11 @@ async def book_update(filename: str, request: Request):
""" """
INSERT INTO library ( INSERT INTO library (
filename, title, author, publisher, has_cover, filename, title, author, publisher, has_cover,
series, series_index, series_suffix, series_volume, publication_status, series, series_index, series_suffix, publication_status,
source_url, publish_date, description, source_url, publish_date, description,
archived, needs_review, updated_at archived, needs_review, updated_at
) )
VALUES (%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, FALSE, FALSE, NOW()) VALUES (%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, FALSE, FALSE, NOW())
ON CONFLICT (filename) DO UPDATE SET ON CONFLICT (filename) DO UPDATE SET
title = EXCLUDED.title, title = EXCLUDED.title,
author = EXCLUDED.author, author = EXCLUDED.author,
@ -1091,7 +874,6 @@ async def book_update(filename: str, request: Request):
series = EXCLUDED.series, series = EXCLUDED.series,
series_index = EXCLUDED.series_index, series_index = EXCLUDED.series_index,
series_suffix = EXCLUDED.series_suffix, series_suffix = EXCLUDED.series_suffix,
series_volume = EXCLUDED.series_volume,
publication_status = EXCLUDED.publication_status, publication_status = EXCLUDED.publication_status,
source_url = EXCLUDED.source_url, source_url = EXCLUDED.source_url,
publish_date = EXCLUDED.publish_date, publish_date = EXCLUDED.publish_date,
@ -1108,7 +890,6 @@ async def book_update(filename: str, request: Request):
series, series,
series_index if series else 0, series_index if series else 0,
series_suffix if series else "", series_suffix if series else "",
series_volume if series else "",
body.get("publication_status", ""), body.get("publication_status", ""),
body.get("source_url", ""), body.get("source_url", ""),
body.get("publish_date") or None, body.get("publish_date") or None,
@ -1147,12 +928,9 @@ async def book_update(filename: str, request: Request):
@router.post("/library/rating/{filename:path}") @router.post("/library/rating/{filename:path}")
async def set_rating(filename: str, request: Request): async def set_rating(filename: str, request: Request):
"""Set (or clear) a 1-5 star rating for a book. rating=0 removes it.""" """Set (or clear) a 1-5 star rating for a book. rating=0 removes it."""
if not is_db_filename(filename):
path = resolve_library_path(filename) path = resolve_library_path(filename)
if path is None or not path.exists(): if path is None or not path.exists():
return JSONResponse({"error": "not found"}, status_code=404) return JSONResponse({"error": "not found"}, status_code=404)
else:
path = None
body = await request.json() body = await request.json()
try: try:
@ -1160,7 +938,6 @@ async def set_rating(filename: str, request: Request):
except (TypeError, ValueError): except (TypeError, ValueError):
return JSONResponse({"error": "invalid rating"}, status_code=400) return JSONResponse({"error": "invalid rating"}, status_code=400)
if path is not None:
ext = path.suffix.lower() ext = path.suffix.lower()
if ext == ".epub": if ext == ".epub":
try: try:
@ -1184,316 +961,17 @@ async def set_rating(filename: str, request: Request):
return JSONResponse({"ok": True, "rating": rating}) return JSONResponse({"ok": True, "rating": rating})
# ---------------------------------------------------------------------------
# Fase 4 — EPUB → DB conversion
# ---------------------------------------------------------------------------
def _epub_body_inner(xhtml: str, z: zf.ZipFile, href: str) -> tuple[str, list[dict]]:
"""Parse an EPUB chapter XHTML, rewrite inline images to imagestore URLs.
Returns (inner_html_without_body_tags, []). Images are written to disk but
not registered in book_images here (that happens in the final DB transaction).
"""
soup = BeautifulSoup(xhtml, "lxml")
body = soup.find("body")
if not body:
return "", []
href_dir = href.rsplit("/", 1)[0] if "/" in href else ""
names = z.namelist()
for img in body.find_all("img"):
src = img.get("src", "")
if not src or src.startswith("http") or src.startswith("data:"):
continue
# Resolve relative path inside ZIP
parts = (href_dir.split("/") if href_dir else []) + src.split("/")
resolved: list[str] = []
for p in parts:
if p == "..":
if resolved:
resolved.pop()
elif p:
resolved.append(p)
zip_path = "/".join(resolved)
img_data: bytes | None = None
if zip_path in names:
img_data = z.read(zip_path)
else:
lo = zip_path.lower()
match = next((n for n in names if n.lower() == lo), None)
if match:
img_data = z.read(match)
if img_data:
ext_s = zip_path.rsplit(".", 1)[-1].lower() if "." in zip_path else "jpg"
mime = {"jpg": "image/jpeg", "jpeg": "image/jpeg", "png": "image/png",
"webp": "image/webp", "gif": "image/gif"}.get(ext_s, "image/jpeg")
_, _, url = write_image_file(img_data, mime)
img["src"] = url
else:
img.decompose()
# Strip leading heading — EPUB chapters often open with the chapter title as
# an <h1>/<h2>/<h3>. The chapter endpoint always prepends its own
# <h2 class="chapter-title">, so keep the stored content heading-free.
for child in list(body.children):
if getattr(child, "name", None) is None:
continue # NavigableString / text node — skip
if not child.get_text(strip=True):
child.decompose()
continue
if child.name in ("h1", "h2", "h3"):
child.decompose()
break
return body.decode_contents(), []
@router.post("/api/library/convert-to-db/{filename:path}")
async def convert_to_db(filename: str):
"""Convert a file-based EPUB to DB storage."""
if is_db_filename(filename):
return JSONResponse({"error": "Already a DB book"}, status_code=400)
old_path = resolve_library_path(filename)
if old_path is None or not old_path.exists():
return JSONResponse({"error": "File not found"}, status_code=404)
if old_path.suffix.lower() != ".epub":
return JSONResponse({"error": "Only EPUB files can be converted"}, status_code=400)
with get_db_conn() as conn:
with conn.cursor() as cur:
cur.execute(
"SELECT title, author, publisher, series, series_index, series_suffix "
"FROM library WHERE filename = %s",
(filename,),
)
row = cur.fetchone()
if not row:
return JSONResponse({"error": "Book not in library"}, status_code=404)
title, author, publisher, series, series_index, series_suffix = row
# Extract chapters from EPUB
try:
spine = _epub_spine(old_path)
chapters = []
with zf.ZipFile(old_path, "r") as z:
for entry in spine:
try:
xhtml = z.read(entry["href"]).decode("utf-8", errors="replace")
except KeyError:
continue
inner, _ = _epub_body_inner(xhtml, z, entry["href"])
if inner.strip():
chapters.append({"title": entry["title"], "content_html": inner})
except Exception as e:
return JSONResponse({"error": f"Failed to extract EPUB: {e}"}, status_code=500)
if not chapters:
return JSONResponse({"error": "No chapters found"}, status_code=400)
base_fn = make_rel_path(
media_type="db",
publisher=publisher or "",
author=author or "",
title=title or "",
series=series or "",
series_index=series_index or 0,
series_suffix=series_suffix or "",
).as_posix()
with get_db_conn() as conn:
with conn:
new_fn = ensure_unique_db_filename(conn, base_fn)
with conn.cursor() as cur:
# Insert new library row
cur.execute(
"""
INSERT INTO library (filename, media_type, storage_type, title, author, publisher,
has_cover, series, series_index, series_suffix, publication_status,
source_url, publish_date, description, archived, want_to_read,
needs_review, rating, created_at, updated_at)
SELECT %s, media_type, 'db', title, author, publisher,
has_cover, series, series_index, series_suffix, publication_status,
source_url, publish_date, description, archived, want_to_read,
needs_review, rating, created_at, NOW()
FROM library WHERE filename = %s
""",
(new_fn, filename),
)
# Migrate child tables
cur.execute("UPDATE book_tags SET filename = %s WHERE filename = %s", (new_fn, filename))
cur.execute("UPDATE reading_progress SET filename = %s WHERE filename = %s", (new_fn, filename))
cur.execute(
"INSERT INTO reading_sessions (filename, read_at) SELECT %s, read_at FROM reading_sessions WHERE filename = %s",
(new_fn, filename),
)
cur.execute("DELETE FROM reading_sessions WHERE filename = %s", (filename,))
cur.execute("UPDATE bookmarks SET filename = %s WHERE filename = %s", (new_fn, filename))
cur.execute(
"INSERT INTO library_cover_cache (filename, mime_type, thumb_webp, updated_at) "
"SELECT %s, mime_type, thumb_webp, updated_at FROM library_cover_cache WHERE filename = %s",
(new_fn, filename),
)
cur.execute("DELETE FROM library_cover_cache WHERE filename = %s", (filename,))
# Insert chapters
for idx, ch in enumerate(chapters):
upsert_chapter(conn, new_fn, idx, ch["title"], ch["content_html"])
with conn.cursor() as cur:
cur.execute("DELETE FROM library WHERE filename = %s", (filename,))
try:
old_path.unlink()
prune_empty_dirs(old_path.parent)
except Exception:
pass
return JSONResponse({"ok": True, "new_filename": new_fn})
# ---------------------------------------------------------------------------
# Fase 5 — DB → EPUB export
# ---------------------------------------------------------------------------
def _rewrite_db_images_for_epub(content_html: str, seen: dict[str, str]) -> tuple[str, list[dict]]:
"""Replace /library/db-images/... img src with EPUB-internal paths.
seen: sha256 epub_path (deduplication across chapters)
Returns (modified_html, new_image_dicts) where dicts have epub_path/data/media_type.
"""
soup = BeautifulSoup(content_html, "html.parser")
new_images: list[dict] = []
for img in soup.find_all("img"):
src = img.get("src", "")
if not src.startswith("/library/db-images/"):
continue
rel = src[len("/library/db-images/"):]
img_file = IMAGES_DIR / rel
if not img_file.exists():
img.decompose()
continue
sha256 = img_file.stem
ext = img_file.suffix.lower()
if sha256 not in seen:
epub_path = f"OEBPS/Images/{sha256}{ext}"
seen[sha256] = epub_path
mime = {".jpg": "image/jpeg", ".png": "image/png",
".webp": "image/webp", ".gif": "image/gif"}.get(ext, "image/jpeg")
new_images.append({"epub_path": epub_path, "data": img_file.read_bytes(), "media_type": mime})
img["src"] = f"../Images/{sha256}{ext}"
return str(soup), new_images
@router.get("/api/library/export-epub/{filename:path}")
async def export_epub(filename: str):
"""Export a DB-stored book as an EPUB download (no file written to disk)."""
if not is_db_filename(filename):
return JSONResponse({"error": "Not a DB book"}, status_code=400)
with get_db_conn() as conn:
with conn.cursor() as cur:
cur.execute(
"""SELECT title, author, publisher, series, series_index, publication_status,
source_url, description, publish_date
FROM library WHERE filename = %s""",
(filename,),
)
meta_row = cur.fetchone()
if not meta_row:
return JSONResponse({"error": "Not found"}, status_code=404)
cur.execute(
"SELECT tag, tag_type FROM book_tags WHERE filename = %s ORDER BY tag_type, tag",
(filename,),
)
tag_rows = cur.fetchall()
cur.execute(
"SELECT chapter_index, title, content FROM book_chapters "
"WHERE filename = %s ORDER BY chapter_index",
(filename,),
)
ch_rows = cur.fetchall()
cur.execute(
"SELECT thumb_webp FROM library_cover_cache WHERE filename = %s",
(filename,),
)
cover_row = cur.fetchone()
title, author, publisher, series, series_index, pub_status, source_url, description, pub_date = meta_row
cover_data: bytes | None = bytes(cover_row[0]) if cover_row and cover_row[0] else None
genres = [t for t, tp in tag_rows if tp == "genre"]
subgenres = [t for t, tp in tag_rows if tp == "subgenre"]
tags = [t for t, tp in tag_rows if tp in ("tag", "subject")]
book_info = {
"genres": genres, "subgenres": subgenres, "tags": tags,
"description": description or "",
"source_url": source_url or "",
"publisher": publisher or "",
"series": series or "",
"series_index": series_index or 1,
"publication_status": pub_status or "",
"updated_date": pub_date.isoformat() if pub_date else "",
}
seen_images: dict[str, str] = {}
chapters = []
for ch_idx, ch_title, ch_content in ch_rows:
# Strip leading h-tags from stored content (same logic as chapter endpoint)
# to prevent double titles when make_chapter_xhtml prepends its own heading.
ch_content = re.sub(r'(?si)^(\s*<h[1-4](?:\s[^>]*)?>.*?</h[1-4]>)+\s*', '', ch_content)
ch_content = re.sub(r'(?si)(<(?:section|div)[^>]*>\s*)<h[1-4][^>]*>.*?</h[1-4]>\s*', r'\1', ch_content, count=1)
modified_html, new_imgs = _rewrite_db_images_for_epub(ch_content, seen_images)
chapter_xhtml = make_chapter_xhtml(ch_title or f"Chapter {ch_idx + 1}", modified_html, ch_idx + 1)
chapters.append({"title": ch_title or f"Chapter {ch_idx + 1}", "xhtml": chapter_xhtml, "images": new_imgs})
try:
break_img_data = open("static/break.png", "rb").read()
except Exception:
break_img_data = b""
book_id = str(uuid.uuid4())
epub_bytes = make_epub(
title or "Untitled", author or "Unknown", chapters,
cover_data, break_img_data, book_id, book_info,
)
safe_title = re.sub(r'[^\w\-. ]', '', (title or "book")).strip() or "book"
return Response(
content=epub_bytes,
media_type="application/epub+zip",
headers={"Content-Disposition": f'attachment; filename="{safe_title}.epub"'},
)
@router.get("/library/read/{filename:path}", response_class=HTMLResponse) @router.get("/library/read/{filename:path}", response_class=HTMLResponse)
async def reader_page(filename: str, request: Request): async def reader_page(filename: str, request: Request):
with get_db_conn() as conn:
with conn.cursor() as cur:
cur.execute("SELECT title FROM library WHERE filename = %s", (filename,))
row = cur.fetchone()
if is_db_filename(filename):
if not row:
return HTMLResponse("Not found", status_code=404)
title = row[0] if row[0] else filename
return templates.TemplateResponse(request, "reader.html", {
"filename": filename,
"title": title,
"format": "epub",
"epub_url": "",
})
path = resolve_library_path(filename) path = resolve_library_path(filename)
if path is None: if path is None:
return HTMLResponse("Not found", status_code=404) return HTMLResponse("Not found", status_code=404)
if not path.exists(): if not path.exists():
return HTMLResponse("Not found", status_code=404) return HTMLResponse("Not found", status_code=404)
with get_db_conn() as conn:
with conn.cursor() as cur:
cur.execute("SELECT title FROM library WHERE filename = %s", (filename,))
row = cur.fetchone()
title = row[0] if row and row[0] else filename title = row[0] if row and row[0] else filename
fmt = path.suffix.lower().lstrip(".") fmt = path.suffix.lower().lstrip(".")
return templates.TemplateResponse(request, "reader.html", { return templates.TemplateResponse(request, "reader.html", {

View File

@ -1,88 +0,0 @@
"""search.py — Full-text search over DB-stored book chapters."""
from fastapi import APIRouter, Request
from fastapi.responses import HTMLResponse, JSONResponse
from shared_templates import templates
from db import get_db_conn
router = APIRouter()
@router.get("/search", response_class=HTMLResponse)
async def search_page(request: Request):
return templates.TemplateResponse(request, "search.html", {"active": "search"})
@router.get("/api/search")
async def api_search(q: str = "", mode: str = "phrase", filter: str = "all"):
q = q.strip()
if not q or len(q) > 500:
return JSONResponse([])
tsquery_fn = "phraseto_tsquery" if mode == "phrase" else "plainto_tsquery"
extra_joins = ""
extra_where = ""
if filter in ("unread_novels", "unread_shorts"):
extra_joins = """
LEFT JOIN reading_sessions rs ON rs.filename = l.filename
LEFT JOIN reading_progress rp ON rp.filename = l.filename"""
extra_where = " AND rs.id IS NULL AND COALESCE(rp.progress, 0) = 0"
if filter == "unread_novels":
extra_where += """
AND NOT EXISTS (
SELECT 1 FROM book_tags bt
WHERE bt.filename = l.filename AND bt.tag = 'Shorts'
AND bt.tag_type IN ('tag', 'subject')
)"""
elif filter == "unread_shorts":
extra_where += """
AND EXISTS (
SELECT 1 FROM book_tags bt
WHERE bt.filename = l.filename AND bt.tag = 'Shorts'
AND bt.tag_type IN ('tag', 'subject')
)"""
with get_db_conn() as conn:
with conn.cursor() as cur:
cur.execute(
f"""
SELECT
l.filename,
l.title,
l.author,
bc.chapter_index,
bc.title AS chapter_title,
ts_headline(
'simple', bc.content,
plainto_tsquery('simple', %s),
'MaxFragments=1, MaxWords=25, MinWords=8, StartSel=<mark>, StopSel=</mark>'
) AS snippet,
ts_rank(bc.content_tsv, plainto_tsquery('simple', %s)) AS rank
FROM book_chapters bc
JOIN library l ON l.filename = bc.filename
{extra_joins}
WHERE (bc.content_tsv @@ {tsquery_fn}('simple', %s)
OR LOWER(bc.title) LIKE LOWER('%%' || %s || '%%'))
AND NOT l.archived
{extra_where}
GROUP BY l.filename, l.title, l.author, bc.chapter_index, bc.title, bc.content, bc.content_tsv
ORDER BY rank DESC, bc.chapter_index ASC
""",
(q, q, q, q),
)
rows = cur.fetchall()
results = [
{
"filename": r[0],
"title": r[1] or "",
"author": r[2] or "",
"chapter_index": r[3],
"chapter_title": r[4] or "",
"snippet": r[5] or "",
"rank": float(r[6]),
}
for r in rows
]
return JSONResponse(results)

View File

@ -1,13 +1,12 @@
import re import re
from fastapi import APIRouter, Request, UploadFile, File from fastapi import APIRouter, Request
from fastapi.responses import HTMLResponse from fastapi.responses import HTMLResponse
from shared_templates import templates from fastapi.templating import Jinja2Templates
from db import get_db_conn from db import get_db_conn
from epub import detect_image_format
from routers.common import write_image_file
templates = Jinja2Templates(directory="templates")
router = APIRouter() router = APIRouter()
@ -96,57 +95,6 @@ async def delete_break_pattern(pid: int):
return {"ok": True} return {"ok": True}
@router.get("/api/app-settings")
async def get_app_settings():
with get_db_conn() as conn:
with conn.cursor() as cur:
cur.execute("SELECT develop_mode, break_image_sha256, break_image_ext FROM app_settings WHERE id = 1")
row = cur.fetchone()
if not row:
return {"develop_mode": False, "break_image_url": None}
sha, ext = row[1], row[2]
break_image_url = f"/library/db-images/{sha[:2]}/{sha}{ext}" if sha and ext else None
return {"develop_mode": bool(row[0]), "break_image_url": break_image_url}
@router.patch("/api/app-settings")
async def update_app_settings(request: Request):
body = await request.json()
with get_db_conn() as conn:
with conn:
with conn.cursor() as cur:
if "develop_mode" in body:
cur.execute(
"UPDATE app_settings SET develop_mode = %s WHERE id = 1",
(bool(body["develop_mode"]),),
)
return {"ok": True}
@router.post("/api/app-settings/break-image")
async def upload_break_image(file: UploadFile = File(...)):
data = await file.read()
if not data:
return {"error": "Empty file"}
_, media_type = detect_image_format(data, file.filename or "break")
sha, ext, _ = write_image_file(data, media_type)
# Also write to static/break.png so EPUB embeds the same image
try:
with open("static/break.png", "wb") as f:
f.write(data)
except Exception:
pass
with get_db_conn() as conn:
with conn:
with conn.cursor() as cur:
cur.execute(
"UPDATE app_settings SET break_image_sha256 = %s, break_image_ext = %s WHERE id = 1",
(sha, ext),
)
url = f"/library/db-images/{sha[:2]}/{sha}{ext}"
return {"ok": True, "url": url}
@router.delete("/api/reading-history") @router.delete("/api/reading-history")
async def reset_reading_history(): async def reset_reading_history():
with get_db_conn() as conn: with get_db_conn() as conn:

View File

@ -1,23 +1,11 @@
from .base import BaseScraper from .base import BaseScraper
from .archiveofourown import ArchiveOfOurOwnScraper
from .awesomedude import AwesomeDudeScraper from .awesomedude import AwesomeDudeScraper
from .codeysworld import CodeysWorldScraper
from .gayauthors import GayAuthorsScraper from .gayauthors import GayAuthorsScraper
from .iomfats import IomfatsScraper
from .nifty import NiftyScraper
from .nifty_new import NiftyNewScraper
from .tedlouis import TedLouisScraper
# Register scrapers in priority order (first match wins) # Register scrapers in priority order (first match wins)
_SCRAPERS: list[type[BaseScraper]] = [ _SCRAPERS: list[type[BaseScraper]] = [
ArchiveOfOurOwnScraper,
AwesomeDudeScraper, AwesomeDudeScraper,
CodeysWorldScraper,
GayAuthorsScraper, GayAuthorsScraper,
IomfatsScraper,
NiftyNewScraper,
NiftyScraper,
TedLouisScraper,
] ]

View File

@ -1,206 +0,0 @@
import re
from urllib.parse import urljoin
import httpx
from bs4 import BeautifulSoup
from .base import BaseScraper
AO3_BASE = "https://archiveofourown.org"
class ArchiveOfOurOwnScraper(BaseScraper):
@classmethod
def matches(cls, url: str) -> bool:
return "archiveofourown.org" in url
def _work_base_url(self, url: str) -> str:
"""Strip chapter segment and query string; return /works/NNNNNN base URL."""
m = re.search(r"(https?://[^/]+/works/\d+)", url)
return m.group(1) if m else url.rstrip("/")
async def login(self, client: httpx.AsyncClient, username: str, password: str) -> bool:
r = await client.get(AO3_BASE + "/users/login")
soup = BeautifulSoup(r.text, "html.parser")
token_el = soup.find("input", {"name": "authenticity_token"})
token = token_el["value"] if token_el else ""
resp = await client.post(
AO3_BASE + "/users/login",
data={
"user[login]": username,
"user[password]": password,
"authenticity_token": token,
"commit": "Log in",
},
)
# Successful login redirects away from the login page
return "/users/login" not in str(resp.url)
async def fetch_book_info(self, client: httpx.AsyncClient, url: str) -> dict:
base_url = self._work_base_url(url)
r = await client.get(base_url, params={"view_adult": "true"})
soup = BeautifulSoup(r.text, "html.parser")
# Title
title_el = soup.find("h2", class_="title")
book_title = title_el.get_text(strip=True) if title_el else "Unknown title"
# Author — can be multiple; Anonymous if no author link
byline = soup.find("h3", class_="byline")
if byline:
author_links = byline.find_all("a", rel="author")
author = ", ".join(a.get_text(strip=True) for a in author_links) if author_links else "Anonymous"
else:
author = "Anonymous"
# Tags from dl.work.meta.group
meta_dl = soup.find("dl", class_="work")
def _tag_list(dl, css_class: str) -> list[str]:
dd = dl.find("dd", class_=css_class) if dl else None
return [a.get_text(strip=True) for a in dd.find_all("a")] if dd else []
fandoms = _tag_list(meta_dl, "fandom")
ratings = _tag_list(meta_dl, "rating")
categories = _tag_list(meta_dl, "category")
relationships = _tag_list(meta_dl, "relationship")
characters = _tag_list(meta_dl, "character")
freeform_tags = _tag_list(meta_dl, "freeform")
# Series
series = ""
series_index_hint = 0
if meta_dl:
series_dd = meta_dl.find("dd", class_="series")
if series_dd:
series_link = series_dd.find("a")
if series_link:
series = series_link.get_text(strip=True)
pos_span = series_dd.find("span", class_="position")
if pos_span:
m = re.search(r"Part\s+(\d+)", pos_span.get_text(), re.I)
if m:
series_index_hint = int(m.group(1))
# Stats (nested dl.stats inside the meta dl)
published = ""
updated_date = ""
publication_status = ""
if meta_dl:
stats_dl = meta_dl.find("dl", class_="stats")
if stats_dl:
pub_dd = stats_dl.find("dd", class_="published")
if pub_dd:
published = pub_dd.get_text(strip=True)
status_dt = stats_dl.find("dt", class_="status")
status_dd = stats_dl.find("dd", class_="status")
if status_dt and status_dd:
updated_date = status_dd.get_text(strip=True)
if "Completed" in status_dt.get_text():
publication_status = "Complete"
else:
publication_status = "Ongoing"
else:
# No status entry — determine from chapters count (N/N = complete)
updated_date = published
chapters_dd = stats_dl.find("dd", class_="chapters")
if chapters_dd:
m = re.match(r"(\d+)/(\d+|\?)", chapters_dd.get_text(strip=True))
if m:
if m.group(2) == "?":
publication_status = "Ongoing"
elif m.group(1) == m.group(2):
publication_status = "Complete"
# Summary
description = ""
summary_div = soup.find("div", class_="summary")
if summary_div:
userstuff = summary_div.find("blockquote", class_="userstuff")
if userstuff:
paras = [p.get_text(strip=True) for p in userstuff.find_all("p") if p.get_text().strip()]
description = "\n\n".join(paras) if paras else userstuff.get_text(strip=True)
# Chapter list via /navigate
chapter_links = []
chapter_method = "html_scan"
try:
nr = await client.get(base_url + "/navigate", params={"view_adult": "true"})
nsoup = BeautifulSoup(nr.text, "html.parser")
chapter_ol = nsoup.find("ol", class_="chapter")
if chapter_ol:
for li in chapter_ol.find_all("li"):
a = li.find("a", href=True)
if a:
chapter_links.append({
"url": urljoin(AO3_BASE, a["href"]),
"title": a.get_text(strip=True),
})
except Exception:
pass
# Fallback: single-chapter work — the work page itself is the content
if not chapter_links:
chapter_method = "fallback_numeric"
chapter_links.append({"url": base_url, "title": book_title})
# Map to Novela tag structure:
# fandoms → genres
# ratings + categories → subgenres
# relationships + characters + freeform → tags
return {
"title": book_title,
"author": author,
"publisher": "Archive of Our Own",
"series": series,
"series_index_hint": series_index_hint,
"genres": fandoms,
"subgenres": ratings + categories,
"tags": relationships + characters + freeform_tags,
"description": description,
"updated_date": updated_date,
"publication_status": publication_status,
"source_url": base_url,
"chapters": chapter_links,
"chapter_method": chapter_method,
}
async def fetch_chapter(self, client: httpx.AsyncClient, ch: dict) -> dict:
r = await client.get(ch["url"], params={"view_adult": "true"})
soup = BeautifulSoup(r.text, "html.parser")
# Chapter title and optional summary from the chapter preface
title = ch["title"]
chapter_summary_bq = None
chapters_div = soup.find("div", id="chapters")
if chapters_div:
chapter_div = chapters_div.find("div", class_="chapter")
if chapter_div:
title_el = chapter_div.find("h3", class_="title")
if title_el:
raw = title_el.get_text(strip=True)
if raw:
title = raw
summary_div = chapter_div.find("div", class_="summary")
if summary_div:
chapter_summary_bq = summary_div.find("blockquote", class_="userstuff")
# Content: div.userstuff inside #chapters (excludes author notes)
content_el = None
if chapters_div:
content_el = chapters_div.find("div", class_="userstuff")
if not content_el:
content_el = soup.find("div", attrs={"role": "article"})
# Prepend chapter summary as blockquote before story content
if chapter_summary_bq and content_el:
content_el.insert(0, chapter_summary_bq)
return {
"title": title,
"content_el": content_el,
"selector_id": content_el.get("id") if content_el else None,
"selector_class": " ".join(content_el.get("class", [])) if content_el else None,
}

View File

@ -1,208 +0,0 @@
import re
from urllib.parse import urljoin, urlparse
import httpx
from bs4 import BeautifulSoup
from .base import BaseScraper
CW_BASE = "https://www.codeysworld.org"
LAYOUT_RE = re.compile(
r"nav|menu|sidebar|header|footer|breadcrumb|pagination|"
r"comment|widget|aside|banner|ad|rating|follow|share",
re.I,
)
class CodeysWorldScraper(BaseScraper):
@classmethod
def matches(cls, url: str) -> bool:
return "codeysworld.org" in url
async def login(self, client: httpx.AsyncClient, username: str, password: str) -> bool:
return True # no login required
async def fetch_book_info(self, client: httpx.AsyncClient, url: str) -> dict:
r = await client.get(url)
soup = BeautifulSoup(r.text, "html.parser")
actual_url = str(r.url)
# Title: <h1>
h1 = soup.find("h1")
book_title = h1.get_text(strip=True) if h1 else "Unknown title"
# Author: <h2> "by Author Name"
author = "Unknown author"
h2 = soup.find("h2")
if h2:
text = h2.get_text(strip=True)
m = re.match(r"^by\s+(.+)$", text, re.I)
if m:
author = m.group(1).strip()
# URL path: /{author_slug}/{category}/filename.htm
tags: list[str] = []
path_parts = urlparse(actual_url).path.strip("/").split("/")
if len(path_parts) >= 3:
author_slug = path_parts[-3]
category = path_parts[-2]
elif len(path_parts) >= 2:
author_slug = path_parts[-2]
category = ""
else:
author_slug = ""
category = ""
# Fallback: derive author from URL slug if not found in page
if author == "Unknown author" and author_slug:
author = author_slug.replace("_", " ").title()
# Category → tag
if category and category.lower() not in ("codey", author_slug.lower()):
tags = [category.replace("_", " ").title()]
# Chapter discovery: links to .htm/.html in the same directory,
# excluding the index page itself and audio/image files.
base_dir = actual_url.rsplit("/", 1)[0] + "/"
chapter_links: list[dict] = []
seen: set[str] = set()
for a in soup.find_all("a", href=True):
href = a["href"]
if re.search(r"\.(mp3|mp4|ogg|wav|jpg|jpeg|png|gif)$", href, re.I):
continue
full = urljoin(actual_url, href)
if (
full.startswith(base_dir)
and re.search(r"\.html?(\?.*)?$", full, re.I)
and full.rstrip("/") != actual_url.rstrip("/")
and full not in seen
):
seen.add(full)
text = re.sub(r"\s+", " ", a.get_text(separator=" ")).strip()
chapter_links.append({"url": full, "title": text, "book_title": book_title, "author": author})
if not chapter_links:
# Single-file story
chapter_links = [{"url": actual_url, "title": book_title, "book_title": book_title, "author": author}]
chapter_method = "single_page"
else:
chapter_method = "html_scan"
for i, c in enumerate(chapter_links, 1):
t = c["title"]
if not t or t.lower() == book_title.lower():
c["title"] = f"Chapter {i}"
elif re.match(r"^\d+$", t):
c["title"] = f"Chapter {t}"
return {
"title": book_title,
"author": author,
"publisher": "codeysworld.org",
"series": "",
"series_index_hint": 0,
"genres": [],
"subgenres": [],
"tags": tags,
"description": "",
"updated_date": "",
"publication_status": "",
"source_url": url,
"chapters": chapter_links,
"chapter_method": chapter_method,
"index_image_url": None,
}
async def fetch_chapter(self, client: httpx.AsyncClient, ch: dict) -> dict:
cr = await client.get(ch["url"])
csoup = BeautifulSoup(cr.text, "html.parser")
title = ch["title"]
book_title_lc = ch.get("book_title", "").lower()
author_lc = ch.get("author", "").lower()
# Refine chapter title from an in-page heading,
# skipping the book title and "by Author" headings.
for tag in csoup.find_all(["h1", "h2", "h3"]):
text = re.sub(r"\s+", " ", tag.get_text(separator=" ")).strip()
if not text or len(text) >= 120:
continue
text_lc = text.lower()
if re.search(r"\s+by\s+", text, re.I):
continue
if book_title_lc and book_title_lc in text_lc:
continue
if author_lc and author_lc in text_lc:
continue
title = text
break
# Content extraction: prefer a content-like wrapper; fall back to body.
content_el = (
csoup.find(id=re.compile(r"^(chapter|story|content|text)[_-]?", re.I))
or csoup.find(class_=re.compile(r"story.?text|chapter.?text|post.?content|entry.?content", re.I))
or csoup.find("article")
)
if not content_el:
candidates = [
el for el in csoup.find_all(["div", "article", "section"])
if not re.search(LAYOUT_RE, " ".join(el.get("class", [])))
and not re.search(LAYOUT_RE, el.get("id", ""))
]
if candidates:
content_el = max(candidates, key=lambda el: len(el.get_text(" ", strip=True)))
body = csoup.find("body")
if body:
body_text_len = len(body.get_text(" ", strip=True))
selected_p_count = len(content_el.find_all("p")) if content_el else 0
selected_text_len = len(content_el.get_text(" ", strip=True)) if content_el else 0
# Codeysworld stores story text as direct <p> children of body.
if not content_el or selected_p_count < 3 or selected_text_len < int(body_text_len * 0.35):
content_el = body
if not content_el:
content_el = body
# Strip site boilerplate: headings (title/byline), navigation links,
# audio links and empty nodes — anywhere in the content element.
if content_el:
# Remove all h1/h2 headings (title and "by Author")
for el in content_el.find_all(["h1", "h2"]):
el.decompose()
# Remove navigation links ("Back to …", "Home", etc.)
for el in content_el.find_all("a", href=True):
text = el.get_text(strip=True)
if re.search(r"back\s+to|<{0,2}\s*back|home", text, re.I):
parent = el.parent
el.decompose()
# Remove the parent too if it's now empty
if parent and not parent.get_text(strip=True):
parent.decompose()
# Remove audio links (links to .mp3 files or containing "listen"/"audio")
for el in content_el.find_all("a", href=True):
href = el.get("href", "")
text = el.get_text(strip=True)
if re.search(r"\.mp3$", href, re.I) or re.search(r"listen|audio", text, re.I):
parent = el.parent
el.decompose()
if parent and not parent.get_text(strip=True):
parent.decompose()
# Remove email links ("Email Author")
for el in content_el.find_all("a", href=re.compile(r"^mailto:", re.I)):
parent = el.parent
el.decompose()
if parent and not parent.get_text(strip=True):
parent.decompose()
return {
"title": title,
"content_el": content_el,
"selector_id": content_el.get("id") if content_el else None,
"selector_class": " ".join(content_el.get("class", [])) if content_el else None,
}

View File

@ -1,267 +0,0 @@
import re
from urllib.parse import urljoin, urlparse
import httpx
from bs4 import BeautifulSoup, NavigableString
from .base import BaseScraper
IOMFATS_BASE = "https://iomfats.org"
class IomfatsScraper(BaseScraper):
@classmethod
def matches(cls, url: str) -> bool:
return "iomfats.org" in url
async def login(self, client: httpx.AsyncClient, username: str, password: str) -> bool:
return True # no login required
def _author_page_url(self, url: str) -> str:
"""Derive the author index page URL from any iomfats.org URL."""
parsed = urlparse(url)
parts = parsed.path.strip("/").split("/")
# Path: storyshelf/hosted/{author}/...
# Author page is the first 3 segments.
if len(parts) >= 3 and parts[0] == "storyshelf" and parts[1] == "hosted":
author_path = "/" + "/".join(parts[:3]) + "/"
return f"{parsed.scheme}://{parsed.netloc}{author_path}"
return url
def _is_author_page(self, url: str) -> bool:
parts = urlparse(url).path.strip("/").split("/")
return (
len(parts) <= 3
and len(parts) >= 2
and parts[0] == "storyshelf"
and parts[1] == "hosted"
)
def _story_folder(self, url: str) -> str | None:
"""Return the story folder segment from a chapter URL, or None."""
parts = urlparse(url).path.strip("/").split("/")
# storyshelf/hosted/{author}/{story-folder}/{chapter}.html
if len(parts) >= 5:
return parts[3]
return None
async def fetch_book_info(self, client: httpx.AsyncClient, url: str) -> dict:
if self._is_author_page(url):
raise ValueError(
"Voer een chapter-URL in, geen author-pagina. "
"Kopieer de URL van het eerste hoofdstuk van het gewenste verhaal."
)
story_folder = self._story_folder(url)
if not story_folder:
raise ValueError(
"Onverwacht URL-formaat voor iomfats.org. "
"Gebruik de URL van een hoofdstuk, bijv. …/grasshopper/dreamchasers/01.html"
)
author_url = self._author_page_url(url)
r = await client.get(author_url)
soup = BeautifulSoup(r.text, "html.parser")
content = soup.find("div", id="content")
if not content:
raise ValueError("Kan de author-pagina niet verwerken (geen #content element).")
# Author name from "by <a>Name</a>" heading
author = "Unknown author"
for el in content.find_all(["h2", "h3"]):
text = el.get_text(strip=True)
m = re.match(r"^by\s+(.+)$", text, re.I)
if m:
author = m.group(1).strip()
break
# Fallback: author slug from URL
if author == "Unknown author":
parts = urlparse(author_url).path.strip("/").split("/")
if len(parts) >= 3:
author = parts[2].replace("_", " ").title()
# Walk the content to find the story matching story_folder.
#
# Two structures on the author page:
#
# Single story:
# <h3>Book Title</h3>
# <ul><li><a href="folder/ch01.html">Chapter 1</a></li>…</ul>
#
# Multi-part series:
# <h3>Series Name</h3>
# <ul>
# <li><h3>Book Title (part 1)</h3>
# <p><small>[status]</small></p>
# <ul><li><a href="folder-part1/ch01.html">…</a></li>…</ul>
# </li>
# <li><h3>Book Title (part 2)</h3>…</li>
# </ul>
book_title = ""
series = ""
series_index_hint = 0
publication_status = ""
chapter_links: list[dict] = []
nodes = list(content.children)
i = 0
while i < len(nodes):
node = nodes[i]
if not hasattr(node, "name"):
i += 1
continue
if node.name == "h3":
outer_title = node.get_text(strip=True)
if re.match(r"^by\s+", outer_title, re.I):
i += 1
continue
# Find the following <ul>
j = i + 1
outer_ul = None
outer_status = ""
while j < len(nodes):
n = nodes[j]
if not hasattr(n, "name"):
j += 1
continue
if n.name == "h3":
break
if n.name == "p":
small = n.find("small")
if small:
outer_status = small.get_text(strip=True).strip("[]")
j += 1
continue
if n.name == "ul":
outer_ul = n
break
j += 1
if not outer_ul:
i += 1
continue
# Check if this <ul> has sub-section <li><h3> entries (multi-part series)
sub_sections = [
li for li in outer_ul.find_all("li", recursive=False)
if li.find("h3")
]
if sub_sections:
# Multi-part series: outer_title = series name
for li in sub_sections:
sub_h3 = li.find("h3")
sub_ul = li.find("ul")
if not sub_h3 or not sub_ul:
continue
chapters = self._collect_chapters(sub_ul, story_folder, author_url)
if chapters:
book_title = sub_h3.get_text(strip=True)
series = outer_title
chapter_links = chapters
# Status from <p><small> inside this <li>
small = li.find("small")
if small:
publication_status = small.get_text(strip=True).strip("[]")
break
else:
# Single story: outer_title = book title
chapters = self._collect_chapters(outer_ul, story_folder, author_url)
if chapters:
book_title = outer_title
publication_status = outer_status
chapter_links = chapters
if chapter_links:
break
i += 1
if not chapter_links:
raise ValueError(
f"Geen hoofdstukken gevonden voor folder '{story_folder}'. "
"Controleer de URL."
)
# Series index from folder name suffix: *-part{N} or *-{N}
m = re.search(r"-part(\d+)$", story_folder, re.I)
if not m:
m = re.search(r"-(\d+)$", story_folder)
if m:
series_index_hint = int(m.group(1))
return {
"title": book_title or story_folder.replace("-", " ").title(),
"author": author,
"publisher": "iomfats.org",
"series": series,
"series_index_hint": series_index_hint,
"genres": [],
"subgenres": [],
"tags": [],
"description": "",
"updated_date": "",
"publication_status": publication_status,
"source_url": url,
"chapters": chapter_links,
"chapter_method": "html_scan",
"index_image_url": None,
}
def _collect_chapters(self, ul, story_folder: str, base_url: str) -> list[dict]:
"""Collect chapter links from a flat <ul>, filtered by story_folder."""
out: list[dict] = []
for li in ul.find_all("li", recursive=False):
a = li.find("a", href=True)
if not a:
continue
full_url = urljoin(base_url, a["href"])
if story_folder not in urlparse(full_url).path:
continue
raw_title = a.get_text(strip=True)
title = f"Chapter {raw_title}" if re.match(r"^\d+$", raw_title) else raw_title
out.append({"url": full_url, "title": title})
return out
async def fetch_chapter(self, client: httpx.AsyncClient, ch: dict) -> dict:
cr = await client.get(ch["url"])
csoup = BeautifulSoup(cr.text, "html.parser")
title = ch["title"]
content_el = csoup.find("div", id="content")
if not content_el:
content_el = csoup.find("body")
if content_el:
# Remove headings (story title, author, chapter number)
for el in content_el.find_all(["h2", "h3"]):
el.decompose()
# Remove chapter navigation divs
for el in content_el.find_all("div", class_=re.compile(r"chapternav", re.I)):
el.decompose()
# Remove footer elements (author note, forum button)
for el in content_el.find_all("div", class_="important"):
el.decompose()
for el in content_el.find_all("a", class_="styled-button"):
parent = el.parent
el.decompose()
if parent and not parent.get_text(strip=True):
parent.decompose()
# Remove anchor tags used as page anchors (<a name="content">)
for el in content_el.find_all("a", attrs={"name": True}):
if not el.get("href"):
el.unwrap()
return {
"title": title,
"content_el": content_el,
"selector_id": "content",
"selector_class": None,
}

View File

@ -1,358 +0,0 @@
import re
from email.utils import parsedate
from html import escape as he
from time import mktime
from datetime import date as _date
from urllib.parse import urljoin, urlparse
import httpx
from bs4 import BeautifulSoup
from .base import BaseScraper
# Email header field names that appear at the top of Nifty classic chapters.
_HEADER_RE = re.compile(
r"^(Date|From|Subject|Reply-To|Message-ID|MIME-Version|Content-Type|X-[\w-]+):",
re.I,
)
# Scene-break patterns in plain text (subset of xhtml.BREAK_PATTERNS for text matching).
_BREAK_RE = re.compile(
r"^("
r"[\*\-]{3,}"
r"|[~=]{3,}"
r"|#{3,}"
r"|[·•◦‣⁃]\s*[·•◦‣⁃]\s*[·•◦‣⁃]"
r"|[-–—]\s*[oO0]\s*[-–—]"
r")$"
)
class NiftyScraper(BaseScraper):
_LEAD_MARKERS = (
"notice this is a work of fiction",
"if it is illegal to read stories",
"if you enjoy this story",
"for my other stories",
"nifty archive",
"code of conduct",
"author note",
"author's note",
"disclaimer",
"this story contains",
"this story includes",
"all characters are",
"all characters depicted",
)
_TAIL_MARKERS = (
"please remember to donate",
"donate",
"support nifty",
"support the archive",
"nifty archive alliance",
"donate.nifty.org",
"nifty.org/donate",
"nifty.org/support",
"patreon",
"buy me a coffee",
"tip jar",
"become a supporter",
)
@classmethod
def matches(cls, url: str) -> bool:
return "nifty.org" in url and "new.nifty.org" not in url
async def login(self, client: httpx.AsyncClient, username: str, password: str) -> bool:
return True # no login required
# ── Helpers ───────────────────────────────────────────────────────────────
def _to_index_url(self, url: str) -> str:
"""Return the story index URL for any Nifty URL (index or chapter).
Nifty path structure:
/nifty/{category}/{subcategory}/{story}/ index (4 segments)
/nifty/{category}/{subcategory}/{story}/{chapter} chapter (5 segments)
"""
parsed = urlparse(url)
parts = [p for p in parsed.path.split("/") if p]
if len(parts) >= 5:
path = "/" + "/".join(parts[:4]) + "/"
else:
path = parsed.path.rstrip("/") + "/"
return f"{parsed.scheme}://{parsed.netloc}{path}"
def _slug_to_title(self, slug: str) -> str:
return slug.replace("-", " ").title()
def _parse_date_header(self, text: str) -> str:
"""Return YYYY-MM-DD from a 'Date: …' line, or ''."""
m = re.search(r"^Date:\s+(.+)$", text, re.M)
if not m:
return ""
try:
parsed = parsedate(m.group(1).strip())
if parsed:
return _date.fromtimestamp(mktime(parsed)).isoformat()
except Exception:
pass
return ""
def _parse_author_header(self, text: str) -> str:
"""Return author name from 'From: Name <email>' line, or ''."""
m = re.search(r"^From:\s+([^<\n]+?)(?:\s*<[^>]+>)?\s*$", text, re.M)
return m.group(1).strip() if m else ""
def _parse_subject_header(self, text: str) -> str:
"""Return the Subject header value, or ''."""
m = re.search(r"^Subject:\s+(.+)$", text, re.M)
return m.group(1).strip() if m else ""
def _normalize(self, text: str) -> str:
"""Normalise text for boilerplate comparison (lowercase, collapsed whitespace)."""
return re.sub(r"\s+", " ", text.lower()).strip()
async def _get_text(self, client: httpx.AsyncClient, url: str) -> tuple[BeautifulSoup, str]:
"""Fetch *url* and return (soup, raw_text).
Nifty classic pages wrap the story content in a <pre> element.
Falls back to the full body text if no <pre> is found.
"""
r = await client.get(url)
soup = BeautifulSoup(r.text, "html.parser")
pre = soup.find("pre")
if pre:
raw = pre.get_text()
else:
body = soup.find("body")
raw = body.get_text("\n") if body else soup.get_text("\n")
return soup, raw
def _strip_email_headers(self, text: str) -> str:
"""Remove the leading email header block (Date/From/Subject/…) from chapter text.
Tolerates blank lines between header fields some Nifty pages place the
Subject on a separate line after a blank line:
Date:
From:
Subject:
"""
lines = text.splitlines()
i = 0
# Skip leading blank lines.
while i < len(lines) and not lines[i].strip():
i += 1
# Only strip if this actually looks like an email header block.
if not any(_HEADER_RE.match(lines[j]) for j in range(i, min(i + 12, len(lines)))):
return text
# Skip header lines, tolerating blank lines between them.
# A blank line ends the block only when no further header line follows.
while i < len(lines):
stripped = lines[i].strip()
if _HEADER_RE.match(stripped):
i += 1
elif not stripped:
# Peek ahead past any blank lines.
j = i + 1
while j < len(lines) and not lines[j].strip():
j += 1
if j < len(lines) and _HEADER_RE.match(lines[j].strip()):
i = j # more headers follow — jump over the blank line(s)
else:
i += 1
break # no more headers — end of block
else:
break # non-header, non-blank line — end of block
# Skip blank lines immediately after the header block.
while i < len(lines) and not lines[i].strip():
i += 1
return "\n".join(lines[i:])
def _text_to_paragraphs(self, text: str) -> list[str]:
"""Split plain text into paragraphs; join hard-wrapped lines within each paragraph.
Nifty classic stories are stored as email submissions: paragraphs are
separated by blank lines, and each line is wrapped at ~70 characters.
This function merges those wrapped lines back into a single line per
paragraph.
"""
text = text.replace("\r\n", "\n").replace("\r", "\n")
blocks = re.split(r"\n{2,}", text)
result = []
for block in blocks:
lines = [l.strip() for l in block.splitlines() if l.strip()]
if lines:
result.append(" ".join(lines))
return result
def _comment_safe(self, text: str) -> str:
return text.replace("--", "- -")
def _plain_text(self, text: str) -> str:
if "<" in text and ">" in text:
return BeautifulSoup(text, "html.parser").get_text(" ", strip=True)
return text
def _looks_like_lead_boilerplate(self, text: str) -> bool:
t = re.sub(r"\s+", " ", self._plain_text(text).lower()).strip()
if not t or len(t) > 4000:
return False
return any(m in t for m in self._LEAD_MARKERS)
def _looks_like_tail_boilerplate(self, text: str) -> bool:
t = re.sub(r"\s+", " ", self._plain_text(text).lower()).strip()
if not t or len(t) > 4000:
return False
return any(m in t for m in self._TAIL_MARKERS)
def _extract_hidden_boilerplate(self, paragraphs: list[str]) -> tuple[list[str], list[str], list[str]]:
visible = list(paragraphs)
leading: list[str] = []
trailing: list[str] = []
while visible and len(leading) < 6 and self._looks_like_lead_boilerplate(visible[0]):
leading.append(visible.pop(0))
while visible and len(trailing) < 6 and self._looks_like_tail_boilerplate(visible[-1]):
trailing.insert(0, visible.pop())
if not visible:
return list(paragraphs), [], []
return visible, leading, trailing
# ── BaseScraper interface ─────────────────────────────────────────────────
async def fetch_book_info(self, client: httpx.AsyncClient, url: str) -> dict:
index_url = self._to_index_url(url)
r = await client.get(index_url)
soup = BeautifulSoup(r.text, "html.parser")
# Title from URL slug.
slug = urlparse(index_url).path.rstrip("/").rsplit("/", 1)[-1]
book_title = self._slug_to_title(slug)
# Genres from URL path: /nifty/{category}/{subcategory}/{story}/
path_parts = [p for p in urlparse(index_url).path.split("/") if p]
category = self._slug_to_title(path_parts[1]) if len(path_parts) > 1 else ""
subcategory = self._slug_to_title(path_parts[2]) if len(path_parts) > 2 else ""
# Chapter links: all <a> tags pointing one level deeper than the index.
chapter_links: list[dict] = []
seen: set[str] = set()
for a in soup.find_all("a", href=True):
full = urljoin(index_url, a["href"])
if (
full.startswith(index_url)
and full.rstrip("/") != index_url.rstrip("/")
and full not in seen
):
seen.add(full)
chapter_links.append({"url": full, "title": a.get_text(strip=True)})
# Sort by trailing chapter number.
def _num(ch: dict) -> int:
m = re.search(r"-(\d+)$", ch["url"].rstrip("/"))
return int(m.group(1)) if m else 0
chapter_links.sort(key=_num)
for i, ch in enumerate(chapter_links, 1):
ch["title"] = f"Chapter {i}"
# Author and dates: extract from email headers in first and last chapters.
author = "Unknown author"
updated_date = ""
preamble_count = 0
if chapter_links:
_, first_text = await self._get_text(client, chapter_links[0]["url"])
author = self._parse_author_header(first_text) or author
pub_date = self._parse_date_header(first_text)
if len(chapter_links) > 1:
_, last_text = await self._get_text(client, chapter_links[-1]["url"])
updated_date = self._parse_date_header(last_text) or pub_date
else:
updated_date = pub_date
# Boilerplate detection: compare leading paragraphs of chapters 1 and 2.
# Paragraphs present in both (after header strip) are repeated preamble.
if len(chapter_links) >= 2:
_, ch2_text = await self._get_text(client, chapter_links[1]["url"])
paras1 = self._text_to_paragraphs(self._strip_email_headers(first_text))
paras2 = self._text_to_paragraphs(self._strip_email_headers(ch2_text))
for p1, p2 in zip(paras1, paras2):
if self._normalize(p1) == self._normalize(p2):
preamble_count += 1
else:
break
for ch in chapter_links:
ch["preamble_count"] = preamble_count
return {
"title": book_title,
"author": author,
"publisher": "nifty.org",
"series": "",
"series_index_hint": 0,
"genres": [],
"subgenres": [],
"tags": [t for t in [category, subcategory] if t],
"description": "",
"updated_date": updated_date,
"publication_status": "",
"source_url": index_url,
"chapters": chapter_links,
"chapter_method": "html_scan",
"index_image_url": None,
}
async def fetch_chapter(self, client: httpx.AsyncClient, ch: dict) -> dict:
_, raw_text = await self._get_text(client, ch["url"])
# Extract Subject before stripping headers; store as invisible comment.
subject = self._parse_subject_header(raw_text)
# Remove email header block.
story_text = self._strip_email_headers(raw_text)
# Convert hard-wrapped plain text to paragraphs.
paragraphs = self._text_to_paragraphs(story_text)
# Skip repeated boilerplate paragraphs at the top of each chapter.
preamble_count = ch.get("preamble_count", 0)
if preamble_count:
paragraphs = paragraphs[preamble_count:]
paragraphs, hidden_lead, hidden_tail = self._extract_hidden_boilerplate(paragraphs)
# Build an HTML fragment: subject as comment, scene-breaks as <hr/>, rest as <p>.
html_parts: list[str] = []
if subject:
html_parts.append(f"<!-- Subject: {self._comment_safe(subject)} -->")
if hidden_lead:
lead_text = " || ".join(re.sub(r"\s+", " ", p).strip() for p in hidden_lead if p.strip())
if lead_text:
html_parts.append(f"<!-- NIFTY_HIDDEN_LEAD: {self._comment_safe(lead_text)} -->")
for para in paragraphs:
if _BREAK_RE.match(para.strip()):
html_parts.append("<hr/>")
else:
html_parts.append(f"<p>{he(para)}</p>")
if hidden_tail:
tail_text = " || ".join(re.sub(r"\s+", " ", p).strip() for p in hidden_tail if p.strip())
if tail_text:
html_parts.append(f"<!-- NIFTY_HIDDEN_TAIL: {self._comment_safe(tail_text)} -->")
wrapper = BeautifulSoup(
"<div>" + "".join(html_parts) + "</div>",
"html.parser",
)
content_el = wrapper.find("div")
return {
"title": ch["title"],
"content_el": content_el,
"selector_id": None,
"selector_class": None,
}

View File

@ -1,310 +0,0 @@
import json
import re
from html import unescape as html_unescape
from urllib.parse import urlparse
import httpx
from bs4 import BeautifulSoup, Comment
from .base import BaseScraper
class NiftyNewScraper(BaseScraper):
_LEAD_MARKERS = (
"notice this is a work of fiction",
"if it is illegal to read stories",
"if you enjoy this story",
"for my other stories",
"nifty archive",
"code of conduct",
"author note",
"author's note",
"disclaimer",
"this story contains",
"this story includes",
"all characters are",
"all characters depicted",
)
_TAIL_MARKERS = (
"please remember to donate",
"donate",
"support nifty",
"support the archive",
"nifty archive alliance",
"donate.nifty.org",
"nifty.org/donate",
"nifty.org/support",
"patreon",
"buy me a coffee",
"tip jar",
"become a supporter",
)
@classmethod
def matches(cls, url: str) -> bool:
return "new.nifty.org" in url
async def login(self, client: httpx.AsyncClient, username: str, password: str) -> bool:
return True # no login required
# ── Helpers ───────────────────────────────────────────────────────────────
def _to_index_url(self, url: str) -> str:
"""Strip trailing chapter number, return story index URL.
/stories/some-slug-83036/3 /stories/some-slug-83036
/stories/some-slug-83036 /stories/some-slug-83036
"""
parsed = urlparse(url)
path = re.sub(r"/\d+$", "", parsed.path.rstrip("/"))
return f"{parsed.scheme}://{parsed.netloc}{path}"
def _parse_date(self, iso: str) -> str:
"""Return YYYY-MM-DD from an ISO datetime string, or ''."""
if not iso:
return ""
return iso[:10]
# ── BaseScraper interface ─────────────────────────────────────────────────
async def fetch_book_info(self, client: httpx.AsyncClient, url: str) -> dict:
index_url = self._to_index_url(url)
r = await client.get(index_url)
soup = BeautifulSoup(r.text, "html.parser")
# Title: <h1>, fallback to <title> (strip "- … - Nifty Archive …" suffix)
h1 = soup.find("h1")
if h1:
title = h1.get_text(strip=True)
else:
title_el = soup.find("title")
raw = title_el.get_text(strip=True) if title_el else ""
title = re.split(r"\s+[-]\s+", raw)[0].strip() if raw else ""
# Author: <strong itemprop="name"> inside /authors/ link
author = "Unknown author"
author_link = soup.find("a", href=re.compile(r"^/authors/\d+"))
if author_link:
name_el = author_link.find("strong", itemprop="name")
if name_el:
author = name_el.get_text(strip=True)
# Dates: <time itemprop="datePublished/dateModified">
pub_el = soup.find("time", itemprop="datePublished")
mod_el = soup.find("time", itemprop="dateModified")
pub_date = self._parse_date(pub_el.get("datetime", "") if pub_el else "")
updated_date = self._parse_date(mod_el.get("datetime", "") if mod_el else "") or pub_date
# Tags: from all <ul aria-label="Tags"> containers (category links + generated tags)
tags: list[str] = []
seen: set[str] = set()
for ul in soup.find_all("ul", attrs={"aria-label": "Tags"}):
for a in ul.find_all("a", href=True):
label = a.get_text(strip=True)
if label and label.lower() not in seen:
seen.add(label.lower())
tags.append(label)
# Description: <meta name="description">
desc = ""
meta_desc = soup.find("meta", attrs={"name": "description"})
if meta_desc and meta_desc.get("content"):
desc = meta_desc["content"].strip()
# Chapters: find /stories/{slug}/N links in the page HTML
slug_path = urlparse(index_url).path # e.g. /stories/some-slug-83036
chapter_pattern = re.compile(r"^" + re.escape(slug_path) + r"/(\d+)$")
chapter_nums: set[int] = set()
for a in soup.find_all("a", href=True):
m = chapter_pattern.match(a["href"])
if m:
chapter_nums.add(int(m.group(1)))
# Fallback: scan RSC stream for chapter index values
if not chapter_nums:
for m in re.finditer(r'"index"\s*:\s*(\d+)', r.text):
chapter_nums.add(int(m.group(1)))
if not chapter_nums:
chapter_nums = {1}
chapters = [
{"url": f"{index_url}/{i}", "title": f"Chapter {i}"}
for i in range(1, max(chapter_nums) + 1)
]
return {
"title": title,
"author": author,
"publisher": "nifty.org",
"series": "",
"series_index_hint": 0,
"genres": [],
"subgenres": [],
"tags": tags,
"description": desc,
"updated_date": updated_date,
"publication_status": "",
"source_url": index_url,
"chapters": chapters,
"chapter_method": "html_scan",
"index_image_url": None,
}
# ── RSC parser ───────────────────────────────────────────────────────────
def _parse_rsc_paragraphs(self, rsc_text: str) -> list[str]:
"""Extract story paragraph text from a Next.js RSC stream.
The RSC format is a series of lines: ``{hex_id}:{json_value}``.
Each line that represents a <p> element looks like:
2c:["$","p",null,{"children":"Paragraph text."}]
"""
paragraphs: list[str] = []
for line in rsc_text.splitlines():
colon = line.find(":")
if colon < 0:
continue
try:
node = json.loads(line[colon + 1:])
except Exception:
continue
paragraphs.extend(self._rsc_find_paragraphs(node))
return paragraphs
def _rsc_find_paragraphs(self, node) -> list[str]:
"""Recursively find <p> text in an RSC component tree node."""
if not isinstance(node, list):
return []
# React element: ["$", tagname, key, props]
if len(node) >= 4 and node[0] == "$" and isinstance(node[1], str):
tag = node[1]
props = node[3] if isinstance(node[3], dict) else {}
if tag == "p":
text = self._rsc_text(props.get("children", ""))
return [text] if text.strip() else []
children = props.get("children")
if children is not None:
return self._rsc_find_paragraphs(children)
return []
# Plain list of child nodes
result: list[str] = []
for item in node:
result.extend(self._rsc_find_paragraphs(item))
return result
def _rsc_text(self, children) -> str:
"""Flatten RSC children (string or nested array) into plain text."""
if isinstance(children, str):
return children if not children.startswith("$") else ""
if isinstance(children, list):
parts: list[str] = []
for item in children:
if isinstance(item, str) and not item.startswith("$"):
parts.append(item)
elif isinstance(item, list) and len(item) >= 4 and item[0] == "$":
inner = item[3] if isinstance(item[3], dict) else {}
parts.append(self._rsc_text(inner.get("children", "")))
return "".join(parts)
return ""
def _extract_escaped_html_paragraphs(self, text: str) -> list[str]:
"""Extract \\u003cp\\u003e...\\u003c/p\\u003e paragraphs from Next payload text."""
paragraphs: list[str] = []
for raw in re.findall(r"\\u003cp\\u003e(.*?)\\u003c/p\\u003e", text, flags=re.S):
try:
decoded = bytes(raw, "utf-8").decode("unicode_escape")
except Exception:
decoded = raw
decoded = html_unescape(decoded)
decoded = re.sub(r"\s+", " ", decoded).strip()
if decoded:
paragraphs.append(decoded)
return paragraphs
def _comment_safe(self, text: str) -> str:
return text.replace("--", "- -")
def _plain_text(self, text: str) -> str:
# Some payload variants contain inline HTML inside paragraph text.
# Convert to plain text before marker matching.
if "<" in text and ">" in text:
return BeautifulSoup(text, "html.parser").get_text(" ", strip=True)
return text
def _looks_like_lead_boilerplate(self, text: str) -> bool:
t = re.sub(r"\s+", " ", self._plain_text(text).lower()).strip()
if not t or len(t) > 4000:
return False
return any(m in t for m in self._LEAD_MARKERS)
def _looks_like_tail_boilerplate(self, text: str) -> bool:
t = re.sub(r"\s+", " ", self._plain_text(text).lower()).strip()
if not t or len(t) > 4000:
return False
return any(m in t for m in self._TAIL_MARKERS)
def _extract_hidden_boilerplate(self, paragraphs: list[str]) -> tuple[list[str], list[str], list[str]]:
visible = list(paragraphs)
leading: list[str] = []
trailing: list[str] = []
while visible and len(leading) < 6 and self._looks_like_lead_boilerplate(visible[0]):
leading.append(visible.pop(0))
while visible and len(trailing) < 6 and self._looks_like_tail_boilerplate(visible[-1]):
trailing.insert(0, visible.pop())
# Never return an empty chapter due to over-eager filtering.
if not visible:
return list(paragraphs), [], []
return visible, leading, trailing
async def fetch_chapter(self, client: httpx.AsyncClient, ch: dict) -> dict:
# Primary path: fetch chapter HTML and read the rendered <article> content.
r = await client.get(ch["url"])
soup = BeautifulSoup(r.text, "html.parser")
paragraphs: list[str] = []
article = soup.find("article")
if article:
for p in article.find_all("p"):
text = p.get_text(" ", strip=True)
if text:
paragraphs.append(text)
# Fallback: paragraph HTML may only appear escaped in Next payload scripts.
if not paragraphs:
paragraphs = self._extract_escaped_html_paragraphs(r.text)
# Last fallback: request ?_rsc=1 and parse both RSC line format + escaped chunks.
if not paragraphs:
r_rsc = await client.get(ch["url"] + "?_rsc=1")
paragraphs = self._parse_rsc_paragraphs(r_rsc.text)
if not paragraphs:
paragraphs = self._extract_escaped_html_paragraphs(r_rsc.text)
paragraphs, hidden_lead, hidden_tail = self._extract_hidden_boilerplate(paragraphs)
# Build a BeautifulSoup <div> with <p> elements.
wrapper = BeautifulSoup("", "html.parser")
div = wrapper.new_tag("div")
if hidden_lead:
lead_text = " || ".join(re.sub(r"\s+", " ", p).strip() for p in hidden_lead if p.strip())
if lead_text:
div.append(Comment(self._comment_safe(f"NIFTY_HIDDEN_LEAD: {lead_text}")))
for text in paragraphs:
p = wrapper.new_tag("p")
p.string = text
div.append(p)
if hidden_tail:
tail_text = " || ".join(re.sub(r"\s+", " ", p).strip() for p in hidden_tail if p.strip())
if tail_text:
div.append(Comment(self._comment_safe(f"NIFTY_HIDDEN_TAIL: {tail_text}")))
return {
"title": ch["title"],
"content_el": div,
"selector_id": None,
"selector_class": None,
}

View File

@ -1,139 +0,0 @@
import re
from urllib.parse import urljoin
import httpx
from bs4 import BeautifulSoup
from .base import BaseScraper
TED_BASE = "https://tedlouis.com/"
class TedLouisScraper(BaseScraper):
@classmethod
def matches(cls, url: str) -> bool:
return "tedlouis.com" in url
async def login(self, client: httpx.AsyncClient, username: str, password: str) -> bool:
return True # no login required
async def fetch_book_info(self, client: httpx.AsyncClient, url: str) -> dict:
r = await client.get(url)
soup = BeautifulSoup(r.text, "html.parser")
# Detect chapter page (wrong entry point)
if soup.find("h1", class_="story-title") and not soup.find("h2", class_="story-page-title"):
raise ValueError(
"Voer de story index-URL in, geen chapter-URL. "
"Kopieer de URL van de verhaal-indexpagina (de pagina met de hoofdstukkenlijst)."
)
# Title: extract only direct NavigableString children from the h2,
# ignoring nested elements like the "Back" link and author byline.
book_title = "Unknown title"
title_el = soup.find("h2", class_="story-page-title")
if title_el:
from bs4 import NavigableString
parts = [
str(c).strip()
for c in title_el.children
if isinstance(c, NavigableString) and str(c).strip()
]
book_title = " ".join(parts) or title_el.get_text(strip=True)
# Author: from byline span (may be inside the h2 or elsewhere)
author = "Unknown author"
byline = soup.find("span", class_="story-author-by-line")
if byline:
a = byline.find("a")
if a:
author = a.get_text(strip=True)
# Publication status
status_el = soup.find("span", class_="story-status-text")
publication_status = ""
if status_el:
raw = status_el.get_text(strip=True)
publication_status = re.sub(r"^Status:\s*", "", raw, flags=re.I).strip()
# Updated date: "Last Updated: Month D, YYYY" → "YYYY-MM-DD"
updated_date = ""
updated_el = soup.find("span", class_="story-last-updated")
if updated_el:
raw = re.sub(r"^Last\s+Updated:\s*", "", updated_el.get_text(strip=True), flags=re.I).strip()
try:
from datetime import datetime
updated_date = datetime.strptime(raw, "%B %d, %Y").strftime("%Y-%m-%d")
except ValueError:
try:
updated_date = datetime.strptime(raw, "%B %Y").strftime("%Y-%m-01")
except ValueError:
pass
# Chapter links from all story-index-list columns
actual_url = str(r.url)
chapter_links: list[dict] = []
seen: set[str] = set()
for ul in soup.find_all("ul", class_="story-index-list"):
for li in ul.find_all("li"):
a = li.find("a", href=True)
if not a:
continue
href = a["href"]
full_url = urljoin(actual_url, href)
if full_url in seen:
continue
seen.add(full_url)
chapter_links.append({"url": full_url, "title": a.get_text(strip=True)})
return {
"title": book_title,
"author": author,
"publisher": "tedlouis.com",
"series": "",
"series_index_hint": 0,
"genres": [],
"subgenres": [],
"tags": [],
"description": "",
"updated_date": updated_date,
"publication_status": publication_status,
"source_url": url,
"chapters": chapter_links,
"chapter_method": "html_scan",
"index_image_url": None,
}
async def fetch_chapter(self, client: httpx.AsyncClient, ch: dict) -> dict:
cr = await client.get(ch["url"])
csoup = BeautifulSoup(cr.text, "html.parser")
title = ch["title"]
# Refine chapter title from <h2 class="chapter-title"><span>…</span></h2>
chapter_h2 = csoup.find("h2", class_="chapter-title")
if chapter_h2:
span = chapter_h2.find("span")
refined = (span or chapter_h2).get_text(strip=True)
if refined:
title = refined
content_el = csoup.find("div", id="chapter")
if content_el:
# Remove story title, chapter title, copyright blocks
for el in content_el.find_all("h1", class_="story-title"):
el.decompose()
for el in content_el.find_all("h2", class_="chapter-title"):
el.decompose()
for el in content_el.find_all("div", class_="chapter-copyright-line"):
el.decompose()
for el in content_el.find_all("div", class_=re.compile(r"chapter-copyright-notice", re.I)):
el.decompose()
return {
"title": title,
"content_el": content_el,
"selector_id": "chapter",
"selector_class": None,
}

View File

@ -1,18 +0,0 @@
from fastapi.templating import Jinja2Templates
from db import get_db_conn
def _develop_mode() -> bool:
try:
with get_db_conn() as conn:
with conn.cursor() as cur:
cur.execute("SELECT develop_mode FROM app_settings WHERE id = 1")
row = cur.fetchone()
return bool(row[0]) if row else False
except Exception:
return False
templates = Jinja2Templates(directory="templates")
templates.env.globals["develop_mode"] = _develop_mode

View File

@ -19,6 +19,23 @@ if (BOOK.has_cover) {
let currentRating = BOOK.rating || 0; let currentRating = BOOK.rating || 0;
async function rateBook(rating) {
const newRating = currentRating === rating ? 0 : rating;
try {
const resp = await fetch(`/library/rating/${encodeURIComponent(filename)}`, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ rating: newRating }),
});
const result = await resp.json();
if (!resp.ok || result.error) return;
currentRating = result.rating;
document.querySelectorAll('#book-stars .star').forEach((el, idx) => {
el.classList.toggle('filled', idx + 1 <= currentRating);
});
} catch {}
}
// ── Want to Read toggle ──────────────────────────────────────────────────── // ── Want to Read toggle ────────────────────────────────────────────────────
async function toggleWtr() { async function toggleWtr() {
@ -148,8 +165,8 @@ class PillInput {
} }
_add(v) { _add(v) {
const parts = v.split(',').map(p => p.trim()).filter(p => p && !this.values.includes(p)); v = v.trim();
if (parts.length) { this.values.push(...parts); this._render(); } if (v && !this.values.includes(v)) { this.values.push(v); this._render(); }
this.input.value = ''; this.input.value = '';
this._hideDropdown(); this._hideDropdown();
} }
@ -288,10 +305,8 @@ async function openEdit() {
document.getElementById('ed-author').value = BOOK.author; document.getElementById('ed-author').value = BOOK.author;
document.getElementById('ed-publisher').value = BOOK.publisher; document.getElementById('ed-publisher').value = BOOK.publisher;
document.getElementById('ed-series').value = BOOK.series; document.getElementById('ed-series').value = BOOK.series;
document.getElementById('ed-series-volume').value = BOOK.series_volume || '';
document.getElementById('ed-series-index').value = BOOK.series_index + (BOOK.series_suffix || ''); document.getElementById('ed-series-index').value = BOOK.series_index + (BOOK.series_suffix || '');
document.getElementById('ed-status').value = BOOK.publication_status || 'Complete'; document.getElementById('ed-status').value = BOOK.publication_status || 'Complete';
document.getElementById('ed-rating').value = String(currentRating);
document.getElementById('ed-url').value = BOOK.source_url; document.getElementById('ed-url').value = BOOK.source_url;
document.getElementById('ed-publish-date').value = BOOK.publish_date; document.getElementById('ed-publish-date').value = BOOK.publish_date;
document.getElementById('ed-description').value = BOOK.description; document.getElementById('ed-description').value = BOOK.description;
@ -304,17 +319,6 @@ async function openEdit() {
document.getElementById('edit-panel').classList.add('open'); document.getElementById('edit-panel').classList.add('open');
} }
function generateTitle() {
const series = document.getElementById('ed-series').value.trim();
if (!series) return;
const volume = document.getElementById('ed-series-volume').value.trim();
const index = document.getElementById('ed-series-index').value.trim();
let title = series;
if (volume) title += ` (${volume})`;
if (index) title += ` #${index}`;
document.getElementById('ed-title').value = title;
}
function closeEdit() { function closeEdit() {
document.getElementById('edit-backdrop').classList.remove('open'); document.getElementById('edit-backdrop').classList.remove('open');
document.getElementById('edit-panel').classList.remove('open'); document.getElementById('edit-panel').classList.remove('open');
@ -332,7 +336,6 @@ async function saveEdit() {
author: document.getElementById('ed-author').value, author: document.getElementById('ed-author').value,
publisher: document.getElementById('ed-publisher').value, publisher: document.getElementById('ed-publisher').value,
series: document.getElementById('ed-series').value, series: document.getElementById('ed-series').value,
series_volume: document.getElementById('ed-series-volume').value,
series_index: document.getElementById('ed-series-index').value, series_index: document.getElementById('ed-series-index').value,
publication_status: document.getElementById('ed-status').value, publication_status: document.getElementById('ed-status').value,
source_url: document.getElementById('ed-url').value, source_url: document.getElementById('ed-url').value,
@ -342,27 +345,17 @@ async function saveEdit() {
subgenres: subgenreInput.getValues(), subgenres: subgenreInput.getValues(),
tags: tagInput.getValues(), tags: tagInput.getValues(),
}; };
const newRating = parseInt(document.getElementById('ed-rating').value, 10) || 0;
const resp = await fetch(`/library/book/${encodeURIComponent(filename)}`, { const resp = await fetch(`/library/book/${encodeURIComponent(filename)}`, {
method: 'PATCH', method: 'PATCH',
headers: { 'Content-Type': 'application/json' }, headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(body), body: JSON.stringify(body),
}); });
const result = await resp.json(); const result = await resp.json();
if (!resp.ok) { if (resp.ok && result.filename) {
alert(result.error || 'Save failed.');
return;
}
if (newRating !== currentRating) {
await fetch(`/library/rating/${encodeURIComponent(result.filename || filename)}`, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ rating: newRating }),
});
}
if (result.filename) {
window.location.href = `/library/book/${encodeURIComponent(result.filename)}`; window.location.href = `/library/book/${encodeURIComponent(result.filename)}`;
} else { } else if (resp.ok) {
window.location.reload(); window.location.reload();
} else {
alert(result.error || 'Save failed.');
} }
} }

View File

@ -71,14 +71,9 @@ function connectConversionStream(job_id) {
document.querySelectorAll('.chapter-item').forEach(el => el.className = 'chapter-item done'); document.querySelectorAll('.chapter-item').forEach(el => el.className = 'chapter-item done');
document.getElementById('result-meta').innerHTML = document.getElementById('result-meta').innerHTML =
`<strong>${esc(d.title)}</strong><br/>${d.chapters} chapters successfully converted`; `<strong>${esc(d.title)}</strong><br/>${d.chapters} chapters successfully converted`;
const dlBtn = document.getElementById('download-btn'); document.getElementById('download-btn').onclick = () => {
if (d.storage_type === 'db') { window.location = `/download/${encodeURIComponent(d.filename)}`;
dlBtn.querySelector('span') && (dlBtn.querySelector('span').textContent = 'Export EPUB'); };
dlBtn.onclick = () => { window.location = `/api/library/export-epub/${encodeURIComponent(d.filename)}`; };
} else {
dlBtn.querySelector('span') && (dlBtn.querySelector('span').textContent = 'Download EPUB');
dlBtn.onclick = () => { window.location = `/download/${encodeURIComponent(d.filename)}`; };
}
document.getElementById('book-detail-btn').onclick = () => { document.getElementById('book-detail-btn').onclick = () => {
window.location = `/library/book/${encodeURIComponent(d.filename)}`; window.location = `/library/book/${encodeURIComponent(d.filename)}`;
}; };

View File

@ -30,13 +30,6 @@ html, body { height: 100%; background: var(--bg); color: var(--text); font-famil
text-align: center; overflow: hidden; text-overflow: ellipsis; white-space: nowrap; text-align: center; overflow: hidden; text-overflow: ellipsis; white-space: nowrap;
} }
.chapter-title-input {
flex: 1; font-size: 0.72rem; font-family: var(--mono); color: var(--text);
background: var(--surface2); border: 1px solid var(--border); border-radius: var(--radius);
padding: 0.25rem 0.5rem; outline: none; min-width: 0;
}
.chapter-title-input:focus { border-color: var(--accent); }
.header-actions { display: flex; align-items: center; gap: 0.5rem; flex-shrink: 0; } .header-actions { display: flex; align-items: center; gap: 0.5rem; flex-shrink: 0; }
.save-status { .save-status {

View File

@ -1,26 +1,20 @@
require.config({ paths: { vs: 'https://cdn.jsdelivr.net/npm/monaco-editor@0.45.0/min/vs' } }); require.config({ paths: { vs: 'https://cdn.jsdelivr.net/npm/monaco-editor@0.45.0/min/vs' } });
const { filename, is_db } = EDITOR; const { filename } = EDITOR;
let editor = null; let editor = null;
let chapters = []; let chapters = [];
let currentIndex = -1; let currentIndex = -1;
let dirty = new Set(); // indices with unsaved changes let dirty = new Set(); // indices with unsaved changes
let pendingContent = new Map(); // index -> modified content not yet saved let pendingContent = new Map(); // index -> modified content not yet saved
let pendingTitles = new Map(); // index -> modified title not yet saved (DB only)
let loadingChapter = false; // suppress dirty events during setValue let loadingChapter = false; // suppress dirty events during setValue
let saving = false; let saving = false;
// ── Init Monaco ─────────────────────────────────────────────────────────────── // ── Init Monaco ───────────────────────────────────────────────────────────────
require(['vs/editor/editor.main'], function () { require(['vs/editor/editor.main'], function () {
if (is_db) {
document.getElementById('header-chapter').style.display = 'none';
document.getElementById('chapter-title-input').style.display = '';
}
editor = monaco.editor.create(document.getElementById('editor-pane'), { editor = monaco.editor.create(document.getElementById('editor-pane'), {
language: is_db ? 'html' : 'xml', language: 'xml',
theme: 'vs-dark', theme: 'vs-dark',
wordWrap: 'on', wordWrap: 'on',
minimap: { enabled: true }, minimap: { enabled: true },
@ -45,19 +39,6 @@ require(['vs/editor/editor.main'], function () {
// Ctrl+S / Cmd+S // Ctrl+S / Cmd+S
editor.addCommand(monaco.KeyMod.CtrlCmd | monaco.KeyCode.KeyS, saveChapter); editor.addCommand(monaco.KeyMod.CtrlCmd | monaco.KeyCode.KeyS, saveChapter);
if (is_db) {
document.getElementById('chapter-title-input').addEventListener('input', () => {
if (currentIndex >= 0) {
pendingTitles.set(currentIndex, document.getElementById('chapter-title-input').value);
dirty.add(currentIndex);
renderChapterList();
setStatus('dirty', 'Unsaved changes');
document.getElementById('btn-save').disabled = false;
updateSaveAll();
}
});
}
loadChapterList(); loadChapterList();
}); });
@ -76,7 +57,6 @@ async function loadChapterList(targetIndex = 0) {
currentIndex = -1; currentIndex = -1;
dirty.clear(); dirty.clear();
pendingContent.clear(); pendingContent.clear();
pendingTitles.clear();
renderChapterList(); renderChapterList();
document.getElementById('header-chapter').textContent = 'No chapters'; document.getElementById('header-chapter').textContent = 'No chapters';
document.getElementById('btn-save').disabled = true; document.getElementById('btn-save').disabled = true;
@ -114,11 +94,6 @@ async function switchChapter(index) {
if (dirty.has(currentIndex) && editor) { if (dirty.has(currentIndex) && editor) {
pendingContent.set(currentIndex, editor.getValue()); pendingContent.set(currentIndex, editor.getValue());
} }
// Preserve title input for DB books
if (is_db && currentIndex >= 0) {
const inp = document.getElementById('chapter-title-input');
if (inp) pendingTitles.set(currentIndex, inp.value);
}
loadChapter(index); loadChapter(index);
} }
@ -127,19 +102,19 @@ async function loadChapter(index) {
document.getElementById('btn-save').disabled = true; document.getElementById('btn-save').disabled = true;
document.getElementById('btn-break').disabled = true; document.getElementById('btn-break').disabled = true;
document.getElementById('btn-del-page').disabled = true; document.getElementById('btn-del-page').disabled = true;
if (!is_db) document.getElementById('header-chapter').textContent = 'Loading…'; document.getElementById('header-chapter').textContent = 'Loading…';
let content, title; let content, title;
if (pendingContent.has(index)) { if (pendingContent.has(index)) {
content = pendingContent.get(index); content = pendingContent.get(index);
title = pendingTitles.has(index) ? pendingTitles.get(index) : (chapters[index]?.title ?? ''); title = chapters[index]?.title ?? '';
} else { } else {
const resp = await fetch(`/api/edit/chapter/${index}/${encodeURIComponent(filename)}`); const resp = await fetch(`/api/edit/chapter/${index}/${encodeURIComponent(filename)}`);
if (!resp.ok) { setStatus('error', 'Load failed'); return; } if (!resp.ok) { setStatus('error', 'Load failed'); return; }
const data = await resp.json(); const data = await resp.json();
content = data.content; content = data.content;
title = pendingTitles.has(index) ? pendingTitles.get(index) : data.title; title = data.title;
} }
currentIndex = index; currentIndex = index;
@ -148,7 +123,6 @@ async function loadChapter(index) {
editor.setValue(content); editor.setValue(content);
editor.setScrollTop(0); editor.setScrollTop(0);
loadingChapter = false; loadingChapter = false;
editor.focus();
// Restore dirty state based on whether we loaded from pending cache // Restore dirty state based on whether we loaded from pending cache
if (dirty.has(index)) { if (dirty.has(index)) {
@ -160,11 +134,7 @@ async function loadChapter(index) {
} }
renderChapterList(); renderChapterList();
if (is_db) {
document.getElementById('chapter-title-input').value = title;
} else {
document.getElementById('header-chapter').textContent = title; document.getElementById('header-chapter').textContent = title;
}
document.getElementById('btn-break').disabled = false; document.getElementById('btn-break').disabled = false;
document.getElementById('btn-del-page').disabled = chapters.length <= 1; document.getElementById('btn-del-page').disabled = chapters.length <= 1;
updateSaveAll(); updateSaveAll();
@ -179,28 +149,18 @@ async function saveChapter() {
setStatus('saving', 'Saving…'); setStatus('saving', 'Saving…');
try { try {
const saveBody = { content: editor.getValue() };
if (is_db) {
const inp = document.getElementById('chapter-title-input');
saveBody.title = inp ? inp.value.trim() : (pendingTitles.get(currentIndex) || '');
}
const resp = await fetch( const resp = await fetch(
`/api/edit/chapter/${currentIndex}/${encodeURIComponent(filename)}`, `/api/edit/chapter/${currentIndex}/${encodeURIComponent(filename)}`,
{ {
method: 'POST', method: 'POST',
headers: { 'Content-Type': 'application/json' }, headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(saveBody), body: JSON.stringify({ content: editor.getValue() }),
} }
); );
const data = await resp.json(); const data = await resp.json();
if (data.ok) { if (data.ok) {
dirty.delete(currentIndex); dirty.delete(currentIndex);
pendingContent.delete(currentIndex); pendingContent.delete(currentIndex);
if (is_db && chapters[currentIndex]) {
const saved = pendingTitles.get(currentIndex) || chapters[currentIndex].title;
chapters[currentIndex].title = saved || chapters[currentIndex].title;
pendingTitles.delete(currentIndex);
}
renderChapterList(); renderChapterList();
setStatus('saved', 'Saved'); setStatus('saved', 'Saved');
setTimeout(() => setStatus('', ''), 2000); setTimeout(() => setStatus('', ''), 2000);
@ -226,13 +186,9 @@ async function saveAllChapters() {
if (btn) btn.disabled = true; if (btn) btn.disabled = true;
setStatus('saving', 'Saving all…'); setStatus('saving', 'Saving all…');
// Flush current editor content and title into pending caches first // Flush current editor content into pendingContent first
if (currentIndex >= 0 && dirty.has(currentIndex)) { if (currentIndex >= 0 && dirty.has(currentIndex)) {
pendingContent.set(currentIndex, editor.getValue()); pendingContent.set(currentIndex, editor.getValue());
if (is_db) {
const inp = document.getElementById('chapter-title-input');
if (inp) pendingTitles.set(currentIndex, inp.value);
}
} }
const indices = [...dirty]; const indices = [...dirty];
@ -240,29 +196,21 @@ async function saveAllChapters() {
const content = pendingContent.has(i) const content = pendingContent.has(i)
? pendingContent.get(i) ? pendingContent.get(i)
: (i === currentIndex ? editor.getValue() : null); : (i === currentIndex ? editor.getValue() : null);
// For DB books, a title-only change has no pendingContent — still need to save if (!content) continue;
const hasTitleChange = is_db && pendingTitles.has(i);
if (!content && !hasTitleChange) continue;
try { try {
const saveBody = { content: content || '' };
if (is_db) saveBody.title = pendingTitles.has(i) ? pendingTitles.get(i) : (chapters[i]?.title || '');
const resp = await fetch( const resp = await fetch(
`/api/edit/chapter/${i}/${encodeURIComponent(filename)}`, `/api/edit/chapter/${i}/${encodeURIComponent(filename)}`,
{ {
method: 'POST', method: 'POST',
headers: { 'Content-Type': 'application/json' }, headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(saveBody), body: JSON.stringify({ content }),
} }
); );
const data = await resp.json(); const data = await resp.json();
if (data.ok) { if (data.ok) {
dirty.delete(i); dirty.delete(i);
pendingContent.delete(i); pendingContent.delete(i);
if (is_db && chapters[i]) {
chapters[i].title = pendingTitles.get(i) || chapters[i].title;
pendingTitles.delete(i);
}
} }
} catch { } catch {
setStatus('error', `Save failed on chapter ${i + 1}`); setStatus('error', `Save failed on chapter ${i + 1}`);
@ -303,11 +251,10 @@ function updateSaveAll() {
function insertBreak() { function insertBreak() {
if (!editor || currentIndex < 0) return; if (!editor || currentIndex < 0) return;
const breakSrc = is_db ? '/static/break.png' : '../Images/break.png';
const pos = editor.getPosition(); const pos = editor.getPosition();
editor.executeEdits('insert-break', [{ editor.executeEdits('insert-break', [{
range: new monaco.Range(pos.lineNumber, pos.column, pos.lineNumber, pos.column), range: new monaco.Range(pos.lineNumber, pos.column, pos.lineNumber, pos.column),
text: `\n<center><img src="${breakSrc}" style="height:15px;"/></center>\n`, text: '\n<center><img src="../Images/break.png" style="height:15px;"/></center>\n',
forceMoveMarkers: true, forceMoveMarkers: true,
}]); }]);
editor.focus(); editor.focus();
@ -339,7 +286,6 @@ async function addChapter() {
dirty.clear(); dirty.clear();
pendingContent.clear(); pendingContent.clear();
pendingTitles.clear();
await loadChapterList(data.index ?? Math.max(currentIndex + 1, 0)); await loadChapterList(data.index ?? Math.max(currentIndex + 1, 0));
setStatus('saved', 'Page added'); setStatus('saved', 'Page added');
setTimeout(() => setStatus('', ''), 1500); setTimeout(() => setStatus('', ''), 1500);
@ -369,7 +315,6 @@ async function deleteChapter() {
dirty.clear(); dirty.clear();
pendingContent.clear(); pendingContent.clear();
pendingTitles.clear();
await loadChapterList(data.index ?? Math.max(currentIndex - 1, 0)); await loadChapterList(data.index ?? Math.max(currentIndex - 1, 0));
setStatus('saved', 'Page deleted'); setStatus('saved', 'Page deleted');
setTimeout(() => setStatus('', ''), 1500); setTimeout(() => setStatus('', ''), 1500);

View File

@ -140,24 +140,6 @@ html, body {
.badge-temporary-hold { color: #c8a03a; } .badge-temporary-hold { color: #c8a03a; }
.badge-long-term-hold { color: #ffa20e; } .badge-long-term-hold { color: #ffa20e; }
/* Archived badge: bottom-left of cover */
.badge-archived {
position: absolute;
bottom: 0.35rem;
left: 0.35rem;
width: 22px;
height: 22px;
border-radius: 50%;
display: flex;
align-items: center;
justify-content: center;
z-index: 2;
background: rgba(15,14,12,0.82);
box-shadow: 0 0 0 2px #0f0e0c;
color: #888;
pointer-events: none;
}
/* Star: want-to-read top-left */ /* Star: want-to-read top-left */
.btn-star { .btn-star {
position: absolute; position: absolute;
@ -376,9 +358,6 @@ html, body {
.slot-missing-inner svg { opacity: 0.5; } .slot-missing-inner svg { opacity: 0.5; }
.slot-missing-inner span { font-family: var(--mono); font-size: 0.65rem; } .slot-missing-inner span { font-family: var(--mono); font-size: 0.65rem; }
/* ── Series detail header ─────────────────────────────────────────────────── */
.series-detail-header { display: flex; justify-content: flex-end; margin-bottom: 1rem; }
/* ── Authors list ─────────────────────────────────────────────────────────── */ /* ── Authors list ─────────────────────────────────────────────────────────── */
.author-list { display: flex; flex-direction: column; gap: 0.3rem; } .author-list { display: flex; flex-direction: column; gap: 0.3rem; }

View File

@ -938,7 +938,6 @@ function renderBooksGrid(books) {
</svg> </svg>
</button> </button>
${statusBadge} ${statusBadge}
${b.archived ? `<div class="badge-archived" title="Archived"><svg width="10" height="10" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2.5"><polyline points="21 8 21 21 3 21 3 8"/><rect x="1" y="3" width="22" height="5"/><line x1="10" y1="12" x2="14" y2="12"/></svg></div>` : ''}
${b.read_count > 0 ? `<div class="read-pill">${b.read_count}\u00d7</div>` : ''} ${b.read_count > 0 ? `<div class="read-pill">${b.read_count}\u00d7</div>` : ''}
${b.progress > 0 ? `<div class="progress-mini"><div class="progress-mini-fill" style="width:${b.progress}%"></div></div>` : ''} ${b.progress > 0 ? `<div class="progress-mini"><div class="progress-mini-fill" style="width:${b.progress}%"></div></div>` : ''}
</div> </div>
@ -989,8 +988,6 @@ function groupBySeries() {
map[b.series].push(b); map[b.series].push(b);
} }
for (const s of Object.values(map)) s.sort((a, b) => { for (const s of Object.values(map)) s.sort((a, b) => {
const va = a.series_volume || '', vb = b.series_volume || '';
if (va !== vb) return va.localeCompare(vb);
if (a.series_index !== b.series_index) return a.series_index - b.series_index; if (a.series_index !== b.series_index) return a.series_index - b.series_index;
return (a.series_suffix || '').localeCompare(b.series_suffix || ''); return (a.series_suffix || '').localeCompare(b.series_suffix || '');
}); });
@ -1085,26 +1082,6 @@ function renderSeriesGrid() {
// ── Series detail ────────────────────────────────────────────────────────── // ── Series detail ──────────────────────────────────────────────────────────
function _slotsForBooks(books, seriesVolume) {
const sorted = [...books].sort((a, b) => {
if (a.series_index !== b.series_index) return a.series_index - b.series_index;
return (a.series_suffix || '').localeCompare(b.series_suffix || '');
});
const byIndex = {};
for (const b of sorted) {
if (!byIndex[b.series_index]) byIndex[b.series_index] = [];
byIndex[b.series_index].push(b);
}
const min = Math.min(...sorted.map(b => b.series_index));
const max = Math.max(...sorted.map(b => b.series_index));
const slots = [];
for (let i = min; i <= max; i++) {
if (byIndex[i]) for (const b of byIndex[i]) slots.push(b);
else slots.push({ missing: true, series_index: i, series_volume: seriesVolume || '' });
}
return slots;
}
function getSeriesSlots(books) { function getSeriesSlots(books) {
// Treat books as indexed (including index 0) only when at least one book // Treat books as indexed (including index 0) only when at least one book
// has series_index > 0 — this preserves the "unindexed flat list" behaviour // has series_index > 0 — this preserves the "unindexed flat list" behaviour
@ -1112,24 +1089,29 @@ function getSeriesSlots(books) {
const hasPositiveIndex = books.some(b => b.series_index > 0); const hasPositiveIndex = books.some(b => b.series_index > 0);
if (!hasPositiveIndex) return books; if (!hasPositiveIndex) return books;
// When series_volume is used, do gap-detection per volume (year) separately. // Sort indexed books by (series_index, series_suffix) so 21 < 21a < 21b < 22.
if (books.some(b => b.series_volume)) { const indexed = [...books].sort((a, b) => {
const byVolume = {}; if (a.series_index !== b.series_index) return a.series_index - b.series_index;
for (const b of books) { return (a.series_suffix || '').localeCompare(b.series_suffix || '');
const vol = b.series_volume || ''; });
if (!byVolume[vol]) byVolume[vol] = [];
byVolume[vol].push(b); // Build slot map keyed by numeric index only (for gap detection).
const byIndex = {};
for (const b of indexed) {
if (!byIndex[b.series_index]) byIndex[b.series_index] = [];
byIndex[b.series_index].push(b);
} }
const min = Math.min(...indexed.map(b => b.series_index));
const max = Math.max(...indexed.map(b => b.series_index));
const slots = []; const slots = [];
for (const vol of Object.keys(byVolume).sort()) { for (let i = min; i <= max; i++) {
for (const slot of _slotsForBooks(byVolume[vol], vol)) slots.push(slot); if (byIndex[i]) for (const b of byIndex[i]) slots.push(b);
else slots.push({ missing: true, series_index: i });
} }
return slots; return slots;
} }
return _slotsForBooks(books, '');
}
function renderSeriesDetail(seriesName) { function renderSeriesDetail(seriesName) {
const map = groupBySeries(); const map = groupBySeries();
const books = map[seriesName] || []; const books = map[seriesName] || [];
@ -1152,9 +1134,7 @@ function renderSeriesDetail(seriesName) {
if (hasPositiveIndex || slot.series_index > 0 || slot.series_suffix) { if (hasPositiveIndex || slot.series_index > 0 || slot.series_suffix) {
const lbl = document.createElement('div'); const lbl = document.createElement('div');
lbl.className = 'slot-index-label'; lbl.className = 'slot-index-label';
lbl.textContent = slot.series_volume lbl.textContent = `#${slot.series_index}${slot.series_suffix || ''}`;
? `(${slot.series_volume}) #${slot.series_index}${slot.series_suffix || ''}`
: `#${slot.series_index}${slot.series_suffix || ''}`;
wrapper.appendChild(lbl); wrapper.appendChild(lbl);
} }
@ -1189,7 +1169,6 @@ function renderSeriesDetail(seriesName) {
<div class="cover-wrap"> <div class="cover-wrap">
<canvas class="cover-canvas"></canvas> <canvas class="cover-canvas"></canvas>
${statusBadge} ${statusBadge}
${b.archived ? `<div class="badge-archived" title="Archived"><svg width="10" height="10" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2.5"><polyline points="21 8 21 21 3 21 3 8"/><rect x="1" y="3" width="22" height="5"/><line x1="10" y1="12" x2="14" y2="12"/></svg></div>` : ''}
${b.read_count > 0 ? `<div class="read-pill">${b.read_count}\u00d7</div>` : ''} ${b.read_count > 0 ? `<div class="read-pill">${b.read_count}\u00d7</div>` : ''}
${b.progress > 0 ? `<div class="progress-mini"><div class="progress-mini-fill" style="width:${b.progress}%"></div></div>` : ''} ${b.progress > 0 ? `<div class="progress-mini"><div class="progress-mini-fill" style="width:${b.progress}%"></div></div>` : ''}
</div> </div>
@ -1219,32 +1198,7 @@ function renderSeriesDetail(seriesName) {
grid.appendChild(wrapper); grid.appendChild(wrapper);
}); });
const allArchived = books.every(b => b.archived);
const header = document.createElement('div');
header.className = 'series-detail-header';
const archiveBtn = document.createElement('button');
archiveBtn.className = 'btn btn-sm';
archiveBtn.textContent = allArchived ? 'Unarchive series' : 'Archive series';
archiveBtn.onclick = async () => {
archiveBtn.disabled = true;
const res = await fetch('/library/archive-series', {
method: 'POST',
headers: {'Content-Type': 'application/json'},
body: JSON.stringify({series: seriesName, archive: !allArchived}),
});
const data = await res.json();
if (data.ok) {
allBooks.forEach(b => { if (b.series === seriesName) b.archived = !allArchived; });
updateCounts();
renderSeriesDetail(seriesName);
} else {
archiveBtn.disabled = false;
}
};
header.appendChild(archiveBtn);
container.innerHTML = ''; container.innerHTML = '';
container.appendChild(header);
container.appendChild(grid); container.appendChild(grid);
} }
@ -1254,7 +1208,7 @@ function renderAuthorsView() {
const container = document.getElementById('grid-container'); const container = document.getElementById('grid-container');
const authorMap = {}; const authorMap = {};
for (const b of allBooks) { for (const b of activeBooks()) {
const a = bookAuthor(b); const a = bookAuthor(b);
if (!a) continue; if (!a) continue;
if (!authorMap[a]) authorMap[a] = []; if (!authorMap[a]) authorMap[a] = [];
@ -1297,7 +1251,7 @@ function renderPublishersView() {
const container = document.getElementById('grid-container'); const container = document.getElementById('grid-container');
const publisherMap = {}; const publisherMap = {};
for (const b of allBooks) { for (const b of activeBooks()) {
const key = bookPublisherKey(b); const key = bookPublisherKey(b);
if (!publisherMap[key]) publisherMap[key] = []; if (!publisherMap[key]) publisherMap[key] = [];
publisherMap[key].push(b); publisherMap[key].push(b);
@ -1543,7 +1497,6 @@ function renderDuplicatesView() {
</svg> </svg>
</button> </button>
${statusBadge} ${statusBadge}
${b.archived ? `<div class="badge-archived" title="Archived"><svg width="10" height="10" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2.5"><polyline points="21 8 21 21 3 21 3 8"/><rect x="1" y="3" width="22" height="5"/><line x1="10" y1="12" x2="14" y2="12"/></svg></div>` : ''}
${b.read_count > 0 ? `<div class="read-pill">${b.read_count}\u00d7</div>` : ''} ${b.read_count > 0 ? `<div class="read-pill">${b.read_count}\u00d7</div>` : ''}
${b.progress > 0 ? `<div class="progress-mini"><div class="progress-mini-fill" style="width:${b.progress}%"></div></div>` : ''} ${b.progress > 0 ? `<div class="progress-mini"><div class="progress-mini-fill" style="width:${b.progress}%"></div></div>` : ''}
</div> </div>
@ -1591,8 +1544,6 @@ function renderAuthorDetail(authorName) {
const sa = a.series || '\uffff'; const sa = a.series || '\uffff';
const sb = b.series || '\uffff'; const sb = b.series || '\uffff';
if (sa !== sb) return sa.localeCompare(sb); if (sa !== sb) return sa.localeCompare(sb);
const va = a.series_volume || '', vb = b.series_volume || '';
if (va !== vb) return va.localeCompare(vb);
if (a.series_index !== b.series_index) return a.series_index - b.series_index; if (a.series_index !== b.series_index) return a.series_index - b.series_index;
if ((a.series_suffix || '') !== (b.series_suffix || '')) return (a.series_suffix || '').localeCompare(b.series_suffix || ''); if ((a.series_suffix || '') !== (b.series_suffix || '')) return (a.series_suffix || '').localeCompare(b.series_suffix || '');
return bookTitle(a).localeCompare(bookTitle(b)); return bookTitle(a).localeCompare(bookTitle(b));
@ -1608,8 +1559,6 @@ function renderPublisherDetail(publisherName) {
const sa = a.series || '\uffff'; const sa = a.series || '\uffff';
const sb = b.series || '\uffff'; const sb = b.series || '\uffff';
if (sa !== sb) return sa.localeCompare(sb); if (sa !== sb) return sa.localeCompare(sb);
const va = a.series_volume || '', vb = b.series_volume || '';
if (va !== vb) return va.localeCompare(vb);
if (a.series_index !== b.series_index) return a.series_index - b.series_index; if (a.series_index !== b.series_index) return a.series_index - b.series_index;
if ((a.series_suffix || '') !== (b.series_suffix || '')) return (a.series_suffix || '').localeCompare(b.series_suffix || ''); if ((a.series_suffix || '') !== (b.series_suffix || '')) return (a.series_suffix || '').localeCompare(b.series_suffix || '');
return bookTitle(a).localeCompare(bookTitle(b)); return bookTitle(a).localeCompare(bookTitle(b));

View File

@ -1,27 +1,3 @@
/* ── Develop banner ── */
.develop-banner {
position: fixed;
top: 26px;
left: -52px;
width: 200px;
background: rgba(194, 120, 20, 0.5);
color: rgba(255, 255, 255, 0.95);
text-align: center;
transform: rotate(-45deg);
z-index: 9999;
pointer-events: none;
}
.develop-banner-text {
display: block;
padding: 6px 0;
font-family: var(--mono);
font-size: 0.62rem;
font-weight: 600;
letter-spacing: 0.2em;
}
/* ── Sidebar ── */ /* ── Sidebar ── */
html { html {
@ -96,10 +72,8 @@ html {
text-decoration: none; text-decoration: none;
transition: background 0.12s, color 0.12s; transition: background 0.12s, color 0.12s;
} }
.sidebar-nav a:visited { color: var(--text-dim); }
.sidebar-nav a:hover { background: var(--surface2); color: var(--text); } .sidebar-nav a:hover { background: var(--surface2); color: var(--text); }
.sidebar-nav a.active, .sidebar-nav a.active { background: var(--surface2); color: var(--accent); }
.sidebar-nav a.active:visited { background: var(--surface2); color: var(--accent); }
.sidebar-nav a svg { flex-shrink: 0; } .sidebar-nav a svg { flex-shrink: 0; }
.sidebar-count { .sidebar-count {

View File

@ -1,7 +1,3 @@
{% if develop_mode() %}
<div class="develop-banner"><span class="develop-banner-text">DEVELOP</span></div>
{% endif %}
<button class="sidebar-toggle" id="sidebar-toggle" onclick="toggleSidebar()" aria-label="Menu"> <button class="sidebar-toggle" id="sidebar-toggle" onclick="toggleSidebar()" aria-label="Menu">
<svg width="16" height="16" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2.5"> <svg width="16" height="16" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2.5">
<line x1="3" y1="6" x2="21" y2="6"/> <line x1="3" y1="6" x2="21" y2="6"/>
@ -156,14 +152,6 @@
<span class="sidebar-count" id="count-duplicates"></span> <span class="sidebar-count" id="count-duplicates"></span>
</a> </a>
</li> </li>
<li>
<a href="/search"{% if active == 'search' %} class="active"{% endif %}>
<svg width="14" height="14" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2">
<circle cx="11" cy="11" r="8"/><path d="M21 21l-4.35-4.35"/>
</svg>
Search
</a>
</li>
<li> <li>
<a href="/stats"{% if active == 'stats' %} class="active"{% endif %}> <a href="/stats"{% if active == 'stats' %} class="active"{% endif %}>
<svg width="14" height="14" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2"> <svg width="14" height="14" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2">

View File

@ -3,7 +3,7 @@
<head> <head>
<meta charset="UTF-8"/> <meta charset="UTF-8"/>
<meta name="viewport" content="width=device-width, initial-scale=1.0"/> <meta name="viewport" content="width=device-width, initial-scale=1.0"/>
<title>Novela{% if develop_mode() %} Develop{% endif %} — Backup</title> <title>Novela - Backup</title>
<link rel="icon" href="/static/favicon.ico" sizes="16x16"/> <link rel="icon" href="/static/favicon.ico" sizes="16x16"/>
<link rel="icon" type="image/png" sizes="32x32" href="/static/favicon-32.png"/> <link rel="icon" type="image/png" sizes="32x32" href="/static/favicon-32.png"/>
<link rel="icon" type="image/png" sizes="256x256" href="/static/favicon-256.png"/> <link rel="icon" type="image/png" sizes="256x256" href="/static/favicon-256.png"/>
@ -236,43 +236,6 @@
</table> </table>
</div> </div>
</section> </section>
<section class="card">
<div class="card-head">Restore</div>
<p class="muted" style="margin-top:0;margin-bottom:0.9rem;">
Browse a snapshot and restore individual books from Dropbox back to disk.
</p>
<div style="display:flex;gap:0.6rem;align-items:center;flex-wrap:wrap;margin-bottom:0.7rem;">
<select class="field-input" id="snapshot-select" style="flex:1;min-width:220px;margin:0;" onchange="onSnapshotChange()">
<option value="">— select snapshot —</option>
</select>
<button class="btn" onclick="loadSnapshots()">Refresh</button>
</div>
<div id="restore-file-panel" style="display:none;">
<input class="field-input" id="restore-search" type="text" placeholder="Filter by filename or path…" oninput="renderRestoreFiles()" style="margin-bottom:0.5rem;"/>
<div class="actions" style="margin-bottom:0.7rem;">
<button class="btn" onclick="selectAllRestoreFiles()">Select all</button>
<button class="btn" onclick="clearRestoreSelection()">Clear</button>
<button class="btn primary" id="btn-restore-selected" onclick="restoreSelected()" disabled>Restore selected</button>
</div>
<div style="overflow:auto;">
<table>
<thead>
<tr>
<th style="width:1.5rem;"></th>
<th style="width:3rem;">Format</th>
<th>Path</th>
<th style="width:5rem;">Size</th>
<th style="width:5rem;">On disk</th>
<th style="width:5rem;"></th>
</tr>
</thead>
<tbody id="restore-file-body"></tbody>
</table>
</div>
</div>
<div class="status-line" id="restore-status"></div>
</section>
</main> </main>
<script src="/static/books.js"></script> <script src="/static/books.js"></script>
@ -540,153 +503,7 @@
} }
async function refreshAll() { async function refreshAll() {
await Promise.all([loadDropboxSettings(), loadHealth(), loadStatus(), loadHistory(), loadSnapshots()]); await Promise.all([loadDropboxSettings(), loadHealth(), loadStatus(), loadHistory()]);
}
// ── Restore ─────────────────────────────────────────────────────────────
let _restoreFiles = [];
async function loadSnapshots() {
const sel = document.getElementById('snapshot-select');
try {
const r = await fetch('/api/backup/snapshots');
const d = await r.json();
if (!d.ok || !d.snapshots.length) {
sel.innerHTML = '<option value="">— no snapshots available —</option>';
return;
}
const current = sel.value;
sel.innerHTML = '<option value="">— select snapshot —</option>' +
d.snapshots.map(s => {
const label = s.created_at
? `${s.name} (${s.created_at.replace('T', ' ').replace('Z', ' UTC')})`
: s.name;
return `<option value="${esc(s.name)}"${s.name === current ? ' selected' : ''}>${esc(label)}</option>`;
}).join('');
} catch (_) {
sel.innerHTML = '<option value="">— Dropbox not configured —</option>';
}
}
async function onSnapshotChange() {
const name = document.getElementById('snapshot-select').value;
const panel = document.getElementById('restore-file-panel');
const status = document.getElementById('restore-status');
if (!name) {
panel.style.display = 'none';
_restoreFiles = [];
status.textContent = '';
return;
}
status.className = 'status-line warn';
status.textContent = 'Loading snapshot files…';
try {
const r = await fetch(`/api/backup/snapshots/${encodeURIComponent(name)}/files`);
const d = await r.json();
if (!d.ok) throw new Error(d.error || 'failed');
_restoreFiles = d.files;
document.getElementById('restore-search').value = '';
panel.style.display = '';
renderRestoreFiles();
status.className = 'status-line ok';
status.textContent = `${d.files.length} file(s) in snapshot.`;
} catch (e) {
status.className = 'status-line err';
status.textContent = `Failed to load snapshot files: ${e}`;
panel.style.display = 'none';
}
}
function fmtBytes(bytes) {
if (!bytes) return '-';
if (bytes >= 1024 * 1024) return (bytes / 1024 / 1024).toFixed(1) + ' MB';
if (bytes >= 1024) return Math.round(bytes / 1024) + ' KB';
return bytes + ' B';
}
function renderRestoreFiles() {
const q = (document.getElementById('restore-search').value || '').toLowerCase().trim();
const body = document.getElementById('restore-file-body');
const filtered = q ? _restoreFiles.filter(f => f.path.toLowerCase().includes(q)) : _restoreFiles;
if (!filtered.length) {
body.innerHTML = '<tr><td colspan="6" style="color:var(--text-dim);padding:0.6rem 0.25rem">No files found.</td></tr>';
document.getElementById('btn-restore-selected').disabled = true;
return;
}
body.innerHTML = filtered.map(f => {
const ext = f.path.split('.').pop().toUpperCase();
const parts = f.path.split('/');
const name = parts[parts.length - 1];
const dir = parts.slice(0, -1).join('/');
const onDisk = f.exists_locally
? '<span class="ok" title="File already on disk">&#10003; exists</span>'
: '<span class="warn">missing</span>';
return `<tr>
<td><input type="checkbox" class="restore-chk" data-path="${esc(f.path)}" onchange="updateRestoreBtn()"/></td>
<td><span style="font-family:var(--mono);font-size:0.68rem;color:var(--text-dim)">${esc(ext)}</span></td>
<td><span style="font-size:0.8rem">${esc(name)}</span><br/><span style="font-size:0.68rem;color:var(--text-dim)">${esc(dir)}</span></td>
<td style="white-space:nowrap;font-family:var(--mono);font-size:0.72rem">${esc(fmtBytes(f.size))}</td>
<td style="font-family:var(--mono);font-size:0.72rem">${onDisk}</td>
<td><button class="btn" style="padding:0.3rem 0.6rem" data-path="${esc(f.path)}" onclick="restoreRowBtn(this)">Restore</button></td>
</tr>`;
}).join('');
updateRestoreBtn();
}
function updateRestoreBtn() {
const checked = document.querySelectorAll('.restore-chk:checked').length;
document.getElementById('btn-restore-selected').disabled = checked === 0;
}
function selectAllRestoreFiles() {
document.querySelectorAll('.restore-chk').forEach(el => { el.checked = true; });
updateRestoreBtn();
}
function clearRestoreSelection() {
document.querySelectorAll('.restore-chk').forEach(el => { el.checked = false; });
updateRestoreBtn();
}
function restoreRowBtn(btn) {
const snapshotName = document.getElementById('snapshot-select').value;
_doRestore(snapshotName, [btn.dataset.path]);
}
function restoreSelected() {
const snapshotName = document.getElementById('snapshot-select').value;
const paths = Array.from(document.querySelectorAll('.restore-chk:checked')).map(el => el.dataset.path);
_doRestore(snapshotName, paths);
}
async function _doRestore(snapshotName, paths) {
if (!paths.length) return;
const status = document.getElementById('restore-status');
status.className = 'status-line warn';
status.textContent = `Restoring ${paths.length} file(s)…`;
try {
const r = await fetch('/api/backup/restore', {
method: 'POST',
headers: {'Content-Type': 'application/json'},
body: JSON.stringify({snapshot_name: snapshotName, files: paths}),
});
const d = await r.json();
if (!d.ok) throw new Error(d.error || 'failed');
const failed = (d.results || []).filter(x => !x.ok);
if (failed.length) {
status.className = 'status-line warn';
status.textContent = `Restored ${d.restored}/${d.total}. Errors: ${failed.map(x => `${x.path}: ${x.error}`).join(' | ')}`;
} else {
status.className = 'status-line ok';
status.textContent = `Restored ${d.restored}/${d.total} file(s) successfully.`;
}
// Refresh exists_locally state
await onSnapshotChange();
} catch (e) {
status.className = 'status-line err';
status.textContent = `Restore failed: ${e}`;
}
} }
refreshAll(); refreshAll();

View File

@ -3,7 +3,7 @@
<head> <head>
<meta charset="UTF-8"/> <meta charset="UTF-8"/>
<meta name="viewport" content="width=device-width, initial-scale=1.0"/> <meta name="viewport" content="width=device-width, initial-scale=1.0"/>
<title>Novela{% if develop_mode() %} Develop{% endif %} — {{ title or filename }}</title> <title>Novela — {{ title or filename }}</title>
<link rel="icon" href="/static/favicon.ico" sizes="16x16"/> <link rel="icon" href="/static/favicon.ico" sizes="16x16"/>
<link rel="icon" type="image/png" sizes="32x32" href="/static/favicon-32.png"/> <link rel="icon" type="image/png" sizes="32x32" href="/static/favicon-32.png"/>
<link rel="icon" type="image/png" sizes="256x256" href="/static/favicon-256.png"/> <link rel="icon" type="image/png" sizes="256x256" href="/static/favicon-256.png"/>
@ -31,9 +31,9 @@
</div> </div>
{% set r = (rating | default(0)) | int %} {% set r = (rating | default(0)) | int %}
<div class="star-row" id="book-stars"> <div class="star-row interactive" id="book-stars">
{% for i in range(1, 6) %} {% for i in range(1, 6) %}
<span class="star {% if i <= r %}filled{% endif %}"></span> <span class="star {% if i <= r %}filled{% endif %}" onclick="rateBook({{ i }})"></span>
{% endfor %} {% endfor %}
</div> </div>
@ -54,7 +54,7 @@
{% if series %} {% if series %}
<div class="meta-row"> <div class="meta-row">
<span class="meta-label">Series</span> <span class="meta-label">Series</span>
<span class="meta-value">{{ series }}{% if series_volume %} ({{ series_volume }}){% endif %}{% if series_index is defined and (series_index or series_suffix or series_is_indexed) %} [{{ series_index }}{{ series_suffix }}]{% endif %}</span> <span class="meta-value">{{ series }}{% if series_index is defined and (series_index or series_suffix or series_is_indexed) %} [{{ series_index }}{{ series_suffix }}]{% endif %}</span>
</div> </div>
{% endif %} {% endif %}
<div class="meta-row"> <div class="meta-row">
@ -155,16 +155,6 @@
Mark as unread Mark as unread
</button> </button>
{% endif %} {% endif %}
{% if storage_type == 'db' %}
<a class="btn-secondary" href="/api/library/export-epub/{{ filename | urlencode }}">
<svg width="14" height="14" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2.5">
<path d="M21 15v4a2 2 0 0 1-2 2H5a2 2 0 0 1-2-2v-4"/>
<polyline points="7 10 12 15 17 10"/>
<line x1="12" y1="15" x2="12" y2="3"/>
</svg>
Export EPUB
</a>
{% else %}
<a class="btn-secondary" href="/download/{{ filename | urlencode }}"> <a class="btn-secondary" href="/download/{{ filename | urlencode }}">
<svg width="14" height="14" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2.5"> <svg width="14" height="14" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2.5">
<path d="M21 15v4a2 2 0 0 1-2 2H5a2 2 0 0 1-2-2v-4"/> <path d="M21 15v4a2 2 0 0 1-2 2H5a2 2 0 0 1-2-2v-4"/>
@ -173,7 +163,6 @@
</svg> </svg>
Download Download
</a> </a>
{% endif %}
<button class="btn-secondary" onclick="openMarkReadModal()"> <button class="btn-secondary" onclick="openMarkReadModal()">
<svg width="14" height="14" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2.5"> <svg width="14" height="14" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2.5">
<polyline points="20 6 9 17 4 12"/> <polyline points="20 6 9 17 4 12"/>
@ -195,7 +184,7 @@
</svg> </svg>
Edit Edit
</button> </button>
{% if filename.endswith('.epub') and storage_type != 'db' %} {% if filename.endswith('.epub') %}
<a class="btn-secondary" href="/library/editor/{{ filename | urlencode }}"> <a class="btn-secondary" href="/library/editor/{{ filename | urlencode }}">
<svg width="14" height="14" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2.5"> <svg width="14" height="14" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2.5">
<polyline points="16 18 22 12 16 6"/> <polyline points="16 18 22 12 16 6"/>
@ -204,23 +193,6 @@
Edit EPUB Edit EPUB
</a> </a>
{% endif %} {% endif %}
{% if storage_type == 'db' %}
<a class="btn-secondary" href="/library/editor/{{ filename | urlencode }}">
<svg width="14" height="14" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2.5">
<path d="M11 4H4a2 2 0 0 0-2 2v14a2 2 0 0 0 2 2h14a2 2 0 0 0 2-2v-7"/>
<path d="M18.5 2.5a2.121 2.121 0 0 1 3 3L12 15l-4 1 1-4 9.5-9.5z"/>
</svg>
Edit chapters
</a>
{% endif %}
{% if filename.endswith('.epub') and storage_type != 'db' %}
<button class="btn-secondary" id="convert-db-btn" onclick="convertToDb()">
<svg width="14" height="14" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2.5">
<path d="M21 16V8a2 2 0 0 0-1-1.73l-7-4a2 2 0 0 0-2 0l-7 4A2 2 0 0 0 3 8v8a2 2 0 0 0 1 1.73l7 4a2 2 0 0 0 2 0l7-4A2 2 0 0 0 21 16z"/>
</svg>
Convert to DB
</button>
{% endif %}
<input type="file" id="cover-input" accept="image/*" style="display:none" onchange="uploadCover(this)"/> <input type="file" id="cover-input" accept="image/*" style="display:none" onchange="uploadCover(this)"/>
<button class="btn-secondary" onclick="document.getElementById('cover-input').click()"> <button class="btn-secondary" onclick="document.getElementById('cover-input').click()">
<svg width="14" height="14" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2.5"> <svg width="14" height="14" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2.5">
@ -254,13 +226,7 @@
</svg> </svg>
</button> </button>
</div> </div>
<div class="edit-field"> <div class="edit-field"><label class="edit-label">Title</label><input class="edit-input" id="ed-title" type="text"/></div>
<label class="edit-label">Title</label>
<div style="display:flex;gap:0.5rem;align-items:center">
<input class="edit-input" id="ed-title" type="text" style="flex:1"/>
<button type="button" class="btn-secondary" style="white-space:nowrap;padding:0.35rem 0.6rem;font-size:0.75rem" onclick="generateTitle()" title="Generate title from series info">Auto</button>
</div>
</div>
<div class="edit-field"> <div class="edit-field">
<label class="edit-label">Author</label> <label class="edit-label">Author</label>
<div class="genre-wrap"> <div class="genre-wrap">
@ -283,8 +249,7 @@
<div class="genre-dropdown" id="series-dropdown" style="display:none"></div> <div class="genre-dropdown" id="series-dropdown" style="display:none"></div>
</div> </div>
</div> </div>
<div class="edit-field"><label class="edit-label">Year/Volume</label><input class="edit-input" id="ed-series-volume" type="text" placeholder="e.g. 2024"/></div> <div class="edit-field"><label class="edit-label">Volume</label><input class="edit-input" id="ed-series-index" type="text" placeholder="e.g. 1 or 21a"/></div>
<div class="edit-field"><label class="edit-label">Number</label><input class="edit-input" id="ed-series-index" type="text" placeholder="e.g. 1 or 21a"/></div>
</div> </div>
<div class="edit-field"> <div class="edit-field">
<label class="edit-label">Status</label> <label class="edit-label">Status</label>
@ -296,17 +261,6 @@
<option value="Long-Term Hold">Long-Term Hold</option> <option value="Long-Term Hold">Long-Term Hold</option>
</select> </select>
</div> </div>
<div class="edit-field">
<label class="edit-label">Rating</label>
<select class="edit-select" id="ed-rating">
<option value="0">— No rating</option>
<option value="1">★ 1</option>
<option value="2">★★ 2</option>
<option value="3">★★★ 3</option>
<option value="4">★★★★ 4</option>
<option value="5">★★★★★ 5</option>
</select>
</div>
<div class="edit-field"><label class="edit-label">Source URL</label><input class="edit-input" id="ed-url" type="url" placeholder="https://…"/></div> <div class="edit-field"><label class="edit-label">Source URL</label><input class="edit-input" id="ed-url" type="url" placeholder="https://…"/></div>
<div class="edit-field"><label class="edit-label">Description</label><textarea class="edit-input edit-textarea" id="ed-description" rows="5" placeholder="Story description…"></textarea></div> <div class="edit-field"><label class="edit-label">Description</label><textarea class="edit-input edit-textarea" id="ed-description" rows="5" placeholder="Story description…"></textarea></div>
<div class="edit-field"><label class="edit-label">Updated</label><input class="edit-input" id="ed-publish-date" type="date" style="color-scheme:dark"/></div> <div class="edit-field"><label class="edit-label">Updated</label><input class="edit-input" id="ed-publish-date" type="date" style="color-scheme:dark"/></div>
@ -367,7 +321,7 @@
<div class="modal-backdrop" id="delete-modal"> <div class="modal-backdrop" id="delete-modal">
<div class="modal"> <div class="modal">
<h3>Delete book</h3> <h3>Delete book</h3>
<p>{% if storage_type == 'db' %}This will permanently delete the book and all its chapters from the database for{% else %}This will permanently delete the file and all reading progress for{% endif %} <strong id="delete-title"></strong>. This cannot be undone.</p> <p>This will permanently delete the EPUB file and all reading progress for <strong id="delete-title"></strong>. This cannot be undone.</p>
<div class="modal-actions"> <div class="modal-actions">
<button class="btn-secondary" onclick="document.getElementById('delete-modal').classList.remove('open')">Cancel</button> <button class="btn-secondary" onclick="document.getElementById('delete-modal').classList.remove('open')">Cancel</button>
<button class="btn-danger" onclick="confirmDelete()">Delete</button> <button class="btn-danger" onclick="confirmDelete()">Delete</button>
@ -376,8 +330,6 @@
</div> </div>
<script> <script>
const STORAGE_TYPE = {{ storage_type | tojson }};
const BOOK = { const BOOK = {
filename: {{ filename | tojson }}, filename: {{ filename | tojson }},
title: {{ (title or filename) | tojson }}, title: {{ (title or filename) | tojson }},
@ -386,7 +338,6 @@
series: {{ (series or '') | tojson }}, series: {{ (series or '') | tojson }},
series_index: {{ series_index or 0 }}, series_index: {{ series_index or 0 }},
series_suffix: {{ (series_suffix or '') | tojson }}, series_suffix: {{ (series_suffix or '') | tojson }},
series_volume: {{ (series_volume or '') | tojson }},
publication_status: {{ (publication_status or '') | tojson }}, publication_status: {{ (publication_status or '') | tojson }},
source_url: {{ (source_url or '') | tojson }}, source_url: {{ (source_url or '') | tojson }},
publish_date: {{ (publish_date or '') | tojson }}, publish_date: {{ (publish_date or '') | tojson }},
@ -398,29 +349,6 @@
rating: {{ rating or 0 }}, rating: {{ rating or 0 }},
}; };
</script> </script>
<script>
async function convertToDb() {
const btn = document.getElementById('convert-db-btn');
if (!confirm('Convert this EPUB to DB storage?\nThe EPUB file will be deleted from disk.\nAll reading progress, bookmarks and ratings are preserved.')) return;
btn.disabled = true;
btn.textContent = 'Converting…';
try {
const resp = await fetch('/api/library/convert-to-db/' + encodeURIComponent(BOOK.filename), { method: 'POST' });
const data = await resp.json();
if (data.ok) {
window.location.href = '/library/book/' + encodeURIComponent(data.new_filename);
} else {
alert('Conversion failed: ' + (data.error || 'unknown error'));
btn.disabled = false;
btn.textContent = 'Convert to DB';
}
} catch (e) {
alert('Conversion failed: ' + e);
btn.disabled = false;
btn.textContent = 'Convert to DB';
}
}
</script>
<script src="/static/books.js"></script> <script src="/static/books.js"></script>
<script src="/static/book.js"></script> <script src="/static/book.js"></script>
</body> </body>

View File

@ -3,7 +3,7 @@
<head> <head>
<meta charset="UTF-8"/> <meta charset="UTF-8"/>
<meta name="viewport" content="width=device-width, initial-scale=1.0"/> <meta name="viewport" content="width=device-width, initial-scale=1.0"/>
<title>Novela{% if develop_mode() %} Develop{% endif %} — Book Builder{% if view == 'editor' %}: {{ draft.title }}{% endif %}</title> <title>Novela — Book Builder{% if view == 'editor' %}: {{ draft.title }}{% endif %}</title>
<link rel="icon" href="/static/favicon.ico" sizes="16x16"/> <link rel="icon" href="/static/favicon.ico" sizes="16x16"/>
<link rel="icon" type="image/png" sizes="32x32" href="/static/favicon-32.png"/> <link rel="icon" type="image/png" sizes="32x32" href="/static/favicon-32.png"/>
<link rel="icon" type="image/png" sizes="256x256" href="/static/favicon-256.png"/> <link rel="icon" type="image/png" sizes="256x256" href="/static/favicon-256.png"/>

View File

@ -3,7 +3,7 @@
<head> <head>
<meta charset="UTF-8"/> <meta charset="UTF-8"/>
<meta name="viewport" content="width=device-width, initial-scale=1.0"/> <meta name="viewport" content="width=device-width, initial-scale=1.0"/>
<title>Novela{% if develop_mode() %} Develop{% endif %} — Bulk Import</title> <title>Novela Bulk Import</title>
<link rel="icon" href="/static/favicon.ico" sizes="16x16"/> <link rel="icon" href="/static/favicon.ico" sizes="16x16"/>
<link rel="icon" type="image/png" sizes="32x32" href="/static/favicon-32.png"/> <link rel="icon" type="image/png" sizes="32x32" href="/static/favicon-32.png"/>
<link rel="icon" type="image/png" sizes="256x256" href="/static/favicon-256.png"/> <link rel="icon" type="image/png" sizes="256x256" href="/static/favicon-256.png"/>
@ -189,7 +189,7 @@
td.td-skip input[type="checkbox"] { cursor: pointer; accent-color: var(--error); } td.td-skip input[type="checkbox"] { cursor: pointer; accent-color: var(--error); }
tbody tr.row-dup { background: rgba(200,90,58,0.06); } tbody tr.row-dup { background: rgba(200,90,58,0.06); }
tbody tr.row-dup:hover { background: rgba(200,90,58,0.10); } tbody tr.row-dup:hover { background: rgba(200,90,58,0.10); }
tbody tr.row-skipped { opacity: 0.38; } tbody tr.row-dup.row-skipped { opacity: 0.38; }
.cnt-dup { color: var(--error); } .cnt-dup { color: var(--error); }
.dup-actions { .dup-actions {
display: flex; gap: 0.5rem; align-items: center; display: flex; gap: 0.5rem; align-items: center;
@ -322,10 +322,6 @@
<div class="suggest-dropdown" id="series-dropdown" style="display:none"></div> <div class="suggest-dropdown" id="series-dropdown" style="display:none"></div>
</div> </div>
</div> </div>
<div>
<label>Year/Vol. <span style="color:var(--text-dim)">(series volume)</span></label>
<input type="text" id="shared-series-volume" autocomplete="off" oninput="updatePreview()" placeholder="e.g. 2024"/>
</div>
<div> <div>
<label>Status</label> <label>Status</label>
<select id="shared-status" oninput="updatePreview()"> <select id="shared-status" oninput="updatePreview()">
@ -344,14 +340,6 @@
<label>Tags <span style="color:var(--text-dim)">(comma-separated)</span></label> <label>Tags <span style="color:var(--text-dim)">(comma-separated)</span></label>
<input type="text" id="shared-tags"/> <input type="text" id="shared-tags"/>
<label style="display:flex;align-items:center;gap:0.5rem;margin-top:0.5rem;cursor:pointer">
<input type="checkbox" id="auto-title" oninput="updatePreview()"/>
Auto-generate titles from series info &mdash; <span style="color:var(--text-dim)">for comics without individual titles</span>
</label>
<p class="hint" id="auto-title-hint" style="display:none">
Format: <code>Series (Year/Vol) #Number</code>. Only fills in rows where title is empty or matches the filename stem.
</p>
</div> </div>
<!-- Card 3: Files --> <!-- Card 3: Files -->
@ -385,13 +373,12 @@
<th style="width:2.5rem">#</th> <th style="width:2.5rem">#</th>
<th>Filename</th> <th>Filename</th>
<th>Series</th> <th>Series</th>
<th>Yr/Vol</th>
<th>Vol</th> <th>Vol</th>
<th>Title</th> <th>Title</th>
<th>Author</th> <th>Author</th>
<th>Publisher</th> <th>Publisher</th>
<th>Year</th> <th>Year</th>
<th style="width:2.5rem" title="Skip this file during import">Skip</th> <th id="th-skip" style="width:2.5rem;display:none" title="Skip this file during import">Skip</th>
<th style="width:1.5rem"></th> <th style="width:1.5rem"></th>
</tr> </tr>
</thead> </thead>
@ -446,7 +433,6 @@
// ── State ────────────────────────────────────────────────────────────────── // ── State ──────────────────────────────────────────────────────────────────
const PLACEHOLDER_META = [ const PLACEHOLDER_META = [
{ key: 'series', label: '%series%', color: 'var(--accent)' }, { key: 'series', label: '%series%', color: 'var(--accent)' },
{ key: 'series_volume', label: '%series_volume%', color: '#d07840' },
{ key: 'volume', label: '%volume%', color: '#4a90b8' }, { key: 'volume', label: '%volume%', color: '#4a90b8' },
{ key: 'title', label: '%title%', color: 'var(--success)' }, { key: 'title', label: '%title%', color: 'var(--success)' },
{ key: 'year', label: '%year%', color: 'var(--warning)' }, { key: 'year', label: '%year%', color: 'var(--warning)' },
@ -458,7 +444,7 @@
]; ];
let selectedFiles = []; let selectedFiles = [];
let parsedRows = []; // [{original_filename, series, series_volume, volume, title, year, author, publisher, status, genres, tags, _warn}] let parsedRows = []; // [{original_filename, series, volume, title, year, author, publisher, status, genres, tags, _warn}]
const BATCH_SIZE = 5; const BATCH_SIZE = 5;
@ -602,30 +588,14 @@
const pattern = document.getElementById('pattern-input').value; const pattern = document.getElementById('pattern-input').value;
const sharedSeries = document.getElementById('shared-series').value.trim(); const sharedSeries = document.getElementById('shared-series').value.trim();
const sharedSeriesVolume = document.getElementById('shared-series-volume').value.trim();
const autoTitle = document.getElementById('auto-title').checked;
document.getElementById('auto-title-hint').style.display = autoTitle ? '' : 'none';
parsedRows = selectedFiles.map(f => { parsedRows = selectedFiles.map(f => {
const stem = f.name.replace(/\.[^.]+$/, ''); const stem = f.name.replace(/\.[^.]+$/, '');
const parsed = parseFilename(stem, pattern); const parsed = parseFilename(stem, pattern);
const series = sharedSeries || parsed.series || '';
const series_volume = sharedSeriesVolume || parsed.series_volume || '';
const volume = parsed.volume || '';
let title = parsed.title || '';
if (autoTitle && series && !title) {
title = series;
if (series_volume) title += ` (${series_volume})`;
if (volume) title += ` #${volume}`;
}
return { return {
original_filename: f.name, original_filename: f.name,
series, series: sharedSeries || parsed.series || '',
series_volume, volume: parsed.volume || '',
volume, title: parsed.title || stem,
title: title || stem,
year: parsed.year || '', year: parsed.year || '',
author: parsed.author || '', author: parsed.author || '',
publisher: parsed.publisher || '', publisher: parsed.publisher || '',
@ -648,7 +618,6 @@
const items = parsedRows.map(r => ({ const items = parsedRows.map(r => ({
title: r.title, title: r.title,
author: r.author || sharedAuthor, author: r.author || sharedAuthor,
series: r.series,
volume: r.volume, volume: r.volume,
})); }));
try { try {
@ -677,6 +646,8 @@
tbody.innerHTML = ''; tbody.innerHTML = '';
const hasDups = parsedRows.some(r => r._duplicate); const hasDups = parsedRows.some(r => r._duplicate);
const thSkip = document.getElementById('th-skip');
if (thSkip) thSkip.style.display = hasDups ? '' : 'none';
let warnCount = 0; let warnCount = 0;
let dupCount = 0; let dupCount = 0;
@ -691,7 +662,7 @@
const classes = []; const classes = [];
if (row._warn) classes.push('row-warn'); if (row._warn) classes.push('row-warn');
if (row._duplicate) classes.push('row-dup'); if (row._duplicate) classes.push('row-dup');
if (row._skip) classes.push('row-skipped'); if (row._duplicate && row._skip) classes.push('row-skipped');
if (classes.length) tr.className = classes.join(' '); if (classes.length) tr.className = classes.join(' ');
// # // #
@ -708,10 +679,8 @@
tr.appendChild(tdFn); tr.appendChild(tdFn);
// Editable fields // Editable fields
const sharedSeriesVolume = document.getElementById('shared-series-volume').value.trim();
const fields = [ const fields = [
{ key: 'series', placeholder: '—' }, { key: 'series', placeholder: '—' },
{ key: 'series_volume', placeholder: sharedSeriesVolume || '—' },
{ key: 'volume', placeholder: '—' }, { key: 'volume', placeholder: '—' },
{ key: 'title', placeholder: 'Title' }, { key: 'title', placeholder: 'Title' },
{ key: 'author', placeholder: sharedAuthor || '—' }, { key: 'author', placeholder: sharedAuthor || '—' },
@ -740,9 +709,11 @@
tr.appendChild(td); tr.appendChild(td);
}); });
// Skip checkbox (always visible) // Skip checkbox (only shown when duplicates exist)
const tdSkip = document.createElement('td'); const tdSkip = document.createElement('td');
tdSkip.className = 'td-skip'; tdSkip.className = 'td-skip';
tdSkip.style.display = hasDups ? '' : 'none';
if (row._duplicate) {
const cb = document.createElement('input'); const cb = document.createElement('input');
cb.type = 'checkbox'; cb.type = 'checkbox';
cb.checked = row._skip; cb.checked = row._skip;
@ -753,6 +724,7 @@
renderPreviewStats(); renderPreviewStats();
}); });
tdSkip.appendChild(cb); tdSkip.appendChild(cb);
}
tr.appendChild(tdSkip); tr.appendChild(tdSkip);
// Warning indicator // Warning indicator
@ -778,9 +750,14 @@
const statsEl = document.getElementById('preview-stats'); const statsEl = document.getElementById('preview-stats');
let stats = `<span class="cnt-ok">${importCount} to import</span>`; let stats = `<span class="cnt-ok">${importCount} to import</span>`;
if (skipCount) stats += ` &nbsp; <span class="cnt-warn">${skipCount} skipped</span>`;
if (warnCount) stats += ` &nbsp; <span class="cnt-warn">${warnCount} to check</span>`; if (warnCount) stats += ` &nbsp; <span class="cnt-warn">${warnCount} to check</span>`;
if (dupCount) stats += ` &nbsp; <span class="cnt-dup">${dupCount} duplicate${dupCount !== 1 ? 's' : ''}</span>`; if (dupCount) {
stats += ` &nbsp; <span class="cnt-dup">${dupCount} duplicate${dupCount !== 1 ? 's' : ''}</span>`;
stats += ` &nbsp; <span class="dup-actions">`;
stats += `<button onclick="setAllDuplicatesSkip(true)">Skip all</button>`;
stats += `<button onclick="setAllDuplicatesSkip(false)">Import all</button>`;
stats += `</span>`;
}
statsEl.innerHTML = stats; statsEl.innerHTML = stats;
document.getElementById('import-btn-label').textContent = document.getElementById('import-btn-label').textContent =
@ -800,7 +777,6 @@
author: document.getElementById('shared-author').value.trim(), author: document.getElementById('shared-author').value.trim(),
publisher: document.getElementById('shared-publisher').value.trim(), publisher: document.getElementById('shared-publisher').value.trim(),
series: document.getElementById('shared-series').value.trim(), series: document.getElementById('shared-series').value.trim(),
series_volume: document.getElementById('shared-series-volume').value.trim(),
status: document.getElementById('shared-status').value, status: document.getElementById('shared-status').value,
genres: document.getElementById('shared-genres').value.trim(), genres: document.getElementById('shared-genres').value.trim(),
tags: document.getElementById('shared-tags').value.trim(), tags: document.getElementById('shared-tags').value.trim(),

View File

@ -3,7 +3,7 @@
<head> <head>
<meta charset="UTF-8"/> <meta charset="UTF-8"/>
<meta name="viewport" content="width=device-width, initial-scale=1.0"/> <meta name="viewport" content="width=device-width, initial-scale=1.0"/>
<title>Novela{% if develop_mode() %} Develop{% endif %} — Changelog</title> <title>Novela — Changelog</title>
<link rel="icon" href="/static/favicon.ico" sizes="16x16"/> <link rel="icon" href="/static/favicon.ico" sizes="16x16"/>
<link rel="icon" type="image/png" sizes="32x32" href="/static/favicon-32.png"/> <link rel="icon" type="image/png" sizes="32x32" href="/static/favicon-32.png"/>
<link rel="icon" type="image/png" sizes="256x256" href="/static/favicon-256.png"/> <link rel="icon" type="image/png" sizes="256x256" href="/static/favicon-256.png"/>

View File

@ -3,7 +3,7 @@
<head> <head>
<meta charset="UTF-8"/> <meta charset="UTF-8"/>
<meta name="viewport" content="width=device-width, initial-scale=1.0"/> <meta name="viewport" content="width=device-width, initial-scale=1.0"/>
<title>Novela{% if develop_mode() %} Develop{% endif %} — Credentials</title> <title>Novela — Credentials</title>
<link rel="icon" href="/static/favicon.ico" sizes="16x16"/> <link rel="icon" href="/static/favicon.ico" sizes="16x16"/>
<link rel="icon" type="image/png" sizes="32x32" href="/static/favicon-32.png"/> <link rel="icon" type="image/png" sizes="32x32" href="/static/favicon-32.png"/>
<link rel="icon" type="image/png" sizes="256x256" href="/static/favicon-256.png"/> <link rel="icon" type="image/png" sizes="256x256" href="/static/favicon-256.png"/>

View File

@ -3,7 +3,7 @@
<head> <head>
<meta charset="UTF-8"/> <meta charset="UTF-8"/>
<meta name="viewport" content="width=device-width, initial-scale=1.0"/> <meta name="viewport" content="width=device-width, initial-scale=1.0"/>
<title>Novela{% if develop_mode() %} Develop{% endif %} — Debug</title> <title>Novela — Debug</title>
<link rel="icon" href="/static/favicon.ico" sizes="16x16"/> <link rel="icon" href="/static/favicon.ico" sizes="16x16"/>
<link rel="icon" type="image/png" sizes="32x32" href="/static/favicon-32.png"/> <link rel="icon" type="image/png" sizes="32x32" href="/static/favicon-32.png"/>
<link rel="icon" type="image/png" sizes="256x256" href="/static/favicon-256.png"/> <link rel="icon" type="image/png" sizes="256x256" href="/static/favicon-256.png"/>

View File

@ -3,7 +3,7 @@
<head> <head>
<meta charset="UTF-8"/> <meta charset="UTF-8"/>
<meta name="viewport" content="width=device-width, initial-scale=1.0"/> <meta name="viewport" content="width=device-width, initial-scale=1.0"/>
<title>Novela{% if develop_mode() %} Develop{% endif %} — Edit {{ title or filename }}</title> <title>Novela — Edit {{ title or filename }}</title>
<link rel="icon" href="/static/favicon.ico" sizes="16x16"/> <link rel="icon" href="/static/favicon.ico" sizes="16x16"/>
<link rel="icon" type="image/png" sizes="32x32" href="/static/favicon-32.png"/> <link rel="icon" type="image/png" sizes="32x32" href="/static/favicon-32.png"/>
<link rel="icon" type="image/png" sizes="256x256" href="/static/favicon-256.png"/> <link rel="icon" type="image/png" sizes="256x256" href="/static/favicon-256.png"/>
@ -24,7 +24,6 @@
{{ (title or filename) | truncate(30, True) }} {{ (title or filename) | truncate(30, True) }}
</a> </a>
<div class="header-chapter" id="header-chapter"></div> <div class="header-chapter" id="header-chapter"></div>
<input class="chapter-title-input" id="chapter-title-input" type="text" placeholder="Chapter title…" style="display:none"/>
<div class="header-actions"> <div class="header-actions">
<button class="btn-add-page" id="btn-add-page" onclick="addChapter()" title="Add new chapter after current"> <button class="btn-add-page" id="btn-add-page" onclick="addChapter()" title="Add new chapter after current">
<svg width="13" height="13" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2"> <svg width="13" height="13" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2">
@ -95,7 +94,6 @@
const EDITOR = { const EDITOR = {
filename: {{ filename | tojson }}, filename: {{ filename | tojson }},
title: {{ (title or filename) | tojson }}, title: {{ (title or filename) | tojson }},
is_db: {{ is_db | tojson }},
}; };
</script> </script>
<script src="https://cdn.jsdelivr.net/npm/monaco-editor@0.45.0/min/vs/loader.js"></script> <script src="https://cdn.jsdelivr.net/npm/monaco-editor@0.45.0/min/vs/loader.js"></script>

View File

@ -3,7 +3,7 @@
<head> <head>
<meta charset="UTF-8"/> <meta charset="UTF-8"/>
<meta name="viewport" content="width=device-width, initial-scale=1.0"/> <meta name="viewport" content="width=device-width, initial-scale=1.0"/>
<title>Novela{% if develop_mode() %} Develop{% endif %} — Following</title> <title>Novela — Following</title>
<link rel="icon" href="/static/favicon.ico" sizes="16x16"/> <link rel="icon" href="/static/favicon.ico" sizes="16x16"/>
<link rel="icon" type="image/png" sizes="32x32" href="/static/favicon-32.png"/> <link rel="icon" type="image/png" sizes="32x32" href="/static/favicon-32.png"/>
<link rel="icon" type="image/png" sizes="256x256" href="/static/favicon-256.png"/> <link rel="icon" type="image/png" sizes="256x256" href="/static/favicon-256.png"/>

View File

@ -3,7 +3,7 @@
<head> <head>
<meta charset="UTF-8"/> <meta charset="UTF-8"/>
<meta name="viewport" content="width=device-width, initial-scale=1.0"/> <meta name="viewport" content="width=device-width, initial-scale=1.0"/>
<title>Novela{% if develop_mode() %} Develop{% endif %}</title> <title>Novela</title>
<link rel="icon" href="/static/favicon.ico" sizes="16x16"/> <link rel="icon" href="/static/favicon.ico" sizes="16x16"/>
<link rel="icon" type="image/png" sizes="32x32" href="/static/favicon-32.png"/> <link rel="icon" type="image/png" sizes="32x32" href="/static/favicon-32.png"/>
<link rel="icon" type="image/png" sizes="256x256" href="/static/favicon-256.png"/> <link rel="icon" type="image/png" sizes="256x256" href="/static/favicon-256.png"/>
@ -214,23 +214,6 @@
} }
.btn-outline:hover { background: var(--surface); color: var(--text); border-color: var(--text-faint); } .btn-outline:hover { background: var(--surface); color: var(--text); border-color: var(--text-faint); }
.storage-toggle {
display: flex; align-items: center; gap: 0.5rem;
margin-bottom: 1rem;
}
.storage-label {
font-family: var(--mono); font-size: 0.75rem; color: var(--text-dim);
}
.storage-opt {
width: auto; padding: 0.3rem 0.75rem;
background: var(--surface2); color: var(--text-dim);
border: 1px solid var(--border); font-size: 0.75rem;
}
.storage-opt:first-of-type { border-radius: var(--radius) 0 0 var(--radius); }
.storage-opt:last-of-type { border-radius: 0 var(--radius) var(--radius) 0; }
.storage-opt.active { background: var(--accent); color: #0f0e0c; border-color: var(--accent); }
.storage-opt:hover:not(.active) { background: var(--surface); color: var(--text); }
.dup-warning { .dup-warning {
display: none; width: 100%; max-width: 620px; margin-bottom: 1.5rem; display: none; width: 100%; max-width: 620px; margin-bottom: 1.5rem;
background: rgba(200,160,58,0.08); border: 1px solid rgba(200,160,58,0.35); background: rgba(200,160,58,0.08); border: 1px solid rgba(200,160,58,0.35);
@ -285,12 +268,6 @@
<div class="cover-filename" id="cover-filename"></div> <div class="cover-filename" id="cover-filename"></div>
</div> </div>
<div class="storage-toggle">
<span class="storage-label">Save as</span>
<button type="button" class="storage-opt active" id="opt-db" onclick="setStorage('db')">DB</button>
<button type="button" class="storage-opt" id="opt-epub" onclick="setStorage('epub')">EPUB file</button>
</div>
<button id="convert-btn" onclick="startConvert()"> <button id="convert-btn" onclick="startConvert()">
<svg width="14" height="14" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2.5"> <svg width="14" height="14" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2.5">
<path d="M5 12h14M12 5l7 7-7 7"/> <path d="M5 12h14M12 5l7 7-7 7"/>
@ -320,7 +297,7 @@
<svg width="14" height="14" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2.5"> <svg width="14" height="14" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2.5">
<path d="M21 15v4a2 2 0 01-2 2H5a2 2 0 01-2-2v-4M7 10l5 5 5-5M12 15V3"/> <path d="M21 15v4a2 2 0 01-2 2H5a2 2 0 01-2-2v-4M7 10l5 5 5-5M12 15V3"/>
</svg> </svg>
<span>Download EPUB</span> Download EPUB
</button> </button>
<button class="btn-outline" id="book-detail-btn"> <button class="btn-outline" id="book-detail-btn">
<svg width="14" height="14" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2.5"> <svg width="14" height="14" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2.5">
@ -338,13 +315,6 @@
<script> <script>
let currentUrl = ''; let currentUrl = '';
let coverB64 = null; let coverB64 = null;
let storageMode = 'db';
function setStorage(mode) {
storageMode = mode;
document.getElementById('opt-db').classList.toggle('active', mode === 'db');
document.getElementById('opt-epub').classList.toggle('active', mode === 'epub');
}
// --- Credential status --- // --- Credential status ---
async function checkUrlCredentials() { async function checkUrlCredentials() {
@ -488,7 +458,7 @@
document.getElementById('log-lines').innerHTML = ''; document.getElementById('log-lines').innerHTML = '';
document.getElementById('progress-bar').style.width = '0%'; document.getElementById('progress-bar').style.width = '0%';
const body = { url: currentUrl, storage_mode: storageMode }; const body = { url: currentUrl };
if (coverB64) body.cover_b64 = coverB64; if (coverB64) body.cover_b64 = coverB64;
const seriesInput = document.getElementById('series-index-input'); const seriesInput = document.getElementById('series-index-input');
if (seriesInput) body.series_index = parseInt(seriesInput.value) || 1; if (seriesInput) body.series_index = parseInt(seriesInput.value) || 1;

View File

@ -3,7 +3,7 @@
<head> <head>
<meta charset="UTF-8"/> <meta charset="UTF-8"/>
<meta name="viewport" content="width=device-width, initial-scale=1.0"/> <meta name="viewport" content="width=device-width, initial-scale=1.0"/>
<title>Novela{% if develop_mode() %} Develop{% endif %} — Home</title> <title>Novela — Home</title>
<link rel="icon" href="/static/favicon.ico" sizes="16x16"/> <link rel="icon" href="/static/favicon.ico" sizes="16x16"/>
<link rel="icon" type="image/png" sizes="32x32" href="/static/favicon-32.png"/> <link rel="icon" type="image/png" sizes="32x32" href="/static/favicon-32.png"/>
<link rel="icon" type="image/png" sizes="256x256" href="/static/favicon-256.png"/> <link rel="icon" type="image/png" sizes="256x256" href="/static/favicon-256.png"/>

View File

@ -3,7 +3,7 @@
<head> <head>
<meta charset="UTF-8"/> <meta charset="UTF-8"/>
<meta name="viewport" content="width=device-width, initial-scale=1.0"/> <meta name="viewport" content="width=device-width, initial-scale=1.0"/>
<title>Novela{% if develop_mode() %} Develop{% endif %}</title> <title>Novela</title>
<link rel="icon" href="/static/favicon.ico" sizes="16x16"/> <link rel="icon" href="/static/favicon.ico" sizes="16x16"/>
<link rel="icon" type="image/png" sizes="32x32" href="/static/favicon-32.png"/> <link rel="icon" type="image/png" sizes="32x32" href="/static/favicon-32.png"/>
<link rel="icon" type="image/png" sizes="256x256" href="/static/favicon-256.png"/> <link rel="icon" type="image/png" sizes="256x256" href="/static/favicon-256.png"/>

View File

@ -3,7 +3,7 @@
<head> <head>
<meta charset="UTF-8"/> <meta charset="UTF-8"/>
<meta name="viewport" content="width=device-width, initial-scale=1.0"/> <meta name="viewport" content="width=device-width, initial-scale=1.0"/>
<title>Novela{% if develop_mode() %} Develop{% endif %} — Library</title> <title>Novela — Library</title>
<link rel="icon" href="/static/favicon.ico" sizes="16x16"/> <link rel="icon" href="/static/favicon.ico" sizes="16x16"/>
<link rel="icon" type="image/png" sizes="32x32" href="/static/favicon-32.png"/> <link rel="icon" type="image/png" sizes="32x32" href="/static/favicon-32.png"/>
<link rel="icon" type="image/png" sizes="256x256" href="/static/favicon-256.png"/> <link rel="icon" type="image/png" sizes="256x256" href="/static/favicon-256.png"/>

View File

@ -3,7 +3,7 @@
<head> <head>
<meta charset="UTF-8"/> <meta charset="UTF-8"/>
<meta name="viewport" content="width=device-width, initial-scale=1.0"/> <meta name="viewport" content="width=device-width, initial-scale=1.0"/>
<title>Novela{% if develop_mode() %} Develop{% endif %} — {{ title }}</title> <title>Novela — {{ title }}</title>
<link rel="icon" href="/static/favicon.ico" sizes="16x16"/> <link rel="icon" href="/static/favicon.ico" sizes="16x16"/>
<link rel="icon" type="image/png" sizes="32x32" href="/static/favicon-32.png"/> <link rel="icon" type="image/png" sizes="32x32" href="/static/favicon-32.png"/>
<link rel="icon" type="image/png" sizes="256x256" href="/static/favicon-256.png"/> <link rel="icon" type="image/png" sizes="256x256" href="/static/favicon-256.png"/>
@ -69,16 +69,6 @@
.btn-header-read:hover { background: rgba(107,170,107,0.08); border-color: var(--success); } .btn-header-read:hover { background: rgba(107,170,107,0.08); border-color: var(--success); }
.btn-header-bm { color: var(--accent); border-color: rgba(255,162,14,0.3); } .btn-header-bm { color: var(--accent); border-color: rgba(255,162,14,0.3); }
.btn-header-bm:hover { background: rgba(255,162,14,0.08); border-color: var(--accent); } .btn-header-bm:hover { background: rgba(255,162,14,0.08); border-color: var(--accent); }
.btn-header-series {
display: none;
color: var(--text-faint); border-color: rgba(255,255,255,0.08);
padding: 0.3rem 0.5rem;
}
.btn-header-series.active {
display: flex;
color: var(--text-dim); border-color: var(--border);
}
.btn-header-series.active:hover { color: var(--text); border-color: var(--text-faint); }
/* ── Bookmark modal ── */ /* ── Bookmark modal ── */
.bm-overlay { .bm-overlay {
@ -197,7 +187,7 @@
/* Chapter content */ /* Chapter content */
#chapter-content { #chapter-content {
font-family: var(--serif); font-family: var(--serif);
font-size: var(--reader-font-size, 1.05rem); font-size: 1.05rem;
line-height: 1.85; line-height: 1.85;
color: var(--text); color: var(--text);
} }
@ -320,14 +310,6 @@
<input type="range" id="width-slider" min="30" max="100" step="1" <input type="range" id="width-slider" min="30" max="100" step="1"
value="65" oninput="applyWidth(this.value)"/> value="65" oninput="applyWidth(this.value)"/>
</div> </div>
<div class="settings-row">
<div class="settings-label">
Font size
<span id="fontsize-value">105%</span>
</div>
<input type="range" id="fontsize-slider" min="80" max="150" step="1"
value="105" oninput="applyFontSize(this.value)"/>
</div>
<div class="settings-row"> <div class="settings-row">
<div class="settings-label">Text colour</div> <div class="settings-label">Text colour</div>
<div class="colour-swatches"> <div class="colour-swatches">
@ -357,18 +339,6 @@
</a> </a>
<div class="header-title" id="header-title"></div> <div class="header-title" id="header-title"></div>
<div class="header-actions"> <div class="header-actions">
<button class="btn-header btn-header-series" id="btn-series-prev" title="" onclick="goSeriesPrev()">
<svg width="13" height="13" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2.5">
<polyline points="19 20 9 12 19 4"/>
<line x1="5" y1="4" x2="5" y2="20"/>
</svg>
</button>
<button class="btn-header btn-header-series" id="btn-series-next" title="" onclick="goSeriesNext()">
<svg width="13" height="13" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2.5">
<polyline points="5 4 15 12 5 20"/>
<line x1="19" y1="4" x2="19" y2="20"/>
</svg>
</button>
<button class="btn-header btn-header-bm" onclick="openBookmarkModal()" title="Add bookmark at current position"> <button class="btn-header btn-header-bm" onclick="openBookmarkModal()" title="Add bookmark at current position">
<svg width="12" height="12" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2"> <svg width="12" height="12" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2">
<path d="M19 21l-7-5-7 5V5a2 2 0 0 1 2-2h10a2 2 0 0 1 2 2z"/> <path d="M19 21l-7-5-7 5V5a2 2 0 0 1 2-2h10a2 2 0 0 1 2 2z"/>
@ -421,7 +391,6 @@
let currentIndex = 0; let currentIndex = 0;
let saveTimer = null; let saveTimer = null;
let scrollTimer = null; let scrollTimer = null;
let seriesNav = { prev: null, next: null };
// ── Width setting ────────────────────────────────────────────── // ── Width setting ──────────────────────────────────────────────
function applyWidth(pct) { function applyWidth(pct) {
@ -451,20 +420,6 @@
applyTextColour(saved); applyTextColour(saved);
} }
// ── Font size ──────────────────────────────────────────────────
function applyFontSize(pct) {
const val = parseInt(pct, 10);
document.documentElement.style.setProperty('--reader-font-size', (val / 100) + 'rem');
document.getElementById('fontsize-value').textContent = val + '%';
document.getElementById('fontsize-slider').value = val;
localStorage.setItem('reader-font-size', val);
}
function loadFontSize() {
const saved = parseInt(localStorage.getItem('reader-font-size') || '105', 10);
applyFontSize(saved);
}
// ── Settings drawer ──────────────────────────────────────────── // ── Settings drawer ────────────────────────────────────────────
function toggleSettings() { function toggleSettings() {
const open = document.getElementById('settings-drawer').classList.toggle('open'); const open = document.getElementById('settings-drawer').classList.toggle('open');
@ -625,8 +580,6 @@
async function init() { async function init() {
loadWidth(); loadWidth();
loadTextColour(); loadTextColour();
loadFontSize();
loadSeriesNav();
const progResp = await fetch(`/library/progress/${encodeURIComponent(filename)}`); const progResp = await fetch(`/library/progress/${encodeURIComponent(filename)}`);
const prog = await progResp.json(); const prog = await progResp.json();
@ -673,45 +626,11 @@
document.getElementById('loading').style.display = 'none'; document.getElementById('loading').style.display = 'none';
} }
// ── Series navigation ──────────────────────────────────────────
async function loadSeriesNav() {
try {
const resp = await fetch(`/api/series-nav/${encodeURIComponent(filename)}`);
seriesNav = await resp.json();
} catch { return; }
const btnPrev = document.getElementById('btn-series-prev');
const btnNext = document.getElementById('btn-series-next');
if (seriesNav.prev) {
const label = seriesNav.prev.index ? `#${seriesNav.prev.index}${seriesNav.prev.suffix} ${seriesNav.prev.title}` : seriesNav.prev.title;
btnPrev.title = label;
btnPrev.classList.add('active');
}
if (seriesNav.next) {
const label = seriesNav.next.index ? `#${seriesNav.next.index}${seriesNav.next.suffix} ${seriesNav.next.title}` : seriesNav.next.title;
btnNext.title = label;
btnNext.classList.add('active');
}
}
function goSeriesPrev() {
if (seriesNav.prev) window.location.href = `/library/read/${encodeURIComponent(seriesNav.prev.filename)}`;
}
function goSeriesNext() {
if (seriesNav.next) window.location.href = `/library/read/${encodeURIComponent(seriesNav.next.filename)}`;
}
async function markRead() { async function markRead() {
clearTimeout(saveTimer); clearTimeout(saveTimer);
await fetch(`/library/mark-read/${encodeURIComponent(filename)}`, { method: 'POST' }); await fetch(`/library/mark-read/${encodeURIComponent(filename)}`, { method: 'POST' });
if (seriesNav.next) {
window.location.href = `/library/read/${encodeURIComponent(seriesNav.next.filename)}`;
} else {
window.location.href = `/library/book/${encodeURIComponent(filename)}`; window.location.href = `/library/book/${encodeURIComponent(filename)}`;
} }
}
// ── Bookmarks ────────────────────────────────────────────────── // ── Bookmarks ──────────────────────────────────────────────────
function openBookmarkModal() { function openBookmarkModal() {

View File

@ -1,189 +0,0 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8"/>
<meta name="viewport" content="width=device-width, initial-scale=1.0"/>
<title>Novela{% if develop_mode() %} Develop{% endif %} — Search</title>
<link rel="icon" href="/static/favicon.ico" sizes="16x16"/>
<link rel="icon" type="image/png" sizes="32x32" href="/static/favicon-32.png"/>
<link rel="icon" type="image/png" sizes="256x256" href="/static/favicon-256.png"/>
<link rel="apple-touch-icon" sizes="180x180" href="/static/apple-touch-icon.png"/>
<link rel="preconnect" href="https://fonts.googleapis.com"/>
<link href="https://fonts.googleapis.com/css2?family=Libre+Baskerville:ital,wght@0,400;0,700;1,400&family=DM+Mono:wght@400;500&display=swap" rel="stylesheet"/>
<link rel="stylesheet" href="/static/theme.css"/>
<link rel="stylesheet" href="/static/sidebar.css"/>
<style>
body { display: flex; min-height: 100vh; background: var(--bg); color: var(--text); font-family: var(--serif); }
.main { margin-left: var(--sidebar); flex: 1; padding: 2rem 2.5rem; max-width: 860px; }
@media (max-width: 768px) { .main { margin-left: 0; padding: 1rem; } }
.page-title { font-size: 1.4rem; font-weight: 700; margin-bottom: 1.5rem; }
.search-bar { display: flex; gap: 0.5rem; margin-bottom: 0.75rem; }
.search-mode { display: flex; gap: 0; margin-bottom: 0.5rem; }
.search-filter { display: flex; gap: 0; margin-bottom: 2rem; }
.mode-btn {
padding: 0.35rem 0.85rem; font-size: 0.8rem; cursor: pointer;
border: 1px solid var(--border); background: var(--surface); color: var(--text-dim);
font-family: var(--mono);
}
.mode-btn:first-child { border-radius: var(--radius) 0 0 var(--radius); }
.mode-btn:last-child { border-radius: 0 var(--radius) var(--radius) 0; border-left: none; }
.mode-btn + .mode-btn { border-left: none; }
.mode-btn.active { background: var(--surface2); color: var(--text); border-color: var(--accent); z-index: 1; }
.search-input {
flex: 1; padding: 0.55rem 0.8rem; border-radius: var(--radius);
border: 1px solid var(--border); background: var(--surface); color: var(--text);
font-size: 1rem; font-family: var(--serif);
}
.search-input:focus { outline: none; border-color: var(--accent); }
.search-btn {
padding: 0.55rem 1.1rem; border-radius: var(--radius); border: none;
background: var(--accent); color: #000; font-weight: 600; cursor: pointer;
font-size: 0.9rem;
}
.search-btn:hover { background: var(--accent2); }
.search-status { color: var(--text-dim); font-size: 0.85rem; margin-bottom: 1rem; font-family: var(--mono); }
.result-list { display: flex; flex-direction: column; gap: 1rem; }
.result-card {
background: var(--surface); border: 1px solid var(--border);
border-radius: var(--radius); padding: 1rem 1.2rem;
}
.result-book { font-size: 1rem; font-weight: 700; margin-bottom: 0.15rem; }
.result-book a { color: var(--text); text-decoration: none; }
.result-book a:hover { color: var(--accent); }
.result-author { font-size: 0.82rem; color: var(--text-dim); margin-bottom: 0.5rem; }
.result-chapter {
font-size: 0.8rem; color: var(--accent); font-family: var(--mono);
margin-bottom: 0.5rem;
}
.result-snippet {
font-size: 0.88rem; color: var(--text); line-height: 1.55;
border-left: 2px solid var(--border); padding-left: 0.75rem;
margin-bottom: 0.75rem;
}
.result-snippet mark {
background: rgba(255,162,14,0.25); color: var(--accent2);
border-radius: 2px; padding: 0 1px;
}
.result-actions { display: flex; gap: 0.5rem; }
.result-link {
display: inline-flex; align-items: center; gap: 0.3rem;
padding: 0.3rem 0.7rem; border-radius: var(--radius); font-size: 0.8rem;
text-decoration: none; border: 1px solid var(--border);
color: var(--text-dim); background: var(--surface2);
}
.result-link:hover { border-color: var(--accent); color: var(--accent); }
.result-link.primary { background: var(--accent); color: #000; border-color: var(--accent); font-weight: 600; }
.result-link.primary:hover { background: var(--accent2); border-color: var(--accent2); }
.empty { color: var(--text-dim); text-align: center; padding: 3rem 0; font-size: 0.95rem; }
</style>
</head>
<body>
{% include "_sidebar.html" %}
<main class="main">
<div class="page-title">Search</div>
<div class="search-bar">
<input class="search-input" id="search-input" type="search"
placeholder="Search inside books…" autocomplete="off"/>
<button class="search-btn" onclick="doSearch()">Search</button>
</div>
<div class="search-mode">
<button class="mode-btn active" id="mode-phrase" onclick="setMode('phrase')">Phrase</button>
<button class="mode-btn" id="mode-words" onclick="setMode('words')">All words</button>
</div>
<div class="search-filter">
<button class="mode-btn active" id="filter-all" onclick="setFilter('all')">All</button>
<button class="mode-btn" id="filter-unread_novels" onclick="setFilter('unread_novels')">Unread novels</button>
<button class="mode-btn" id="filter-unread_shorts" onclick="setFilter('unread_shorts')">Unread shorts</button>
</div>
<div class="search-status" id="search-status"></div>
<div class="result-list" id="result-list"></div>
</main>
<script>
const input = document.getElementById('search-input');
const statusEl = document.getElementById('search-status');
const listEl = document.getElementById('result-list');
let searchMode = 'phrase';
let searchFilter = 'all';
function setMode(mode) {
searchMode = mode;
document.getElementById('mode-phrase').classList.toggle('active', mode === 'phrase');
document.getElementById('mode-words').classList.toggle('active', mode === 'words');
}
function setFilter(filter) {
searchFilter = filter;
document.getElementById('filter-all').classList.toggle('active', filter === 'all');
document.getElementById('filter-unread_novels').classList.toggle('active', filter === 'unread_novels');
document.getElementById('filter-unread_shorts').classList.toggle('active', filter === 'unread_shorts');
}
input.addEventListener('keydown', e => { if (e.key === 'Enter') doSearch(); });
// Auto-run if ?q= param provided
const urlParams = new URLSearchParams(location.search);
const urlQ = urlParams.get('q');
const urlMode = urlParams.get('mode');
const urlFilter = urlParams.get('filter');
if (urlMode === 'words') setMode('words');
if (urlFilter === 'unread_novels' || urlFilter === 'unread_shorts') setFilter(urlFilter);
if (urlQ) { input.value = urlQ; doSearch(); }
async function doSearch() {
const q = input.value.trim();
if (!q) { listEl.innerHTML = ''; statusEl.textContent = ''; return; }
history.replaceState(null, '', '/search?q=' + encodeURIComponent(q) + '&mode=' + searchMode + '&filter=' + searchFilter);
statusEl.textContent = 'Searching…';
listEl.innerHTML = '';
try {
const resp = await fetch('/api/search?q=' + encodeURIComponent(q) + '&mode=' + searchMode + '&filter=' + searchFilter);
if (!resp.ok) throw new Error('Search failed');
const results = await resp.json();
render(results, q);
} catch (e) {
statusEl.textContent = 'Search failed.';
}
}
function render(results, q) {
if (!results.length) {
statusEl.textContent = 'No results.';
listEl.innerHTML = '<div class="empty">No matches found for this query.</div>';
return;
}
statusEl.textContent = results.length + ' result' + (results.length === 1 ? '' : 's');
listEl.innerHTML = results.map(r => {
const enc = encodeURIComponent(r.filename);
const readUrl = '/library/read/' + enc + '?bm_ch=' + r.chapter_index + '&bm_scroll=0';
return `
<div class="result-card">
<div class="result-book"><a href="/library/book/${enc}">${esc(r.title || r.filename)}</a></div>
${r.author ? `<div class="result-author">${esc(r.author)}</div>` : ''}
<div class="result-chapter">Chapter ${r.chapter_index + 1}${r.chapter_title ? ' — ' + esc(r.chapter_title) : ''}</div>
<div class="result-snippet">${r.snippet}</div>
<div class="result-actions">
<a class="result-link primary" href="${readUrl}">Read here</a>
<a class="result-link" href="/library/book/${enc}">Book detail</a>
</div>
</div>`;
}).join('');
}
function esc(s) {
return String(s).replace(/&/g,'&amp;').replace(/</g,'&lt;').replace(/>/g,'&gt;').replace(/"/g,'&quot;');
}
</script>
<script src="/static/books.js"></script>
</body>
</html>

View File

@ -3,7 +3,7 @@
<head> <head>
<meta charset="UTF-8"/> <meta charset="UTF-8"/>
<meta name="viewport" content="width=device-width, initial-scale=1.0"/> <meta name="viewport" content="width=device-width, initial-scale=1.0"/>
<title>Novela{% if develop_mode() %} Develop{% endif %} — Settings</title> <title>Novela — Settings</title>
<link rel="icon" href="/static/favicon.ico" sizes="16x16"/> <link rel="icon" href="/static/favicon.ico" sizes="16x16"/>
<link rel="icon" type="image/png" sizes="32x32" href="/static/favicon-32.png"/> <link rel="icon" type="image/png" sizes="32x32" href="/static/favicon-32.png"/>
<link rel="icon" type="image/png" sizes="256x256" href="/static/favicon-256.png"/> <link rel="icon" type="image/png" sizes="256x256" href="/static/favicon-256.png"/>
@ -169,22 +169,6 @@
<main class="main"> <main class="main">
<div class="main-title">Settings</div> <div class="main-title">Settings</div>
<!-- Develop mode -->
<div class="card">
<div class="card-title">Develop mode</div>
<div class="card-desc">
Mark this as a development instance. Adds a <strong>DEVELOP</strong> banner to every page
and shows <strong>Novela Develop</strong> in the browser tab title.
</div>
<label style="display:flex;align-items:center;gap:0.75rem;cursor:pointer">
<input type="checkbox" id="develop-mode-toggle" style="width:16px;height:16px;accent-color:var(--accent);cursor:pointer"
{% if develop_mode() %}checked{% endif %}
onchange="toggleDevelopMode(this.checked)"/>
<span style="font-family:var(--mono);font-size:0.8rem">Enable develop mode</span>
</label>
<div class="feedback" id="develop-feedback"></div>
</div>
<!-- Reading history reset --> <!-- Reading history reset -->
<div class="card"> <div class="card">
<div class="card-title">Reading history</div> <div class="card-title">Reading history</div>
@ -204,30 +188,6 @@
</button> </button>
<div class="feedback" id="reset-feedback"></div> <div class="feedback" id="reset-feedback"></div>
</div> </div>
<!-- Break image -->
<div class="card">
<div class="card-title">Break image</div>
<div class="card-desc">
The image used as a scene break in converted books.
Applies to all newly imported books. Upload a PNG, JPG or WebP image.
</div>
<div id="break-image-preview" style="margin-bottom:1rem;display:none">
<img id="break-image-img" src="" alt="break image"
style="max-height:40px;border:1px solid var(--border);border-radius:4px;padding:4px;background:var(--surface)"/>
</div>
<label class="btn" style="background:var(--surface);border:1px solid var(--border);color:var(--text);cursor:pointer">
<svg width="13" height="13" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2.5">
<path d="M21 15v4a2 2 0 0 1-2 2H5a2 2 0 0 1-2-2v-4"/>
<polyline points="17 8 12 3 7 8"/>
<line x1="12" y1="3" x2="12" y2="15"/>
</svg>
Upload break image
<input type="file" id="break-image-file" accept="image/png,image/jpeg,image/webp"
style="display:none" onchange="uploadBreakImage(this)"/>
</label>
<div class="feedback" id="break-image-feedback"></div>
</div>
<!-- Break detection patterns --> <!-- Break detection patterns -->
<div class="card"> <div class="card">
<div class="card-title">Break detection</div> <div class="card-title">Break detection</div>
@ -296,26 +256,6 @@
<script src="/static/books.js"></script> <script src="/static/books.js"></script>
<script> <script>
// ── Develop mode ───────────────────────────────────────────────────────────
async function toggleDevelopMode(enabled) {
const fb = document.getElementById('develop-feedback');
fb.className = 'feedback';
fb.textContent = '';
const resp = await fetch('/api/app-settings', {
method: 'PATCH',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ develop_mode: enabled }),
});
const data = await resp.json();
if (data.ok) {
location.reload();
} else {
fb.className = 'feedback err';
fb.textContent = 'Could not save setting.';
document.getElementById('develop-mode-toggle').checked = !enabled;
}
}
// ── Break patterns ───────────────────────────────────────────────────────── // ── Break patterns ─────────────────────────────────────────────────────────
let bpPatterns = []; let bpPatterns = [];
@ -443,45 +383,12 @@
// Enter key in add inputs // Enter key in add inputs
// ── Break image ────────────────────────────────────────────────────────────
async function loadBreakImagePreview() {
const resp = await fetch('/api/app-settings');
const data = await resp.json();
if (data.break_image_url) {
document.getElementById('break-image-img').src = data.break_image_url + '?t=' + Date.now();
document.getElementById('break-image-preview').style.display = 'block';
}
}
async function uploadBreakImage(input) {
const file = input.files[0];
if (!file) return;
const fb = document.getElementById('break-image-feedback');
fb.className = 'feedback';
fb.textContent = 'Uploading…';
const form = new FormData();
form.append('file', file);
const resp = await fetch('/api/app-settings/break-image', { method: 'POST', body: form });
const data = await resp.json();
input.value = '';
if (data.ok) {
fb.className = 'feedback ok';
fb.textContent = 'Break image updated.';
document.getElementById('break-image-img').src = data.url + '?t=' + Date.now();
document.getElementById('break-image-preview').style.display = 'block';
} else {
fb.className = 'feedback err';
fb.textContent = data.error || 'Upload failed.';
}
}
document.addEventListener('DOMContentLoaded', () => { document.addEventListener('DOMContentLoaded', () => {
document.getElementById('bp-text-input').addEventListener('keydown', e => { if (e.key === 'Enter') addPatternFromText(); }); document.getElementById('bp-text-input').addEventListener('keydown', e => { if (e.key === 'Enter') addPatternFromText(); });
document.getElementById('bp-regex-input').addEventListener('keydown', e => { if (e.key === 'Enter') addPattern('regex'); }); document.getElementById('bp-regex-input').addEventListener('keydown', e => { if (e.key === 'Enter') addPattern('regex'); });
document.getElementById('bp-css-input').addEventListener('keydown', e => { if (e.key === 'Enter') addPattern('css_class'); }); document.getElementById('bp-css-input').addEventListener('keydown', e => { if (e.key === 'Enter') addPattern('css_class'); });
document.getElementById('bp-test-input').addEventListener('keydown', e => { if (e.key === 'Enter') testBreak(); }); document.getElementById('bp-test-input').addEventListener('keydown', e => { if (e.key === 'Enter') testBreak(); });
loadBreakPatterns(); loadBreakPatterns();
loadBreakImagePreview();
}); });
// ── Reading history ──────────────────────────────────────────────────────── // ── Reading history ────────────────────────────────────────────────────────

View File

@ -3,7 +3,7 @@
<head> <head>
<meta charset="UTF-8"/> <meta charset="UTF-8"/>
<meta name="viewport" content="width=device-width, initial-scale=1.0"/> <meta name="viewport" content="width=device-width, initial-scale=1.0"/>
<title>Novela{% if develop_mode() %} Develop{% endif %} — Statistics</title> <title>Novela — Statistics</title>
<link rel="icon" href="/static/favicon.ico" sizes="16x16"/> <link rel="icon" href="/static/favicon.ico" sizes="16x16"/>
<link rel="icon" type="image/png" sizes="32x32" href="/static/favicon-32.png"/> <link rel="icon" type="image/png" sizes="32x32" href="/static/favicon-32.png"/>
<link rel="icon" type="image/png" sizes="256x256" href="/static/favicon-256.png"/> <link rel="icon" type="image/png" sizes="256x256" href="/static/favicon-256.png"/>

View File

@ -1,7 +1,7 @@
import re import re
from html import escape as he from html import escape as he
from bs4 import Comment, NavigableString, Tag from bs4 import NavigableString, Tag
BREAK_PATTERNS = [ BREAK_PATTERNS = [
re.compile(r"^\s*[\*\-]{3,}\s*$"), # *** of --- re.compile(r"^\s*[\*\-]{3,}\s*$"), # *** of ---
@ -111,12 +111,6 @@ def element_to_xhtml(el, break_img_path: str = "../Images/break.png", empty_p_is
result += "\n" + trailer result += "\n" + trailer
return result return result
if isinstance(el, Comment):
# Preserve HTML comments as XML comments (e.g. scraper metadata).
# Sanitise "--" sequences which are illegal inside XML comments.
body = str(el).replace("--", "- -")
return f"<!-- {body} -->"
if isinstance(el, NavigableString): if isinstance(el, NavigableString):
text = str(el) text = str(el)
if text.strip(): if text.strip():

View File

@ -6,7 +6,6 @@ It is the primary technical reference for the current implementation.
## Architecture ## Architecture
- Stack: FastAPI, Jinja2 templates, plain JavaScript, PostgreSQL 16, Docker. - Stack: FastAPI, Jinja2 templates, plain JavaScript, PostgreSQL 16, Docker.
- All routers import `templates` from `shared_templates.py` (a single `Jinja2Templates` instance). This module registers a `develop_mode()` callable as a Jinja2 global, making it available in every template without passing it explicitly per route.
- Startup lifecycle (`main.py`): - Startup lifecycle (`main.py`):
1. `init_pool()` 1. `init_pool()`
2. `run_migrations()` 2. `run_migrations()`
@ -27,23 +26,22 @@ All files are stored under `library/` (relative to the app working directory, ma
| Format | Path pattern | | Format | Path pattern |
|--------|-------------| |--------|-------------|
| EPUB (no series) | `library/epub/{publisher}/{author}/Stories/{title}.epub` | | EPUB (no series) | `library/epub/{publisher}/{author}/Stories/{title}.epub` |
| EPUB (series) | `library/epub/{publisher}/{author}/Series/{series}/{idx:03d}_-_{title}.epub` | | EPUB (series) | `library/epub/{publisher}/{author}/Series/{series}/{idx:03d} - {title}.epub` |
| PDF | `library/pdf/{publisher}/{author}/{title}.pdf` | | PDF | `library/pdf/{publisher}/{author}/{title}.pdf` |
| CBR (no series) | `library/comics/{publisher}/{author}/{title}.cbr` | | CBR (no series) | `library/comics/{publisher}/{author}/{title}.cbr` |
| CBR (series) | `library/comics/{publisher}/{author}/Series/{series}/{idx:03d}_-_{title}.cbr` | | CBR (series) | `library/comics/{publisher}/{author}/Series/{series}/{idx:03d} - {title}.cbr` |
| CBZ (no series) | `library/comics/{publisher}/{author}/{title}.cbz` | | CBZ (no series) | `library/comics/{publisher}/{author}/{title}.cbz` |
| CBZ (series) | `library/comics/{publisher}/{author}/Series/{series}/{idx:03d}_-_{title}.cbz` | | CBZ (series) | `library/comics/{publisher}/{author}/Series/{series}/{idx:03d} - {title}.cbz` |
- Segments are sanitised: special chars stripped, spaces replaced with `_`, max lengths applied (publisher/author 80, title 140, series 80). - Segments are sanitised: special chars stripped, max lengths applied (publisher/author 80, title 140, series 80).
- Series index is zero-padded to 3 digits (`001`, `002`, …), clamped to 1999. - Series index is zero-padded to 3 digits (`001`, `002`, …), clamped to 1999.
- Duplicate filenames get a `(2)`, `(3)`, … suffix. - Duplicate filenames get a `(2)`, `(3)`, … suffix.
- After any file move, empty parent directories are pruned up to `LIBRARY_ROOT`. - After any file move, empty parent directories are pruned up to `LIBRARY_ROOT`.
### Path logic ### Path logic
- `common.make_rel_path(media_type, publisher, author, title, series, series_index, series_suffix, ext)` — used by import and grabber. - `common.make_rel_path(media_type, publisher, author, title, series, series_index, ext)` — used by import and grabber.
- `reader.py _make_rel_path(publisher, author, title, series, series_index, series_suffix, ext)` — used by metadata PATCH; same logic, uses actual file extension. - `reader.py _make_rel_path(publisher, author, title, series, series_index, ext)` — used by metadata PATCH; same logic, uses actual file extension.
- `series_volume` is not part of the file path; it is stored in DB and OPF only.
- Both functions produce identical paths for all formats. - Both functions produce identical paths for all formats.
### Metadata save behaviour per format ### Metadata save behaviour per format
@ -68,10 +66,9 @@ All files are stored under `library/` (relative to the app working directory, ma
- `GET /download/{filename}` — download file with `Content-Disposition: attachment` - `GET /download/{filename}` — download file with `Content-Disposition: attachment`
- `GET /library/cover/{filename}` — serve cover (EPUB from file; PDF/CBR from cache) - `GET /library/cover/{filename}` — serve cover (EPUB from file; PDF/CBR from cache)
- `GET /library/cover-cached/{filename}` — serve cover from DB cache only - `GET /library/cover-cached/{filename}` — serve cover from DB cache only
- `POST /library/cover/{filename}` — upload/replace cover; for EPUB files: embeds cover in the EPUB and updates cache; for DB-stored books: stores cover directly in `library_cover_cache` and sets `has_cover = TRUE` - `POST /library/cover/{filename}` — upload/replace cover (EPUB only)
- `POST /library/want-to-read/{filename}` — toggle want-to-read flag - `POST /library/want-to-read/{filename}` — toggle want-to-read flag
- `POST /library/archive/{filename}` — toggle archived flag - `POST /library/archive/{filename}` — toggle archived flag
- `POST /library/archive-series` — set `archived` for all books in a series; body: `{"series": "…", "archive": true|false}`; returns `{ok, archived, count}`
- `POST /library/new/mark-reviewed` — bulk set `needs_review=false` - `POST /library/new/mark-reviewed` — bulk set `needs_review=false`
- `POST /library/bulk-delete` — delete multiple files; accepts `{"filenames": [...]}`, removes files from disk and DB in one query per batch; returns `{ok, deleted, skipped}` - `POST /library/bulk-delete` — delete multiple files; accepts `{"filenames": [...]}`, removes files from disk and DB in one query per batch; returns `{ok, deleted, skipped}`
- `POST /library/rating/{filename}` — set/clear star rating `{"rating": 0-5}` - `POST /library/rating/{filename}` — set/clear star rating `{"rating": 0-5}`
@ -80,7 +77,7 @@ All files are stored under `library/` (relative to the app working directory, ma
- `GET /stats` — statistics page - `GET /stats` — statistics page
- `GET /api/stats` — statistics data JSON - `GET /api/stats` — statistics data JSON
- `GET /api/disk` — partition usage for the library directory: `{total, used, free, pct_used}` - `GET /api/disk` — partition usage for the library directory: `{total, used, free, pct_used}`
- `POST /api/bulk-check-duplicates` — accepts `{"items": [{title, author, series, volume}, ...]}`, returns `{"duplicates": [bool, ...]}`checks by title+author+series_index; also checks by series+author+series_index as fallback (catches duplicate detection when title format changed); when volume is absent, matches on title+author only - `POST /api/bulk-check-duplicates` — accepts `{"items": [{title, author, volume}, ...]}`, returns `{"duplicates": [bool, ...]}`when `volume` is a number, requires title+author+series_index to all match; when volume is absent, matches on title+author only
- `GET /library/list` — compat alias - `GET /library/list` — compat alias
`GET /api/library` runs in fast-path mode by default (DB-only, no full disk rescan). `GET /api/library` runs in fast-path mode by default (DB-only, no full disk rescan).
@ -109,12 +106,9 @@ Home read sections are ordered oldest-first:
- `novels_read`: `ORDER BY MAX(read_at) ASC` - `novels_read`: `ORDER BY MAX(read_at) ASC`
### `routers/reader.py` ### `routers/reader.py`
- `GET /library/db-images/{path:path}` — serve image from content-addressed imagestore (`library/images/`); security: path must be under `IMAGES_DIR`
- `POST /api/library/convert-to-db/{filename:path}` — convert on-disk EPUB to a DB-stored book; extracts chapters via `_epub_body_inner` (stores images in imagestore, rewrites src to `/library/db-images/…`), migrates all child tables (INSERT new library row → UPDATE children → DELETE old row), deletes EPUB file; returns `{ok, new_filename}`
- `GET /api/library/export-epub/{filename:path}` — build and stream an EPUB from a DB-stored book; `_rewrite_db_images_for_epub` rewrites `/library/db-images/…` back to `OEBPS/Images/…` paths (dedup by sha256); returns as `Content-Disposition: attachment`
- `GET /library/epub/{filename}` — serve EPUB inline (no attachment header) - `GET /library/epub/{filename}` — serve EPUB inline (no attachment header)
- `GET /library/chapters/{filename}` — EPUB spine as JSON; for `storage_type='db'` books returns chapters from `book_chapters` - `GET /library/chapters/{filename}` — EPUB spine as JSON
- `GET /library/chapter/{index}/{filename}` — single chapter as HTML fragment; for `storage_type='db'` books reads from `book_chapters` - `GET /library/chapter/{index}/{filename}` — single EPUB chapter as HTML fragment
- `GET /library/chapter-img/{path}?filename=…` — image extracted from EPUB ZIP; `path` is the full internal ZIP path (e.g. `OEBPS/Images/cover.jpg` or `EPUB/images/cover.jpg`); case-insensitive fallback for mismatched folder names - `GET /library/chapter-img/{path}?filename=…` — image extracted from EPUB ZIP; `path` is the full internal ZIP path (e.g. `OEBPS/Images/cover.jpg` or `EPUB/images/cover.jpg`); case-insensitive fallback for mismatched folder names
- `GET /library/pdf/{filename}?page=N&dpi=150` — render PDF page as PNG - `GET /library/pdf/{filename}?page=N&dpi=150` — render PDF page as PNG
- `GET /api/pdf/info/{filename}``{"page_count": N}` - `GET /api/pdf/info/{filename}``{"page_count": N}`
@ -125,10 +119,9 @@ Home read sections are ordered oldest-first:
- `POST /library/mark-read/{filename}` — mark as read (with optional date) - `POST /library/mark-read/{filename}` — mark as read (with optional date)
- `GET /library/book/{filename}` — book detail page - `GET /library/book/{filename}` — book detail page
- `GET /api/genres` — all tags from `book_tags` (optional `?type=genre|subgenre|tag`) - `GET /api/genres` — all tags from `book_tags` (optional `?type=genre|subgenre|tag`)
- `PATCH /library/book/{filename}` — update metadata + tags; moves file if path fields change; DB-only for non-EPUB; for `storage_type='db'` books: recomputes synthetic `db/…` filename, FK-safe rename (INSERT→UPDATE children→DELETE old), updates `book_chapters` + `bookmarks` as well - `PATCH /library/book/{filename}` — update metadata + tags; moves file if path fields change; DB-only for non-EPUB
- `POST /library/rating/{filename}` — set/clear 15 star rating; writes to EPUB OPF / CBZ ComicInfo.xml; DB-only for CBR/PDF - `POST /library/rating/{filename}` — set/clear 15 star rating; writes to EPUB OPF / CBZ ComicInfo.xml; DB-only for CBR/PDF
- `GET /library/read/{filename}` — reader page (EPUB or PDF); supports `?bm_ch=N&bm_scroll=F` to jump to bookmark position - `GET /library/read/{filename}` — reader page (EPUB or PDF); supports `?bm_ch=N&bm_scroll=F` to jump to bookmark position
- `GET /api/series-nav/{filename}` — returns `{prev, next}` (`{filename, title, index, suffix}` or `null`) for the adjacent books in the same series ordered by `series_index ASC, series_suffix ASC`; used by the reader for series navigation buttons and `markRead()` redirect
- `GET /library/bookmarks/{filename}` — list bookmarks for a book - `GET /library/bookmarks/{filename}` — list bookmarks for a book
- `POST /library/bookmarks/{filename}` — add bookmark `{chapter_index, scroll_frac, chapter_title, note}` - `POST /library/bookmarks/{filename}` — add bookmark `{chapter_index, scroll_frac, chapter_title, note}`
- `PATCH /library/bookmarks/{id}` — update bookmark note - `PATCH /library/bookmarks/{id}` — update bookmark note
@ -139,14 +132,14 @@ Home read sections are ordered oldest-first:
- `GET /bulk-import` — Bulk Import page - `GET /bulk-import` — Bulk Import page
- `POST /library/bulk-import` — import files with pre-parsed metadata; accepts multipart `files[]`, `rows` (JSON array of per-file metadata), `shared` (JSON with author/publisher/status/genres/tags applied to all files) - `POST /library/bulk-import` — import files with pre-parsed metadata; accepts multipart `files[]`, `rows` (JSON array of per-file metadata), `shared` (JSON with author/publisher/status/genres/tags applied to all files)
Filename parsing is done client-side in `bulk_import.html`. The page uses a free-text `%placeholder%` pattern (e.g. `%series% - %series_volume% - %volume% - %title% - %year%`). Available placeholders: `%series%` `%series_volume%` `%volume%` `%title%` `%year%` `%month%` `%day%` `%author%` `%publisher%` `%ignore%`. Colored chips can be clicked (insert at cursor) or dragged onto the input. Pattern is converted to a regex at parse time. Shared metadata fields (including "Year/Vol." for `series_volume`) override filename-parsed values. "Auto-generate titles" checkbox fills empty title cells as `Series (Year/Vol) #Number`. Skip checkbox is always visible for every row; skipped rows are excluded from import. Files are uploaded in batches of 5 with a progress bar. Filename parsing is done client-side in `bulk_import.html`. The page uses a free-text `%placeholder%` pattern (e.g. `%series% - %volume% - %title% - %year%`). Available placeholders: `%series%` `%volume%` `%title%` `%year%` `%month%` `%day%` `%author%` `%publisher%` `%ignore%`. Colored chips can be clicked (insert at cursor) or dragged onto the input. Pattern is converted to a regex at parse time. Shared metadata fields override filename-parsed values. Files are uploaded in batches of 5 with a progress bar.
### `routers/editor.py` ### `routers/editor.py`
- `GET /library/editor/{filename}` — chapter editor page; supports both EPUB files and DB-stored books (`db/…` filenames); passes `is_db` flag to template; DB branch queries `library` table directly (no file check) - `GET /library/editor/{filename}`EPUB chapter editor page
- `GET /api/edit/chapter/{index}/{filename}` — get chapter content; DB branch reads from `book_chapters` and returns `{index, href, title, content}` - `GET /api/edit/chapter/{index}/{filename}` — get chapter HTML
- `POST /api/edit/chapter/{index}/{filename}` — save chapter; DB branch accepts `{content, title}`, calls `upsert_chapter` (updates `content_tsv` too) - `POST /api/edit/chapter/{index}/{filename}` — save chapter HTML
- `POST /api/edit/chapter/add/{filename}` — add new chapter after `after_index`; DB branch shifts `chapter_index` up via `UPDATE … SET chapter_index = chapter_index + 1 WHERE chapter_index >= insert_idx` then inserts - `POST /api/edit/chapter/add/{filename}` — add new chapter
- `DELETE /api/edit/chapter/{index}/{filename}` — delete chapter; DB branch deletes and re-indexes via `UPDATE … SET chapter_index = chapter_index - 1 WHERE chapter_index > index` - `DELETE /api/edit/chapter/{index}/{filename}` — delete chapter
### `routers/grabber.py` ### `routers/grabber.py`
- `GET /grabber` — grabber page - `GET /grabber` — grabber page
@ -158,135 +151,17 @@ Filename parsing is done client-side in `bulk_import.html`. The page uses a free
- `POST /credentials` — save credential - `POST /credentials` — save credential
- `DELETE /credentials/{site}` — delete credential - `DELETE /credentials/{site}` — delete credential
- `POST /preload` — preload book info from URL - `POST /preload` — preload book info from URL
- `POST /convert` — run scrape; body may include `storage_mode: "db"` (default) or `"epub"` to control output format - `POST /convert` — run scrape + convert to EPUB
- `GET /events/{job_id}` — SSE stream for job progress; `done` event includes `storage_type` (`'db'` or `'file'`) - `GET /events/{job_id}` — SSE stream for job progress
Scrape/convert flow (DB storage — default):
1. Fetch book info + chapters via scraper
2. Per chapter: download images → write to `library/images/{sha2}/{sha256}{ext}` (content-addressed) → rewrite `img[src]` to `/library/db-images/...`; break images replaced with `<hr>` before `element_to_xhtml` runs → build `content_html` via `element_to_xhtml` with `break_img_path="/static/break.png"`
3. One DB transaction: `ensure_unique_db_filename``upsert_book` (storage_type='db') → `upsert_chapter` for each chapter → `upsert_cover_cache` if cover provided
4. Synthetic filename: `db/{publisher}/{author}/{title}` (or `db/{pub}/{auth}/Series/{series}/{idx} - {title}` for series)
Scrape/convert flow (EPUB file — `storage_mode: "epub"`):
12. Same as DB flow; `break_img_path="../Images/break.png"` passed to `element_to_xhtml`
3. Chapters converted to XHTML via `make_chapter_xhtml`; EPUB file built via `make_epub` (embeds `static/break.png` as `OEBPS/Images/break.png`) and written to `library/epub/…`
4. `upsert_book` called with `storage_type='file'`
### Scrapers (`scrapers/`)
All scrapers inherit `BaseScraper` and implement `matches(url)`, `login()`, `fetch_book_info()`, `fetch_chapter()`. Registration order in `scrapers/__init__.py` determines priority (first match wins).
| Scraper | Domain | Login | Notes |
|---|---|---|---|
| `ArchiveOfOurOwnScraper` | archiveofourown.org | Optional | Uses authenticity token; adult content gate via `?view_adult=true` |
| `AwesomeDudeScraper` | awesomedude.org | No | Chapter discovery via `.htm/.html` links in same directory; content extracted from largest non-layout block |
| `CodeysWorldScraper` | codeysworld.org | No | See below |
| `GayAuthorsScraper` | gayauthors.org | Optional | Genres + subgenres from `itemprop="genre"` links; tags from `ipsTags` list |
| `IomfatsScraper` | iomfats.org | No | See below; requires chapter URL as entry point |
| `NiftyNewScraper` | new.nifty.org | No | See below; registered before NiftyScraper |
| `NiftyScraper` | nifty.org (classic) | No | See below; excludes new.nifty.org; category/subcategory stored as tags |
| `TedLouisScraper` | tedlouis.com | No | Story index URL required as entry point; all pages use `?t=TOKEN` routing; chapter links in `<ul class="story-index-list">` |
#### NiftyNewScraper
`new.nifty.org` is a Next.js RSC application. Pages render proper HTML with semantic markup — no plain-text email format.
- URL normalisation: `_to_index_url()` strips a trailing `/N` (chapter index) so any URL (index or chapter) can be passed as entry point. Story URL pattern: `/stories/{slug}-{id}`.
- `fetch_book_info()`:
- Title from `<h1>`; fallback: `<title>` with ` - … - Nifty Archive …` suffix stripped.
- Author from `<strong itemprop="name">` inside `<a href="/authors/{id}">`.
- Publication date from `<time itemprop="datePublished" datetime="…">`, updated date from `<time itemprop="dateModified" datetime="…">`; both truncated to `YYYY-MM-DD`.
- Tags from all `<ul aria-label="Tags">` containers on the page — covers both the story category links (`/collections/…`) and the AI-generated content tags (`/search?query=tags%3A…`); deduplicated; `genres` and `subgenres` are always empty.
- Description from `<meta name="description">`.
- Chapter list: `<a>` links matching `/stories/{slug}/N` collected from page HTML; fallback: regex scan of RSC stream for `"index": N` values. URLs generated as `{index_url}/1``{index_url}/max`.
- `fetch_chapter()`:
- Content extraction order:
1. Chapter HTML (`{url}`): read `<article>` and collect `<p>` text
2. Fallback on same HTML: extract escaped Next payload paragraphs (`\u003cp...\u003c/p`)
3. Last fallback (`{url}?_rsc=1`): parse RSC line format (`{hex_id}:{json}`) for `["$","p",…]` nodes, then escaped paragraph fallback
- Chapter title uses the precomputed chapter dict title (`Chapter N`).
- Lead/tail boilerplate detection for common Nifty intro/donate text. Removed boilerplate is preserved as invisible HTML comments in chapter content:
- `<!-- NIFTY_HIDDEN_LEAD: ... -->`
- `<!-- NIFTY_HIDDEN_TAIL: ... -->`
- No email-header stripping and no plain-text line-joining (those are specific to Nifty classic).
#### NiftyScraper
Nifty classic pages are plain-text email submissions wrapped in a `<pre>` element.
- URL normalisation: `_to_index_url()` strips the chapter segment so any URL (index or chapter) can be passed as the entry point. Path structure: `/nifty/{category}/{subcategory}/{story}/` (index, 4 segments) vs `/nifty/{category}/{subcategory}/{story}/{chapter}` (chapter, 5 segments).
- `fetch_book_info()` performs up to 3 extra HTTP requests: chapter 1 (author + publication date), last chapter (`updated_date`), chapter 2 (boilerplate detection). Author and dates are extracted from the email headers (`From:`, `Date:`) embedded at the top of each chapter file. Date is parsed via `email.utils.parsedate``YYYY-MM-DD`.
- Boilerplate detection: leading paragraphs of chapters 1 and 2 (after email-header strip) are compared using normalised text (lowercase, whitespace collapsed). Consecutive matching paragraphs are recorded as `preamble_count` and stored in each chapter dict; `fetch_chapter()` skips them.
- `fetch_chapter()` pipeline:
1. Extract `<pre>` text (fallback: full body text)
2. Parse `Subject:` header → store as `<!-- Subject: … -->` comment in chapter content (invisible in reader, extractable later)
3. Strip email header block (up to first blank line after `Date:`/`From:`/`Subject:` lines)
4. Skip first `preamble_count` paragraphs
5. Split on blank lines → paragraphs; join hard-wrapped lines within each paragraph with a space
6. Detect and remove lead/tail boilerplate (common notice/disclaimer/author promo/donate blocks)
7. Persist removed boilerplate as invisible comments:
- `<!-- NIFTY_HIDDEN_LEAD: ... -->`
- `<!-- NIFTY_HIDDEN_TAIL: ... -->`
8. Scene-break patterns (`***`, `---`, `~~~`, `• • •`, etc.) → `<hr/>`
9. Build `content_el` as a BeautifulSoup `<div>` of comments + `<p>` + `<hr/>` nodes
- Genres/subgenres from URL path: `category` (e.g. `gay``Gay`) and `subcategory` (e.g. `young-friends``Young Friends`).
#### CodeysWorldScraper
- Entry point: any `codeysworld.org` URL.
- Title from `<h1>`; author from `<h2>` matching `"by …"` pattern; fallback: URL path segment `/{author}/{category}/filename`.
- Category from URL path (second-to-last segment, e.g. `remembrances` → tag `"Remembrances"`).
- Chapter discovery: `.htm/.html` links in the same directory as the entry URL; audio/image links skipped. No chapter links → single-file story (entry URL is the only chapter).
- `fetch_chapter()`: removes all `<h1>`/`<h2>` headings, back-navigation links, audio links (`.mp3`), mailto links; falls back to `<body>` when no content wrapper is found.
#### IomfatsScraper
All stories by an author are listed on a single author page (`/storyshelf/hosted/{author}/`). Individual story pages do not exist.
- Entry point must be a **chapter URL** (`/storyshelf/hosted/{author}/{story-folder}/{chapter}.html`). Passing the author page URL raises a `ValueError` with a user-visible message.
- On load: navigates to the author page and scans `<div id="content">` for the matching story.
- Two page structures detected:
- **Single story**: outer `<h3>` = book title; chapters are direct `<li><a>` children of the following `<ul>`.
- **Multi-part series**: outer `<h3>` = series name; nested `<li><h3>` = book title per part; chapters in the sub-`<ul>` matching `story_folder`.
- Series index extracted from folder name suffix: `*-part{N}` or `*-{N}`.
- Publication status from `<p><small>[…]</small></p>` after the book title heading.
- `fetch_chapter()`: content from `<div id="content">`; removes `<h2>`/`<h3>` headings, `.chapternav` divs, `div.important` footer blocks, anchor-name elements.
#### TedLouisScraper
All pages on `tedlouis.com` use opaque token-based routing: `https://tedlouis.com/?t=<TOKEN>`. There are no predictable URL patterns — tokens must be followed from the story index page.
- Entry point must be a **story index URL** (the page listing all chapters). Passing a chapter URL raises a `ValueError` with a user-visible message. Detection: story index has `<h2 class="story-page-title">`, chapter page has `<h1 class="story-title">`.
- `fetch_book_info()`:
- Title from direct `NavigableString` children of `<h2 class="story-page-title">` — the element also contains a "Back" button (`<a class="btn">`) and the author byline (`<span class="story-author-by-line">`), which are skipped.
- Author from `<span class="story-author-by-line"> <a>`.
- Publication status from `<span class="story-status-text">` with "Status: " prefix stripped.
- Updated date from `<span class="story-last-updated">` ("Last Updated: Month D, YYYY") → `YYYY-MM-DD`.
- Chapter list from all `<ul class="story-index-list">` elements (three columns on the page); relative `?t=TOKEN` hrefs resolved to absolute URLs. Order preserved; duplicates deduplicated.
- No genres, subgenres, tags or description available on the page.
- `fetch_chapter()`: content from `<div id="chapter">`; strips `<h1 class="story-title">`, `<h2 class="chapter-title">`, `div.chapter-copyright-line`, and `div.chapter-copyright-notice-text` blocks. Chapter title refined from `<h2 class="chapter-title"> <span>`.
#### `xhtml.element_to_xhtml()` — Comment handling
`bs4.Comment` objects (a `NavigableString` subclass) are now emitted as XML comments: `<!-- … -->`. The `--` sequence (illegal inside XML comments) is sanitised to `- -`. This allows scrapers to embed invisible metadata (e.g. the Nifty `Subject:` header) in chapter content without it appearing in the rendered reader.
### `routers/search.py`
- `GET /search` — full-text search page (`search.html`); Enter-to-search, `?q=` param auto-runs on load
- `GET /api/search?q=…&mode=phrase|words&filter=all|unread_novels|unread_shorts` — FTS over `book_chapters.content_tsv`; `mode=phrase` (default) uses `phraseto_tsquery` (words in order); `mode=words` uses `plainto_tsquery` (all words present, any order); `ts_rank` and `ts_headline` always use `plainto_tsquery`; also matches chapters whose `title` contains the query (case-insensitive LIKE fallback); no result limit; excludes archived books; `filter=unread_novels` restricts to books with no reading sessions/progress and no `Shorts` tag; `filter=unread_shorts` restricts to books with no reading sessions/progress and a `Shorts` tag; results include `filename`, `title`, `author`, `chapter_index`, `chapter_title`, `snippet`, `rank`
### `routers/settings.py` ### `routers/settings.py`
- `GET /settings` — settings page - `GET /settings` — settings page
- `GET /api/app-settings` — returns `{"develop_mode": bool, "break_image_url": str|null}`
- `PATCH /api/app-settings` — accepts `{"develop_mode": bool}`, persists to `app_settings` table
- `POST /api/app-settings/break-image` — multipart file upload (PNG/JPG/WebP); stores image in imagestore + overwrites `static/break.png`; saves `break_image_sha256` + `break_image_ext` to `app_settings`; returns `{"ok": true, "url": "/library/db-images/…"}`
- `GET /api/break-patterns` — list chapter-break patterns - `GET /api/break-patterns` — list chapter-break patterns
- `POST /api/break-patterns` — add break pattern (type: `regex` or `css_class`) - `POST /api/break-patterns` — add break pattern (type: `regex` or `css_class`)
- `PATCH /api/break-patterns/{id}` — update pattern (enable/disable or change value) - `PATCH /api/break-patterns/{id}` — update pattern (enable/disable or change value)
- `DELETE /api/break-patterns/{id}` — delete pattern - `DELETE /api/break-patterns/{id}` — delete pattern
- `DELETE /api/reading-history` — wipe all reading sessions - `DELETE /api/reading-history` — wipe all reading sessions
`app_settings` table (single row, `id = 1`): `develop_mode BOOLEAN`, `break_image_sha256 VARCHAR(64)`, `break_image_ext VARCHAR(10)`.
### `routers/builder.py` ### `routers/builder.py`
- `GET /builder` — Book Builder index (draft list + new draft form) - `GET /builder` — Book Builder index (draft list + new draft form)
- `POST /builder` — create new draft; redirects to `/builder/{id}` - `POST /builder` — create new draft; redirects to `/builder/{id}`
@ -325,9 +200,6 @@ URL is stored in the `authors` table (`name` unique, `url`, `created_at`, `updat
- `GET /api/backup/history` — backup run history (last 20) - `GET /api/backup/history` — backup run history (last 20)
- `GET /api/backup/progress` — live progress of running backup `{running, done, total, phase}` - `GET /api/backup/progress` — live progress of running backup `{running, done, total, phase}`
- `POST /api/backup/run` — trigger backup (background task) - `POST /api/backup/run` — trigger backup (background task)
- `GET /api/backup/snapshots` — list available snapshots `{ok, snapshots: [{name, created_at}]}`
- `GET /api/backup/snapshots/{snapshot_name}/files` — list files in a snapshot with local existence check `{ok, snapshot, files: [{path, size, sha256, exists_locally}]}`
- `POST /api/backup/restore` — restore files from a snapshot: `{snapshot_name, files: [rel_paths]}`; downloads from Dropbox, writes to disk, re-indexes via `scan_media` + `upsert_book`; returns `{ok, restored, total, results: [{path, ok, error?}]}`
--- ---
@ -483,21 +355,16 @@ Loaded by `index.html` (Convert page) and `grabber.html` (Grabber page). Require
- Amber: filled `#c8a03a`, unfilled `rgba(200, 160, 58, 0.25)`. - Amber: filled `#c8a03a`, unfilled `rgba(200, 160, 58, 0.25)`.
- Reader settings (hamburger menu): - Reader settings (hamburger menu):
- Content width slider (30100 vw), persisted as `reader-content-width-pct`. - Content width slider (30100 vw), persisted as `reader-content-width-pct`.
- Font size slider (80150%, default 105%), persisted as `reader-font-size`; applied via `--reader-font-size` CSS custom property on `#chapter-content`.
- Text colour: 5 warm-tone presets `#e8e2d9``#938d86`, persisted as `reader-text-colour`. - Text colour: 5 warm-tone presets `#e8e2d9``#938d86`, persisted as `reader-text-colour`.
- Hamburger and back-link separated with `margin-left: 1rem` on `.header-back`. - Hamburger and back-link separated with `margin-left: 1rem` on `.header-back`.
- Reader supports EPUB, PDF, and CBR/CBZ: - Reader supports EPUB and PDF:
- EPUB: chapter-text rendering; progress = `{chapterIndex}:{scrollFrac}`; progress % = `(chapterIndex + scrollFrac) / total * 100`. - EPUB: chapter-text rendering; progress = `{chapterIndex}:{scrollFrac}`; progress % = `(chapterIndex + scrollFrac) / total * 100`.
- PDF: page-image rendering via `/library/pdf/{filename}?page=N`; page count from `/api/pdf/info/{filename}`; progress = `{pageIndex}:0`; keyboard/button navigation identical. - PDF: page-image rendering via `/library/pdf/{filename}?page=N`; page count from `/api/pdf/info/{filename}`; progress = `{pageIndex}:0`; keyboard/button navigation identical.
- `reader.html` branches on `FORMAT` variable injected by the server. - `reader.html` branches on `FORMAT` variable injected by the server.
- Series navigation: on load, `loadSeriesNav()` fetches `/api/series-nav/{filename}` and activates prev/next volume buttons in the header (hidden when no series); `markRead()` redirects to `/library/read/{next.filename}` when a next volume exists, otherwise to the book detail page.
- `Edit EPUB` button in Book Detail is only shown for `.epub` files. - `Edit EPUB` button in Book Detail is only shown for `.epub` files.
- Backup page supports: manual run, dry-run, Dropbox root, retention count, schedule (on/off + hours), status + history. - Backup page supports: manual run, dry-run, Dropbox root, retention count, schedule (on/off + hours), status + history.
- Bookmarks: saved per book via `POST /library/bookmarks/{filename}`; shown in Library sidebar section; navigated via `?bm_ch=N&bm_scroll=F` URL params on reader page. - Bookmarks: saved per book via `POST /library/bookmarks/{filename}`; shown in Library sidebar section; navigated via `?bm_ch=N&bm_scroll=F` URL params on reader page.
- Convert page: after loading metadata, if a book with the same title+author already exists in the library, a warning banner is shown (with a link to the existing book); user can still proceed with conversion. Check is done server-side in `/preload` response (`already_exists`, `existing_books`). - Convert page: after loading metadata, if a book with the same title+author already exists in the library, a warning banner is shown (with a link to the existing book); user can still proceed with conversion. Check is done server-side in `/preload` response (`already_exists`, `existing_books`).
- Authors view (`#authors`): lists all authors across `allBooks` (active + archived); authors whose books are all archived still appear. Sidebar counter (`count-authors`) counts only active-book authors. Author detail view (`#authors/{name}`) also uses `allBooks`; archived books show the `.badge-archived` overlay on their cover.
- Publishers view (`#publishers`): same rule — `allBooks` (active + archived); publishers with only archived books still appear. Sidebar counter uses active books only. Publisher detail also uses `allBooks`.
- Series detail view (`#series/{name}`): shows all books in a series as a cover grid. Header contains an "Archive series" / "Unarchive series" button — calls `POST /library/archive-series` to set `archived` for every book in the series at once; the button label reflects whether any book is still active.
- Duplicates view (`#duplicates`): groups non-archived books by `(title, author)` (case-insensitive); shows only groups with ≥ 2 copies; counter in sidebar shows total number of duplicate books. Detection is entirely client-side from the existing library data. - Duplicates view (`#duplicates`): groups non-archived books by `(title, author)` (case-insensitive); shows only groups with ≥ 2 copies; counter in sidebar shows total number of duplicate books. Detection is entirely client-side from the existing library data.
- Incomplete view (`#incomplete`): shows all non-archived books where `publication_status` is not `Complete` (Ongoing, Temporary Hold, Long-Term Hold, or blank); sidebar counter included. - Incomplete view (`#incomplete`): shows all non-archived books where `publication_status` is not `Complete` (Ongoing, Temporary Hold, Long-Term Hold, or blank); sidebar counter included.
- Following page (`/following`): dedicated page in its own sidebar section between Library and Tools; shows all library authors with their external URL; two tabs — Following (authors with URL set) and All Authors; inline URL editing with keyboard support (Enter = save, Escape = cancel); clicking Visit opens the external URL in a new tab. Author URLs are stored in the `authors` table. Sidebar counter shows number of followed authors. - Following page (`/following`): dedicated page in its own sidebar section between Library and Tools; shows all library authors with their external URL; two tabs — Following (authors with URL set) and All Authors; inline URL editing with keyboard support (Enter = save, Escape = cancel); clicking Visit opens the external URL in a new tab. Author URLs are stored in the `authors` table. Sidebar counter shows number of followed authors.
@ -505,17 +372,6 @@ Loaded by `index.html` (Convert page) and `grabber.html` (Grabber page). Require
--- ---
## Develop Mode
When enabled, every page shows a diagonal **DEVELOP** ribbon in the top-left corner and the browser tab title becomes **Novela Develop — …** instead of **Novela — …**.
- Persisted in `app_settings` table (single row, `id = 1`); created by `migrate_create_app_settings()`.
- `shared_templates._develop_mode()` reads this value from DB on every template render and is registered as a Jinja2 global (`develop_mode`), so all templates can use `{% if develop_mode() %}` without explicit context injection.
- Banner CSS lives in `static/sidebar.css` (`.develop-banner` / `.develop-banner-text`); rendered at the top of `templates/_sidebar.html`.
- Toggled via the **Develop mode** card on the Settings page (`/settings`); saving reloads the page so the banner and title take effect immediately.
---
## Known Conventions ## Known Conventions
- Book deletion flow: `unlink` file → `prune_empty_dirs(parent)``DELETE FROM library` (cascade removes child rows). - Book deletion flow: `unlink` file → `prune_empty_dirs(parent)``DELETE FROM library` (cascade removes child rows).
- Empty dir pruning: `prune_empty_dirs(start)` walks up from `start` to `LIBRARY_ROOT`, removing each dir if empty; stops at first non-empty dir. - Empty dir pruning: `prune_empty_dirs(start)` walks up from `start` to `LIBRARY_ROOT`, removing each dir if empty; stops at first non-empty dir.
@ -529,7 +385,6 @@ When enabled, every page shows a diagonal **DEVELOP** ribbon in the top-left cor
- CBR/PDF: DB only - CBR/PDF: DB only
- `upsert_book` uses `CASE WHEN EXCLUDED.rating > 0 THEN EXCLUDED.rating ELSE library.rating END` to restore rating from file without overwriting existing DB value. - `upsert_book` uses `CASE WHEN EXCLUDED.rating > 0 THEN EXCLUDED.rating ELSE library.rating END` to restore rating from file without overwriting existing DB value.
- Tag types in `book_tags`: `genre`, `subgenre`, `tag`, `subject`. No direct `genres`/`subgenres` fields on book objects; always use helpers `bookGenres()`, `bookSubgenres()`, `bookPlainTags()`. - Tag types in `book_tags`: `genre`, `subgenre`, `tag`, `subject`. No direct `genres`/`subgenres` fields on book objects; always use helpers `bookGenres()`, `bookSubgenres()`, `bookPlainTags()`.
- `series_volume` (e.g. `"1982"`) is used for annual comic series where issue numbers restart each year. It is separate from `series_index` (issue number within the year) and `series_suffix` (letter variant like `"a"`). Stored in DB and EPUB OPF (`novela:series_volume`); not reflected in the file path. Sort order: `series → series_volume → series_index → series_suffix`. In `getSeriesSlots`, gap-detection runs per volume independently when any book has `series_volume` set; slot labels show as `(year) #index`.
--- ---
@ -550,62 +405,9 @@ When enabled, every page shows a diagonal **DEVELOP** ribbon in the top-left cor
--- ---
## DB-Stored Books
Books scraped via the grabber are stored entirely in PostgreSQL (`storage_type = 'db'`). No EPUB file is written.
### New tables
| Table | Key columns | Notes |
|---|---|---|
| `book_chapters` | `filename FK, chapter_index, title, content TEXT, content_tsv TSVECTOR` | Unique on `(filename, chapter_index)`; GIN index on `content_tsv` for FTS; `content_tsv` is `to_tsvector('simple', title || ' ' || stripped_html)` — title included for title-based FTS matches |
| `book_images` | `sha256 PK, ext, media_type, size_bytes` | Content-addressed; files live at `library/images/{sha256[:2]}/{sha256}{ext}` |
### `library.storage_type`
| Value | Meaning |
|---|---|
| `'file'` | Book lives on disk (EPUB/PDF/CBR/CBZ); default for all existing books |
| `'db'` | Book content lives in `book_chapters`; no file on disk |
### Synthetic filename for DB books
`db/{publisher}/{author}/{title}` — or for series: `db/{publisher}/{author}/Series/{series}/{idx:03d} - {title}`
Same sanitization rules as file-based paths. Uniqueness enforced via `ensure_unique_db_filename` (DB lookup, not filesystem).
### Chapter editor for DB books
`GET /library/editor/{filename}` supports DB-stored books. The Monaco editor shows `language: 'html'` for DB books (vs `'xml'` for EPUB). The header shows a title input instead of a read-only chapter name. Unsaved content and titles are preserved across chapter switches via `pendingContent` and `pendingTitles` maps. `editor.focus()` is called after every content load so the editor is immediately interactive.
### Imagestore
Images embedded in chapter HTML are stored content-addressed at `library/images/{sha256[:2]}/{sha256}{ext}`.
- Served via `GET /library/db-images/{path:path}`
- URLs embedded in `book_chapters.content` as absolute paths: `/library/db-images/...`
- `book_images` table registers each unique image (auto-deduplication via sha256)
### EPUB → DB conversion
`POST /api/library/convert-to-db/{filename}` converts an on-disk EPUB to `storage_type='db'`:
1. Parse EPUB spine → per item: extract body HTML via `_epub_body_inner`, store images in imagestore via `write_image_file`, rewrite `img[src]` to `/library/db-images/…`
2. Compute new synthetic `db/…` filename via `make_rel_path(media_type="db", …)` + `ensure_unique_db_filename`
3. DB transaction: INSERT new library row (storage_type='db') → UPDATE all child tables (book_tags, reading_progress, reading_sessions, bookmarks, library_cover_cache, book_chapters) → DELETE old library row
4. Delete EPUB file from disk + `prune_empty_dirs`
### DB → EPUB export
`GET /api/library/export-epub/{filename}` streams an EPUB built from DB content:
1. Query metadata, tags, chapters, cover from DB
2. Per chapter: `_rewrite_db_images_for_epub` strips `/library/db-images/` prefix, reads files from `IMAGES_DIR`, deduplicates by sha256, assigns `OEBPS/Images/{sha256}{ext}` paths, rewrites `img[src]` to `../Images/…`
3. Build EPUB via `make_epub()`; return as `Content-Disposition: attachment`
---
## Known Bugs Fixed ## Known Bugs Fixed
- `renderGenreView` and `renderSearchResults` in `library.js` referenced `b.genres` (non-existent). Fixed: use `bookGenres()`, `bookSubgenres()`, `bookPlainTags()`. - `renderGenreView` and `renderSearchResults` in `library.js` referenced `b.genres` (non-existent). Fixed: use `bookGenres()`, `bookSubgenres()`, `bookPlainTags()`.
- `PillInput` in `book.js` did not handle comma as delimiter and did not flush on save. Fixed: comma keydown + `flush()` in `saveEdit()`. - `PillInput` in `book.js` did not handle comma as delimiter and did not flush on save. Fixed: comma keydown + `flush()` in `saveEdit()`.
- `PillInput._add` in `book.js` added a pasted comma-separated list as one tag instead of splitting it. Fixed: `_add` now splits the value on commas and pushes each trimmed, non-empty, non-duplicate part individually.
- `PATCH /library/book` failed for PDFs: `_sync_epub_metadata` tried to open PDF as ZIP. Fixed: only called for `.epub`. - `PATCH /library/book` failed for PDFs: `_sync_epub_metadata` tried to open PDF as ZIP. Fixed: only called for `.epub`.
- `_make_rel_path` in `reader.py` lacked format prefix (`epub/`, `pdf/`, `comics/`). Fixed: aligned with `common.make_rel_path`. - `_make_rel_path` in `reader.py` lacked format prefix (`epub/`, `pdf/`, `comics/`). Fixed: aligned with `common.make_rel_path`.
- `common.make_rel_path` always generated `.cbr` extension for CBZ files (both map to `media_type="cbr"`). Fixed: accepts optional `ext` parameter; `library.py` import now passes actual suffix. - `common.make_rel_path` always generated `.cbr` extension for CBZ files (both map to `media_type="cbr"`). Fixed: accepts optional `ext` parameter; `library.py` import now passes actual suffix.

View File

@ -1,275 +1,5 @@
# Develop Changelog # Develop Changelog
## 2026-04-15 (1)
- Reader: font size control in reading settings
- New "Font size" slider (80150%, default 105%) in the settings drawer, between "Content width" and "Text colour"
- Applies via CSS custom property `--reader-font-size` on `#chapter-content`
- Persisted per-device in `localStorage` as `reader-font-size` — iPad and desktop each remember their own preference
## 2026-04-13 (1)
- Edit metadata: comma-separated tag input fix
- `PillInput._add` in `book.js` now splits the incoming value on commas before adding — each trimmed, non-empty, non-duplicate part is pushed individually
- Applies to genres, subgenres and plain tags; pasting e.g. `Fiction, Thriller, Adventure` adds 3 separate pills instead of 1
---
*Released as v0.1.11 on 2026-04-13*
## 2026-04-12 (2)
- Series navigation in the reader
- New `GET /api/series-nav/{filename}` endpoint: returns `{prev, next}` with `{filename, title, index, suffix}` for adjacent books in the same series, ordered by `series_index ASC, series_suffix ASC`; returns `{prev: null, next: null}` for books without a series
- Reader header now shows prev/next series buttons (skip-to-start/skip-to-end icons); hidden for books without a series, visible once `loadSeriesNav()` resolves
- Hovering a series button shows a tooltip: `#<index><suffix> <title>` (e.g. `#4 Batman: Year One`)
- `markRead()` in the reader redirects to `/library/read/{next.filename}` when a next volume exists, so reading continues without leaving the reader; falls back to the book detail page when the series is complete or the book has no series
## 2026-04-12 (1)
- Comics: series_volume support for annual series (issue numbers restart each year)
- New `series_volume VARCHAR(20)` column on `library` (migration `migrate_series_volume`); default `''`
- Stored in EPUB OPF as `<meta name="novela:series_volume" content="…"/>`; read back by `scan_epub`
- `upsert_book` inserts/preserves `series_volume` with the same COALESCE strategy as `series_suffix`
- `list_library_json` ORDER BY now: `publisher → author → series → series_volume → series_index → title`
- `PATCH /library/book/{filename}` reads `series_volume` from request body; persists to DB and OPF for both file-based and DB-stored books
- Book detail page: displays `(year)` after series name when `series_volume` is set (e.g. `Donald Duck (1982) [15]`); edit panel has a new "Year/Volume" input field
- `book.js`: "Auto" button next to the Title field generates `Series (Year/Vol) #Number` from the current series fields
- Bulk importer: `series_volume` support
- New `%series_volume%` placeholder token (orange) for filename pattern parsing
- New "Year/Vol." shared metadata field (applies to all files; overridden by per-row value or pattern)
- Preview table has a new "Yr/Vol" column
- New "Auto-generate titles from series info" checkbox: when enabled, rows without a parsed title get title `Series (Year/Vol) #Number`
- Skip checkbox now always visible for every row (previously only shown when duplicates were detected); any file can be manually excluded before import; skipped rows are dimmed; stats bar shows "X skipped"
- `POST /api/bulk-check-duplicates` extended: now also checks `(series, series_index, author)` as fallback — detects duplicates even when the title format has changed; items must include `series` field
- Library front-end: sorting within a series now respects `series_volume` before `series_index`
- `groupBySeries`, `renderAuthorDetail`, `renderPublisherDetail` all sort by `series_volume → series_index → series_suffix`
- `getSeriesSlots` refactored: when any book in the series has `series_volume` set, gap-detection runs per volume (year) independently — prevents `#5 (1982)` and `#5 (1983)` from colliding in the same slot
- Slot index label shows `(year) #index` for annual series (e.g. `(1982) #5`); unchanged for regular series
---
*Released as v0.1.10 on 2026-04-12*
## 2026-04-08 (13)
- Library: archive series in one action
- "Archive series" / "Unarchive series" button in the series detail view (`#series/{name}`)
- New endpoint `POST /library/archive-series` — sets `archived` for all books in the series via a single SQL UPDATE; body: `{"series": "…", "archive": true|false}`; returns `{ok, archived, count}`
- Button label reflects current state: "Archive series" when any book is active, "Unarchive series" when all are archived
- After the call, `allBooks` is updated in place and sidebar counters are recalculated without a full page reload
## 2026-04-08 (12)
- TedLouisScraper: title extraction fix
- `<h2 class="story-page-title">` also contains a "Back" button (`<a class="btn">`) and the author byline (`<span class="story-author-by-line">`)
- Fix: only direct `NavigableString` children of the h2 are used as the title, so link and span text is skipped
## 2026-04-08 (11)
- New scraper: `TedLouisScraper` (`scrapers/tedlouis.py`) for `tedlouis.com`
- Matches all `tedlouis.com` URLs; no login required
- Entry point is the story index page (e.g. `?t=CWYSqpOryu2rQmT1`); raises an error when a chapter URL is used as entry point
- Title from direct text nodes of `<h2 class="story-page-title">`
- Author from `<span class="story-author-by-line"> <a>`
- Status from `<span class="story-status-text">` (strips "Status: " prefix)
- Updated date from `<span class="story-last-updated">``YYYY-MM-DD`
- Chapters from all `<ul class="story-index-list">` elements (three columns); relative `?t=TOKEN` links resolved to absolute URLs
- `fetch_chapter()`: content from `<div id="chapter">`; removes story title, chapter title and copyright blocks
## 2026-04-08 (10)
- Settings: break image upload added
- New "Break image" card on the settings page: upload a PNG/JPG/WebP as the scene break image
- Stored in the imagestore (sha256-addressed) and overwrites `static/break.png` so EPUB export uses the same image
- `app_settings` extended with `break_image_sha256` and `break_image_ext` columns (migration `migrate_app_settings_break_image`)
- New endpoints: `POST /api/app-settings/break-image`; `GET /api/app-settings` now also returns `break_image_url`
- Preview of the current break image visible on the settings page
## 2026-04-08 (9)
- Grabber: break image fix for DB-stored books
- Break images (`<center><img src="../Images/break.png">`) contain a relative EPUB path that does not exist in the DB context
- Fix: `storage_mode` determined earlier in `_run_scrape`; for DB mode `/static/break.png` is passed as `break_img_path` to `element_to_xhtml`, for EPUB mode `../Images/break.png` is kept
- Additional fix: when a break image is detected in the image loop, the parent (`<center>`) is replaced with `<hr>` instead of decomposing the `<img>`; this allows `element_to_xhtml` to correctly detect the break after the image loop
## 2026-04-08 (8)
- New scraper: `IomfatsScraper` (`scrapers/iomfats.py`) for `iomfats.org`
- Matches all `iomfats.org` URLs; no login required
- Entry point is a chapter URL (e.g. `.../grasshopper/justhitsend-part1/justhitsend01.html`); automatically navigates to the author page to fetch all metadata and chapters
- Author page as entry point: raises a user-visible error message
- Detects two structures on the author page:
- Single story: outer `<h3>` = book title; chapters directly in `<ul>`
- Multi-part series: outer `<h3>` = series name; nested `<li><h3>` = book title per part; chapters in sub-`<ul>`
- Series index from folder name: `*-part{N}` or `*-{N}``series_index_hint`
- Publication status from `<p><small>[...]</small></p>` after book title
- `fetch_chapter()`: content via `<div id="content">`; removes `<h2>`/`<h3>` headings, chapternav divs, footer elements
## 2026-04-08 (7)
- New scraper: `CodeysWorldScraper` (`scrapers/codeysworld.py`) for `codeysworld.org`
- Matches all `codeysworld.org` URLs; no login required
- Title from `<h1>`; author from `<h2>` ("by …"); fallback to URL slug (`/{author}/{category}/filename`)
- Category from URL path → tag (e.g. "Remembrances")
- Single-file stories (no chapter links): the page itself is the only chapter
- Multi-chapter: links to `.htm`/`.html` files in the same directory (audio/image links skipped)
- `fetch_chapter()`: removes all `<h1>`/`<h2>` headings, navigation links ("Back to …"), audio links (`.mp3`), mailto links
- Nifty scraper: category/subcategory moved from `genres`/`subgenres` to `tags`
## 2026-04-08 (6)
- NiftyNewScraper: chapter extraction made more robust
- `fetch_chapter()` now first tries the standard chapter HTML (`{url}`) and reads `<article><p>` directly
- Fallback added for Next payloads with escaped paragraphs (`\u003cp...\u003c/p`) via `_extract_escaped_html_paragraphs()`
- Last fallback remains `?_rsc=1`: first `_parse_rsc_paragraphs()`, then escaped-paragraph fallback again
- Nifty (classic + new): standard boilerplate no longer visible in reader, but preserved in chapter
- Lead/tail detection added for common blocks (e.g. `NOTICE This is a work of fiction…`, `If you enjoy this story…`, donate text)
- Detected intro/closing boilerplate removed from visible paragraphs
- Removed text stored as invisible HTML comment:
- `<!-- NIFTY_HIDDEN_LEAD: ... -->`
- `<!-- NIFTY_HIDDEN_TAIL: ... -->`
- Detection also works when text contains inline HTML (e.g. `<a>` or `<svg>` in donate links)
## 2026-04-08 (5)
- NiftyNewScraper: chapter content fix
- `new.nifty.org` uses Next.js RSC — chapter content is not present in the static HTML but in the RSC payload (React component tree)
- `fetch_chapter()` now fetches `{url}?_rsc=1` instead of the regular HTML page
- RSC parser added: `_parse_rsc_paragraphs()`, `_rsc_find_paragraphs()`, `_rsc_text()` — parse the RSC stream line by line (format: `{hex_id}:{json}`), recursively search for `["$","p",null,{...}]` nodes and extract text from `children`
## 2026-04-08 (4)
- NiftyNewScraper added (`scrapers/nifty_new.py`) for `new.nifty.org`
- Matches all `new.nifty.org` URLs; no login required
- `_to_index_url()`: strips trailing `/N` (chapter number) so both index and chapter URLs can be used as entry point
- `fetch_book_info()`: title from `<h1>`, author from `<strong itemprop="name">` in author link, dates from `<time itemprop="datePublished/dateModified">`, tags from all `<ul aria-label="Tags">` containers (category links and generated tags, deduplicated), description from `<meta name="description">`, chapter list via `<a>` links matching `/stories/{slug}/N` (RSC stream regex as fallback)
- `fetch_chapter()`: title from JSON-LD `@type: "Chapter"`, content from `<article>`; no email header stripping, no line joining, no boilerplate detection
- `NiftyScraper.matches()` updated: excludes `new.nifty.org` (`"nifty.org" in url and "new.nifty.org" not in url`)
- `NiftyNewScraper` registered before `NiftyScraper` in `scrapers/__init__.py`
## 2026-04-08 (3)
- Settings: develop mode added
- New `app_settings` table (single row, `id = 1`) with `develop_mode` boolean; created via `migrate_create_app_settings()`
- `shared_templates.py`: shared `Jinja2Templates` instance for all routers; `develop_mode()` registered as Jinja2 global so all templates can access it without explicit context injection
- All 11 routers now import `templates` from `shared_templates` instead of each creating their own instance
- New endpoints in `routers/settings.py`: `GET /api/app-settings` and `PATCH /api/app-settings`
- Diagonal **DEVELOP** banner in the top-left of every page (CSS in `static/sidebar.css`, HTML in `templates/_sidebar.html`); only visible when develop mode is enabled
- All 17 HTML templates: `<title>` shows **Novela Develop — …** when develop mode is active
- Settings page: new card with checkbox toggle; page reloads after saving so banner and title apply immediately
## 2026-04-08 (2)
- Nifty scraper: fix `_strip_email_headers` now tolerates blank lines between header fields
- Some Nifty pages place `Subject:` after a blank line (`Date:\nFrom:\n\nSubject:\n`) — the previous implementation stopped at the first blank line causing `Subject:` to appear as a paragraph in chapter text
- Fix: on a blank line, look ahead to check whether a header field still follows; if so, skip the blank line(s) and continue stripping
## 2026-04-08 (1)
- Nifty scraper added (`scrapers/nifty.py`)
- Matches all `nifty.org` URLs; no login required
- `fetch_book_info`: accepts index or chapter URL; normalises to index; title from URL slug, author and publication date from email headers of chapter 1, `updated_date` from email headers of the last chapter; genres/subgenres from URL path (`/nifty/{category}/{subcategory}/…`)
- Boilerplate detection: compares first paragraphs of chapters 1 and 2 after header stripping; matching paragraphs are skipped in all chapters (`preamble_count` per chapter dict)
- `fetch_chapter`: retrieves `<pre>` content (fallback: body); subject header stored as invisible HTML comment `<!-- Subject: … -->` at the top of chapter content; email headers stripped; hard line breaks within paragraphs joined into a single line; scene breaks (`***`, `---`, etc.) → `<hr/>`
- Date parsing via `email.utils.parsedate``YYYY-MM-DD`
- `xhtml.py`: `element_to_xhtml` now handles `bs4.Comment` objects as XML comments (`<!-- … -->`); `--` in the body is sanitised to `- -` (illegal in XML comments)
## 2026-04-06 (3)
- Book detail: rating moved from clickable stars to dropdown in the Edit metadata panel
- Stars on the detail page are now purely visual (no longer clickable)
- New `<select id="ed-rating">` field in the edit panel, directly below Status
- `openEdit()` populates the select with the current rating; `saveEdit()` calls `POST /library/rating/…` if the value has changed
- `rateBook()` function removed from `book.js`
- Overview pages unchanged
## 2026-04-06 (2)
- Library: cover upload now also supported for DB-stored books
- `POST /library/cover/{filename}` previously returned an error for DB books (`"File not found"`) because no physical file exists
- Fix: DB books are now detected via `is_db_filename`; cover is stored directly in `library_cover_cache` and `has_cover = TRUE` set in the `library` table — `add_cover_to_epub` is not called
## 2026-04-06 (1)
- Search: filter for unread novels / unread shorts
- New `filter` parameter on `GET /api/search?q=…&filter=all|unread_novels|unread_shorts` (default: `all`)
- `unread_novels`: restricts to books with no reading sessions/progress and no `Shorts` tag
- `unread_shorts`: restricts to books with no reading sessions/progress and with a `Shorts` tag
- UI: second toggle row (All / Unread novels / Unread shorts) below the Phrase/All words toggle
- Filter persisted in URL (`?filter=…`) and restored on page load
## 2026-04-05 (4)
- Backup: large files (> 148 MB) now uploaded via Dropbox upload session in 100 MB chunks
- `_dropbox_upload_bytes`: files ≤ 148 MB go via `files_upload` (unchanged); larger files via `upload_session_start``upload_session_append_v2``upload_session_finish`
- Fixes `ApiError: UploadError('payload_too_large')`
## 2026-04-05 (3)
- Filenames: spaces now replaced with underscores when saving new files
- `clean_segment` (common.py) and `_clean_segment` (reader.py): `\s+``_` instead of space
- Series separator changed from ` - ` to `_-_` (applies to EPUB, CBR, CBZ and DB filenames)
- Existing files are not renamed
## 2026-04-05 (2)
- Search: removed result limit; added Phrase / All words mode toggle
- `LIMIT 30` removed — all matching chapters are returned
- New `mode` parameter on `GET /api/search?q=…&mode=phrase|words`: `phrase` (default) requires words in order (`phraseto_tsquery`); `words` requires all words present in any order (`plainto_tsquery`)
- Toggle in the UI (Phrase / All words) above the results; mode is included in the URL
## 2026-04-05 (1)
- Export EPUB: double chapter titles fixed — same heading-stripping logic as the reader now applied before passing content to `make_chapter_xhtml`
- Library: authors and publishers with only archived books now remain visible in the Authors and Publishers list views
- `renderAuthorsView` and `renderPublishersView` switched from `activeBooks()` to `allBooks` — consistent with `renderAuthorDetail` and `renderPublisherDetail`
## 2026-04-04 (2)
- Reader: fixed double chapter titles for books where the heading is wrapped in a `<section>` element (pandoc-style)
- Previous regex only stripped `<h1>``<h4>` at the very start of content; pandoc-converted EPUBs wrap headings as `<section …><h1>…</h1>` — heading was not at position 0 so the regex didn't fire
- Added a second `re.sub` pass that removes the first `<h1>``<h4>` found directly after an opening `<section>` or `<div>` tag, preserving the wrapper element
- Same stripping applied in the DB→EPUB export (`export_epub`) before passing content to `make_chapter_xhtml`
- Search: switched from `plainto_tsquery` to `phraseto_tsquery` in the FTS WHERE clause
- `plainto_tsquery` ANDs all words but treats them as independent terms (any order, any distance) — multi-word queries like "4 years later" matched chapters where the words appeared far apart
- `phraseto_tsquery` requires all words to appear in sequence; `ts_rank` and `ts_headline` still use `plainto_tsquery` for correct scoring and highlighting
## 2026-04-04 (1)
- Reader: fixed double chapter titles in DB-stored books
- Chapter endpoint (`GET /library/chapter/{index}/{filename}`) now strips all leading `<h1>``<h4>` tags from stored content before prepending its own `<h2 class="chapter-title">` — books scraped before front-matter stripping was added to the scraper showed the title (and chapter heading) twice
- Library: archived books now visible in author and publisher detail views, with an indicator badge on the cover
- `renderAuthorDetail` and `renderPublisherDetail` reverted to use `allBooks` (including archived) so archived books remain accessible from the author/publisher pages
- New `.badge-archived` overlay (bottom-left of cover): dark circle with archive icon, consistent with existing `badge-status` and `read-pill` overlays; added to all three card-rendering code paths
## 2026-04-03 (3)
- DB chapter editor: Monaco-based editor now supports DB-stored books
- `GET /library/editor/{filename}` handles `db/…` filenames; `is_db` flag passed to template
- `GET /api/edit/chapter/{index}/{filename}` and `POST …`: DB branches query/update `book_chapters` directly; save calls `upsert_chapter` (updates `content_tsv` too)
- `POST /api/edit/chapter/add/{filename}` and `DELETE …`: DB branches insert/delete with `chapter_index` shift via `UPDATE … SET chapter_index = chapter_index ± 1`
- Title editing: header chapter-name replaced with a text input for DB books; `pendingTitles` map preserves unsaved titles across chapter switches (parallel to `pendingContent`); title-only dirty chapters correctly saved in Save All
- `insertBreak`: scene-break image path is `/static/break.png` for DB books (vs `../Images/break.png` for EPUB)
- Fix: `editor.focus()` called after content load so Monaco receives keyboard focus immediately
- Fix: `header-chapter` "Loading…" text suppressed for DB books where that element is hidden
- `book.html`: "Edit chapters" button shown for `storage_type = 'db'` books
- Search: chapter titles now included in FTS
- `upsert_chapter` prepends title to the plain-text input for `to_tsvector`: `title + " " + stripped_html`
- `GET /api/search`: added `OR LOWER(bc.title) LIKE LOWER('%…%')` fallback for chapters whose title matches but content doesn't
- Startup migration `migrate_rebuild_chapter_tsv_with_title()` rebuilds existing `content_tsv` values to include titles
- Grabber: added DB/EPUB storage toggle on the Convert page
- UI toggle above Convert button ("Save as: DB | EPUB file"); `storageMode` JS variable sent in POST body
- `POST /convert`: reads `storage_mode` from body; stored in job as `'db'` or `'epub'`
- `_run_scrape`: EPUB path builds chapters via `make_chapter_xhtml`, calls `make_epub`, writes file, calls `upsert_book(storage_type='file')`; DB path unchanged
- `done` SSE event includes `storage_type`; `conversion.js` updates the download button label/action accordingly
- EPUB → DB conversion: fixed double chapter title
- `_epub_body_inner` strips the first `<h1>`/`<h2>`/`<h3>` heading from each chapter body before storing; the editor prepends its own heading, so storing the EPUB heading too caused it to appear twice
- Fix for `NavigableString` crash: `getattr(child, "name", None) is None` used instead of `hasattr(child, "name")``NavigableString` has `name = None` but no `decompose()` method
- Sidebar: Search link styling fixed
- Stray `<li>Search</li>` moved inside the Library `<ul class="sidebar-nav">` (was outside, causing incorrect HTML structure)
- `sidebar.css`: added `a:visited { color: var(--text-dim) }` and `a.active:visited { color: var(--accent) }` to prevent the browser's default purple visited color
## 2026-04-03 (2)
- DB-stored books (Fase 46): EPUB→DB conversion, DB→EPUB export, full-text search
- **Fase 4** — EPUB-to-DB conversion: `POST /api/library/convert-to-db/{filename}` converts an existing on-disk EPUB to a DB-stored book; extracts chapters via `_epub_body_inner` (rewrites img src to imagestore URLs), migrates all child rows (book_tags, reading_progress, reading_sessions, bookmarks, library_cover_cache) to the new `db/…` filename using INSERT→UPDATE→DELETE to respect FK constraints, then deletes the EPUB file
- **Fase 5** — DB→EPUB export: `GET /api/library/export-epub/{filename}` builds and streams an EPUB from DB content; `_rewrite_db_images_for_epub` rewrites `/library/db-images/…` URLs back to `OEBPS/Images/…` paths, deduplicating by sha256; `Content-Disposition: attachment` response
- **Fase 6** — Full-text search: new `routers/search.py` with `GET /search` (page) and `GET /api/search?q=…` (FTS over `book_chapters.content_tsv` via `plainto_tsquery('simple', q)`, `ts_headline` for snippets, `ts_rank` for ordering, LIMIT 30, excludes archived); new `templates/search.html` with highlight (`<mark>`), "Read here" link (`?bm_ch=N&bm_scroll=0`), and "Book detail" link; Search entry added to sidebar
- `book.html`: DB books show "Export EPUB" instead of "Download"; "Edit EPUB" and "Convert to DB" buttons only shown for `.epub` files; delete modal text differs for DB vs file books
- `PATCH /library/book/{filename}`: DB book branch added — skips file move, recomputes synthetic `db/…` filename via `make_rel_path`, applies same FK-safe rename pattern, updates `book_chapters` and `bookmarks` in addition to standard child tables
## 2026-04-03 (1)
- DB-stored books (Fase 13): grabber now stores scraped books in PostgreSQL instead of EPUB files on disk
- New `book_chapters` table: `filename FK, chapter_index, title, content TEXT, content_tsv TSVECTOR`; GIN index on `content_tsv` for future FTS
- New `book_images` table: `sha256 PK, ext, media_type, size_bytes`; content-addressed imagestore at `library/images/{sha2}/{sha256}{ext}`
- New `storage_type VARCHAR(10) DEFAULT 'file'` column on `library`; DB-stored books use `'db'`
- New utilities in `common.py`: `is_db_filename`, `write_image_file`, `store_db_image`, `html_to_plain`, `upsert_chapter`, `ensure_unique_db_filename`; `make_rel_path` now handles `media_type="db"` → synthetic `db/{pub}/{auth}/...` filename
- `upsert_book` and `list_library_json` updated to include `storage_type`
- Grabber: `_run_scrape` stores chapters in `book_chapters`, chapter images in imagestore (absolute `/library/db-images/` URLs embedded in HTML), cover in `library_cover_cache`; no EPUB file written
- New `GET /library/db-images/{path:path}` endpoint serves imagestore files
- Reader: `GET /library/chapters/` and `GET /library/chapter/` have DB branches for `storage_type='db'` books (query `book_chapters` directly)
- Reader page (`/library/read/`), book detail page, mark-read, and rating endpoints all handle DB filenames (no file existence required)
- Cover endpoints (`/library/cover/`, `/library/cover-cached/`) serve DB books from `library_cover_cache`
## 2026-04-02 (1)
- Added Restore functionality to the Backup page
- New `GET /api/backup/snapshots` endpoint: lists available Dropbox snapshots (name + date parsed from filename, no downloads needed)
- New `GET /api/backup/snapshots/{snapshot_name}/files` endpoint: loads a snapshot from Dropbox and returns all files with path, size, sha256, and whether the file currently exists locally
- New `POST /api/backup/restore` endpoint: downloads file objects from Dropbox, writes to disk, and re-indexes via `scan_media` + `upsert_book`; returns per-file result with errors
- New "Restore" card on the backup page: snapshot dropdown (auto-loaded on page open), file list with filter/search, per-file "Restore" button, multi-select + "Restore selected", on-disk indicator, inline status feedback
- After restore, the file list refreshes to reflect updated on-disk state
## 2026-03-29 (10) ## 2026-03-29 (10)
- Duplicates: fixed `updateCounts` crashing with a TypeError (`g.books.length` → `g.length`); the crash prevented `renderGrid` from running, so the duplicates view never rendered and the counter was stale - Duplicates: fixed `updateCounts` crashing with a TypeError (`g.books.length` → `g.length`); the crash prevented `renderGrid` from running, so the duplicates view never rendered and the counter was stale

View File

@ -1,131 +1,5 @@
# Changelog # Changelog
## v0.1.12 — 2026-04-15
### New features
- Reader: **font size slider** in the reading settings drawer — adjust text size from 80% to 150%; setting is saved per device so iPad and desktop each remember their own preference
---
## v0.1.11 — 2026-04-13
### Bug fixes
- Edit metadata: pasting or typing a comma-separated list in the genre, subgenre or tag input now adds each value as a separate tag instead of one combined tag
---
## v0.1.10 — 2026-04-12
### New features
- Reader: **prev/next volume buttons** in the header for books that are part of a series — buttons appear automatically when the book has adjacent volumes; tooltip shows the volume number and title; marking a book as read redirects directly to the next volume in the reader instead of the book detail page
- Comics: **series_volume field** for annual series where issue numbers restart each year (e.g. Donald Duck (1982) [15]) — stored in the database and EPUB OPF; displayed as `(year)` after the series name on the book detail page; sorting respects `series_volume` before `series_index`; supported in Bulk Import via `%series_volume%` placeholder and a "Year/Vol." shared field
- Library: **archive a series in one click** — "Archive series" / "Unarchive series" button in the series detail view; updates all books in the series via a single SQL UPDATE and recalculates sidebar counters without a page reload
### Bug fixes
- TedLouis scraper: title extraction no longer includes the "Back" button text or the author byline — only direct text nodes of the title heading are used
---
## v0.1.9 — 2026-04-08
### New features
- New scraper: **Nifty.org (classic)** — scrapes plain-text email-format stories; email headers stripped, boilerplate paragraphs auto-detected and hidden, scene-break patterns converted to break images
- New scraper: **new.nifty.org** — scrapes the Next.js version of Nifty; reads chapter content from the RSC payload when the static HTML does not include it; boilerplate detection shared with classic Nifty
- New scraper: **codeysworld.org** — single-file and multi-chapter stories; title and author extracted from heading elements; category from URL path stored as tag; navigation links and audio links stripped from chapter content
- New scraper: **iomfats.org** — all stories are listed on a single author page; provide any chapter URL and the scraper finds the correct story automatically; supports single stories and multi-part series (series name, book title, and series index derived from the page structure)
- New scraper: **tedlouis.com** — all pages use opaque token-based routing (`?t=TOKEN`); provide the story index URL and the scraper collects all chapter links from the three-column chapter list
- Settings: break image upload — upload a custom PNG/JPG/WebP to use as the scene break image in all converted books; stored in the imagestore and applied to both DB-stored and EPUB-format books
- Settings: develop mode toggle — shows a DEVELOP banner and updates the page title across all pages when enabled
### Bug fixes
- Break images were not displayed in DB-stored books — the image path `../Images/break.png` is a relative EPUB path that does not exist for DB content; DB mode now uses `/static/break.png`
- Break images were silently lost during import — the image was decomposed before `element_to_xhtml` ran, leaving an empty wrapper; the wrapper is now replaced with `<hr>` so the break is correctly rendered
---
## v0.1.8 — 2026-04-06
### Bug fixes
- Library: cover upload now works for DB-stored books — the upload endpoint previously returned "File not found" because DB books have no file on disk; the cover is now stored directly in the cover cache
### Improvements
- Book detail: rating moved from clickable stars to a dropdown in the Edit metadata panel — avoids touch-input issues on iPad where hover state caused all stars to appear filled
---
## v0.1.7 — 2026-04-06
### New feature
- Search: filter on unread novels or unread shorts — a second toggle row (All / Unread novels / Unread shorts) restricts results to books with no reading history; filter is preserved in the URL
### Bug fixes
- Backup: files larger than 148 MB now upload correctly — chunked upload session (100 MB per chunk) replaces the single-call upload that hit Dropbox's payload size limit
### Improvements
- File paths: spaces in new filenames are now replaced with underscores (publisher, author, title, series segments); series separator changed from ` - ` to `_-_`
---
## v0.1.6 — 2026-04-05
### Bug fixes
- Export EPUB: double chapter titles fixed — same heading-stripping logic as the reader now applied before passing content to `make_chapter_xhtml`
- Library: authors and publishers with only archived books now remain visible in the Authors and Publishers list views
---
## v0.1.5 — 2026-04-04
### Bug fixes
- Reader: double chapter titles for pandoc-converted books — headings wrapped in a `<section>` element were not stripped by the previous regex; now also removes the first heading found directly inside an opening `<section>` or `<div>`
- Search: multi-word queries no longer match chapters where the words appear far apart — switched to `phraseto_tsquery` so all words must appear in order
---
## v0.1.4 — 2026-04-04
### Bug fixes
- Reader: double chapter titles in DB-stored books — the chapter endpoint now strips all leading headings from stored content before prepending its own chapter title; affects books scraped before front-matter stripping was added
- Library: archived books were missing from author and publisher detail views — detail views now include all books (active and archived); archived books have a badge on their cover so they remain distinguishable
---
## v0.1.3 — 2026-04-03
### New feature
- DB-stored books: scraped books are now stored as chapters in PostgreSQL instead of EPUB files on disk — full-text search, content deduplication, and backup coverage are all handled automatically
- Grabber stores chapters in a `book_chapters` table and images in a content-addressed imagestore (sha256-based, automatic deduplication)
- EPUB → DB conversion: "Convert to DB" button on any EPUB book detail page — extracts chapters, migrates all metadata and child rows (tags, progress, bookmarks, cover), removes the EPUB file
- DB → EPUB export: "Export EPUB" button on DB-stored books — builds and streams a standards-compliant EPUB without writing a file to disk
- Full-text search (`/search`): searches across all DB-stored chapter content via PostgreSQL FTS (`tsvector` / `plainto_tsquery`), returns highlighted snippets with direct links to the chapter position in the reader
- Chapter editor supports DB-stored books: Monaco-based editor reads and writes `book_chapters` directly; chapter titles editable inline; title-only changes correctly included in Save All
- Grabber: storage toggle on the Convert page — choose between DB storage and EPUB file before converting
---
## v0.1.2 — 2026-04-02
### New feature
- Restore functionality on the Backup page: browse any available Dropbox snapshot, see which files are currently missing from disk, and restore individual books or a selection back to the library — file is written to disk and immediately re-indexed
---
## v0.1.1 — 2026-03-31 ## v0.1.1 — 2026-03-31
Bug fixes, volume-aware duplicate detection, shared code cleanup, and a new Changelog page. Bug fixes, volume-aware duplicate detection, shared code cleanup, and a new Changelog page.

View File

@ -1 +1 @@
v0.2.8 v0.1.2