17 Commits

Author SHA1 Message Date
11fccdb05f ci(goreleaser): Install templ before generating templates
Adds a `go install` hook to `.goreleaser.yaml` to ensure the `templ` binary is
installed and up-to-date before `templ generate` is executed. This prevents
potential build failures in CI/CD environments where `templ` might not be
pre-installed or could be an outdated version, making the release process
more robust and self-contained.
2025-10-06 18:20:47 +11:00
af968cbc9a ci(workflow): Upgrade GitHub Actions in release workflow
Updated the major versions of several GitHub Actions used in the release workflow:
- actions/checkout from v4 to v5
- actions/setup-go from v4 to v5
- goreleaser/goreleaser-action from v5 to v6

This ensures we are using the latest features, bug fixes, and security updates provided by these actions.
2025-10-06 18:11:27 +11:00
08e6280c34 feat: Add NFPM packaging and defer Playwright installation
This commit introduces NFPM configuration in `.goreleaser.yaml` to
generate native packages for various Linux distributions (e.g., .deb,
.rpm, .apk). This provides a more streamlined installation experience
for Linux users.

The Playwright browser installation logic has been moved from `main.go`
to the `Run` function of the `download` command. This change ensures
that Playwright binaries are only downloaded and installed when the
`download` command is actually invoked, improving initial application
startup performance and reducing unnecessary overhead for other commands.

The Goreleaser configuration has also been updated to version 2 syntax
and the `arm` architecture has been removed from builds.
2025-10-06 18:07:54 +11:00
34179b4dc0 Create LICENSE 2025-10-06 18:03:04 +11:00
b0f8f31dcc feat: Add concurrency and headless options for downloads
This commit introduces new features for controlling the download process:

-   **Concurrency**: Users can now specify the number of concurrent volume downloads using the `--concurrency` flag. This significantly speeds up the download of entire novels.
-   **Headless Mode**: A `--headless` flag has been added to control whether the browser operates in headless mode (without a visible UI). This is useful for debugging or running in environments without a display.

**Changes include:**

-   Updated `download` command to accept `--concurrency` and `--headless` flags.
-   Refactored `bilinovel` downloader to support `BilinovelNewOption` for configuring headless mode and concurrency.
-   Implemented a page pool and concurrency control mechanism within the `bilinovel` downloader to manage concurrent browser page usage.
-   Added `DownloadNovel` and `DownloadVolume` methods to the `bilinovel` downloader, utilizing goroutines and wait groups for parallel processing.
-   Updated `.vscode/launch.json` with new configurations for testing novel and volume downloads with the new options.
2025-10-06 10:20:36 +11:00
6084386989 refactor(bilinovel): Migrate browser automation from Chromedp to Playwright
This commit replaces the `chromedp` library with `playwright-go` for browser automation within the Bilinovel downloader.

Changes include:
*   Updated `Bilinovel` struct to manage Playwright browser, context, and page instances.
*   Rewrote `initBrowser` and `Close` methods to use Playwright's API for browser lifecycle management.
*   Refactored `processContentWithChromedp` to `processContentWithPlaywright`, adapting the logic to use Playwright's page evaluation capabilities.
*   Removed unused `context` and `time` imports.
*   Added HTML cleanup in `getChapterByPage` to remove `class` attributes from images and `data-k` attributes from all elements, improving content consistency.
2025-10-06 07:58:31 +11:00
f1320cb978 Merge pull request #2 from sarymo/patch-3
fix: normalize path separators in wrapper.go
2025-09-03 13:02:07 +10:00
sarymo
434d5f54bd Update wrapper.go 2025-09-03 08:39:30 +08:00
b8cd053b00 refactor: improve network event handling and cleanup of hidden elements in Bilinovel processing 2025-08-24 20:51:09 +10:00
560cdfdec9 refactor: streamline download process and enhance browser handling in Bilinovel 2025-08-24 19:04:00 +10:00
ed5440f5fb Update dependencies and templates: bump templ to v0.3.943, update pflag to v1.0.7, and golang.org/x/net to v0.43.0; adjust XML declaration formatting in cover template 2025-08-24 17:11:14 +10:00
26f82dd9ea fix: trim whitespace from text before writing to chapter file 2025-08-24 16:57:51 +10:00
e9fbe5c5db refactor 2025-08-24 16:53:05 +10:00
75745b9431 fix: kavita 无法正确加载字体 2025-07-17 14:03:10 +08:00
ca3fdf8980 Update templ dependency to v0.3.906 and adjust XML declaration formatting in templates 2025-07-16 21:02:15 +08:00
042b383988 u 2025-05-03 13:40:27 +10:00
c9a7853cef fix font error 2025-04-21 14:59:51 +10:00
32 changed files with 1296 additions and 860 deletions

28
.github/workflows/release.yml vendored Normal file
View File

@@ -0,0 +1,28 @@
name: release
on:
push:
tags:
- "v*"
permissions:
contents: write
jobs:
goreleaser:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v5
- name: Set up Go
uses: actions/setup-go@v5
- name: Run GoReleaser
uses: goreleaser/goreleaser-action@v6
with:
distribution: goreleaser
version: latest
args: release --clean
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

View File

@@ -1,6 +1,8 @@
version: 2
project_name: bilinovel-downloader
before:
hooks:
- go install github.com/a-h/templ/cmd/templ@latest
- templ generate
builds:
- env:
@@ -12,16 +14,15 @@ builds:
goarch:
- amd64
- arm64
- arm
- "386"
ldflags:
- -s -w -X bilinovel-downloader/cmd.Version={{ .Version }}
flags:
- -trimpath
archives:
- format: tar.gz
- formats: ["tar.gz"]
format_overrides:
- format: zip
- formats: ["zip"]
goos: windows
wrap_in_directory: true
release:
@@ -29,3 +30,17 @@ release:
upx:
- enabled: true
compress: best
nfpms:
- id: bilinovel-downloader
homepage: https://github.com/bestnite/bilinovel-downloader
maintainer: Nite <admin@nite07.com>
license: "MIT"
formats:
- apk
- deb
- rpm
- termux.deb
- archlinux
provides:
- bilinovel-downloader

23
.vscode/launch.json vendored
View File

@@ -2,12 +2,31 @@
"version": "0.2.0",
"configurations": [
{
"name": "Debug download volume",
"name": "novel",
"type": "go",
"request": "launch",
"mode": "auto",
"program": "${workspaceFolder}",
"args": ["download", "volume", "-n", "2013", "-v", "165880"]
"args": [
"download",
"-n",
"2727",
"--concurrency",
"5"
]
},
{
"name": "volume",
"type": "go",
"request": "launch",
"mode": "auto",
"program": "${workspaceFolder}",
"args": [
"download",
"-n=2727",
"-v=150098",
"--headless=false"
]
}
]
}

21
LICENSE Normal file
View File

@@ -0,0 +1,21 @@
MIT License
Copyright (c) 2025 Nite
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

View File

@@ -1,24 +1,24 @@
# BiliNovel Downloader
# Bilinovel Downloader
这是一个用于下载和生成轻小说 EPUB 电子书的工具。
这是一个用于从 Bilinovel 下载和生成轻小说 EPUB 电子书的工具。
生成的 EPUB 文件完全符合 EPUB 标准,可以在 Calibre 检查中无错误通过。
## 功能特点
## 使用示例
- 支持下载轻小说并转换为标准 EPUB 格式
- 自动处理图片和文本内容
- 生成符合 EPUB 3.0 规范的电子书文件
- 支持多章节内容的组织和管理
- 保留原有插图和排版格式
1. 下载整本 `https://www.bilinovel.com/novel/2388.html`
## 使用说明
```bash
bilinovel-downloader download -n 2388
```
1. 确保系统环境满足要求
2. 运行下载器获取小说内容
3. 程序会自动处理并生成标准格式的 EPUB 文件
4. 生成的电子书文件可以在任何支持 EPUB 3.0 的阅读器中打开
2. 下载单卷 `https://www.bilinovel.com/novel/2388/vol_84522.html`
## 注意事项
```bash
bilinovel-downloader download -n 2388 -v 84522
```
- 生成的 EPUB 文件严格遵循 EPUB 3.0 规范
- 建议使用支持 EPUB 3.0 的阅读器以获得最佳阅读体验
- 请遵守相关法律法规,合理使用下载的内容
3. 对自动生成的 epub 格式不满意可以自行修改后使用命令打包
```bash
bilinovel-downloader pack -d <目录路径>
```

View File

@@ -1,9 +1,19 @@
package cmd
import (
"bilinovel-downloader/downloader"
"bilinovel-downloader/downloader/bilinovel"
"bilinovel-downloader/epub"
"bilinovel-downloader/model"
"bilinovel-downloader/text"
"encoding/json"
"fmt"
"io"
"log"
"os"
"path/filepath"
"github.com/playwright-community/playwright-go"
"github.com/spf13/cobra"
)
@@ -11,74 +21,182 @@ var downloadCmd = &cobra.Command{
Use: "download",
Short: "Download a novel or volume",
Long: "Download a novel or volume",
Run: func(cmd *cobra.Command, args []string) {
log.Println("Installing playwright")
err := playwright.Install(&playwright.RunOptions{
Browsers: []string{"chromium"},
Stdout: io.Discard,
})
if err != nil {
log.Panicf("failed to install playwright")
}
err = runDownloadNovel()
if err != nil {
log.Printf("failed to download novel: %v", err)
}
},
}
var downloadNovelCmd = &cobra.Command{
Use: "novel",
Short: "Download a novel, default download all volumes",
Long: "Download a novel, default download all volumes",
RunE: runDownloadNovel,
}
var downloadVolumeCmd = &cobra.Command{
Use: "volume",
Short: "Download a volume",
Long: "Download a volume",
RunE: runDownloadVolume,
}
type downloadNovelArgs struct {
NovelId int `validate:"required"`
outputPath string
}
type downloadVolumeArgs struct {
NovelId int `validate:"required"`
VolumeId int `validate:"required"`
outputPath string
type downloadCmdArgs struct {
NovelId int `validate:"required"`
VolumeId int `validate:"required"`
outputPath string
outputType string
headless bool
concurrency int
}
var (
novelArgs downloadNovelArgs
volumeArgs downloadVolumeArgs
downloadArgs downloadCmdArgs
)
func init() {
downloadNovelCmd.Flags().IntVarP(&novelArgs.NovelId, "novel-id", "n", 0, "novel id")
downloadNovelCmd.Flags().StringVarP(&novelArgs.outputPath, "output-path", "o", "./novels", "output path")
downloadVolumeCmd.Flags().IntVarP(&volumeArgs.NovelId, "novel-id", "n", 0, "novel id")
downloadVolumeCmd.Flags().IntVarP(&volumeArgs.VolumeId, "volume-id", "v", 0, "volume id")
downloadVolumeCmd.Flags().StringVarP(&volumeArgs.outputPath, "output-path", "o", "./novels", "output path")
downloadCmd.AddCommand(downloadNovelCmd)
downloadCmd.AddCommand(downloadVolumeCmd)
downloadCmd.Flags().IntVarP(&downloadArgs.NovelId, "novel-id", "n", 0, "novel id")
downloadCmd.Flags().IntVarP(&downloadArgs.VolumeId, "volume-id", "v", 0, "volume id")
downloadCmd.Flags().StringVarP(&downloadArgs.outputPath, "output-path", "o", "novels", "output path")
downloadCmd.Flags().StringVarP(&downloadArgs.outputType, "output-type", "t", "epub", "output type, epub or text")
downloadCmd.Flags().BoolVar(&downloadArgs.headless, "headless", true, "headless mode")
downloadCmd.Flags().IntVar(&downloadArgs.concurrency, "concurrency", 3, "concurrency of downloading volumes")
RootCmd.AddCommand(downloadCmd)
}
func runDownloadNovel(cmd *cobra.Command, args []string) error {
if novelArgs.NovelId == 0 {
func runDownloadNovel() error {
downloader, err := bilinovel.New(bilinovel.BilinovelNewOption{
Headless: downloadArgs.headless,
Concurrency: downloadArgs.concurrency,
})
if err != nil {
return fmt.Errorf("failed to create downloader: %v", err)
}
// 确保在函数结束时关闭资源
defer func() {
if closeErr := downloader.Close(); closeErr != nil {
log.Printf("Failed to close downloader: %v", closeErr)
}
}()
if downloadArgs.NovelId == 0 {
return fmt.Errorf("novel id is required")
}
err := bilinovel.DownloadNovel(novelArgs.NovelId, novelArgs.outputPath)
if err != nil {
return fmt.Errorf("failed to download novel: %v", err)
if downloadArgs.VolumeId == 0 {
// 下载整本小说
err := downloadNovel(downloader, downloadArgs.NovelId)
if err != nil {
return fmt.Errorf("failed to get novel: %v", err)
}
} else {
// 下载单卷
err = downloadVolume(downloader, downloadArgs.VolumeId)
if err != nil {
return fmt.Errorf("failed to download volume: %v", err)
}
}
return nil
}
func runDownloadVolume(cmd *cobra.Command, args []string) error {
if volumeArgs.NovelId == 0 {
return fmt.Errorf("novel id is required")
}
if volumeArgs.VolumeId == 0 {
return fmt.Errorf("volume id is required")
}
err := bilinovel.DownloadVolume(volumeArgs.NovelId, volumeArgs.VolumeId, volumeArgs.outputPath)
func downloadNovel(downloader downloader.Downloader, novelId int) error {
novelInfo, err := downloader.GetNovel(novelId, true, nil)
if err != nil {
return fmt.Errorf("failed to download volume: %v", err)
return fmt.Errorf("failed to get novel info: %w", err)
}
skipVolumes := make([]int, 0)
for _, volume := range novelInfo.Volumes {
jsonPath := filepath.Join(downloadArgs.outputPath, fmt.Sprintf("volume-%d-%d.json", downloadArgs.NovelId, volume.Id))
err = os.MkdirAll(filepath.Dir(jsonPath), 0755)
if err != nil {
return fmt.Errorf("failed to create directory: %v", err)
}
_, err = os.Stat(jsonPath)
if err == nil {
// 已经下载
skipVolumes = append(skipVolumes, volume.Id)
}
}
novel, err := downloader.GetNovel(novelId, false, skipVolumes)
if err != nil {
return fmt.Errorf("failed to download novel: %w", err)
}
for _, volume := range novel.Volumes {
jsonPath := filepath.Join(downloadArgs.outputPath, fmt.Sprintf("volume-%d-%d.json", downloadArgs.NovelId, volume.Id))
err = os.MkdirAll(filepath.Dir(jsonPath), 0755)
if err != nil {
return fmt.Errorf("failed to create directory: %v", err)
}
jsonFile, err := os.Create(jsonPath)
if err != nil {
return fmt.Errorf("failed to create json file: %v", err)
}
err = json.NewEncoder(jsonFile).Encode(volume)
if err != nil {
return fmt.Errorf("failed to encode json file: %v", err)
}
switch downloadArgs.outputType {
case "epub":
err = epub.PackVolumeToEpub(volume, downloadArgs.outputPath, downloader.GetStyleCSS(), downloader.GetExtraFiles())
if err != nil {
return fmt.Errorf("failed to pack volume: %v", err)
}
case "text":
err = text.PackVolumeToText(volume, downloadArgs.outputPath)
if err != nil {
return fmt.Errorf("failed to pack volume: %v", err)
}
}
}
return nil
}
func downloadVolume(downloader downloader.Downloader, volumeId int) error {
jsonPath := filepath.Join(downloadArgs.outputPath, fmt.Sprintf("volume-%d-%d.json", downloadArgs.NovelId, volumeId))
err := os.MkdirAll(filepath.Dir(jsonPath), 0755)
if err != nil {
return fmt.Errorf("failed to create directory: %v", err)
}
_, err = os.Stat(jsonPath)
volume := &model.Volume{}
if err != nil {
if os.IsNotExist(err) {
volume, err = downloader.GetVolume(downloadArgs.NovelId, volumeId, false)
if err != nil {
return fmt.Errorf("failed to get volume: %v", err)
}
jsonFile, err := os.Create(jsonPath)
if err != nil {
return fmt.Errorf("failed to create json file: %v", err)
}
err = json.NewEncoder(jsonFile).Encode(volume)
if err != nil {
return fmt.Errorf("failed to encode json file: %v", err)
}
} else {
return fmt.Errorf("failed to get volume: %v", err)
}
} else {
jsonFile, err := os.Open(jsonPath)
if err != nil {
return fmt.Errorf("failed to open json file: %v", err)
}
defer jsonFile.Close()
err = json.NewDecoder(jsonFile).Decode(volume)
if err != nil {
return fmt.Errorf("failed to decode json file: %v", err)
}
}
switch downloadArgs.outputType {
case "epub":
err = epub.PackVolumeToEpub(volume, downloadArgs.outputPath, downloader.GetStyleCSS(), downloader.GetExtraFiles())
if err != nil {
return fmt.Errorf("failed to pack volume: %v", err)
}
case "text":
err = text.PackVolumeToText(volume, downloadArgs.outputPath)
if err != nil {
return fmt.Errorf("failed to pack volume: %v", err)
}
}
return nil
}

View File

@@ -1,7 +1,7 @@
package cmd
import (
"bilinovel-downloader/downloader/bilinovel"
"bilinovel-downloader/epub"
"fmt"
"github.com/spf13/cobra"
@@ -28,7 +28,7 @@ func init() {
}
func runPackage(cmd *cobra.Command, args []string) error {
err := bilinovel.CreateEpub(pArgs.DirPath)
err := epub.PackEpub(pArgs.DirPath)
if err != nil {
return fmt.Errorf("failed to create epub: %v", err)
}

View File

@@ -4,4 +4,6 @@ import (
"github.com/spf13/cobra"
)
var RootCmd = &cobra.Command{}
var RootCmd = &cobra.Command{
Use: "bilinovel-downloader",
}

Binary file not shown.

View File

@@ -2,12 +2,10 @@ package bilinovel
import (
"bilinovel-downloader/model"
"bilinovel-downloader/template"
"bilinovel-downloader/utils"
"bytes"
"context"
"crypto/sha256"
_ "embed"
"encoding/json"
"fmt"
"log"
"net/http"
@@ -15,19 +13,121 @@ import (
"path"
"path/filepath"
"regexp"
"slices"
"strconv"
"strings"
"time"
"sync"
"github.com/PuerkitoBio/goquery"
"github.com/google/uuid"
mapper "github.com/bestnite/font-mapper"
"github.com/playwright-community/playwright-go"
)
func GetNovel(novelId int) (*model.Novel, error) {
novelUrl := fmt.Sprintf("https://www.bilinovel.com/novel/%v.html", novelId)
resp, err := utils.Request().Get(novelUrl)
//go:embed read.ttf
var readTTF []byte
//go:embed "MI LANTING.ttf"
var miLantingTTF []byte
type Bilinovel struct {
fontMapper *mapper.GlyphOutlineMapper
textOnly bool
restyClient *utils.RestyClient
// 浏览器实例复用
browser playwright.Browser
browserContext playwright.BrowserContext
pages map[string]playwright.Page
concurrency int
concurrentChan chan any
}
type BilinovelNewOption struct {
Headless bool
Concurrency int
}
func New(option BilinovelNewOption) (*Bilinovel, error) {
fontMapper, err := mapper.NewGlyphOutlineMapper(readTTF, miLantingTTF)
if err != nil {
return nil, fmt.Errorf("failed to get novel info: %v", err)
return nil, fmt.Errorf("failed to create font mapper: %v", err)
}
restyClient := utils.NewRestyClient(50)
b := &Bilinovel{
fontMapper: fontMapper,
textOnly: false,
restyClient: restyClient,
pages: make(map[string]playwright.Page),
concurrency: option.Concurrency,
concurrentChan: make(chan any, option.Concurrency),
}
// 初始化浏览器实例
err = b.initBrowser(option.Headless)
if err != nil {
return nil, fmt.Errorf("failed to init browser: %v", err)
}
return b, nil
}
func (b *Bilinovel) SetTextOnly(textOnly bool) {
b.textOnly = textOnly
}
func (b *Bilinovel) GetExtraFiles() []model.ExtraFile {
return nil
}
// initBrowser 初始化浏览器实例
func (b *Bilinovel) initBrowser(headless bool) error {
pw, err := playwright.Run()
if err != nil {
return fmt.Errorf("could not start playwright: %w", err)
}
b.browser, err = pw.Chromium.Launch(playwright.BrowserTypeLaunchOptions{
Headless: playwright.Bool(headless),
})
if err != nil {
return fmt.Errorf("could not launch browser: %w", err)
}
b.browserContext, err = b.browser.NewContext()
if err != nil {
return fmt.Errorf("could not create browser context: %w", err)
}
log.Println("Browser initialized successfully")
return nil
}
// Close 清理资源
func (b *Bilinovel) Close() error {
if b.browser != nil {
if err := b.browser.Close(); err != nil {
log.Printf("could not close browser: %v", err)
}
b.browser = nil
b.browserContext = nil
}
return nil
}
//go:embed style.css
var styleCSS []byte
func (b *Bilinovel) GetStyleCSS() string {
return string(styleCSS)
}
func (b *Bilinovel) GetNovel(novelId int, skipChapterContent bool, skipVolumes []int) (*model.Novel, error) {
log.Printf("Getting novel %v\n", novelId)
novelUrl := fmt.Sprintf("https://www.bilinovel.com/novel/%v.html", novelId)
resp, err := b.restyClient.R().Get(novelUrl)
if err != nil {
return nil, fmt.Errorf("failed to get novel info: %w", err)
}
if resp.StatusCode() != http.StatusOK {
return nil, fmt.Errorf("failed to get novel info: %v", resp.Status())
@@ -51,7 +151,7 @@ func GetNovel(novelId int) (*model.Novel, error) {
novel.Authors = append(novel.Authors, strings.TrimSpace(s.Text()))
})
volumes, err := getNovelVolumes(novelId)
volumes, err := b.getAllVolumes(novelId, skipChapterContent, skipVolumes)
if err != nil {
return nil, fmt.Errorf("failed to get novel volumes: %v", err)
}
@@ -60,11 +160,13 @@ func GetNovel(novelId int) (*model.Novel, error) {
return novel, nil
}
func GetVolume(novelId int, volumeId int) (*model.Volume, error) {
func (b *Bilinovel) GetVolume(novelId int, volumeId int, skipChapterContent bool) (*model.Volume, error) {
log.Printf("Getting volume %v of novel %v\n", volumeId, novelId)
novelUrl := fmt.Sprintf("https://www.bilinovel.com/novel/%v/catalog", novelId)
resp, err := utils.Request().Get(novelUrl)
resp, err := b.restyClient.R().Get(novelUrl)
if err != nil {
return nil, fmt.Errorf("failed to get novel info: %v", err)
return nil, fmt.Errorf("failed to get novel info: %w", err)
}
if resp.StatusCode() != http.StatusOK {
return nil, fmt.Errorf("failed to get novel info: %v", resp.Status())
@@ -89,7 +191,7 @@ func GetVolume(novelId int, volumeId int) (*model.Volume, error) {
}
volumeUrl := fmt.Sprintf("https://www.bilinovel.com/novel/%v/vol_%v.html", novelId, volumeId)
resp, err = utils.Request().Get(volumeUrl)
resp, err = b.restyClient.R().Get(volumeUrl)
if err != nil {
return nil, fmt.Errorf("failed to get novel info: %v", err)
}
@@ -109,9 +211,14 @@ func GetVolume(novelId int, volumeId int) (*model.Volume, error) {
volume.SeriesIdx = seriesIdx
volume.Title = strings.TrimSpace(doc.Find(".book-title").First().Text())
volume.Description = strings.TrimSpace(doc.Find(".book-summary>content").First().Text())
volume.Cover = doc.Find(".book-cover").First().AttrOr("src", "")
volume.Url = volumeUrl
volume.Chapters = make([]*model.Chapter, 0)
volume.CoverUrl = doc.Find(".book-cover").First().AttrOr("src", "")
cover, err := b.getImg(volume.CoverUrl)
if err != nil {
return nil, fmt.Errorf("failed to get cover: %v", err)
}
volume.Cover = cover
doc.Find(".authorname>a").Each(func(i int, s *goquery.Selection) {
volume.Authors = append(volume.Authors, strings.TrimSpace(s.Text()))
@@ -119,7 +226,6 @@ func GetVolume(novelId int, volumeId int) (*model.Volume, error) {
doc.Find(".illname>a").Each(func(i int, s *goquery.Selection) {
volume.Authors = append(volume.Authors, strings.TrimSpace(s.Text()))
})
doc.Find(".chapter-li.jsChapter").Each(func(i int, s *goquery.Selection) {
volume.Chapters = append(volume.Chapters, &model.Chapter{
Title: s.Find("a").Text(),
@@ -127,12 +233,36 @@ func GetVolume(novelId int, volumeId int) (*model.Volume, error) {
})
})
idRegexp := regexp.MustCompile(`/novel/(\d+)/(\d+).html`)
if !skipChapterContent {
for i := range volume.Chapters {
matches := idRegexp.FindStringSubmatch(volume.Chapters[i].Url)
if len(matches) > 0 {
chapterId, err := strconv.Atoi(matches[2])
if err != nil {
return nil, fmt.Errorf("failed to convert chapter id: %v", err)
}
chapter, err := b.GetChapter(novelId, volumeId, chapterId)
if err != nil {
return nil, fmt.Errorf("failed to get chapter: %v", err)
}
chapter.Id = chapterId
volume.Chapters[i] = chapter
} else {
return nil, fmt.Errorf("failed to get chapter id: %v", volume.Chapters[i].Url)
}
}
}
return volume, nil
}
func getNovelVolumes(novelId int) ([]*model.Volume, error) {
func (b *Bilinovel) getAllVolumes(novelId int, skipChapterContent bool, skipVolumes []int) ([]*model.Volume, error) {
log.Printf("Getting all volumes of novel %v\n", novelId)
catelogUrl := fmt.Sprintf("https://www.bilinovel.com/novel/%v/catalog", novelId)
resp, err := utils.Request().Get(catelogUrl)
resp, err := b.restyClient.R().Get(catelogUrl)
if err != nil {
return nil, fmt.Errorf("failed to get catelog: %v", err)
}
@@ -156,232 +286,94 @@ func getNovelVolumes(novelId int) ([]*model.Volume, error) {
}
})
volumes := make([]*model.Volume, 0)
volumes := make([]*model.Volume, len(volumeIds))
var wg sync.WaitGroup
var mu sync.Mutex // 保护 volumes 写入的互斥锁
for i, volumeIdStr := range volumeIds {
volumeId, err := strconv.Atoi(volumeIdStr)
if err != nil {
return nil, fmt.Errorf("failed to convert volume id: %v", err)
}
volume, err := GetVolume(novelId, volumeId)
if err != nil {
return nil, fmt.Errorf("failed to get volume info: %v", err)
}
volume.SeriesIdx = i
volumes = append(volumes, volume)
}
wg.Add(1)
b.concurrentChan <- struct{}{} // 获取一个并发槽
return volumes, nil
}
go func(i int, volumeIdStr string) {
defer wg.Done()
defer func() { <-b.concurrentChan }() // 释放并发槽
func DownloadNovel(novelId int, outputPath string) error {
log.Printf("Downloading Novel: %v", novelId)
novel, err := GetNovel(novelId)
if err != nil {
return fmt.Errorf("failed to get novel info: %v", err)
}
outputPath = filepath.Join(outputPath, utils.CleanDirName(novel.Title))
err = os.MkdirAll(outputPath, 0755)
if err != nil {
return fmt.Errorf("failed to create output directory: %v", err)
}
for _, volume := range novel.Volumes {
err := downloadVolume(volume, outputPath)
if err != nil {
return fmt.Errorf("failed to download volume: %v", err)
}
}
return nil
}
func DownloadVolume(novelId, volumeId int, outputPath string) error {
volume, err := GetVolume(novelId, volumeId)
if err != nil {
return fmt.Errorf("failed to get volume info: %v", err)
}
err = downloadVolume(volume, outputPath)
if err != nil {
return fmt.Errorf("failed to download volume: %v", err)
}
return nil
}
func downloadVolume(volume *model.Volume, outputPath string) error {
log.Printf("Downloading Volume: %s", volume.Title)
outputPath = filepath.Join(outputPath, utils.CleanDirName(volume.Title))
err := os.MkdirAll(outputPath, 0755)
if err != nil {
return fmt.Errorf("failed to create output directory: %v", err)
}
_, err = os.Stat(filepath.Join(outputPath, "volume.json"))
if os.IsNotExist(err) {
for idx, chapter := range volume.Chapters {
err := DownloadChapter(idx, chapter, outputPath)
volumeId, err := strconv.Atoi(volumeIdStr)
if err != nil {
return fmt.Errorf("failed to download chapter: %v", err)
log.Printf("failed to convert volume id %s: %v", volumeIdStr, err)
return
}
}
} else {
jsonBytes, err := os.ReadFile(filepath.Join(outputPath, "volume.json"))
if err != nil {
return fmt.Errorf("failed to read volume: %v", err)
}
err = json.Unmarshal(jsonBytes, volume)
if err != nil {
return fmt.Errorf("failed to unmarshal volume: %v", err)
}
for idx, chapter := range volume.Chapters {
file, err := os.Create(filepath.Join(outputPath, fmt.Sprintf("OEBPS/Text/chapter-%03v.xhtml", idx+1)))
if slices.Contains(skipVolumes, volumeId) {
return
}
volume, err := b.GetVolume(novelId, volumeId, skipChapterContent)
if err != nil {
return fmt.Errorf("failed to create chapter file: %v", err)
log.Printf("failed to get volume info for novel %d, volume %d: %v", novelId, volumeId, err)
return
}
err = template.ContentXHTML(chapter).Render(context.Background(), file)
if err != nil {
return fmt.Errorf("failed to render text file: %v", err)
volume.SeriesIdx = i
// 关闭浏览器标签页
pwPageKey := fmt.Sprintf("%v-%v", novelId, volumeId)
if pwPage, ok := b.pages[pwPageKey]; ok {
_ = pwPage.Close()
delete(b.pages, pwPageKey)
}
mu.Lock()
volumes[i] = volume
mu.Unlock()
}(i, volumeIdStr)
}
wg.Wait()
// 过滤掉获取失败的 nil volume
filteredVolumes := make([]*model.Volume, 0, len(volumes))
for _, vol := range volumes {
if vol != nil {
filteredVolumes = append(filteredVolumes, vol)
}
}
for i := range volume.Chapters {
volume.Chapters[i].ImageFullPaths = utils.Unique(volume.Chapters[i].ImageFullPaths)
volume.Chapters[i].ImageOEBPSPaths = utils.Unique(volume.Chapters[i].ImageOEBPSPaths)
}
jsonBytes, err := json.Marshal(volume)
if err != nil {
return fmt.Errorf("failed to marshal volume: %v", err)
}
err = os.WriteFile(filepath.Join(outputPath, "volume.json"), jsonBytes, 0644)
if err != nil {
return fmt.Errorf("failed to write volume: %v", err)
}
coverPath := filepath.Join(outputPath, "OEBPS/Images/cover.jpg")
err = os.MkdirAll(path.Dir(coverPath), 0755)
if err != nil {
return fmt.Errorf("failed to create cover directory: %v", err)
}
err = DownloadImg(volume.Cover, coverPath)
if err != nil {
return fmt.Errorf("failed to download cover: %v", err)
}
coverXHTMLPath := filepath.Join(outputPath, "OEBPS/Text/cover.xhtml")
err = os.MkdirAll(path.Dir(coverXHTMLPath), 0755)
if err != nil {
return fmt.Errorf("failed to create cover directory: %v", err)
}
file, err := os.Create(coverXHTMLPath)
if err != nil {
return fmt.Errorf("failed to create cover file: %v", err)
}
err = template.ContentXHTML(&model.Chapter{
Title: "封面",
Content: fmt.Sprintf(`<img src="../Images/cover%s" />`, path.Ext(volume.Cover)),
}).Render(context.Background(), file)
if err != nil {
return fmt.Errorf("failed to render cover: %v", err)
}
err = DownloadFont(filepath.Join(outputPath, "OEBPS/Fonts"))
if err != nil {
return fmt.Errorf("failed to download font: %v", err)
}
contentsXHTMLPath := filepath.Join(outputPath, "OEBPS/Text/contents.xhtml")
err = os.MkdirAll(path.Dir(contentsXHTMLPath), 0755)
if err != nil {
return fmt.Errorf("failed to create contents directory: %v", err)
}
file, err = os.Create(contentsXHTMLPath)
if err != nil {
return fmt.Errorf("failed to create contents file: %v", err)
}
contents := strings.Builder{}
contents.WriteString(`<nav epub:type="toc" id="toc">`)
contents.WriteString(`<ol>`)
for _, chapter := range volume.Chapters {
contents.WriteString(fmt.Sprintf(`<li><a href="%s">%s</a></li>`, strings.TrimPrefix(chapter.TextOEBPSPath, "Text/"), chapter.Title))
}
contents.WriteString(`</ol>`)
contents.WriteString(`</nav>`)
err = template.ContentXHTML(&model.Chapter{
Title: "目录",
Content: contents.String(),
}).Render(context.Background(), file)
if err != nil {
return fmt.Errorf("failed to render contents: %v", err)
}
err = CreateContainerXML(outputPath)
if err != nil {
return fmt.Errorf("failed to create container xml: %v", err)
}
u, err := uuid.NewV7()
if err != nil {
return fmt.Errorf("failed to generate uuid: %v", err)
}
err = CreateContentOPF(outputPath, u.String(), volume)
if err != nil {
return fmt.Errorf("failed to create content opf: %v", err)
}
err = CreateTocNCX(outputPath, u.String(), volume)
if err != nil {
return fmt.Errorf("failed to create toc ncx: %v", err)
}
err = CreateEpub(outputPath)
if err != nil {
return fmt.Errorf("failed to create epub: %v", err)
}
return nil
return filteredVolumes, nil
}
func DownloadChapter(chapterIdx int, chapter *model.Chapter, outputPath string) error {
chapter.TextFullPath = filepath.Join(outputPath, fmt.Sprintf("OEBPS/Text/chapter-%03v.xhtml", chapterIdx+1))
chapter.TextOEBPSPath = fmt.Sprintf("Text/chapter-%03v.xhtml", chapterIdx+1)
err := os.MkdirAll(path.Dir(chapter.TextFullPath), 0755)
if err != nil {
return fmt.Errorf("failed to create text directory: %v", err)
}
func (b *Bilinovel) GetChapter(novelId int, volumeId int, chapterId int) (*model.Chapter, error) {
log.Printf("Getting chapter %v of novel %v\n", chapterId, novelId)
page := 1
pageNum := 1
chapter := &model.Chapter{
Id: chapterId,
NovelId: novelId,
VolumeId: volumeId,
Url: fmt.Sprintf("https://www.bilinovel.com/novel/%v/%v.html", novelId, chapterId),
}
for {
hasNext, err := downloadChapterByPage(page, chapterIdx, chapter, outputPath)
pwPageKey := fmt.Sprintf("%v-%v", novelId, volumeId)
if _, ok := b.pages[pwPageKey]; !ok {
pwPage, err := b.browserContext.NewPage()
if err != nil {
return nil, fmt.Errorf("failed to create browser page: %w", err)
}
b.pages[pwPageKey] = pwPage
}
hasNext, err := b.getChapterByPage(b.pages[pwPageKey], chapter, pageNum)
if err != nil {
return fmt.Errorf("failed to download chapter: %v", err)
return nil, fmt.Errorf("failed to download chapter: %w", err)
}
if !hasNext {
break
}
page++
time.Sleep(time.Second)
pageNum++
}
file, err := os.Create(chapter.TextFullPath)
if err != nil {
return fmt.Errorf("failed to create text file: %v", err)
}
err = template.ContentXHTML(chapter).Render(context.Background(), file)
if err != nil {
return fmt.Errorf("failed to render text file: %v", err)
}
return nil
return chapter, nil
}
func downloadChapterByPage(page, chapterIdx int, chapter *model.Chapter, outputPath string) (bool, error) {
Url := strings.TrimSuffix(chapter.Url, ".html") + fmt.Sprintf("_%v.html", page)
log.Printf("Downloading Chapter: %s", Url)
func (b *Bilinovel) getChapterByPage(pwPage playwright.Page, chapter *model.Chapter, pageNum int) (bool, error) {
log.Printf("Getting chapter %v by page %v\n", chapter.Id, pageNum)
Url := strings.TrimSuffix(chapter.Url, ".html") + fmt.Sprintf("_%v.html", pageNum)
hasNext := false
headers := map[string]string{
@@ -389,298 +381,193 @@ func downloadChapterByPage(page, chapterIdx int, chapter *model.Chapter, outputP
"Accept-Language": "zh-CN,zh;q=0.9,en-GB;q=0.8,en;q=0.7,zh-TW;q=0.6",
"Cookie": "night=1;",
}
resp, err := utils.Request().SetHeaders(headers).Get(Url)
resp, err := b.restyClient.R().SetHeaders(headers).Get(Url)
if err != nil {
return hasNext, err
return false, fmt.Errorf("failed to get chapter: %w", err)
}
if resp.StatusCode() != http.StatusOK {
return hasNext, fmt.Errorf("failed to get chapter: %v", resp.Status())
return false, fmt.Errorf("failed to get chapter: %v", resp.Status())
}
if strings.Contains(resp.String(), `<a onclick="window.location.href = ReadParams.url_next;">下一頁</a>`) {
hasNext = true
}
doc, err := goquery.NewDocumentFromReader(bytes.NewReader(resp.Body()))
html := resp.Body()
// 解决乱序问题
resortedHtml, err := b.processContentWithPlaywright(pwPage, string(html))
if err != nil {
fmt.Println(err)
return hasNext, err
return false, fmt.Errorf("failed to process html: %w", err)
}
doc, err := goquery.NewDocumentFromReader(strings.NewReader(resortedHtml))
if err != nil {
return false, fmt.Errorf("failed to parse html: %w", err)
}
imgSavePath := fmt.Sprintf("OEBPS/Images/chapter-%03v", chapterIdx+1)
if pageNum == 1 {
chapter.Title = doc.Find("#atitle").Text()
}
content := doc.Find("#acontent").First()
content.Find(".cgo").Remove()
content.Find("center").Remove()
content.Find(".google-auto-placed").Remove()
content.Find("p").Last().AddClass("read-font")
content.Find("img").Each(func(i int, s *goquery.Selection) {
if strings.Contains(resp.String(), `font-family: "read"`) {
html, err := content.Find("p").Last().Html()
if err != nil {
return
return false, fmt.Errorf("failed to get html: %v", err)
}
imgUrl := s.AttrOr("data-src", "")
if imgUrl == "" {
imgUrl = s.AttrOr("src", "")
if imgUrl == "" {
return
builder := strings.Builder{}
for _, r := range html {
_, newRune, ok := b.fontMapper.MappingRune(r)
if ok {
builder.WriteRune(newRune)
}
}
fileName := filepath.Join(imgSavePath, fmt.Sprintf("%03v%s", len(chapter.ImageFullPaths)+1, path.Ext(imgUrl)))
err = DownloadImg(imgUrl, filepath.Join(outputPath, fileName))
if err == nil {
s.SetAttr("src", "../"+strings.TrimPrefix(fileName, "OEBPS/"))
s.RemoveAttr("class")
s.RemoveAttr("data-src")
chapter.ImageFullPaths = append(chapter.ImageFullPaths, filepath.Join(outputPath, fileName))
chapter.ImageOEBPSPaths = append(chapter.ImageOEBPSPaths, strings.TrimPrefix(fileName, "OEBPS/"))
}
})
if err != nil {
return false, fmt.Errorf("failed to download img: %v", err)
content.Find("p").Last().SetHtml(builder.String())
}
html, err := content.Html()
if b.textOnly {
content.Find("img").Remove()
} else {
content.Find("img").Each(func(i int, s *goquery.Selection) {
imgUrl := s.AttrOr("data-src", "")
if imgUrl == "" {
imgUrl = s.AttrOr("src", "")
if imgUrl == "" {
return
}
}
imageHash := sha256.Sum256([]byte(imgUrl))
imageFilename := fmt.Sprintf("%x%s", string(imageHash[:]), path.Ext(imgUrl))
s.SetAttr("src", imageFilename)
s.SetAttr("alt", imgUrl)
s.RemoveAttr("class")
img, err := b.getImg(imgUrl)
if err != nil {
return
}
if chapter.Content == nil {
chapter.Content = &model.ChaperContent{}
}
if chapter.Content.Images == nil {
chapter.Content.Images = make(map[string][]byte)
}
chapter.Content.Images[imageFilename] = img
})
}
doc.Find("*").Each(func(i int, s *goquery.Selection) {
if len(s.Nodes) > 0 && len(s.Nodes[0].Attr) > 0 {
// 遍历元素的所有属性
for _, attr := range s.Nodes[0].Attr {
// 3. 检查属性名是否以 "data-k" 开头,且属性值是否为空
if strings.HasPrefix(attr.Key, "data-k") {
// 4. 如果满足条件,就移除这个属性
s.RemoveAttr(attr.Key)
}
}
}
})
htmlStr, err := content.Html()
if err != nil {
return false, fmt.Errorf("failed to get html: %v", err)
}
chapter.Content += strings.TrimSpace(html)
if chapter.Content == nil {
chapter.Content = &model.ChaperContent{}
}
chapter.Content.Html += strings.TrimSpace(htmlStr)
return hasNext, nil
}
func DownloadImg(url string, fileName string) error {
_, err := os.Stat(fileName)
if !os.IsNotExist(err) {
func (b *Bilinovel) getImg(url string) ([]byte, error) {
log.Printf("Getting img %v\n", url)
resp, err := b.restyClient.R().SetHeader("Referer", "https://www.bilinovel.com").Get(url)
if err != nil {
return nil, err
}
return resp.Body(), nil
}
// processContentWithPlaywright 使用复用的浏览器实例处理内容
func (b *Bilinovel) processContentWithPlaywright(page playwright.Page, htmlContent string) (string, error) {
tempFile, err := os.CreateTemp("", "bilinovel-temp-*.html")
if err != nil {
return "", fmt.Errorf("failed to create temp file: %w", err)
}
defer os.Remove(tempFile.Name())
_, err = tempFile.WriteString(htmlContent)
if err != nil {
return "", fmt.Errorf("failed to write temp file: %w", err)
}
tempFile.Close()
tempFilePath := tempFile.Name()
_, err = page.ExpectResponse(func(url string) bool {
return strings.Contains(url, "chapterlog.js")
}, func() error {
_, err = page.Goto("file://" + filepath.ToSlash(tempFilePath))
if err != nil {
return fmt.Errorf("could not navigate to file: %w", err)
}
return nil
}
log.Printf("Downloading Image: %s", url)
dir := filepath.Dir(fileName)
err = os.MkdirAll(dir, 0755)
if err != nil {
return err
}
resp, err := utils.Request().SetHeader("Referer", "https://www.bilinovel.com").Get(url)
if err != nil {
return err
}
err = os.WriteFile(fileName, resp.Body(), 0644)
if err != nil {
return err
}
return nil
}
func CreateContainerXML(dirPath string) error {
containerPath := filepath.Join(dirPath, "META-INF/container.xml")
err := os.MkdirAll(path.Dir(containerPath), 0755)
if err != nil {
return fmt.Errorf("failed to create container directory: %v", err)
}
file, err := os.Create(containerPath)
if err != nil {
return fmt.Errorf("failed to create container file: %v", err)
}
err = template.ContainerXML().Render(context.Background(), file)
if err != nil {
return fmt.Errorf("failed to render container: %v", err)
}
return nil
}
func CreateContentOPF(dirPath string, uuid string, volume *model.Volume) error {
creators := make([]model.DCCreator, 0)
for _, author := range volume.Authors {
creators = append(creators, model.DCCreator{
Value: author,
})
}
dc := &model.DublinCoreMetadata{
Titles: []model.DCTitle{
{
Value: volume.Title,
},
},
Identifiers: []model.DCIdentifier{
{
Value: fmt.Sprintf("urn:uuid:%s", uuid),
ID: "book-id",
// Scheme: "UUID",
},
},
Languages: []model.DCLanguage{
{
Value: "zh-TW",
},
},
Descriptions: []model.DCDescription{
{
Value: volume.Description,
},
},
Creators: creators,
Metas: []model.DublinCoreMeta{
{
Name: "cover",
Content: "images-cover" + path.Ext(volume.Cover),
},
{
Property: "dcterms:modified",
Value: time.Now().UTC().Format("2006-01-02T15:04:05Z"),
},
{
Name: "calibre:series",
Content: volume.NovelTitle,
},
{
Name: "calibre:series_index",
Content: strconv.Itoa(volume.SeriesIdx),
},
},
}
manifest := &model.Manifest{
Items: make([]model.ManifestItem, 0),
}
manifest.Items = append(manifest.Items, model.ManifestItem{
ID: "ncx",
Link: "toc.ncx",
Media: "application/x-dtbncx+xml",
}, playwright.PageExpectResponseOptions{
Timeout: playwright.Float(5000),
})
manifest.Items = append(manifest.Items, model.ManifestItem{
ID: "cover.xhtml",
Link: "Text/cover.xhtml",
Media: "application/xhtml+xml",
if err != nil {
return "", fmt.Errorf("failed to wait for network request finish")
}
err = page.Locator("#acontent").WaitFor(playwright.LocatorWaitForOptions{
State: playwright.WaitForSelectorStateVisible,
})
manifest.Items = append(manifest.Items, model.ManifestItem{
ID: "contents.xhtml",
Link: "Text/contents.xhtml",
Media: "application/xhtml+xml",
Properties: "nav",
})
manifest.Items = append(manifest.Items, model.ManifestItem{
ID: "images-cover" + path.Ext(volume.Cover),
Link: fmt.Sprintf("Images/cover%s", path.Ext(volume.Cover)),
Media: fmt.Sprintf("image/%s", strings.ReplaceAll(strings.TrimPrefix(path.Ext(volume.Cover), "."), "jpg", "jpeg")),
})
manifest.Items = append(manifest.Items, model.ManifestItem{
ID: "read.ttf",
Link: "Fonts/read.ttf",
Media: "application/vnd.ms-opentype",
})
for _, chapter := range volume.Chapters {
manifest.Items = append(manifest.Items, model.ManifestItem{
ID: path.Base(chapter.TextOEBPSPath),
Link: chapter.TextOEBPSPath,
Media: "application/xhtml+xml",
})
for _, image := range chapter.ImageOEBPSPaths {
item := model.ManifestItem{
ID: strings.Join(strings.Split(strings.ToLower(image), string(filepath.Separator)), "-"),
Link: image,
if err != nil {
return "", fmt.Errorf("could not wait for #acontent: %w", err)
}
// 遍历所有 #acontent 的子元素, 通过 window.getComputedStyle().display 检测是否是 none, 如果是 none 则从页面删除这个元素
result, err := page.Evaluate(`
(function() {
const acontent = document.getElementById('acontent');
if (!acontent) {
return 'acontent element not found';
}
item.Media = fmt.Sprintf("image/%s", strings.ReplaceAll(strings.TrimPrefix(path.Ext(volume.Cover), "."), "jpg", "jpeg"))
manifest.Items = append(manifest.Items, item)
}
}
manifest.Items = append(manifest.Items, model.ManifestItem{
ID: "style",
Link: "Styles/style.css",
Media: "text/css",
})
spine := &model.Spine{
Items: make([]model.SpineItem, 0),
}
for _, item := range manifest.Items {
if filepath.Ext(item.Link) == ".xhtml" {
spine.Items = append(spine.Items, model.SpineItem{
IDref: item.ID,
})
}
}
contentOPFPath := filepath.Join(dirPath, "OEBPS/content.opf")
err := os.MkdirAll(path.Dir(contentOPFPath), 0755)
let removedCount = 0;
const elements = acontent.querySelectorAll('*');
// 从后往前遍历,避免删除元素时影响索引
for (let i = elements.length - 1; i >= 0; i--) {
const element = elements[i];
const computedStyle = window.getComputedStyle(element);
if (computedStyle.display === 'none' || computedStyle.transform == 'matrix(0, 0, 0, 0, 0, 0)') {
element.remove();
removedCount++;
}
}
return 'Removed ' + removedCount + ' hidden elements';
})()
`)
if err != nil {
return fmt.Errorf("failed to create content directory: %v", err)
return "", fmt.Errorf("failed to remove hidden elements: %w", err)
}
file, err := os.Create(contentOPFPath)
log.Printf("Hidden elements removal result: %s", result)
processedHTML, err := page.Content()
if err != nil {
return fmt.Errorf("failed to create content file: %v", err)
return "", fmt.Errorf("could not get page content: %w", err)
}
err = template.ContentOPF("book-id", dc, manifest, spine, nil).Render(context.Background(), file)
if err != nil {
return fmt.Errorf("failed to render content: %v", err)
}
return nil
}
func CreateTocNCX(dirPath string, uuid string, volume *model.Volume) error {
navMap := &model.NavMap{Points: make([]*model.NavPoint, 0)}
navMap.Points = append(navMap.Points, &model.NavPoint{
Id: "cover",
PlayOrder: 1,
Label: "封面",
Content: model.NavPointContent{Src: "Text/cover.xhtml"},
})
navMap.Points = append(navMap.Points, &model.NavPoint{
Id: "contents",
PlayOrder: 2,
Label: "目录",
Content: model.NavPointContent{Src: "Text/contents.xhtml"},
})
for idx, chapter := range volume.Chapters {
navMap.Points = append(navMap.Points, &model.NavPoint{
Id: fmt.Sprintf("chapter-%03v", idx+1),
PlayOrder: len(navMap.Points) + 1,
Label: chapter.Title,
Content: model.NavPointContent{Src: chapter.TextOEBPSPath},
})
}
head := &model.TocNCXHead{
Meta: []model.TocNCXHeadMeta{
{Name: "dtb:uid", Content: fmt.Sprintf("urn:uuid:%s", uuid)},
},
}
ncxPath := filepath.Join(dirPath, "OEBPS/toc.ncx")
err := os.MkdirAll(path.Dir(ncxPath), 0755)
if err != nil {
return fmt.Errorf("failed to create toc directory: %v", err)
}
file, err := os.Create(ncxPath)
if err != nil {
return fmt.Errorf("failed to create toc file: %v", err)
}
err = template.TocNCX(volume.Title, head, navMap).Render(context.Background(), file)
if err != nil {
return fmt.Errorf("failed to render toc: %v", err)
}
return nil
}
//go:embed read.ttf
var readTTF []byte
func DownloadFont(outputPath string) error {
log.Printf("Writing Font: %s", outputPath)
fontPath := filepath.Join(outputPath, "read.ttf")
err := os.MkdirAll(path.Dir(fontPath), 0755)
if err != nil {
return fmt.Errorf("failed to create font directory: %v", err)
}
err = os.WriteFile(fontPath, readTTF, 0644)
if err != nil {
return fmt.Errorf("failed to write font: %v", err)
}
return nil
return processedHTML, nil
}

View File

@@ -1,122 +0,0 @@
package bilinovel
import (
"archive/zip"
"io"
"log"
"os"
"path/filepath"
)
func CreateEpub(path string) error {
log.Printf("Creating epub for %s", path)
savePath := path + ".epub"
zipFile, err := os.Create(savePath)
if err != nil {
return err
}
defer zipFile.Close()
zipWriter := zip.NewWriter(zipFile)
defer zipWriter.Close()
err = addStringToZip(zipWriter, "mimetype", "application/epub+zip", zip.Store)
if err != nil {
return err
}
err = addDirContentToZip(zipWriter, path, zip.Deflate)
if err != nil {
return err
}
err = addStringToZip(zipWriter, "OEBPS/Styles/style.css", StyleCSS, zip.Deflate)
if err != nil {
return err
}
return nil
}
// func addFileToZip(zipWriter *zip.Writer, filename string, relPath string, method uint16) error {
// file, err := os.Open(filename)
// if err != nil {
// return err
// }
// defer file.Close()
// info, err := file.Stat()
// if err != nil {
// return err
// }
// header, err := zip.FileInfoHeader(info)
// if err != nil {
// return err
// }
// header.Name = relPath
// header.Method = method
// writer, err := zipWriter.CreateHeader(header)
// if err != nil {
// return err
// }
// _, err = io.Copy(writer, file)
// return err
// }
func addStringToZip(zipWriter *zip.Writer, relPath, content string, method uint16) error {
header := &zip.FileHeader{
Name: relPath,
Method: method,
}
writer, err := zipWriter.CreateHeader(header)
if err != nil {
return err
}
_, err = writer.Write([]byte(content))
return err
}
func addDirContentToZip(zipWriter *zip.Writer, dirPath string, method uint16) error {
return filepath.Walk(dirPath, func(filePath string, info os.FileInfo, err error) error {
if filepath.Base(filePath) == "volume.json" {
return nil
}
if err != nil {
return err
}
if info.IsDir() {
return nil
}
relPath, err := filepath.Rel(dirPath, filePath)
if err != nil {
return err
}
file, err := os.Open(filePath)
if err != nil {
return err
}
defer file.Close()
header, err := zip.FileInfoHeader(info)
if err != nil {
return err
}
header.Name = relPath
header.Method = method
writer, err := zipWriter.CreateHeader(header)
if err != nil {
return err
}
_, err = io.Copy(writer, file)
return err
})
}

View File

@@ -1,15 +1,3 @@
package bilinovel
const StyleCSS = `
@font-face {
font-family: "MI LANTING";
src: url(../Fonts/read.ttf);
}
.read-font {
font-family: "MI LANTING", serif !important;
}
body > div {
margin: 0 auto;
padding: 20px;
@@ -49,4 +37,3 @@ img {
margin-top: 1em;
margin-bottom: 1em;
}
`

12
downloader/downloader.go Normal file
View File

@@ -0,0 +1,12 @@
package downloader
import "bilinovel-downloader/model"
type Downloader interface {
GetNovel(novelId int, skipChapterContent bool, skipVolumes []int) (*model.Novel, error)
GetVolume(novelId int, volumeId int, skipChapterContent bool) (*model.Volume, error)
GetChapter(novelId int, volumeId int, chapterId int) (*model.Chapter, error)
GetStyleCSS() string
GetExtraFiles() []model.ExtraFile
Close() error
}

365
epub/wrapper.go Normal file
View File

@@ -0,0 +1,365 @@
package epub
import (
"archive/zip"
"bilinovel-downloader/model"
"bilinovel-downloader/template"
"bilinovel-downloader/utils"
"context"
"fmt"
"io"
"os"
"path"
"path/filepath"
"strconv"
"strings"
"time"
"github.com/google/uuid"
)
func PackVolumeToEpub(volume *model.Volume, outputPath string, styleCSS string, extraFiles []model.ExtraFile) error {
outputPath = filepath.Join(outputPath, utils.CleanDirName(volume.Title))
_, err := os.Stat(outputPath)
if err != nil {
if os.IsNotExist(err) {
err = os.MkdirAll(outputPath, 0755)
if err != nil {
return fmt.Errorf("failed to create output directory: %v", err)
}
} else {
return fmt.Errorf("failed to get output directory: %v", err)
}
} else {
err = os.RemoveAll(outputPath)
if err != nil {
return fmt.Errorf("failed to remove output directory: %v", err)
}
err = os.MkdirAll(outputPath, 0755)
if err != nil {
return fmt.Errorf("failed to create output directory: %v", err)
}
}
// 将文字写入 OEBPS/Text/chapter-%03v.xhtml
// 将图片写入 OEBPS/Images/chapter-%03v/
for i, chapter := range volume.Chapters {
imageNames := make([]string, 0)
for imgName, imgData := range chapter.Content.Images {
imageNames = append(imageNames, imgName)
imgPath := filepath.Join(outputPath, fmt.Sprintf("OEBPS/Images/chapter-%03v/%s", i, imgName))
err := os.MkdirAll(filepath.Dir(imgPath), 0755)
if err != nil {
return fmt.Errorf("failed to create image directory: %v", err)
}
err = os.WriteFile(imgPath, imgData, 0644)
if err != nil {
return fmt.Errorf("failed to write image: %v", err)
}
}
chapterPath := filepath.Join(outputPath, fmt.Sprintf("OEBPS/Text/chapter-%03v.xhtml", i))
err = os.MkdirAll(filepath.Dir(chapterPath), 0755)
if err != nil {
return fmt.Errorf("failed to create chapter directory: %v", err)
}
file, err := os.Create(chapterPath)
if err != nil {
return fmt.Errorf("failed to create chapter file: %v", err)
}
defer file.Close()
text := chapter.Content.Html
for _, imgName := range imageNames {
text = strings.ReplaceAll(text, imgName, fmt.Sprintf("../Images/chapter-%03v/%s", i, imgName))
}
err = template.ContentXHTML(chapter.Title, text).Render(context.Background(), file)
if err != nil {
return fmt.Errorf("failed to write chapter: %v", err)
}
}
// 将 Cover 写入
coverPath := filepath.Join(outputPath, fmt.Sprintf("cover%s", filepath.Ext(volume.CoverUrl)))
err = os.WriteFile(coverPath, volume.Cover, 0644)
if err != nil {
return fmt.Errorf("failed to write cover: %v", err)
}
// 将 CoverXHTML 写入 OEBPS/Text/cover.xhtml
coverXHTMLPath := filepath.Join(outputPath, "OEBPS/Text/cover.xhtml")
file, err := os.Create(coverXHTMLPath)
if err != nil {
return fmt.Errorf("failed to create cover XHTML file: %v", err)
}
defer file.Close()
err = template.CoverXHTML(fmt.Sprintf("../../%s", filepath.Base(coverPath))).Render(context.Background(), file)
if err != nil {
return fmt.Errorf("failed to render cover XHTML: %v", err)
}
// OEBPS/Text/contents.xhtml 目录
contentsXHTMLPath := filepath.Join(outputPath, "OEBPS/Text/contents.xhtml")
file, err = os.Create(contentsXHTMLPath)
if err != nil {
return fmt.Errorf("failed to create contents XHTML file: %v", err)
}
defer file.Close()
contents := strings.Builder{}
contents.WriteString(`<nav epub:type="toc" id="toc">`)
contents.WriteString(`<ol>`)
for i, chapter := range volume.Chapters {
contents.WriteString(fmt.Sprintf(`<li><a href="chapter-%03v.xhtml">%s</a></li>`, i, chapter.Title))
}
contents.WriteString(`</ol>`)
contents.WriteString(`</nav>`)
err = template.ContentXHTML("目录", contents.String()).Render(context.Background(), file)
if err != nil {
return fmt.Errorf("failed to render contents XHTML: %v", err)
}
// ContainerXML
containerPath := filepath.Join(outputPath, "META-INF/container.xml")
err = os.MkdirAll(filepath.Dir(containerPath), 0755)
if err != nil {
return fmt.Errorf("failed to create container directory: %v", err)
}
file, err = os.Create(containerPath)
if err != nil {
return fmt.Errorf("failed to create container file: %v", err)
}
defer file.Close()
err = template.ContainerXML().Render(context.Background(), file)
if err != nil {
return fmt.Errorf("failed to render container: %v", err)
}
// ContentOPF
u := uuid.New()
err = CreateContentOPF(outputPath, u.String(), volume, extraFiles)
if err != nil {
return fmt.Errorf("failed to create content OPF: %v", err)
}
// 写入 CSS
cssPath := filepath.Join(outputPath, "style.css")
err = os.WriteFile(cssPath, []byte(styleCSS), 0644)
if err != nil {
return fmt.Errorf("failed to write CSS: %v", err)
}
// 写入 extraFiles
for _, file := range extraFiles {
extraFilePath := filepath.Join(outputPath, file.Path)
err = os.WriteFile(extraFilePath, file.Data, 0644)
if err != nil {
return fmt.Errorf("failed to write extra file: %v", err)
}
}
// 打包成 epub 文件
err = PackEpub(outputPath)
if err != nil {
return fmt.Errorf("failed to pack epub: %v", err)
}
return nil
}
func CreateContentOPF(outputPath string, uuid string, volume *model.Volume, extraFiles []model.ExtraFile) error {
creators := make([]model.DCCreator, 0)
for _, author := range volume.Authors {
creators = append(creators, model.DCCreator{
Value: author,
})
}
dc := &model.DublinCoreMetadata{
Titles: []model.DCTitle{
{
Value: volume.Title,
},
},
Identifiers: []model.DCIdentifier{
{
Value: fmt.Sprintf("urn:uuid:%s", uuid),
ID: "book-id",
// Scheme: "UUID",
},
},
Languages: []model.DCLanguage{
{
Value: "zh-CN",
},
},
Descriptions: []model.DCDescription{
{
Value: volume.Description,
},
},
Creators: creators,
Metas: []model.DublinCoreMeta{
{
Name: "cover",
Content: "cover",
},
{
Property: "dcterms:modified",
Value: time.Now().UTC().Format("2006-01-02T15:04:05Z"),
},
{
Name: "calibre:series",
Content: volume.NovelTitle,
},
{
Name: "calibre:series_index",
Content: strconv.Itoa(volume.SeriesIdx),
},
},
}
manifest := &model.Manifest{
Items: make([]model.ManifestItem, 0),
}
manifest.Items = append(manifest.Items, model.ManifestItem{
ID: "cover.xhtml",
Link: "OEBPS/Text/cover.xhtml",
Media: "application/xhtml+xml",
})
manifest.Items = append(manifest.Items, model.ManifestItem{
ID: "contents.xhtml",
Link: "OEBPS/Text/contents.xhtml",
Media: "application/xhtml+xml",
Properties: "nav",
})
manifest.Items = append(manifest.Items, model.ManifestItem{
ID: "cover",
Link: fmt.Sprintf("cover%s", filepath.Ext(volume.CoverUrl)),
Media: fmt.Sprintf("image/%s", strings.ReplaceAll(strings.TrimPrefix(filepath.Ext(volume.CoverUrl), "."), "jpg", "jpeg")),
Properties: "cover-image",
})
for i, chapter := range volume.Chapters {
manifest.Items = append(manifest.Items, model.ManifestItem{
ID: fmt.Sprintf("chapter-%03v.xhtml", i),
Link: fmt.Sprintf("OEBPS/Text/chapter-%03v.xhtml", i),
Media: "application/xhtml+xml",
})
for filename := range chapter.Content.Images {
item := model.ManifestItem{
ID: fmt.Sprintf("chapter-%03v-%s", i, filepath.Base(filename)),
Link: fmt.Sprintf("OEBPS/Images/chapter-%03v/%s", i, filepath.Base(filename)),
Media: fmt.Sprintf("image/%s", strings.ReplaceAll(strings.TrimPrefix(filepath.Ext(filename), "."), "jpg", "jpeg")),
}
manifest.Items = append(manifest.Items, item)
}
}
manifest.Items = append(manifest.Items, model.ManifestItem{
ID: "style",
Link: "style.css",
Media: "text/css",
})
// ExtraFiles
for _, file := range extraFiles {
manifest.Items = append(manifest.Items, file.ManifestItem)
}
spine := &model.Spine{
Items: make([]model.SpineItem, 0),
}
for _, item := range manifest.Items {
if filepath.Ext(item.Link) == ".xhtml" {
spine.Items = append(spine.Items, model.SpineItem{
IDref: item.ID,
})
}
}
contentOPFPath := filepath.Join(outputPath, "content.opf")
err := os.MkdirAll(path.Dir(contentOPFPath), 0755)
if err != nil {
return fmt.Errorf("failed to create content directory: %v", err)
}
file, err := os.Create(contentOPFPath)
if err != nil {
return fmt.Errorf("failed to create content file: %v", err)
}
err = template.ContentOPF("book-id", dc, manifest, spine, nil).Render(context.Background(), file)
if err != nil {
return fmt.Errorf("failed to render content: %v", err)
}
return nil
}
func PackEpub(dirPath string) error {
savePath := strings.TrimSuffix(dirPath, string(filepath.Separator)) + ".epub"
zipFile, err := os.Create(savePath)
if err != nil {
return err
}
defer zipFile.Close()
zipWriter := zip.NewWriter(zipFile)
defer zipWriter.Close()
err = addStringToZip(zipWriter, "mimetype", "application/epub+zip", zip.Store)
if err != nil {
return err
}
err = addDirContentToZip(zipWriter, dirPath, zip.Deflate)
if err != nil {
return err
}
return nil
}
func addStringToZip(zipWriter *zip.Writer, relPath, content string, method uint16) error {
header := &zip.FileHeader{
Name: relPath,
Method: method,
}
writer, err := zipWriter.CreateHeader(header)
if err != nil {
return err
}
_, err = writer.Write([]byte(content))
return err
}
func addDirContentToZip(zipWriter *zip.Writer, dirPath string, method uint16) error {
return filepath.Walk(dirPath, func(filePath string, info os.FileInfo, err error) error {
if filepath.Base(filePath) == "volume.json" {
return nil
}
if err != nil {
return err
}
if info.IsDir() {
return nil
}
relPath, err := filepath.Rel(dirPath, filePath)
if err != nil {
return err
}
relPath = filepath.ToSlash(relPath)
file, err := os.Open(filePath)
if err != nil {
return err
}
defer file.Close()
header, err := zip.FileInfoHeader(info)
if err != nil {
return err
}
header.Name = relPath
header.Method = method
writer, err := zipWriter.CreateHeader(header)
if err != nil {
return err
}
_, err = io.Copy(writer, file)
return err
})
}

13
go.mod
View File

@@ -4,15 +4,22 @@ go 1.24.2
require (
github.com/PuerkitoBio/goquery v1.10.3
github.com/a-h/templ v0.3.857
github.com/a-h/templ v0.3.943
github.com/bestnite/font-mapper v0.0.0-20250823155658-56c76d820267
github.com/go-resty/resty/v2 v2.16.5
github.com/google/uuid v1.6.0
github.com/playwright-community/playwright-go v0.5200.1
github.com/spf13/cobra v1.9.1
)
require (
github.com/andybalholm/cascadia v1.3.3 // indirect
github.com/deckarep/golang-set/v2 v2.8.0 // indirect
github.com/go-jose/go-jose/v3 v3.0.4 // indirect
github.com/go-stack/stack v1.8.1 // indirect
github.com/golang/freetype v0.0.0-20170609003504-e2365dfdc4a0 // indirect
github.com/inconshreveable/mousetrap v1.1.0 // indirect
github.com/spf13/pflag v1.0.6 // indirect
golang.org/x/net v0.39.0 // indirect
github.com/spf13/pflag v1.0.7 // indirect
golang.org/x/image v0.30.0 // indirect
golang.org/x/net v0.43.0 // indirect
)

39
go.sum
View File

@@ -1,23 +1,48 @@
github.com/PuerkitoBio/goquery v1.10.3 h1:pFYcNSqHxBD06Fpj/KsbStFRsgRATgnf3LeXiUkhzPo=
github.com/PuerkitoBio/goquery v1.10.3/go.mod h1:tMUX0zDMHXYlAQk6p35XxQMqMweEKB7iK7iLNd4RH4Y=
github.com/a-h/templ v0.3.857 h1:6EqcJuGZW4OL+2iZ3MD+NnIcG7nGkaQeF2Zq5kf9ZGg=
github.com/a-h/templ v0.3.857/go.mod h1:qhrhAkRFubE7khxLZHsBFHfX+gWwVNKbzKeF9GlPV4M=
github.com/a-h/templ v0.3.943 h1:o+mT/4yqhZ33F3ootBiHwaY4HM5EVaOJfIshvd5UNTY=
github.com/a-h/templ v0.3.943/go.mod h1:oCZcnKRf5jjsGpf2yELzQfodLphd2mwecwG4Crk5HBo=
github.com/andybalholm/cascadia v1.3.3 h1:AG2YHrzJIm4BZ19iwJ/DAua6Btl3IwJX+VI4kktS1LM=
github.com/andybalholm/cascadia v1.3.3/go.mod h1:xNd9bqTn98Ln4DwST8/nG+H0yuB8Hmgu1YHNnWw0GeA=
github.com/bestnite/font-mapper v0.0.0-20250823155658-56c76d820267 h1:nmUTJV2u/0XmVjQ++VIy/Hu+MtxdpQvOevvcSZtUATA=
github.com/bestnite/font-mapper v0.0.0-20250823155658-56c76d820267/go.mod h1:cfB1e9YhoI/QWrXPp3h6QVAKU6iCI2ifbjRPHP3xf/0=
github.com/cpuguy83/go-md2man/v2 v2.0.6/go.mod h1:oOW0eioCTA6cOiMLiUPZOpcVxMig6NIQQ7OS05n1F4g=
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/deckarep/golang-set/v2 v2.8.0 h1:swm0rlPCmdWn9mESxKOjWk8hXSqoxOp+ZlfuyaAdFlQ=
github.com/deckarep/golang-set/v2 v2.8.0/go.mod h1:VAky9rY/yGXJOLEDv3OMci+7wtDpOF4IN+y82NBOac4=
github.com/go-jose/go-jose/v3 v3.0.4 h1:Wp5HA7bLQcKnf6YYao/4kpRpVMp/yf6+pJKV8WFSaNY=
github.com/go-jose/go-jose/v3 v3.0.4/go.mod h1:5b+7YgP7ZICgJDBdfjZaIt+H/9L9T/YQrVfLAMboGkQ=
github.com/go-resty/resty/v2 v2.16.5 h1:hBKqmWrr7uRc3euHVqmh1HTHcKn99Smr7o5spptdhTM=
github.com/go-resty/resty/v2 v2.16.5/go.mod h1:hkJtXbA2iKHzJheXYvQ8snQES5ZLGKMwQ07xAwp/fiA=
github.com/go-stack/stack v1.8.1 h1:ntEHSVwIt7PNXNpgPmVfMrNhLtgjlmnZha2kOpuRiDw=
github.com/go-stack/stack v1.8.1/go.mod h1:dcoOX6HbPZSZptuspn9bctJ+N/CnF5gGygcUP3XYfe4=
github.com/golang/freetype v0.0.0-20170609003504-e2365dfdc4a0 h1:DACJavvAHhabrF08vX0COfcOBJRhZ8lUbR+ZWIs0Y5g=
github.com/golang/freetype v0.0.0-20170609003504-e2365dfdc4a0/go.mod h1:E/TSTwGwJL78qG/PmXZO1EjYhfJinVAhrmmHX6Z8B9k=
github.com/google/go-cmp v0.5.9/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=
github.com/google/go-cmp v0.6.0 h1:ofyhxvXcZhMsU5ulbFiLKl/XBFqE1GSq7atu8tAmTRI=
github.com/google/go-cmp v0.6.0/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=
github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/inconshreveable/mousetrap v1.1.0 h1:wN+x4NVGpMsO7ErUn/mUI3vEoE6Jt13X2s0bqwp9tc8=
github.com/inconshreveable/mousetrap v1.1.0/go.mod h1:vpF70FUmC8bwa3OWnCshd2FqLfsEA9PFc4w1p2J65bw=
github.com/mitchellh/go-ps v1.0.0 h1:i6ampVEEF4wQFF+bkYfwYgY+F/uYJDktmvLPf7qIgjc=
github.com/mitchellh/go-ps v1.0.0/go.mod h1:J4lOc8z8yJs6vUwklHw2XEIiT4z4C40KtWVN3nvg8Pg=
github.com/playwright-community/playwright-go v0.5200.1 h1:Sm2oOuhqt0M5Y4kUi/Qh9w4cyyi3ZIWTBeGKImc2UVo=
github.com/playwright-community/playwright-go v0.5200.1/go.mod h1:UnnyQZaqUOO5ywAZu60+N4EiWReUqX1MQBBA3Oofvf8=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/russross/blackfriday/v2 v2.1.0/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM=
github.com/spf13/cobra v1.9.1 h1:CXSaggrXdbHK9CF+8ywj8Amf7PBRmPCOJugH954Nnlo=
github.com/spf13/cobra v1.9.1/go.mod h1:nDyEzZ8ogv936Cinf6g1RU9MRY64Ir93oCnqb9wxYW0=
github.com/spf13/pflag v1.0.6 h1:jFzHGLGAlb3ruxLB8MhbI6A8+AQX/2eW4qeyNZXNp2o=
github.com/spf13/pflag v1.0.6/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=
github.com/spf13/pflag v1.0.7 h1:vN6T9TfwStFPFM5XzjsvmzZkLuaLX+HS+0SeFLRgU6M=
github.com/spf13/pflag v1.0.7/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.10.0 h1:Xv5erBjTwe/5IxqUQTdXv5kgmIvbHo3QQyRwhJsOfJA=
github.com/stretchr/testify v1.10.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY=
github.com/yuin/goldmark v1.4.13/go.mod h1:6yULJ656Px+3vBD8DxQVa3kxgyrAnzto9xy5taEt/CY=
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
golang.org/x/crypto v0.0.0-20210921155107-089bfa567519/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc=
@@ -25,6 +50,8 @@ golang.org/x/crypto v0.13.0/go.mod h1:y6Z2r+Rw4iayiXXAIxJIDAJ1zMW4yaTpebo8fPOliY
golang.org/x/crypto v0.19.0/go.mod h1:Iy9bg/ha4yyC70EfRS8jz+B6ybOBKMaSxLj6P6oBDfU=
golang.org/x/crypto v0.23.0/go.mod h1:CKFgDieR+mRhux2Lsu27y0fO304Db0wZe70UKqHu0v8=
golang.org/x/crypto v0.31.0/go.mod h1:kDsLvtWBEx7MV9tJOj9bnXsPbxwJQ6csT/x4KIN4Ssk=
golang.org/x/image v0.30.0 h1:jD5RhkmVAnjqaCUXfbGBrn3lpxbknfN9w2UhHHU+5B4=
golang.org/x/image v0.30.0/go.mod h1:SAEUTxCCMWSrJcCy/4HwavEsfZZJlYxeHLc6tTiAe/c=
golang.org/x/mod v0.6.0-dev.0.20220419223038-86c51ed26bb4/go.mod h1:jJ57K6gSWd91VN4djpZkiMVwK6gcyfeH4XE8wZrZaV4=
golang.org/x/mod v0.8.0/go.mod h1:iBbtSCu2XBx23ZKBPSOrRkjjQPZFPuis4dIYUhu/chs=
golang.org/x/mod v0.12.0/go.mod h1:iBbtSCu2XBx23ZKBPSOrRkjjQPZFPuis4dIYUhu/chs=
@@ -39,8 +66,8 @@ golang.org/x/net v0.15.0/go.mod h1:idbUs1IY1+zTqbi8yxTbhexhEEk5ur9LInksu6HrEpk=
golang.org/x/net v0.21.0/go.mod h1:bIjVDfnllIU7BJ2DNgfnXvpSvtn8VRwhlsaeUTyUS44=
golang.org/x/net v0.25.0/go.mod h1:JkAGAh7GEvH74S6FOH42FLoXpXbE/aqXSrIQjXgsiwM=
golang.org/x/net v0.33.0/go.mod h1:HXLR5J+9DxmrqMwG9qjGCxZ+zKXxBru04zlTvWlWuN4=
golang.org/x/net v0.39.0 h1:ZCu7HMWDxpXpaiKdhzIfaltL9Lp31x/3fCP11bc6/fY=
golang.org/x/net v0.39.0/go.mod h1:X7NRbYVEA+ewNkCNyJ513WmMdQ3BineSwVtN2zD/d+E=
golang.org/x/net v0.43.0 h1:lat02VYK2j4aLzMzecihNvTlJNQUq316m2Mr9rnM6YE=
golang.org/x/net v0.43.0/go.mod h1:vhO1fvI4dGsIjh73sWfUVjj3N7CA9WkKJNQm2svM6Jg=
golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20220722155255-886fb9371eb4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.1.0/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
@@ -87,4 +114,6 @@ golang.org/x/tools v0.13.0/go.mod h1:HvlwmtVNQAhOuCjW7xxvovg8wbNq7LwfXh/k7wXUl58
golang.org/x/tools v0.21.1-0.20240508182429-e35e4ccd0d2d/go.mod h1:aiJjzUbINMkxbQROHiO6hDPo2LHcIPhhQsa9DLh0yGk=
golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=

View File

@@ -2,11 +2,8 @@ package main
import (
"bilinovel-downloader/cmd"
"log"
)
func main() {
if err := cmd.RootCmd.Execute(); err != nil {
log.Fatalf("Error executing command: %v", err)
}
_ = cmd.RootCmd.Execute()
}

View File

@@ -1,6 +1,14 @@
package model
import "encoding/xml"
import (
"encoding/xml"
)
type ExtraFile struct {
Data []byte
Path string
ManifestItem ManifestItem
}
type DublinCoreMetadata struct {
XMLName xml.Name `xml:"metadata"`
@@ -158,7 +166,6 @@ type Spine struct {
}
func (s *Spine) Marshal() (string, error) {
s.Toc = "ncx"
xmlBytes, err := xml.Marshal(s)
if err != nil {
return "", err

View File

@@ -1,13 +1,17 @@
package model
type ChaperContent struct {
Html string
Images map[string][]byte
}
type Chapter struct {
Title string
Url string
Content string
ImageOEBPSPaths []string
ImageFullPaths []string
TextOEBPSPath string
TextFullPath string
Id int
NovelId int
VolumeId int
Title string
Url string
Content *ChaperContent
}
type Volume struct {
@@ -15,7 +19,8 @@ type Volume struct {
SeriesIdx int
Title string
Url string
Cover string
CoverUrl string
Cover []byte
Description string
Authors []string
Chapters []*Chapter

View File

@@ -1,47 +0,0 @@
package model
import "encoding/xml"
type TocNCXHead struct {
XMLName xml.Name `xml:"head"`
Meta []TocNCXHeadMeta `xml:"meta"`
}
type TocNCXHeadMeta struct {
XMLName xml.Name `xml:"meta"`
Content string `xml:"content,attr"`
Name string `xml:"name,attr"`
}
func (h *TocNCXHead) Marshal() (string, error) {
xmlBytes, err := xml.Marshal(h)
if err != nil {
return "", err
}
return string(xmlBytes), nil
}
type NavPoint struct {
Id string `xml:"id,attr"`
PlayOrder int `xml:"playOrder,attr"`
Label string `xml:"navLabel>text"`
Content NavPointContent `xml:"content"`
NavPoints []*NavPoint `xml:"navPoint"`
}
type NavPointContent struct {
Src string `xml:"src,attr"`
}
type NavMap struct {
XMLName xml.Name `xml:"navMap"`
Points []*NavPoint `xml:"navPoint"`
}
func (n *NavMap) Marshal() (string, error) {
xmlBytes, err := xml.Marshal(n)
if err != nil {
return "", err
}
return string(xmlBytes), nil
}

View File

@@ -1,10 +1,10 @@
package template
templ ContainerXML() {
@templ.Raw(`<?xml version="1.0" encoding="UTF-8"?>`)
<container version="1.0" xmlns="urn:oasis:names:tc:opendocument:xmlns:container">
@templ.Raw(`<?xml version='1.0' encoding='utf-8'?>`)
<container xmlns="urn:oasis:names:tc:opendocument:xmlns:container" version="1.0">
<rootfiles>
<rootfile full-path="OEBPS/content.opf" media-type="application/oebps-package+xml"></rootfile>
<rootfile full-path="content.opf" media-type="application/oebps-package+xml"></rootfile>
</rootfiles>
</container>
}

View File

@@ -1,6 +1,6 @@
// Code generated by templ - DO NOT EDIT.
// templ: version: v0.3.857
// templ: version: v0.3.943
package template
//lint:file-ignore SA4006 This context is only used if a nested component is present.
@@ -29,11 +29,11 @@ func ContainerXML() templ.Component {
templ_7745c5c3_Var1 = templ.NopComponent
}
ctx = templ.ClearChildren(ctx)
templ_7745c5c3_Err = templ.Raw(`<?xml version="1.0" encoding="UTF-8"?>`).Render(ctx, templ_7745c5c3_Buffer)
templ_7745c5c3_Err = templ.Raw(`<?xml version='1.0' encoding='utf-8'?>`).Render(ctx, templ_7745c5c3_Buffer)
if templ_7745c5c3_Err != nil {
return templ_7745c5c3_Err
}
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 1, "<container version=\"1.0\" xmlns=\"urn:oasis:names:tc:opendocument:xmlns:container\"><rootfiles><rootfile full-path=\"OEBPS/content.opf\" media-type=\"application/oebps-package+xml\"></rootfile></rootfiles></container>")
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 1, "<container xmlns=\"urn:oasis:names:tc:opendocument:xmlns:container\" version=\"1.0\"><rootfiles><rootfile full-path=\"content.opf\" media-type=\"application/oebps-package+xml\"></rootfile></rootfiles></container>")
if templ_7745c5c3_Err != nil {
return templ_7745c5c3_Err
}

View File

@@ -3,7 +3,7 @@ package template
import "bilinovel-downloader/model"
templ ContentOPF(uniqueIdentifier string, dc *model.DublinCoreMetadata, manifest *model.Manifest, spine *model.Spine, guide *model.Guide) {
@templ.Raw(`<?xml version="1.0" encoding="UTF-8"?>`)
@templ.Raw(`<?xml version='1.0' encoding='utf-8'?>`)
<package version="3.0" xmlns="http://www.idpf.org/2007/opf" xmlns:dc="http://purl.org/dc/elements/1.1/" unique-identifier={ uniqueIdentifier }>
if dc != nil {
{{ metadata, err := dc.Marshal() }}

View File

@@ -1,6 +1,6 @@
// Code generated by templ - DO NOT EDIT.
// templ: version: v0.3.857
// templ: version: v0.3.943
package template
//lint:file-ignore SA4006 This context is only used if a nested component is present.
@@ -31,7 +31,7 @@ func ContentOPF(uniqueIdentifier string, dc *model.DublinCoreMetadata, manifest
templ_7745c5c3_Var1 = templ.NopComponent
}
ctx = templ.ClearChildren(ctx)
templ_7745c5c3_Err = templ.Raw(`<?xml version="1.0" encoding="UTF-8"?>`).Render(ctx, templ_7745c5c3_Buffer)
templ_7745c5c3_Err = templ.Raw(`<?xml version='1.0' encoding='utf-8'?>`).Render(ctx, templ_7745c5c3_Buffer)
if templ_7745c5c3_Err != nil {
return templ_7745c5c3_Err
}

View File

@@ -1,21 +1,19 @@
package template
import "bilinovel-downloader/model"
templ ContentXHTML(content *model.Chapter) {
@templ.Raw(`<?xml version="1.0" encoding="utf-8" standalone="no"?>`)
@templ.Raw(`<!DOCTYPE html>`)
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="zh-CN" xmlns:epub="http://www.idpf.org/2007/ops" xmlns:xml="http://www.w3.org/XML/1998/namespace">
templ ContentXHTML(title, content string) {
@templ.Raw(`<?xml version='1.0' encoding='utf-8'?>`)
// @templ.Raw(`<!DOCTYPE html>`)
<html xmlns="http://www.w3.org/1999/xhtml" xmlns:epub="http://www.idpf.org/2007/ops" xml:lang="zh-CN">
<head>
<title>{ content.Title }</title>
@templ.Raw(`<link href="../Styles/style.css" rel="stylesheet" type="text/css"/>`)
<title>{ title }</title>
@templ.Raw(`<link href="../../style.css" rel="stylesheet" type="text/css"/>`)
</head>
<body>
<div class="chapter">
<h1>{ content.Title }</h1>
<h1>{ title }</h1>
@templ.Raw(`<hr/>`)
<div class="content">
@templ.Raw(content.Content)
@templ.Raw(content)
</div>
</div>
</body>

View File

@@ -1,6 +1,6 @@
// Code generated by templ - DO NOT EDIT.
// templ: version: v0.3.857
// templ: version: v0.3.943
package template
//lint:file-ignore SA4006 This context is only used if a nested component is present.
@@ -8,9 +8,7 @@ package template
import "github.com/a-h/templ"
import templruntime "github.com/a-h/templ/runtime"
import "bilinovel-downloader/model"
func ContentXHTML(content *model.Chapter) templ.Component {
func ContentXHTML(title, content string) templ.Component {
return templruntime.GeneratedTemplate(func(templ_7745c5c3_Input templruntime.GeneratedComponentInput) (templ_7745c5c3_Err error) {
templ_7745c5c3_W, ctx := templ_7745c5c3_Input.Writer, templ_7745c5c3_Input.Context
if templ_7745c5c3_CtxErr := ctx.Err(); templ_7745c5c3_CtxErr != nil {
@@ -31,22 +29,18 @@ func ContentXHTML(content *model.Chapter) templ.Component {
templ_7745c5c3_Var1 = templ.NopComponent
}
ctx = templ.ClearChildren(ctx)
templ_7745c5c3_Err = templ.Raw(`<?xml version="1.0" encoding="utf-8" standalone="no"?>`).Render(ctx, templ_7745c5c3_Buffer)
templ_7745c5c3_Err = templ.Raw(`<?xml version='1.0' encoding='utf-8'?>`).Render(ctx, templ_7745c5c3_Buffer)
if templ_7745c5c3_Err != nil {
return templ_7745c5c3_Err
}
templ_7745c5c3_Err = templ.Raw(`<!DOCTYPE html>`).Render(ctx, templ_7745c5c3_Buffer)
if templ_7745c5c3_Err != nil {
return templ_7745c5c3_Err
}
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 1, "<html xmlns=\"http://www.w3.org/1999/xhtml\" xml:lang=\"zh-CN\" xmlns:epub=\"http://www.idpf.org/2007/ops\" xmlns:xml=\"http://www.w3.org/XML/1998/namespace\"><head><title>")
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 1, "<html xmlns=\"http://www.w3.org/1999/xhtml\" xmlns:epub=\"http://www.idpf.org/2007/ops\" xml:lang=\"zh-CN\"><head><title>")
if templ_7745c5c3_Err != nil {
return templ_7745c5c3_Err
}
var templ_7745c5c3_Var2 string
templ_7745c5c3_Var2, templ_7745c5c3_Err = templ.JoinStringErrs(content.Title)
templ_7745c5c3_Var2, templ_7745c5c3_Err = templ.JoinStringErrs(title)
if templ_7745c5c3_Err != nil {
return templ.Error{Err: templ_7745c5c3_Err, FileName: `template/content.xhtml.templ`, Line: 10, Col: 25}
return templ.Error{Err: templ_7745c5c3_Err, FileName: `template/content.xhtml.templ`, Line: 8, Col: 17}
}
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var2))
if templ_7745c5c3_Err != nil {
@@ -56,7 +50,7 @@ func ContentXHTML(content *model.Chapter) templ.Component {
if templ_7745c5c3_Err != nil {
return templ_7745c5c3_Err
}
templ_7745c5c3_Err = templ.Raw(`<link href="../Styles/style.css" rel="stylesheet" type="text/css"/>`).Render(ctx, templ_7745c5c3_Buffer)
templ_7745c5c3_Err = templ.Raw(`<link href="../../style.css" rel="stylesheet" type="text/css"/>`).Render(ctx, templ_7745c5c3_Buffer)
if templ_7745c5c3_Err != nil {
return templ_7745c5c3_Err
}
@@ -65,9 +59,9 @@ func ContentXHTML(content *model.Chapter) templ.Component {
return templ_7745c5c3_Err
}
var templ_7745c5c3_Var3 string
templ_7745c5c3_Var3, templ_7745c5c3_Err = templ.JoinStringErrs(content.Title)
templ_7745c5c3_Var3, templ_7745c5c3_Err = templ.JoinStringErrs(title)
if templ_7745c5c3_Err != nil {
return templ.Error{Err: templ_7745c5c3_Err, FileName: `template/content.xhtml.templ`, Line: 15, Col: 23}
return templ.Error{Err: templ_7745c5c3_Err, FileName: `template/content.xhtml.templ`, Line: 13, Col: 15}
}
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var3))
if templ_7745c5c3_Err != nil {
@@ -85,7 +79,7 @@ func ContentXHTML(content *model.Chapter) templ.Component {
if templ_7745c5c3_Err != nil {
return templ_7745c5c3_Err
}
templ_7745c5c3_Err = templ.Raw(content.Content).Render(ctx, templ_7745c5c3_Buffer)
templ_7745c5c3_Err = templ.Raw(content).Render(ctx, templ_7745c5c3_Buffer)
if templ_7745c5c3_Err != nil {
return templ_7745c5c3_Err
}

View File

@@ -0,0 +1,37 @@
package template
templ CoverXHTML(coverPath string) {
@templ.Raw(`<?xml version='1.0' encoding='utf-8'?>`)
<html xmlns="http://www.w3.org/1999/xhtml" xmlns:epub="http://www.idpf.org/2007/ops"
xml:lang="zh-CN">
<head>
<title>Cover</title>
</head>
<style type="text/css">
@page {
padding: 0pt;
margin: 0pt
}
body {
text-align: center;
padding: 0pt;
margin: 0pt;
}
</style>
<body>
<div>
<svg
xmlns="http://www.w3.org/2000/svg"
xmlns:xlink="http://www.w3.org/1999/xlink"
version="1.1"
width="100%"
height="100%"
viewBox="0 0 400 581"
preserveAspectRatio="none"
>
<image width="400" height="581" xlink:href={ coverPath }></image>
</svg>
</div>
</body>
</html>
}

View File

@@ -1,6 +1,6 @@
// Code generated by templ - DO NOT EDIT.
// templ: version: v0.3.857
// templ: version: v0.3.943
package template
//lint:file-ignore SA4006 This context is only used if a nested component is present.
@@ -8,9 +8,7 @@ package template
import "github.com/a-h/templ"
import templruntime "github.com/a-h/templ/runtime"
import "bilinovel-downloader/model"
func TocNCX(title string, head *model.TocNCXHead, navMap *model.NavMap) templ.Component {
func CoverXHTML(coverPath string) templ.Component {
return templruntime.GeneratedTemplate(func(templ_7745c5c3_Input templruntime.GeneratedComponentInput) (templ_7745c5c3_Err error) {
templ_7745c5c3_W, ctx := templ_7745c5c3_Input.Writer, templ_7745c5c3_Input.Context
if templ_7745c5c3_CtxErr := ctx.Err(); templ_7745c5c3_CtxErr != nil {
@@ -31,54 +29,24 @@ func TocNCX(title string, head *model.TocNCXHead, navMap *model.NavMap) templ.Co
templ_7745c5c3_Var1 = templ.NopComponent
}
ctx = templ.ClearChildren(ctx)
templ_7745c5c3_Err = templ.Raw(`<?xml version="1.0" encoding="UTF-8"?>`).Render(ctx, templ_7745c5c3_Buffer)
templ_7745c5c3_Err = templ.Raw(`<?xml version='1.0' encoding='utf-8'?>`).Render(ctx, templ_7745c5c3_Buffer)
if templ_7745c5c3_Err != nil {
return templ_7745c5c3_Err
}
templ_7745c5c3_Err = templ.Raw(`<!DOCTYPE ncx PUBLIC "-//NISO//DTD ncx 2005-1//EN" "http://www.daisy.org/z3986/2005/ncx-2005-1.dtd">`).Render(ctx, templ_7745c5c3_Buffer)
if templ_7745c5c3_Err != nil {
return templ_7745c5c3_Err
}
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 1, "<ncx xmlns=\"http://www.daisy.org/z3986/2005/ncx/\" version=\"2005-1\">")
if templ_7745c5c3_Err != nil {
return templ_7745c5c3_Err
}
if head != nil {
head, err := head.Marshal()
if err == nil {
templ_7745c5c3_Err = templ.Raw(head).Render(ctx, templ_7745c5c3_Buffer)
if templ_7745c5c3_Err != nil {
return templ_7745c5c3_Err
}
}
}
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 2, "<docTitle><text>")
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 1, "<html xmlns=\"http://www.w3.org/1999/xhtml\" xmlns:epub=\"http://www.idpf.org/2007/ops\" xml:lang=\"zh-CN\"><head><title>Cover</title></head><style type=\"text/css\">\n\t\t@page {\n\t\tpadding: 0pt;\n\t\tmargin: 0pt\n\t\t}\n\t\tbody {\n\t\ttext-align: center;\n\t\tpadding: 0pt;\n\t\tmargin: 0pt;\n\t\t}\n\t</style><body><div><svg xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\" version=\"1.1\" width=\"100%\" height=\"100%\" viewBox=\"0 0 400 581\" preserveAspectRatio=\"none\"><image width=\"400\" height=\"581\" xlink:href=\"")
if templ_7745c5c3_Err != nil {
return templ_7745c5c3_Err
}
var templ_7745c5c3_Var2 string
templ_7745c5c3_Var2, templ_7745c5c3_Err = templ.JoinStringErrs(title)
templ_7745c5c3_Var2, templ_7745c5c3_Err = templ.JoinStringErrs(coverPath)
if templ_7745c5c3_Err != nil {
return templ.Error{Err: templ_7745c5c3_Err, FileName: `template/toc.ncx.templ`, Line: 16, Col: 16}
return templ.Error{Err: templ_7745c5c3_Err, FileName: `template/cover.xhtml.templ`, Line: 32, Col: 58}
}
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var2))
if templ_7745c5c3_Err != nil {
return templ_7745c5c3_Err
}
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 3, "</text></docTitle> ")
if templ_7745c5c3_Err != nil {
return templ_7745c5c3_Err
}
if navMap != nil {
navMap, err := navMap.Marshal()
if err == nil {
templ_7745c5c3_Err = templ.Raw(navMap).Render(ctx, templ_7745c5c3_Buffer)
if templ_7745c5c3_Err != nil {
return templ_7745c5c3_Err
}
}
}
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 4, "</ncx>")
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 2, "\"></image></svg></div></body></html>")
if templ_7745c5c3_Err != nil {
return templ_7745c5c3_Err
}

View File

@@ -1,25 +0,0 @@
package template
import "bilinovel-downloader/model"
templ TocNCX(title string, head *model.TocNCXHead, navMap *model.NavMap) {
@templ.Raw(`<?xml version="1.0" encoding="UTF-8"?>`)
@templ.Raw(`<!DOCTYPE ncx PUBLIC "-//NISO//DTD ncx 2005-1//EN" "http://www.daisy.org/z3986/2005/ncx-2005-1.dtd">`)
<ncx xmlns="http://www.daisy.org/z3986/2005/ncx/" version="2005-1">
if head != nil {
{{ head, err := head.Marshal() }}
if err == nil {
@templ.Raw(head)
}
}
<docTitle>
<text>{ title }</text>
</docTitle>
if navMap != nil {
{{ navMap, err := navMap.Marshal() }}
if err == nil {
@templ.Raw(navMap)
}
}
</ncx>
}

59
test/bilinovel_test.go Normal file
View File

@@ -0,0 +1,59 @@
package test
import (
"bilinovel-downloader/downloader/bilinovel"
"encoding/json"
"fmt"
"testing"
)
func TestBilinovel_GetNovel(t *testing.T) {
bilinovel, err := bilinovel.New(bilinovel.BilinovelNewOption{Headless: false, Concurrency: 5})
bilinovel.SetTextOnly(true)
if err != nil {
t.Fatalf("failed to create bilinovel: %v", err)
}
novel, err := bilinovel.GetNovel(2727, false, nil)
if err != nil {
t.Fatalf("failed to get novel: %v", err)
}
jsonBytes, err := json.Marshal(novel)
if err != nil {
t.Fatalf("failed to marshal novel: %v", err)
}
fmt.Println(string(jsonBytes))
}
func TestBilinovel_GetVolume(t *testing.T) {
bilinovel, err := bilinovel.New(bilinovel.BilinovelNewOption{Headless: false, Concurrency: 1})
bilinovel.SetTextOnly(true)
if err != nil {
t.Fatalf("failed to create bilinovel: %v", err)
}
volume, err := bilinovel.GetVolume(2727, 129092, false)
if err != nil {
t.Fatalf("failed to get volume: %v", err)
}
jsonBytes, err := json.Marshal(volume)
if err != nil {
t.Fatalf("failed to marshal volume: %v", err)
}
fmt.Println(string(jsonBytes))
}
func TestBilinovel_GetChapter(t *testing.T) {
bilinovel, err := bilinovel.New(bilinovel.BilinovelNewOption{Headless: false, Concurrency: 1})
bilinovel.SetTextOnly(true)
if err != nil {
t.Fatalf("failed to create bilinovel: %v", err)
}
chapter, err := bilinovel.GetChapter(2727, 129092, 129094)
if err != nil {
t.Fatalf("failed to get chapter: %v", err)
}
jsonBytes, err := json.Marshal(chapter)
if err != nil {
t.Fatalf("failed to marshal chapter: %v", err)
}
fmt.Println(string(jsonBytes))
}

55
text/wrapper.go Normal file
View File

@@ -0,0 +1,55 @@
package text
import (
"bilinovel-downloader/model"
"bilinovel-downloader/utils"
"fmt"
"os"
"path/filepath"
"strings"
"github.com/PuerkitoBio/goquery"
)
func PackVolumeToText(volume *model.Volume, outputPath string) error {
outputPath = filepath.Join(outputPath, utils.CleanDirName(volume.Title))
_, err := os.Stat(outputPath)
if err != nil {
if os.IsNotExist(err) {
err = os.MkdirAll(outputPath, 0755)
if err != nil {
return fmt.Errorf("failed to create output directory: %v", err)
}
} else {
return fmt.Errorf("failed to get output directory: %v", err)
}
} else {
err = os.RemoveAll(outputPath)
if err != nil {
return fmt.Errorf("failed to remove output directory: %v", err)
}
err = os.MkdirAll(outputPath, 0755)
if err != nil {
return fmt.Errorf("failed to create output directory: %v", err)
}
}
for i, chapter := range volume.Chapters {
chapterPath := filepath.Join(outputPath, fmt.Sprintf("%03d-%s.txt", i, chapter.Title))
chapterFile, err := os.Create(chapterPath)
if err != nil {
return fmt.Errorf("failed to create chapter file: %v", err)
}
defer chapterFile.Close()
doc, err := goquery.NewDocumentFromReader(strings.NewReader(chapter.Content.Html))
if err != nil {
return fmt.Errorf("failed to create chapter file: %v", err)
}
doc.Find("img").Remove()
text := doc.Text()
_, err = chapterFile.WriteString(strings.TrimSpace(text))
if err != nil {
return fmt.Errorf("failed to write chapter file: %v", err)
}
}
return nil
}

View File

@@ -9,11 +9,19 @@ import (
"github.com/go-resty/resty/v2"
)
var client *resty.Client
type RestyClient struct {
client *resty.Client
concurrency int
sem chan struct{}
}
func init() {
client = resty.New()
client.SetTransport(&http.Transport{
func NewRestyClient(concurrency int) *RestyClient {
client := &RestyClient{
client: resty.New(),
concurrency: concurrency,
sem: make(chan struct{}, concurrency),
}
client.client.SetTransport(&http.Transport{
DialContext: func(ctx context.Context, network, addr string) (net.Conn, error) {
if addr == "www.bilinovel.com:443" {
addr = "64.140.161.52:443"
@@ -24,7 +32,16 @@ func init() {
},
TLSHandshakeTimeout: 10 * time.Second,
})
client.SetRetryCount(10).
client.client.
OnBeforeRequest(func(c *resty.Client, req *resty.Request) error {
client.sem <- struct{}{}
return nil
}).
OnAfterResponse(func(c *resty.Client, resp *resty.Response) error {
<-client.sem
return nil
})
client.client.SetRetryCount(10).
SetRetryWaitTime(3 * time.Second).
SetRetryAfter(func(client *resty.Client, resp *resty.Response) (time.Duration, error) {
if resp.StatusCode() == http.StatusTooManyRequests {
@@ -43,10 +60,13 @@ func init() {
AddRetryCondition(func(r *resty.Response, err error) bool {
return err != nil || r.StatusCode() == http.StatusTooManyRequests
})
client.client.SetLogger(disableLogger{}).SetHeader("Accept-Charset", "utf-8").SetHeader("User-Agent", "Mozilla/5.0 (X11; Linux x86_64; rv:133.0) Gecko/20100101 Firefox/133.0")
return client
}
func Request() *resty.Request {
return client.R().SetLogger(disableLogger{}).SetHeader("Accept-Charset", "utf-8").SetHeader("User-Agent", "Mozilla/5.0 (X11; Linux x86_64; rv:133.0) Gecko/20100101 Firefox/133.0")
func (c *RestyClient) R() *resty.Request {
return c.client.R()
}
type disableLogger struct{}