refactor: switch index from SQLite to JSON with relative paths
manage_index.py: - Complete rewrite from SQLite to JSON file format - All file paths stored as relative (relative to video directory) - Moving the entire video directory does not break references - Paths outside video dir stored as absolute (editable in text editor) - Same 8 subcommands, same CLI interface SKILL.md: - Step 5: SQLite schema replaced with JSON structure example - Verification checklist: .vedit.db → .vedit.json README.md + README.zh.md: - Features: 'SQLite database' → 'JSON file with relative paths' - Project structure: 'SQLite index management' → 'JSON index management' - 'Index Database' section → 'Index File' with relative path explanation .gitignore: - *.vedit.db → *.vedit.json
This commit is contained in:
@@ -102,47 +102,55 @@ uv run --directory SKILL_DIR/scripts/transcription funasr-nano VIDEO_PATH --list
|
||||
|
||||
### Step 5 — Manage index file
|
||||
|
||||
An SQLite database tracks all processing artifacts to avoid duplicate work. It lives **next to the video file**.
|
||||
A JSON file tracks all processing artifacts to avoid duplicate work. It lives **next to the video file** and is human-readable and editable.
|
||||
|
||||
```
|
||||
<video_dir>/<video_stem>.vedit.db
|
||||
<video_dir>/<video_stem>.vedit.json
|
||||
```
|
||||
|
||||
Schema (managed by `scripts/index/manage_index.py`):
|
||||
**All file paths are stored as relative paths** (relative to the video directory). Moving the entire directory does not break any references. If files are split across directories, paths outside the video directory are stored as absolute paths — the user can edit the JSON directly with a text editor to fix them.
|
||||
|
||||
```sql
|
||||
CREATE TABLE IF NOT EXISTS transcription (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
json_path TEXT NOT NULL,
|
||||
track_index INTEGER NOT NULL,
|
||||
created_at REAL NOT NULL, -- Unix timestamp
|
||||
duration_s REAL
|
||||
);
|
||||
Structure (managed by `scripts/index/manage_index.py`):
|
||||
|
||||
CREATE TABLE IF NOT EXISTS clips (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
start_time REAL NOT NULL, -- seconds from video start
|
||||
end_time REAL NOT NULL,
|
||||
path TEXT NOT NULL,
|
||||
created_at REAL NOT NULL, -- Unix timestamp
|
||||
reason TEXT -- why this clip was extracted
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS frames (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
clip_id INTEGER NOT NULL REFERENCES clips(id),
|
||||
timestamp REAL NOT NULL, -- seconds from video start
|
||||
path TEXT NOT NULL,
|
||||
scene_score REAL, -- ffmpeg scene score if available
|
||||
method TEXT NOT NULL, -- 'scene' | 'sample' | 'both'
|
||||
created_at REAL NOT NULL -- Unix timestamp
|
||||
);
|
||||
```json
|
||||
{
|
||||
"transcriptions": [
|
||||
{
|
||||
"id": 1,
|
||||
"json_path": "video_track1_fun-asr-nano.json",
|
||||
"track_index": 1,
|
||||
"created_at": 1783997794.78,
|
||||
"duration_s": 120.5
|
||||
}
|
||||
],
|
||||
"clips": [
|
||||
{
|
||||
"id": 1,
|
||||
"start_time": 30.0,
|
||||
"end_time": 60.0,
|
||||
"path": "video_clips/clip_30-60.mkv",
|
||||
"created_at": 1783997794.85,
|
||||
"reason": "user asked about this segment"
|
||||
}
|
||||
],
|
||||
"frames": [
|
||||
{
|
||||
"id": 1,
|
||||
"clip_id": 1,
|
||||
"timestamp": 32.0,
|
||||
"path": "video_frames/frame_0001.jpg",
|
||||
"scene_score": 0.45,
|
||||
"method": "scene",
|
||||
"created_at": 1783997795.0
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
Use `scripts/index/manage_index.py` for all CRUD operations:
|
||||
|
||||
```bash
|
||||
# Initialize database (idempotent)
|
||||
# Initialize index file (idempotent)
|
||||
uv run --directory SKILL_DIR python scripts/index/manage_index.py init --video VIDEO_PATH
|
||||
|
||||
# Add transcription record
|
||||
@@ -165,6 +173,9 @@ uv run --directory SKILL_DIR python scripts/index/manage_index.py list
|
||||
|
||||
# Check if a time range has already been extracted
|
||||
uv run --directory SKILL_DIR python scripts/index/manage_index.py check-range --start S --end E
|
||||
|
||||
# Remove records for files that no longer exist on disk
|
||||
uv run --directory SKILL_DIR python scripts/index/manage_index.py clean
|
||||
```
|
||||
|
||||
**Completion criteria:** index file exists and all produced artifacts are recorded in it.
|
||||
@@ -349,7 +360,7 @@ Check only the items relevant to the steps actually executed. Not all items appl
|
||||
- [ ] Video path and audio track index(es) confirmed. *(Step 2 — required when video processing is needed)*
|
||||
- [ ] User's editing goal understood. *(Step 3 — skip if not relevant to the user's request)*
|
||||
- [ ] Transcription completed for all requested tracks, or reused from cache. *(Step 4)*
|
||||
- [ ] Index file initialized at `<video_dir>/<video_stem>.vedit.db`. *(Step 5 — always required when any processing is done)*
|
||||
- [ ] Index file initialized at `<video_dir>/<video_stem>.vedit.json`. *(Step 5 — always required when any processing is done)*
|
||||
- [ ] All extracted clips and frames recorded in the index. *(Step 6 — only if frames were extracted)*
|
||||
- [ ] Vision analysis performed where needed. *(Step 7 — only if frames were extracted)*
|
||||
- [ ] Edit plan produced as Markdown table with overall recommendation. *(Step 8 — skip if not relevant to the user's request)*
|
||||
|
||||
Reference in New Issue
Block a user