DB-backed incremental redesign: find tracks missing locally
Replace the JSON-file pipeline with a SQLite/SQLAlchemy store (Postgres via DATABASE_URL) and an incremental, ID-based matcher whose goal is "which Spotify/iTunes tracks am I missing locally". - src/db: models (Track/MbCache/FileState/IngestState) + engine/session. - src/ingest: upsert prefetched spotify/itunes/local (work_dir/latest-1) by (source, source_id); pulls ISRC from raw Spotify; preserves enrichment. - src/enrich: MusicBrainz recording-MBID resolver. Resolves via NORMALIZED text search (so the same song across sources lands on one MBID + one cached lookup), ISRC as exact fallback. Only touches unresolved tracks, caches positive AND negative results, rate-limited and resumable. - src/match: normalization + cross-source matcher with tiers ISRC > MBID > normalized text; reports present/missing and the match method. - main.py: CLI (ingest / enrich / match). Gui.py: missing-tracks viewer. - README + requirements (SQLAlchemy, musicbrainzngs); ignore *.db. Incremental everywhere: re-running updates only what changed; enrich continues where it stopped. Live Spotify/local backends are left intact. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
parent
83f782d125
commit
0e64d439c4
16 changed files with 784 additions and 274 deletions
12
Env.py
12
Env.py
|
|
@ -11,3 +11,15 @@ work_dir = Path(os.environ.get("WORK_DIR", "./work_dir/new"))
|
|||
# export SPOTIFY_CLIENT_SECRET=...
|
||||
client_id = os.environ.get("SPOTIFY_CLIENT_ID")
|
||||
client_secret = os.environ.get("SPOTIFY_CLIENT_SECRET")
|
||||
|
||||
# Persistent store. Defaults to a local SQLite file; point DATABASE_URL at a
|
||||
# Postgres instance (postgresql+psycopg://user:pass@host/db) to switch with no
|
||||
# code changes.
|
||||
database_url = os.environ.get("DATABASE_URL", "sqlite:///./musicindexer.db")
|
||||
|
||||
# Folder of prefetched JSON used to (re)build the DB without hitting the network.
|
||||
prefetched_dir = Path(os.environ.get("PREFETCHED_DIR", "./work_dir/latest-1"))
|
||||
|
||||
# Contact email sent to MusicBrainz in the User-Agent (be a good API citizen).
|
||||
mb_contact = os.environ.get("MB_CONTACT", "music-indexer@example.com")
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue