MusicIndexer

Pull your Spotify library and listening stats via the Spotify Web API, index a local music library (audio tags + iTunes XML), and match the two so you can see which tracks you own, what's missing, and your top artists/tracks/playlists — in a Streamlit GUI.

It is database-backed and incremental: data lives in a DB (SQLite by default, Postgres optional) and every step UPSERTs, so re-running only updates what changed and nothing already fetched is lost. The headline use case is seeing which Spotify/iTunes tracks you don't have locally, decided by real recording identifiers (ISRC / MusicBrainz MBID) with a normalized-text fallback.

Pipeline

ingest  ->  enrich (MBIDs)  ->  match  ->  GUI

Step	Command	Incremental rule
ingest	python main.py ingest	upsert tracks by (source, source_id); never drops enrichment
enrich	python main.py enrich	query MusicBrainz only for tracks with no MBID yet; every lookup (incl. "not found") cached; rate-limited & resumable
match	python main.py match	classify each Spotify/iTunes track: present locally (ISRC > MBID > text) or missing
GUI	./gui.sh	browse missing/present with the match method shown

How matching works

Each track is resolved to a MusicBrainz recording MBID by searching with the normalized artist+title, so the same song from different sources lands on the same MBID (and shares one cached lookup). A Spotify/iTunes track counts as "have" if its ISRC, its MBID, or its normalized artist|title matches a local track — otherwise it's missing. The GUI labels each match isrc/mbid (ID-certain) or text (fuzzy).

Note: the local library here has no ISRC/MBID tags, so before enrich runs, matching is normalized-text only (still useful). enrich upgrades matches to ID-certain. Tagging local files with Picard/AcoustID (ISRC) would make the ID tiers exact end-to-end.

Setup

Requires Python 3.13+.

python -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt

Credentials

Create an app at https://developer.spotify.com/dashboard and add http://127.0.0.1:8888 as a Redirect URI. Credentials are read from the environment — never commit them.

cp .env.example .env
# edit .env with your client id/secret
source .env

Env.py reads SPOTIFY_CLIENT_ID / SPOTIFY_CLIENT_SECRET (only needed for live fetching), plus optional DATABASE_URL, PREFETCHED_DIR and MB_CONTACT.

Database

Defaults to a local SQLite file (musicindexer.db). To use Postgres instead, set one env var — no code changes:

export DATABASE_URL=postgresql+psycopg://user:pass@localhost:5432/musicindexer

Usage

python main.py ingest        # load prefetched data (work_dir/latest-1) into the DB
python main.py enrich        # resolve MusicBrainz MBIDs (resumable; ~1 req/s)
python main.py match         # print present/missing summary
./gui.sh                     # browse what you're missing

ingest and the GUI work immediately (text-based matching). enrich queries MusicBrainz only for tracks that don't have an MBID yet, caches every result, and is resumable — if it's interrupted, just run it again to continue. Run python main.py enrich --limit 300 to do a quick partial pass, or --source local to restrict to one source.

Live fetching (SpotifyWebAPI, the local indexer) is unchanged and still available; this redesign reads prefetched JSON so the pipeline can be built and demoed without network access.

Layout

main.py                 CLI: ingest / enrich / match
Gui.py / gui.sh         Streamlit GUI (missing-tracks viewer)
Env.py                  config (DB URL, paths, credentials from env)
src/
  db/                   SQLAlchemy models + engine/session
  ingest/               load prefetched JSON into the DB (upsert)
  enrich/               MusicBrainz MBID resolver (cached, resumable)
  match/                normalization + cross-source matcher
  backends/             live fetchers/parsers (spotify, local, itunes, ...)
  Library.py            in-memory data model used by the backends
work_dir/latest-1/      prefetched JSON used by `ingest` (gitignored)
musicindexer.db         the database (gitignored)

Notes

• The DB is the source of truth; work_dir/ JSON is only an ingest source.
• Only currently-supported Spotify endpoints are used (no deprecated audio-features/recommendations/etc.).