# MusicIndexer

Pull your Spotify library and listening stats via the Spotify Web API, index a
local music library (audio tags + iTunes XML), and match the two so you can see
which tracks you own, what's missing, and your top artists/tracks/playlists — in
a Streamlit GUI.

It is **database-backed and incremental**: data lives in a DB (SQLite by
default, Postgres optional) and every step UPSERTs, so re-running only updates
what changed and nothing already fetched is lost. The headline use case is
seeing **which Spotify/iTunes tracks you don't have locally**, decided by real
recording identifiers (ISRC / MusicBrainz MBID) with a normalized-text fallback.

## Pipeline

```
ingest  ->  enrich (MBIDs)  ->  match  ->  GUI
```

| Step | Command | Incremental rule |
|------|---------|------------------|
| **ingest** | `python main.py ingest` | upsert tracks by `(source, source_id)`; never drops enrichment |
| **enrich** | `python main.py enrich` | query MusicBrainz only for tracks with no MBID yet; every lookup (incl. "not found") cached; rate-limited & resumable |
| **match** | `python main.py match` | classify each Spotify/iTunes track: present locally (ISRC > MBID > text) or missing |
| **GUI** | `./gui.sh` | browse missing/present with the match method shown |

### How matching works

Each track is resolved to a MusicBrainz **recording MBID** by searching with the
*normalized* artist+title, so the same song from different sources lands on the
same MBID (and shares one cached lookup). A Spotify/iTunes track counts as
"have" if its ISRC, its MBID, or its normalized artist|title matches a local
track — otherwise it's **missing**. The GUI labels each match `isrc`/`mbid`
(ID-certain) or `text` (fuzzy).

> Note: the local library here has no ISRC/MBID tags, so before `enrich` runs,
> matching is normalized-text only (still useful). `enrich` upgrades matches to
> ID-certain. Tagging local files with Picard/AcoustID (ISRC) would make the ID
> tiers exact end-to-end.

## Setup

Requires Python 3.13+.

```bash
python -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
```

### Credentials

Create an app at https://developer.spotify.com/dashboard and add
`http://127.0.0.1:8888` as a **Redirect URI**. Credentials are read from the
environment — never commit them.

```bash
cp .env.example .env
# edit .env with your client id/secret
source .env
```

`Env.py` reads `SPOTIFY_CLIENT_ID` / `SPOTIFY_CLIENT_SECRET` (only needed for
live fetching), plus optional `DATABASE_URL`, `PREFETCHED_DIR` and `MB_CONTACT`.

### Database

Defaults to a local SQLite file (`musicindexer.db`). To use Postgres instead,
set one env var — no code changes:

```bash
export DATABASE_URL=postgresql+psycopg://user:pass@localhost:5432/musicindexer
```

## Usage

```bash
python main.py ingest        # load prefetched data (work_dir/latest-1) into the DB
python main.py enrich        # resolve MusicBrainz MBIDs (resumable; ~1 req/s)
python main.py match         # print present/missing summary
./gui.sh                     # browse what you're missing
```

`ingest` and the GUI work immediately (text-based matching). `enrich` queries
MusicBrainz only for tracks that don't have an MBID yet, caches every result,
and is **resumable** — if it's interrupted, just run it again to continue. Run
`python main.py enrich --limit 300` to do a quick partial pass, or
`--source local` to restrict to one source.

> Live fetching (`SpotifyWebAPI`, the local indexer) is unchanged and still
> available; this redesign reads prefetched JSON so the pipeline can be built
> and demoed without network access.

## Layout

```
main.py                 CLI: ingest / enrich / match
Gui.py / gui.sh         Streamlit GUI (missing-tracks viewer)
Env.py                  config (DB URL, paths, credentials from env)
src/
  db/                   SQLAlchemy models + engine/session
  ingest/               load prefetched JSON into the DB (upsert)
  enrich/               MusicBrainz MBID resolver (cached, resumable)
  match/                normalization + cross-source matcher
  backends/             live fetchers/parsers (spotify, local, itunes, ...)
  Library.py            in-memory data model used by the backends
work_dir/latest-1/      prefetched JSON used by `ingest` (gitignored)
musicindexer.db         the database (gitignored)
```

## Notes

- The DB is the source of truth; `work_dir/` JSON is only an ingest source.
- Only currently-supported Spotify endpoints are used (no deprecated
  audio-features/recommendations/etc.).