Real backends + fetch/display workflows; config paths

Wire the incremental pipeline to real sources and expose two workflows.

Config (Env.py): three explicit paths — LOCAL_LIBRARY_PATH (scan source),
ITUNES_XML_PATH (manual export), DATA_DIR (storage root: DB + Spotify cache).

Real, incremental ingesters:
- local_scan: walks the library, skips files whose mtime/size are unchanged,
  prunes deleted ones, and reads embedded ISRC + recording MBID from tags
  (tagged files get real IDs with no network), via tags.py.
- itunes_xml: parse the iTunes Library.xml as a plist; skip if mtime unchanged.
- spotify_raw: ingest raw Spotify JSON (ISRC from external_ids); live fetch
  wrapper for when credentials are set.

Workflows (src/workflows.py + main.py):
- `fetch`  — pull + scan + parse + enrich + match, with readable step logs.
  First run is slow (full scan + MB lookups); later runs only touch changes
  (verified: ~71s first, ~0.6s second).
- `display` — launch the GUI.

Also: enrich uses ISRC-first again (local MBIDs now come from tags), fix
--enrich-limit 0, quiet musicbrainzngs XML warnings, ignore /data/.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

README.md

@@ -48,68 +48,73 @@ source .venv/bin/activate
 pip install -r requirements.txt
 ```
 
-### Credentials
-
-Create an app at https://developer.spotify.com/dashboard and add
-`http://127.0.0.1:8888` as a **Redirect URI**. Credentials are read from the
-environment — never commit them.
+### Configure (`.env`)
 
 ```bash
-cp .env.example .env
-# edit .env with your client id/secret
-source .env
+cp .env.example .env       # then edit and `source .env`
 ```
 
-`Env.py` reads `SPOTIFY_CLIENT_ID` / `SPOTIFY_CLIENT_SECRET` (only needed for
-live fetching), plus optional `DATABASE_URL`, `PREFETCHED_DIR` and `MB_CONTACT`.
+Three things you point at your own data:
 
-### Database
+| Variable | What |
+|----------|------|
+| `LOCAL_LIBRARY_PATH` | folder of audio files to scan (ISRC/MusicBrainz tags are read) |
+| `ITUNES_XML_PATH` | the `Library.xml` you export from Music (File → Library → Export Library…) |
+| `DATA_DIR` | storage root — the database and cached Spotify responses live here |
 
-Defaults to a local SQLite file (`musicindexer.db`). To use Postgres instead,
-set one env var — no code changes:
+`SPOTIFY_CLIENT_ID` / `SPOTIFY_CLIENT_SECRET` are only needed for live Spotify
+fetching (register `http://127.0.0.1:8888` as a Redirect URI in your
+[Spotify app](https://developer.spotify.com/dashboard)). Set
+`DATABASE_URL=postgresql+psycopg://…` to use Postgres instead of SQLite — no
+code change.
+
+## Two workflows
 
 ```bash
-export DATABASE_URL=postgresql+psycopg://user:pass@localhost:5432/musicindexer
+python main.py fetch      # do all the work, incrementally, and report
+python main.py display    # open the GUI to see what you're missing
 ```
 
-## Usage
+**`fetch`** pulls Spotify, scans the local library, parses the iTunes export,
+resolves MBIDs, and matches — logging progress at each step. The **first run is
+slow** (full scan + MusicBrainz lookups at ~1 req/s); **later runs are quick**
+because only changed files, the iTunes export if it changed, and still-unresolved
+MBIDs are touched. Useful flags/helpers:
 
 ```bash
-python main.py ingest        # load prefetched data (work_dir/latest-1) into the DB
-python main.py enrich        # resolve MusicBrainz MBIDs (resumable; ~1 req/s)
-python main.py match         # print present/missing summary
-./gui.sh                     # browse what you're missing
+python main.py fetch --enrich-limit 500   # cap MBID lookups this run
+python main.py enrich                      # resume MBID resolution only
+python main.py match                       # print the present/missing summary
 ```
 
-`ingest` and the GUI work immediately (text-based matching). `enrich` queries
-MusicBrainz only for tracks that don't have an MBID yet, caches every result,
-and is **resumable** — if it's interrupted, just run it again to continue. Run
-`python main.py enrich --limit 300` to do a quick partial pass, or
-`--source local` to restrict to one source.
-
-> Live fetching (`SpotifyWebAPI`, the local indexer) is unchanged and still
-> available; this redesign reads prefetched JSON so the pipeline can be built
-> and demoed without network access.
+> ISRC/MBID matching works as soon as `fetch` runs (local tags + Spotify ISRC).
+> MusicBrainz enrichment fills in the rest (iTunes, untagged local files); it's
+> cached and resumable, so you can stop/continue any time.
 
 ## Layout
 
 ```
-main.py                 CLI: ingest / enrich / match
+main.py                 CLI: fetch / display / enrich / match
 Gui.py / gui.sh         Streamlit GUI (missing-tracks viewer)
-Env.py                  config (DB URL, paths, credentials from env)
+Env.py                  config: LOCAL_LIBRARY_PATH / ITUNES_XML_PATH / DATA_DIR
 src/
+  workflows.py          run_fetch (the whole pipeline) + run_display
   db/                   SQLAlchemy models + engine/session
-  ingest/               load prefetched JSON into the DB (upsert)
+  ingest/
+    spotify_raw.py      live fetch + ingest raw Spotify JSON (ISRC)
+    local_scan.py       incremental file scan (reads ISRC/MBID tags)
+    itunes_xml.py       parse the iTunes Library.xml (plist)
+    tags.py             read ISRC / recording MBID / duration from a file
   enrich/               MusicBrainz MBID resolver (cached, resumable)
   match/                normalization + cross-source matcher
-  backends/             live fetchers/parsers (spotify, local, itunes, ...)
-  Library.py            in-memory data model used by the backends
-work_dir/latest-1/      prefetched JSON used by `ingest` (gitignored)
-musicindexer.db         the database (gitignored)
+  backends/             lower-level Spotify Web API client / OAuth
+$DATA_DIR/              database + cached Spotify responses (gitignored)
 ```
 
 ## Notes
 
-- The DB is the source of truth; `work_dir/` JSON is only an ingest source.
+- The DB is the source of truth; everything is incremental and re-runnable.
+- Local matching is real-ID where files are tagged (ISRC/MusicBrainz, e.g. via
+  Picard); untagged files fall back to MusicBrainz text lookup, then text.
 - Only currently-supported Spotify endpoints are used (no deprecated
   audio-features/recommendations/etc.).