186 lines
12 KiB
Markdown
186 lines
12 KiB
Markdown
# 🎵 Ainulindale
|
|
|
|
> *"There was Eru, the One [...] and he made first the Ainur [...] and they sang before him, and he was glad."*
|
|
> — J.R.R. Tolkien, The Silmarillion
|
|
|
|
A fully self-hosted, automated music discovery and streaming system.
|
|
No Spotify. No YouTube Music. No ads. Just your music, your server, your rules.
|
|
|
|
---
|
|
|
|
## What This Stack Does
|
|
|
|
- **Streams** your library to any device through Navidrome
|
|
- **Discovers** new music every week from **two independent sources**: ListenBrainz Weekly Exploration and Last.fm recommendations. If one of them is down, the other still delivers
|
|
- **Downloads** the discoveries from Soulseek at high quality (FLAC/WAV preferred, 320 kbps minimum), through a VPN
|
|
- **Keeps only this week and last week** of discoveries. Older playlists and their files are removed, except tracks you starred or added to a playlist of your own
|
|
- **Heals itself**: a Soulseek session that drops is reconnected, and a stuck container is restarted, without you noticing
|
|
|
|
## Components
|
|
|
|
| Component | Role |
|
|
|---|---|
|
|
| **Navidrome** | Music server (installed separately, e.g. Unraid Community Apps) |
|
|
| **Gluetun** | VPN gateway. All Soulseek traffic exits via ProtonVPN |
|
|
| **slskd** | Soulseek client, living inside gluetun's network |
|
|
| **autoheal** | Restarts gluetun / slskd when Docker marks them unhealthy |
|
|
| **Explo** | Searches slskd, downloads, builds the Navidrome playlist |
|
|
| **discovery/discover.py** | Runs inside the Explo container and decides *what* Explo imports and *when*; adds Last.fm, outage handling and retention |
|
|
| **Lidarr** | Optional, for whole-album grabs |
|
|
|
|
## How Discovery Works
|
|
|
|
Once a day (`DISCOVERY_SCHEDULE`, default 00:15) the orchestrator runs a **tick**. A tick only does work that is still missing, so it is safe at any frequency and after any restart:
|
|
|
|
```
|
|
tick
|
|
├─ ListenBrainz: is there a Weekly Exploration playlist I have not imported yet?
|
|
│ yes → check slskd is logged in → Explo imports it → "Weekly-Exploration-2026-Week38"
|
|
│ no → nothing to do LB down → note it, try again next tick
|
|
├─ Last.fm: does this week's playlist exist yet?
|
|
│ no → pull recommendations → hand them to Explo → "Lastfm-Recommended-2026-Week38"
|
|
└─ Prune: keep the newest KEEP_WEEKS per source, remove the rest
|
|
```
|
|
|
|
Things worth knowing:
|
|
|
|
- **ListenBrainz is tracked by playlist ID, not by calendar.** If ListenBrainz publishes late, or is down on Tuesday, the playlist is simply picked up on the first tick after it appears. If it never appears, last week's playlist is *not* re-imported under a new name. A playlist that keeps failing is attempted 3 times, then left alone.
|
|
- **Last.fm recommendations** come from the same feed the Last.fm website player uses (`last.fm/player/station/user/<you>/recommended`). It is public, needs no login, and returns Last.fm's own picks of artists you have not played. It is not an official API, so with `LASTFM_API_KEY` set there is a second path built purely on the official API (artists similar to your last three months, minus everyone you already know). Tracks recommended in earlier weeks are remembered and not repeated.
|
|
- **`LASTFM_MODE`**: `always` gives you two playlists a week; `fallback` only steps in, from Wednesday on, in weeks where ListenBrainz has produced nothing; `off` disables it.
|
|
- **slskd preflight.** Before Explo starts, the orchestrator checks that slskd is really logged in to Soulseek and nudges it if not. If it stays down, the run is postponed to the next tick. Without this, one bad night means fifty YouTube rips instead of fifty FLACs.
|
|
- **No jams.** Weekly Jams and Daily Jams are gone. `PRUNE_LEGACY_JAMS=true` cleans up what the old setup left behind.
|
|
|
|
### Retention
|
|
|
|
With `KEEP_WEEKS=2` you always have this week's and last week's playlist per source. On each tick, anything older goes: the Navidrome playlist and the matching folder under `music/explo/`.
|
|
|
|
Before a folder is deleted, every file in it is checked against your **starred tracks and every playlist that is staying**. Matches are moved to `music/explo/Keepers/` instead of deleted, so starring a track is how you say "keep this one". Only folders the pipeline created (`Weekly-Exploration-*`, `Custom-Lastfm-*`) are ever touched.
|
|
|
|
Preview it first:
|
|
|
|
```bash
|
|
docker exec explo python3 /ainulindale/discover.py prune --dry-run
|
|
```
|
|
|
|
`PRUNE_FILES=false` removes old playlists but leaves all files on disk.
|
|
|
|
## Why slskd Stays Connected Now
|
|
|
|
The old setup had three gaps, which together explain "slskd drops and autoheal does nothing":
|
|
|
|
1. **slskd's healthcheck never looked at Soulseek.** It only asks whether the web server answers (and the image gives itself a 60 minute start period). Logged out of Soulseek with the UI up counts as healthy, so autoheal had nothing to act on. `slskd/healthcheck.sh` replaces it and reports the Soulseek login state. On a drop it first asks slskd to reconnect; only three failed checks in a row lead to a restart.
|
|
2. **A gluetun restart strands slskd.** slskd borrows gluetun's network namespace. When gluetun restarts, slskd is left holding the old, dead one: localhost still works (so the old check passed) but nothing reaches the internet. The new check notices that gluetun's control server has vanished from localhost and fails, and autoheal restarts slskd into the new namespace.
|
|
3. **slskd did not know about the VPN.** slskd has a native gluetun integration that was not switched on. It now polls gluetun every few seconds, drops the Soulseek session when the tunnel drops, logs in again when it is back, and applies the forwarded port by itself when port forwarding is on.
|
|
|
|
When the tunnel itself is down, the healthcheck stays green on purpose: restarting slskd cannot fix a VPN, and gluetun repairs its own tunnel.
|
|
|
|
One thing no healthcheck can fix: **do not log in with the same Soulseek account anywhere else** (Nicotine+, SoulseekQt on a desktop). The server kicks the older session, and the two clients will keep throwing each other out.
|
|
|
|
---
|
|
|
|
## Setup
|
|
|
|
### 1. Clone and configure
|
|
|
|
```bash
|
|
git clone https://github.com/Quinta0/Ainulindale
|
|
cd Ainulindale
|
|
cp .env.example .env
|
|
```
|
|
|
|
Everything lives in that one `.env`: VPN key, Soulseek login, Navidrome login, ListenBrainz and Last.fm names, retention. Generate the two internal keys with `openssl rand -hex 24`.
|
|
|
|
Run `docker compose` from this folder. The compose file mounts `./discovery` and `./slskd/healthcheck.sh` by relative path.
|
|
|
|
### 2. Folders
|
|
|
|
```bash
|
|
APPDATA_PATH=/mnt/user/appdata; MUSIC_PATH=/mnt/user/data/media/music # same as in .env
|
|
mkdir -p $APPDATA_PATH/{gluetun,slskd,explo/config}
|
|
mkdir -p $MUSIC_PATH/{explo,slskd/incomplete}
|
|
touch $MUSIC_PATH/slskd/incomplete/.ndignore # Navidrome skips half-downloaded files
|
|
cp slskd/slskd.yml $APPDATA_PATH/slskd/slskd.yml
|
|
```
|
|
|
|
```
|
|
$APPDATA_PATH/
|
|
├── gluetun/
|
|
├── slskd/slskd.yml ← tuning only, no secrets
|
|
└── explo/config/ ← Explo cache + ainulindale-state.json (must persist)
|
|
|
|
$MUSIC_PATH/
|
|
├── slskd/ ← slskd downloads (manual ones show up in Navidrome from here)
|
|
│ └── incomplete/
|
|
├── explo/
|
|
│ ├── Weekly-Exploration-2026-Week38/
|
|
│ ├── Custom-Lastfm-2026-38/
|
|
│ └── Keepers/ ← rescued favourites
|
|
└── (your library)
|
|
```
|
|
|
|
### 3. ProtonVPN key
|
|
|
|
[account.proton.me](https://account.proton.me) → VPN → Downloads → WireGuard, pick a P2P server, copy `PrivateKey` into `.env`.
|
|
|
|
### 4. Navidrome
|
|
|
|
| Variable | Value |
|
|
|---|---|
|
|
| `ND_LASTFM_ENABLED` | `true` |
|
|
| `ND_LASTFM_APIKEY` / `ND_LASTFM_SECRET` | your Last.fm API key and secret |
|
|
| `ND_SCANSCHEDULE` | `1m` |
|
|
|
|
Link both Last.fm and ListenBrainz in Navidrome's user settings so both services learn from what you play.
|
|
|
|
### 5. Start and verify
|
|
|
|
```bash
|
|
docker compose up -d
|
|
docker compose ps # gluetun and slskd should turn "healthy"
|
|
docker inspect slskd --format '{{range .State.Health.Log}}{{.Output}}{{end}}' | tail -1
|
|
# → ok: Connected, LoggedIn
|
|
docker exec explo python3 /ainulindale/discover.py status
|
|
docker exec explo python3 /ainulindale/discover.py lastfm-preview # what Last.fm would give you
|
|
docker exec explo python3 /ainulindale/discover.py run # a full tick now, instead of waiting for 00:15
|
|
```
|
|
|
|
## Migrating From the Previous Version
|
|
|
|
1. `docker compose down`
|
|
2. **Replace `slskd.yml`.** slskd lets the YAML file override environment variables, so an old file with `soulseek:`, `web:` or `directories:` blocks silently wins over `.env`. Back it up, copy the new one in.
|
|
3. Move your settings from `appdata/explo/.env` into the root `.env` (names differ slightly, see `.env.example`). The old file is no longer read.
|
|
4. Set `GLUETUN_API_KEY` and `SLSKD_API_KEY`. Gluetun's control port 8000 is no longer published on the host; nothing outside the stack needs it.
|
|
5. New downloads land in `music/slskd/`. Existing files stay where they are. Set `SLSKD_DOWNLOADS_PATH=$MUSIC_PATH` to keep the old behaviour.
|
|
6. `docker compose up -d`, then run `prune --dry-run` (above). The first real prune will remove every `Weekly-Exploration-*` week except the newest two, so **star what you want to keep first**. Set `PRUNE_LEGACY_JAMS=true` for one run to clear the old jams playlists and folders.
|
|
|
|
The first tick re-creates this week's ListenBrainz playlist once (it has no record of the old setup having imported it). Tracks already on disk are recognised, not downloaded again.
|
|
|
|
## Troubleshooting
|
|
|
|
**slskd keeps restarting.** `docker inspect slskd --format '{{json .State.Health.Log}}'` shows what the healthcheck saw. "cannot read Soulseek state" means `SLSKD_API_KEY` is not the key slskd is using (old `slskd.yml` still in place?). "logged out of Soulseek" every time, with `docker logs slskd` mentioning another client, means the account is in use elsewhere.
|
|
|
|
**slskd is stuck in "Restarting".** It validates its settings on start and exits on the first bad one; `docker logs slskd --tail 40` names it. Usual causes: `PORT_FORWARDING` set to `on` instead of `true`, a missing `music/slskd/incomplete` folder, an `SLSKD_API_KEY` under 16 characters.
|
|
|
|
**slskd never logs in, log says it is waiting for the VPN.** `GLUETUN_API_KEY` is empty or differs between the two containers, or `PORT_FORWARDING=true` on a Proton plan/server without port forwarding.
|
|
|
|
**Tick says "slskd did not log in ... Skipping this run".** Working as intended; it retries on the next tick. `REQUIRE_SLSKD=false` lets runs go ahead on YouTube alone.
|
|
|
|
**Last.fm gives 0 tracks.** Open `https://www.last.fm/player/station/user/<you>/recommended` in a browser. Empty JSON means the account has too few scrobbles, or the profile's listening data is private. Set `LASTFM_API_KEY` to enable the API-based path.
|
|
|
|
**Prune says it could not remove a folder.** Files from the old root-owned setup while Explo now runs with `PUID`/`PGID`. `chown -R` the `music/explo` folder.
|
|
|
|
**Upgrading Explo.** The Last.fm bridge writes Explo's playlist cache (`config/cache/custom-*.json`), an internal format. It is pinned to `v1.2.0`; after bumping, run `discover.py run` once and watch the log. If Explo ever ships Last.fm support itself ([issue #135](https://github.com/LumePart/Explo/issues/135)), the bridge can retire.
|
|
|
|
## Tips
|
|
|
|
- **ListenBrainz needs a few weeks** of scrobbles before Weekly Exploration appears. Last.fm recommendations work with whatever history your account already has, which makes it a good bridge in the meantime.
|
|
- **Manual downloads**: search in the slskd UI at `http://your-nas-ip:5030`; results land in `music/slskd/`.
|
|
- **Sharing**: slskd shares your library back. The more you share, the better your queue position with other peers.
|
|
- **Port forwarding** noticeably improves Soulseek results. If your Proton plan has it, set `PORT_FORWARDING=true` (one switch for gluetun and slskd; it must be `true`/`false`, slskd does not start on `on`/`off`).
|
|
|
|
---
|
|
|
|
## License
|
|
|
|
MIT. Do whatever you want with it.
|