Surprise VS Navidrome Music Server

Something is wrong with the repo. The compiler fails with this error clang: error: no such file or directory: 'surprise/similarities.c' If you go to the repo, you'll see the file is indeed missing: https://github.com/NicolasHug/Surprise/tree/master/surprise

Scikit-surprise is a useful package and has pretty good documentation to help make the leap from conceptual understanding to code. If you want to understand the various implementations, the package is open source and available on GitHub. I can’t speak for optimal computational efficiency but I think that it’s premature to worry about that while you’re still making the transition from concept to functionality.

I spent the latter half of 2019 trying to build this as a startup. Ultimately I pivoted (now I do newsletter recommendations instead), but if I hadn't made some mistakes I think it could've gotten more traction. Mostly I should've simplified the idea to make it easier to build. If anyone's interested in working on this, here's what I would do:

(But first some background: The way I saw it, you can split music recommendation into two tasks: (1) picking a song you already know that should be played right now, and (2) picking a new song you've never heard of before. (Music recommendation is unique in this way since in most other domains there isn't much value in re-recommending items). I think #1 is more important, and if you nail that, you can do a so-so job of #2 and still have a good system.)

Make a website that imports your Last.fm history. Organize the history into sessions (say, groups of listen events with a >= 30 minute gap in between). Feed those sessions into a collaborative filtering library like Surprise[1], as a CSV of `, , 1` (1 being a rating--in this case we only have positive ratings). Then make some UI that lets people create and export playlists. e.g. I pick a couple seed songs from my listening history, then the app uses Surprise to suggest more songs. Present a list of 10 songs at a time. Click a song to add it, and have a "skip all" button that gets a new list of songs. Save these interactions as ratings--e.g. if I skip a song, that's a -1 rating for this playlist. For some percentage of the suggestions (20% by default? Make it configurable), use Last.fm's or Spotify's API to pick a new song not in your history, based on the songs in the current playlist. Also sometimes include songs that were added to the playlist previously--if you skip them, they get removed from the playlist. Then you can spend a couple minutes every week refreshing your playlists. Export the playlists to Spotify/Apple Music/whatever.

As you get more users, you can do "regular" collaborative filtering (i.e. with different users) to recommend new songs instead of relying on external APIs. There are probably lots of other things you could do too--e.g. scrape wikipedia to figure out what artists have done collaborations or something. In general I think the right approach is to build a model for artist similarity rather than individual song similarity. At recommendation time, you pick an artist and then suggest their top songs (and sometimes pick an artist already in the user's history, and suggest songs they haven't heard yet--that's even easier).

This is the simplest thing I can think of that would solve my "I love music but I listen to the same old songs everyday because I'm busy and don't want to futz around with curating my music library" problem. You wouldn't have to waste time building a crappy custom music app, and users won't have to use said crappy custom music app (speaking from personal experience...). You wouldn't have to deal with music rights or integrating with Spotify/Apple Music since you're not actually playing any music.

If you want to go further with it, you could get traction first and then launch your own streaming service or something. (Reminds me a bit of Readwise starting with just highlights and then launching their own reader recently). I think it'd be neat to make an indie streaming service--kind of like Bandcamp but with an algorithm to help you find the good stuff. Let users upload and listen to their own MP3s so it can still work with popular music. Of course it'd be nicer for users in the short term if you just made deals with the big record labels, however this would help you not end up in Spotify's position of pivoting to podcasts so you can get out of paying record labels. And then maybe in a few decades all the good music won't be on the big labels anyway :).

Anyway if anyone is remotely interested in building something like this, I'll be your first user. I really need it. Otherwise I'll probably build it myself at some point in the next year or two as a side project.

[1] http://surpriselib.com/

I'm planning to build a business on this, so probably won't open-source it--but I'm always looking for interesting things to write about! I write a weekly newsletter called Future of Discovery[1]; I might write up some more implementation details there in a week or two. In the mean time, most of the heavy lifting is done by the Surprise python lib[2]. It's pretty easy to play around with, just give it a csv of , , and then you can start making rating predictions. Also fastText[3] is easy to mess around with too. Most of the code I've written just layers things on top of that, e.g. to handle exploration-vs-exploitation as discussed in another thread here.

Recently I've been factoring out the ML code into a separate recommendation service so it can different kinds of apps (I just barely made this essay recommender system[4] start using it for example).

I'm happy to chat about recommender systems also if you like, email's in my profile.

[1] https://findka.com

[2] http://surpriselib.com/

[3] https://fasttext.cc/

[4] https://essays.findka.com

My primary way to play music is from my self-hosted navidrome[1] server with my collection of albums I've mostly purchased from bandcamp. I can stream it to many different devices at home or on the go.

But sitting next to my bed is a Walkman (actually a $10 Jensen version) with a few of my favorite cassettes in the nightstand drawer. Granted, I listen to raw black metal, so the format fits the music well, but I really enjoy just popping in a cassette and hitting play. When I "metaltate", I listen to full albums and do not want to ever be interrupted or have skipping audio due to bluetooth or anything else. It is a really simple and great experience.

Would I ever take my walkman with me or want to carry around a bunch of tapes on a trip? Of course not! But it does have a time and place that is valuable.

When friends come over, we use either vinyl or my custom built RFID cards. There is more of a ceremony to digging through a physical stack of albums and being forced to listen to the album front to back.

[1] https://www.navidrome.org/

Basically, you run a server on your Mac that scans your music collection and "broadcasts" it to the network (LAN or WAN) via either the venerable UPnP/DLNA[1] family of technologies or the newish Subsonic API[2]. Of course, there are others, like DAAP or AURA, etc..

From there, you need to point a compatible player to said server to play your music on any supported device.

If you want to listen to your music on the go, pairing a Subsonic-compatible server on your Mac and a Subsonic-compatible app on your iPhone is probably the way to go. On the server side, I have only used the original Subsonic[3], which was good, but Navidrome[4] seems to be OK. But be aware that the whole "scene" is super messy and fragmented, with the usual abandoned forks of open source alternatives of everything.

Note that this means opening your local network, which comes with its own complexity.

This r/selfhosted thread[5] should give you an idea.

My use case is slightly different. I only care about streaming to my Denon CEOL mini system, which only supports UPnP/DLNA, so my current setup is:

- All my music is stored on a 2011 Mac Mini,

- I use Kazoo Server[6] (not perfect but reliable) to stream it to my audio system,

- which I control via the HEOS app provided by Denon.

Whatever stack you choose, make sure your files are tagged correctly and consistently.

[1] https://en.wikipedia.org/wiki/DLNA

[2] http://www.subsonic.org/pages/api.jsp

[3] http://www.subsonic.org/pages/index.jsp

[4] https://www.navidrome.org/

[5] https://old.reddit.com/r/selfhosted/comments/pz9dpb/lets_mak...

[6] https://docs.linn.co.uk/wiki/index.php/Kazoo_Server_setup_Ma...

[Scanner] Fix Windows scanner (#2633). Thanks @caiocotts

EDIT: This version has a bug when running on Windows that breaks your database! I deleted the Windows binary from the download page and will publish a fix very soon. For details see: https://github.com/navidrome/navidrome/issues/2630

Jellyfin is great for movies & shows. As an anecdote, it's not so good for music if you're a collector. I personally use Navidrome for that[0].

Anyway, Sonarr[1] makes use of .NET, too. Very reliable software, in my experience.

[0]: https://github.com/navidrome/navidrome

I am running ND via the docker container (deluan/navidrome:latest which is 0.49.3 (8b93962f) at the time of this writing) and interact with ND using the web interface.