When I first started to use Spotify, I wasn’t super impressed.

Music was really important to me at the time, because it was one of the best ways I had to reliably manage my emotional state. I was constantly selecting music and judging my reaction to it, as kind of a constant feedback loop, and, along with exercise, helped me avoid prolonged bouts of anxiety and depression. I had a well organized MP3 collection, sorted into directories by artist and then by album, so I could easily select which albums I wanted to.  It was, and is, good medicine.


When Spotify first came out, it didn’t have the ability to organize folders. It just had playlists, flat lists of files without any additional structure, and I didn’t like it. I could have just made a list of playlists, but it was clumsy and time consuming to choose what I wanted to hear. The product just seemed incomplete, so I kind of dismissed it as a novelty. It wasn’t helping me do what I wanted to do – manage the feedback loop between my ears and my heart – but I didn’t see it replacing my existing tools: GrooveShark, Foobar2000, and Windows Media Player.

There was another tool, too: a script that I’d written to specifically to help me discover new music. At the time, just a couple of years ago, finding music was possible but it wasn’t easy. I wanted a tool that would help me find more music by artists I was already listening to, and I never found one. So I decided to make one.

The Basic Problem: Finding More Of What I Want

Let’s say that I’m a fan of a particular band. Let’s use the British trip-hop outfit Morcheeba as an example. I’ve been a fan of Morcheeba for a long time, and I’d love to know when they release a new album. How can I know when they do? Surely Apple would also like me to know when Morcheeba releases a new album, so that I can go buy it on iTunes. It’s in their interest to sell me MP3s, and it’s in my interest to buy them. But no luck. I could never understand why this wasn’t a stock feature in iTunes.

iTunes did offer something use: a weekly RSS feed of new music available to purchase. In case you don’t already know this, virtually all new music is released weekly on Fridays. This is a relatively new development: for decades new music was released in the United States on Tuesdays. And so every Tuesday, iTunes would publish an RSS feed with a list of new albums and singles. The list was about 300-400 items long.

One day, an idea begins the quick journey from one side of my brain to the other. It’s a short trip, you see. I thought: could I search the iTunes RSS feed for bands I already knew about, like Morcheeba? Apple couldn’t be bothered to do this for me, but maybe with a little work I could create a tool that would do this automatically. I figured I could probably write a basic script in Perl that would download the RSS feed, parse the XML, and do some searches.

via GIPHY  Dramatic reenactment of Russ attempting to develop software

Steps involved:

  1. Get a list of all the MP3s on my drive, and scrape the metadata (specifically artist name and album name)
  2. Put that data in a table on a MySQL database
  3. Scrape the iTunes RSS feed, and get a list of of artists releasing new albums
  4. Put that data into a table on a MySQL database (specifically artist name, album name, and release date)
  5. Run a join query of the two tables and see which artist names the two tables had in common

This was just my proof of concept. And it worked. I proved the concept. I’d created a minimum viable product and it improved my life. I was quite proud of myself.

Step 2: Making A Good Thing Better

After a bit more research, I discovered a website called allmusic.com. And I discovered that they also published a weekly list of new music, and that it was as good as iTunes’ RSS feed, if not better. I immediately adapted my Perl script to scrape both sources and put both resources into my MySQL db. I thought this was awesome. My then-girlfriend continued to think I was weird.

But why’d you start this post talking about Spotify?

I wasn’t impressed with Spotify at first, but I kept my subscription because it offered unprecedented access at an affordable price. I couldn’t organize it well, but at least I could listen to loads and loads of stuff. I figured it was useful enough to keep. I guess.

And I’m glad I did, because Spotify continued to improve over time. I started to use it for more and more music. Eventually, at some point in 2012, I realized that I was playing more music in Spotify than not, and fortunately, they started allowing users to make folder trees. I immediately made a folder, called it “ALBUMS”, and made a subfolders for each artist. Under each of these, I made a playlist for albums.

    • Morcheeba
      • Who Can You Trust?
      • Big Calm
      • Charango
      • Fragments Of Freedom

And so on.

I made these by hand, bit by bit, over time. I still do. It’s something I just make time for. I don’t understand why everyone doesn’t do this. What, you’re too busy with your jobs and families and lives?

But one thing I didn’t want to do was do a bunch of data entry in the database. I certainly didn’t want to take the time to maintain my list by hand in Spotify, and maintain a matching list in MySQL, and then do the work of making sure the two lists matched. I do, in fact, have better things to do. So this presented a real problem.

Then I realized that Spotify offered an API, and I could once again write another script do to the work that computers are meant to do – making my life easy. The new process uses two scripts, and looks like this:

Script #1:

  • Query Spotify for all my playlists
  • For each playlist, query the database to see if a record for the playlist already exists.
    • If it does, skip to the next playlist
    • If it doesn’t, create a record in the database for the artist name, album name, and track name

Script #2:

  • Query the database for a list of artists
  • For each artist:
    • Query Spotify a list of albums that artist has released
    • Query the database for a list of albums I already know about
    • Compare the two album lists. If Spotify lists an album that’s not in my database, then it must be a new album!
  • Run a SQL query to get the list of artists who have released new albums this week

That’s it. It’s not too complicated, but complicated enough. I’ve run this script for years, and it reliably notifies me of new albums. It’s not perfect, and maybe it’s not even good enough to show people, but I’ve learned about a lot lot lot of music that I never would have otherwise known about.




Categories: awesome


Leave a Reply

Avatar placeholder