Building a music finder with Golang, Apache CouchDB and RabbitMQ #1
I started a YouTube journey recently and I immediately encountered a challenge. How to obtain music that is good and legal to use commercially in YouTube videos ?
Sound Cloud has a search filter that allows one to search on commercially usable music. I have not seen this ability on Spotify or YouTube music. One problem I found with that feature was when I found a song I liked, and looked at the related songs, they’re all illegal to use commercially. So the license based filters don’t apply to related songs. This makes the process very manual and crawling these songs manually is time consuming.
I asked myself: “What if I could build a system that I could train to find music automatically?”
At the same time, I could use this project to learn and vlog about new tech that I come across. I pulled out a whiteboard and started drawing up a design.
In order to train a system, It would need to be able to perform the basic instructions. Like downloading a song. I need the ability to give the system a song request and it would need to consume that request, download the song and store the audio binary and metadata in a storage layer for later use.
I built a song request API in Go. It’s a lightweight service that takes a song URL to any music platform and pushes it to a message queue. For my message queue, I used RabbitMQ. The documentation on RabbitMQ is super dope and it has great support for Golang. I also love its UI as it has many statistics that are all self explanatory.
The music consumer is the application that subscribes to the queue and performs the task of downloading the song and places it in storage.
For storage I chose Apache CouchDB as recommended by a few articles as a great binary store. I also store the metadata document as an attachment in the database. This will help me to build a data warehouse of music and write a few algorithms to find related songs that fall under Creative Commons and free to use licenses.
Check out Part 1 of my Music finder project series on YouTube:
Hope you enjoy!
Peace!
