Post

Side projects, for me, aren't usually only about making the thing you want to build, but is also usually about learning how to do something new or better.

I'm building an application that will scrape gold product prices (current interest of mine) from different online gold brokers.

In scraping these sites, the end game is to be able to have a digestible way to compare products easily to find the lowest premium or deals.

But mostly I wanted to build something that was waaaay over engineered to get better with some tech I've used but am not good with yet. Namely, kafka and OTel.

Additionally, I've gotten a jeagar collector running and an OTel package for tracing to provide observability into my code which is cool. And yesterday and today, I worked on getting a sarama consumer/producer stood up so I can start sending messages.

I got a lot of, if not all, of my architecture set up to be run in a makefile but it's a pain running everything locally when I need to switch between my laptop and desktop... I think I'll take some time to set up a server sitting on my local network to handle my db, kafka broker, and OTel collector