Organizing Cron Jobs with Node.js + Express

While working on a project recently, I needed to write a handful of scheduled tasks to collect or manipulate data. Each task would share some common elements, like connections to MongoDB or RabbitMQ servers, among other things. Ultimately I wanted a single connection to each external service with the tasks using them as needed. Some jobs would be scheduled to run every five minutes so I didn't want them having to connect and disconnect that often. Let's see...

How about using a small web server? All the server has to do is expose API routes on localhost. The logic for various tasks can be organized in the route handler functions. Each Cron job then becomes a simple shell script that curls a local URL. Sounds pretty good to me.

Since I was already using a lot of JavaScript on related projects, I chose to build things with Node.js and Express. This could just as easily have been done with another language or framework. If I were to do this in Python I'd definitely use Flask or Bottle not something big like Django. Micro frameworks are much more flexible.

A full example of how I put this together is on Github: https://github.com/projectweekend/Holly-Jobs. Everything for the web server is found in the /app directory and any script called by a Cron job can be found in /cron.

Riot Fest: Day 2

Conditions were much better on the second day of Riot Fest. The sky was clear and there was no rain in sight. The air was a little chilly, but it was actually kind of nice. Highlights for the day included two groups I had seen previously: Wu-Tang Clan and The Flaming Lips.

It had been almost nine years since I last saw The Flaming Lips. During that time technology finally caught up with their stage show. Hundreds of strands of RGB LEDs dangled above the stage with a giant LED matrix as backdrop. The animations and colors were synchronized to the music in psychedelic perfection.

It was an amazing spectacle, even given a major hiccup at the beginning of the show when the lights kicked into high gear. Evidently it was too much for the power setup. Everything went dark and it took event staff about 10 minutes to get things back in working order. I've never seen anything like that before. When they returned the set featured some more recent material, but also included a handful of songs from one of my favorite albums: Yoshimi Battles The Pink Robots.



Riot Fest: Day 1

Evidently summer ended two weeks early in Chicago. It was cold, rainy, and very muddy for the first night of Riot Fest in Humboldt Park. Facing the elements, I thought: "I'm getting too old for this". The chaos of a large music festival was far more magical a decade ago. With that said there was no turning back. I had a mission to accomplish.

The goal: 33 year old me needed to see one of 13 year old me's favorite albums performed live. I'm glad the old man side of me toughed it out because it sounded great. I'm also happy to say that there's still a little bit of that 13 year old in me that hasn't been beaten into submission by adulthood. Rock on.