Highlighting Vagrantfiles in Sublime Text

So, Sublime Text is awesome, just ignore the badass Vim guy in the back corner. But one of the biggest drags I’ve run into is its inability to bind file types to files without extensions. You’ve got your Vagrantfiles, your Thorfiles, etcetera etcetera etcetera, and they’re all just being treated as un-highlighted plain text.

Given my one-man crusade that Everybody Should Be Using DSLs For Configuration, this has made me sad. Ed is sad. Sad.

But! Now I have a plugin that solves this for me, and yes it’s been out there forever but now I know about it and now you will know about it and we can all be happy. It’s called ApplySyntax, and it comes out of the box very happy to splash Ruby syntax all over your Foofiles.

asger – an event handler for AWS autoscaling groups

So in our adventures at Leaf, building out our new environment, Will and I ran into a persistent problem – EC2 doesn’t guarantee that an instance will ever be shut down gracefully. No guarantees for upstart scripts, /etc/init.d, whatever. This is particularly problematic for dealing with Chef, where an instance needs to be deleted from the Chef datastore when it goes away. If you don’t, knife search will happily return tens or hundreds of nodes that no longer exist. Which is great.

No, wait. It’s crap.

Enter asger. Since autoscaling groups can publish notifications to SNS, asger will watch an SQS queue subscribed to SNS and execute arbitrary tasks based on the up/down pattern. Created nodes invoke up functions, terminated nodes invoke down functions, life is good. Currently there’s a task for deregistering Chef nodes, but asger is pretty flexible–Route 53 subscriptions, tie-ins to systems like Consul, that sort of thing. You can get asger via RubyGems with gem install asger or on GitHub; use it in good health.

Terraframe: Making HashiCorp’s Terraform Awesome

So, Terraform is a cool infrastructure-as-code system with a boatload of functionality. It makes building out entire clouds on AWS super easy and super fun. I am a fan. I started my new job as Lead Platform Engineer at Leaf this week, and I was really happy to have the chance to use Terraform in a greenfield project where I could play with it.

But I wasn’t happy with it, and so I’ve done a thing.
read more

Docker Web Proxy with SSL support

So Mesos is pretty cool, but it isn’t a universal solution. Not much reason to run it as a singleton cluster, for example, so this blog is running on a Linode running all its apps in Docker containers. This is new to me, but the upside is that it’s new to everybody else too—Docker’s still pretty new and there’s a lot of green field to play with in the containerization space. While there’s that sense of openness there’s also very little to suggest what the Right Course Of Action really is. Sometimes you gotta get inventive

Speaking of inventivity, I found myself needing a way to route HTTP requests to Docker containers based on vhosts. This isn’t a new problem to have, and Jason Wilder has a neat proxy container written in Go. I needed something that could handle HTTPS as well, though. I tried extending that container to do what I wanted, but I learned two things along the way.

  • Go’s templating options are Not Ready For Prime Time. Compared to ERB, it’s straight-up miserable.
  • The list of things I would rather do than use Go is not small and includes scenarios that involve nailing pieces of myself to other pieces of myself.

Enter docker-web-proxy, which I put together while I was on vacation in Maine this week. It’s a simple Ruby-based app that polls Docker for connections and splats out the appropriate nginx config before SIGHUP’ing the service; arguments are passed via the env vars given to the containers that need to be made visible. It supports HTTP, HTTPS, or HTTPS with HTTP forwarding, and optionally supports forwarding www.example.com to example.com if that there’s your thing.

After doing some nontrivial work with Docker, I’ve come around to the opinion that Docker would really benefit from a way to tag containers dynamically at runtime—think AWS—without bringing in something like etcd or Consul (my own personal favorite in this space, HashiCorp is cool people). I get the argument for separation of concerns, but, ehh.

So, part one of this little getting-acquainted with Mesos left me with one (1) Mesos server, running apps in an app-ish sort of way via the Marathon framework. Which is cool. But there’s a lot of places to go from here.

I could install additional frameworks onto the box, if I wanted–I could bolt in Chronos to give me a cron replacement or I could wire up Spark to map and reduce things on HDFS. But those aren’t really interesting to me, not least because I don’t have any jobs that have a burning need to run at 3AM, nor do I have a few terabytes of interesting data lying around to gnaw on. So instead I’m going to stick with Mesos and see what I can do about expanding my little cluster past a singleton server.
read more

A Virtual Mesos Cluster with Vagrant and Chef

Part 2 is here.

So, blog reboot, like, thirty-four. I’m bad at blogs. But this time, I come bearing neat stuff to talk about, so maybe this’ll stick.

Anyway, since starting at Localytics, I’ve found myself thrown into a bunch of new-ish stuff. My prior ops experience was much more “developer moonlighting as a sysadmin”, rather than buzzword-compliant DevOps Ninja Powers. At Localytics I’ve been leveling those up pretty quick, though, and there’s some fun stuff I’d like to talk about a little. But we need to figure out what we want to make public first, and it’ll probably end up on the company blog before it ends up here, so I’m going to natter on a bit about something I’m doing in my spare time: setting up a Mesos cluster and populating it with apps for a side project or two.
read more