I’ve been doing a lot of thinking and talking with a lot of folks lately about what I’ve been calling “devops” and now mostly call platform engineering: the care and feeding of the underlying systems that power scalability, fault tolerance, and developer productivity. There’s a great deal of value to be mined from treating your infrastructure as a product, with strongly guaranteed interfaces and APIs for other product teams to use. Starting early on the kind of infrastructural work that can empower early developers and enable more effective early development, while building towards the long-term scalability needs of startups heading towards the “elbow” of exponential growth, where you Get Noticed and your lives get interesting.
Much of this doc is cadged from everybody else’s notebooks about platform engineering, pick-and-choosing best practices from others and from my own experience, to put together what I’m calling a Healthy Platform Checklist. Unlike the Joel Test, however, I don’t intend this to apply to everybody everywhere; this is a set of characteristics that expect the need for scalability and value developer agility and productivity, and not all businesses need that. I emphatically don’t view this as being a purely prescriptive “you suck if you aren’t doing this right now”, but instead something to pin on the wall, a set of guiding principles to keep in mind and ideal states to work towards. read more
So Mesos is pretty cool, but it isn’t a universal solution. Not much reason to run it as a singleton cluster, for example, so this blog is running on a Linode running all its apps in Docker containers. This is new to me, but the upside is that it’s new to everybody else too—Docker’s still pretty new and there’s a lot of green field to play with in the containerization space. While there’s that sense of openness there’s also very little to suggest what the Right Course Of Action really is. Sometimes you gotta get inventive
Speaking of inventivity, I found myself needing a way to route HTTP requests to Docker containers based on vhosts. This isn’t a new problem to have, and Jason Wilder has a neat proxy container written in Go. I needed something that could handle HTTPS as well, though. I tried extending that container to do what I wanted, but I learned two things along the way.
Go’s templating options are Not Ready For Prime Time. Compared to ERB, it’s straight-up miserable.
The list of things I would rather do than use Go is not small and includes scenarios that involve nailing pieces of myself to other pieces of myself.
Enter docker-web-proxy, which I put together while I was on vacation in Maine this week. It’s a simple Ruby-based app that polls Docker for connections and splats out the appropriate nginx config before SIGHUP’ing the service; arguments are passed via the env vars given to the containers that need to be made visible. It supports HTTP, HTTPS, or HTTPS with HTTP forwarding, and optionally supports forwarding www.example.com to example.com if that there’s your thing.
After doing some nontrivial work with Docker, I’ve come around to the opinion that Docker would really benefit from a way to tag containers dynamically at runtime—think AWS—without bringing in something like etcd or Consul (my own personal favorite in this space, HashiCorp is cool people). I get the argument for separation of concerns, but, ehh.