Infrastructure Automation with Chef

By now you’ve at least heard about Chef or perhaps you’ve thought about evaluating it. If you haven’t heard about Chef, no worries, let me explain. Chef is an open source infrastructure automation framework written in Ruby by the guys at OpsCode for developers. In my opinion, OpsCode has hit a grand slam home run on this one and I want to send them a thank you. So the big questions. Why Chef, why another framework, why should you be interested? It boils down to this, developer tooling and infrastructure automation is generally one of the most overlooked areas in software. Let’s be honest, developers never get the appropriate amount of time while working on a project to do this usually because management and stakeholders don’t understand the value it brings to a project. To the business it is totally intangible, or is it? You might have heard complaints about how they hate that it takes so long for a new developer to get setup and hit the ground running. Maybe you’ve gotten complaints about how long, unpredictable, or “manual” deployments have gotten. Perhaps you’ve been wondering how you could rapidly and consistently clone image clusters of servers. Chef makes this dead easy.

The Quick and Dirty

Chef has an organization structure of many cookbooks. Cookbooks are just folders for organization of a particular set of related recipes. Developers write recipes in a Ruby Chef DSL. You can write recipes to do just about anything. Directory creation, library installation, gem installation, file permissions, OS package management can be controlled via the Ruby DSL Chef provides. It gets better. There is probably a recipe or cookbook already written for what you want to do. There are a growing number of Chef cookbooks and recipes shared on Github. Want a Nginx and Passenger recipe? Already written. Want memcached installed? Already written.  You name it, chances are there is a recipe waiting for you. Instantly velocity! Now while this is all fine and dandy for most, recipe support across the various flavors of *nix is growing. Right now there is fantastic cookbook and recipe support if you’re running a Debian, Ubuntu, CentOS, or RedHat Linux. If you’re running another flavor of *nix, it is really easy to do whatever you want. Chef bends and flexes to your needs. Anything you can do with the Ruby language can be used to your advantage while writing a recipe. That reminds me, I’ve got a sweet CentOS recipe for Nginx installation that I have to contribute. Without rambling further you can get the most up to date information on the Chef wiki.

Common Uses

Writing a recipe for developer machine setup is a breeze and you actually can use the Chef gem that comes with the chef-solo command to kick off the recipe.

Are you still using some sucky deployment tool or script written a million years ago? Maybe you’re using Capistrano. Try out the Chef “deploy” resource. The Chef deploy resource by default encapsulates all of the best practices for Rails deployment, it doesn’t get simpler than this. Capistrano was an excellent tool for it’s time, you must try deployment on Chef. It makes rollbacks a snap also. Not using Rails or have some custom deployment requirements? The deploy resource is still awesome. You can pick and choose what you want and configure whatever customizations you need.

Take deployment and rollback even a step further with a tool like Hudson or TeamCity. Configure a job for them to run and wire up the Chef deployment script. The “easy button” deployment and rollback.

Chef combined with some server virtualization management tool you could easily image and setup a whole cluster of servers.

Still not convinced? Community adoption is growing fast and some of the most popular hosting companies in the Ruby sphere of influence are already using Chef. In fact, EngineYard uses Chef for application deployment on their cloud platform.

Apr 14, 2010 | Software | 3 Comments »

Bright Future for Dynamic Languages

I find myself fascinated by the increasing popularity of dynamic languages. For many of us dynamic language introduction came by way of Ruby, Python, and/or Javascript. While my fascination with these languages hasn’t ended, I am more enthralled with dynamic languages whose strength is concurrency. Indeed, these are exciting times and there is a shift happening in the software community. Honestly, who hasn’t bragged on Twitter that they just finished reading the latest book on Clojure or Erlang? Clearly there is momentum and change in the air. The future of software and computing in the next 10 or 15 years is being shaped and molded with the underlying fundamentals and principles of these languages.

I’m excited about the evolution happening. There is a higher degree of maturity, professionalism, and most importantly scientific and mathematical infused creativity. We’ve all heard the anecdotal comparison of a craft like architecture versus software. Yes software as a craft is merely a fetus when compared to architecture. For centuries architecture has used equal parts science and artistry to build some of the most breathtaking sites in the world. I believe the software community has really begun using more scientific and mathematical backed artsy creativity. We must flex both sides of our brains to imagine the crazy, the impossible, and make it reality. I digress…back to the languages.

Why am I so fascinated? The bottom line is we are evolving and learning.

• Language composition and leverage
• Developer productivity
• Functional programming and concurrency

Clojure and JRuby both run on the JVM. Undoubtedly there are statistics somewhere showing that Java is one of the most popular languages of our time and it is probably running in your software shop in some capacity. As a result the barrier to entry for integrating and using these new languages has been lowered. It is comparatively easier than the transition when Java gained über popularity in the 90′s. However, this isn’t as important and _fascinating_ as how these languages integrate directly with the Java language. The number of possibilities for composing and leveraging preexisting Java libraries with these languages seems endless. This eliminates many reasons to reinvent a library or API. When you wish you could use some Java component, framework, or API, you can – easily! I’ve used this fact to my advantage using JRuby numerous times. The other really neat possibility that I haven’t messed around with yet is using Clojure in JRuby.

The next reason for my interest is directly related to why Ruby is hot and will remain so for a long time. On a side note, I feel the need to disclose that I am not a Ruby “fan boy”, I’ve spent my time in the ranks wrestling with C and Java and still have an fond appreciation for both. Back to what is important, developer productivity. Here are some of the things these dynamic languages really excel in, while keeping in mind that correlation doesn’t imply causation:

• It will make your developers happy to write less elaborate wordy boilerplate code
• Writing less boilerplate code usually results in writing more human readable and elegant code (it has been said before that code should be human readable first and just coincidentally interpreted by a machine)
• More human readable code can improve maintainability, health, and longevity of the code base
• Maintainability can result in greater agility to output more business value

Mathematicians have been joking that they’ve known that functional programming rocks for a pretty long time now. How could they not? Functions are the premise of their craft. It is true they’ve been using Lisp, Sage, and Python for years. I remember my MCS professors telling us in the late 90′s and early 2000′s that we better learn why these languages are kick ass (all the while they were teaching us the principles of OOP with this new language called Java). In fact, I noticed the other day that the new course outlines for MCS students at UIC have plenty of functional programming aspects using Python and Lisp. Sure Python isn’t a purist’s functional language, but anything with support for closures will do. Joining the bandwagon, I am trying to get good with Clojure at the moment with the trusty Pragmatic Programming Clojure book at my side. I am not going to lie, it has been tough to wrap my head around this concept (but doesn’t that generally happen with anything outside your comfort zone?). I would be remiss if I didn’t mention Erlang or Haskell in the context of popular functional languages. I suppose I could of chosen either one of those. I felt like Clojure was a better fit for me since it runs on the JVM, it’s dynamic, and the application of my learnings would be easier to integrate at work. So I have written fewer lines of code and as a result things are just concise. I am not sitting there worrying about immutability and concurrency they are just the fruits of my labor. I hope to extract some of the cool ideas out of the O’Reilly Collective Intelligence book and write them in Clojure.

I’m definitely stoked about the future with these languages. There is certainly so many possibilities and cool things to come. So the question is what will you be doing with these languages?

Apr 10, 2010 | Software | 2 Comments »