Rack API Middleware
A couple of Rack API middlewares for API versioning and API key authentication.
sidekiq-bossman
This might be one of my lazier posts, but I wrote a straightforward programatic utility for starting and stopping Sidekiq workers. It’s honestly nothing special, but I find it useful.
Log4R In A Rails Application
Logging is arguably one of the best things you can do for your future self. Getting the best and clearest picture of what is happening inside your application isn’t always easy with the default Rails logger. In this post I will show you how to setup log4r in a Rails application. Specifically, a Rails 3 app.
First let me explain why I am dissatisfied with the default logging that comes with Rails. One of the biggest peeves I have with the default Rails logger is that it doesn’t indicate the severity level of the log message. Nor does it include the PID. Nor does it include a consistent timestamp on every log message. After all, this sort of stuff is the tip of the iceberg. It seems logical to me that you’d always want to see this sort of detail when your bug hunting. Luckily, log4r is easily configurable and provides all these things and much much more.
In order to get started, create a file called “log4r_init.rb” under your $RAILS_ROOT/config folder. As a sample to get started try something like this inside your log4r_init.rb file.
require 'log4r' require 'log4r/yamlconfigurator' require 'log4r/outputter/fileoutputter' require 'log4r/outputter/datefileoutputter' log4r_config = YAML.load(ERB.new(File.read(File.expand_path(File.dirname(__FILE__) + "/log4r.yml"))).result) Log4r::YamlConfigurator.decode_yaml(log4r_config['log4r_config']) NameOfYourProject::Application.config.logger = Log4r::Logger[Rails.env]
You’ll notice that I am loading a YAML file called “log4r.yml”. It is in here that you setup the formatting and outputs for your logger. Here is a sample of what I typically configure:
log4r_config:
loggers:
- name : development
level : DEBUG
additive : 'false'
trace : 'true'
outputters:
- standard_log_outputter
- name : test
level : DEBUG
additive : 'false'
trace : 'true'
outputters:
- standard_log_outputter
- name : staging
level : INFO
additive : 'false'
trace : 'true'
outputters:
- standard_log_outputter
- name : production
level : INFO
additive : 'false'
trace : 'true'
outputters:
- standard_log_outputter
outputters:
- type : StderrOutputter
name : stderr_outputter
level : INFO
formatter:
date_pattern: '%Y-%m-%d %H:%M:%S'
pattern : "[%d] PID:%p %l %m"
type : PatternFormatter
- type : FileOutputter
name : standard_log_outputter
trunc : 'false'
filename : <%= "#{Rails.root}/log/#{Rails.env}.log" %>
formatter :
date_pattern: '%Y-%m-%d %H:%M:%S'
pattern : "[%d] %l %m"
type : PatternFormatter
- type : FileOutputter
name : standard_plus_trace_log_outputter
trunc : 'false'
filename : <%= "#{Rails.root}/log/#{Rails.env}.log" %>
formatter :
date_pattern: '%Y-%m-%d %H:%M:%S'
pattern : "[%d] PID:%p %t %l %m"
type : PatternFormatterThe name of the loggers match the “Rails environment” just like the default Rails logger. Of course, you can name them whatever you’d like, but I find it best not to veer too far off the path of convention.
The next thing you’ll observe is the minimum severity level to appear in your logs. In development and test I like to see everything so I set it at “DEBUG”, but then tone it down to “INFO” for staging and production.
Take note of the “outputters” part of the configuration. This is another advantage log4r has over the default Rails logger. log4r allows your application to have many outputters. Beyond standard file output, there are “outputters” for email, UDP, or you can roll your own! I’ve heard many hipster hackers are putting their logs in document stores. I actually agree that aggregating all of your logging data into a central data store is both convenient and smart. Rather than SSH’ing and grep’ing on a ton of servers you can go to one place and apply some type of SQL to intelligently extract the info. Not only that, all your logged data is now archived for free.
Lastly each “outputter” needs a formatter. I like to include a proper timestamp, the PID, the severity, and the actual message. You can also include the “trace” that gives you insight to exactly where the logger was called.
Alright, the final piece of the puzzle is adding the “log4r_init.rb” to your “application.rb”. Notice that I do not put it inside the scope of the “NameOfYourProject::Application”, but instead I add to the last line of the “application.rb”. Also, notice that I intentionally didn’t add the code “log4r_init.rb” to an “initializer”.
I’ve tried all sorts of combinations, and this is the only way I can ever get log4r and Rails to work together.
If anyone has figured out a cleaner way of doing log4r setup, please let me know.
module NameOfYourProject class Application < Rails::Application ## # all the things end end require File.expand_path(File.dirname(__FILE__) + "/log4r_init.rb")
Hope this helps and happy logging!
Clojure Applications as Daemons
Running your Clojure app as a daemon is very useful especially when you’re ready to take it live for several reasons.
- You need a way to easily and consistently “background” or disassociate from the controlling tty for every deployment.
- Systems often start daemons at boot time (think about when your server is rebooted intentionally and more often…unintentionally).
- Something a monitor script can execute if your application becomes unresponsive or unruly.
There may be others, but those are the main reasons.
Those unfamiliar with the Java ecosystem may feel especially lost in this area, but fear not because here is an awesome solution. I am going to walk you through getting your application running as a daemon using a great piece of software made by Tanuki. I do not work for Tanuki, nor do I even know anyone at Tanuki, and so trust me this isn’t some lame plug for a product. I’ve used it for many of my Java applications and now I use it with my Clojure applications. It is quite simply a great piece of software. They call it a “Java Service Wrapper”, but that is a bit of a misnomer. It is really a JVM Service Wrapper.
In order to start using it, you’ll need to bundle your application as a Jar file. As you’ll read from the documentation on the Tanuki site there are several ways to “integrate” with the service wrapper. However, I find that using their fourth approach is the best. It is not coupled in any way to your application and there is no code to be written. Generating a Jar file is of course a breeze if you’re using Leiningen or Cake. I’m most familiar with Leiningen so you’ll want to build an “uberjar” or an equivalent Jar file that contains all of its dependencies. Wait, you’re already quandering…”will it work if I have a web app?” Yes! You can make it a War file, if you are using an embedded Jetty server. In fact, you can make your web application a Jar file if you’re using the Ring Jetty adapter and skip the complexity of a War file. Of course this is beyond the scope of the post. Before you jump off into the deep end with the service wrapper be sure that your application starts properly. Remember you’ll need a “main” function registered in the manifest of your Jar file. On the command line start it up like this:
java -jar your-nifty-clojure-app.jarPerfect. Now you’ll need to download the service wrapper from here. You’ll see that they have thought of just about every OS distribution. If your development and production OS are the same, simply choose the right download. Or if you develop on a Mac and deploy to a Linux server like me, then choose the “Delta Pack”. The “Delta Pack” is cross platform support for those of us who develop and deploy to a multitude of OS’s. I forgot to mention I use the community edition and that is what I will be using for this setup.
I prefer to create a folder in my project called “daemon” and unzip the service wrapper contents there. Once unzipped you’ll see a directory structure like this. Begin trimming the unnecessary files by deleting the README’s, doc, and jdoc folders.
Now go into the src/bin directory and find a file called “sh.script.in”. Copy it into the bin folder of daemon directory, rename it, and lastly make it executable.
cd src/bin cp sh.script.in ../../bin/your-nifty-clojure-app cd ../../bin chmod +x your-nifty-clojure-app
Open the executable and edit the following three variables to match your application name.
APP_NAME="your-nifty-clojure-app" APP_LONG_NAME="Your Nifty Clojure App" ... PIDDIR="./"
Naturally you can change the PIDDIR to write where ever you want or conditionally change it based on the target environment. For starters though just put the PID file where you start the script.
Now go into the src/conf directory and find a file called “wrapper.conf.in”. Copy it into the conf folder of daemon directory, and rename it “wrapper.conf”.
cp src/conf/wrapper.conf.in conf/wrapper.conf
You’ll to make several edits to the wrapper.conf file.
Begin by telling the wrapper script that it should use the fourth integration method. Specifically it will use the WrapperJarApp as the main method and then subsequently execute your main method.
... wrapper.java.mainclass=org.tanukisoftware.wrapper.WrapperJarApp ...
I like to set a reusable variable that sets the path to your application Jar file.
... # Reusable path to the main JAR file for your application set.APP_JAR_PATH=../../your-nifty-clojure-app.jar ...
Now you’ll need to add your Jar file to the classpath as defined in the wrapper.conf.
# Java Classpath (include wrapper.jar) Add class path elements as # needed starting from 1 wrapper.java.classpath.1=../lib/wrapper.jar wrapper.java.classpath.2=%APP_JAR_PATH%
You should adjust the JVM minimum and maximum heap sizes.
# Initial Java Heap Size (in MB) wrapper.java.initmemory=256 # Set it to whatever you see fit... # Maximum Java Heap Size (in MB) wrapper.java.maxmemory=512 # Set it to whatever you see fit...
Lastly, tell the wrapper script what Jar file to execute. These work like command line arguments. If your application requires arguments, feel free to enumerate them
... wrapper.app.parameter.1=%APP_JAR_PATH% #wrapper.app.parameter.2=some argument #wrapper.app.parameter.3=another argument ...
Now you should be able to start,stop, and restart your application using the bin/your-nifty-clojure-app
./bin/your-nifty-clojure-app start|stop|restart
Tanuki offers a bunch of other options that I haven’t even scratched the surface on and they are worth a look. I hope this helps you daemonize your Clojure application!
Short and Sweet Disclojure
After experimenting and bouncing back and forth on several relatively new languages, I was undecided on what new language I wanted to commit myself to. It wasn’t until I began reading Clojure in Action that I really started to grok and see through its otherwise intimidating syntax. Admittedly, Clojure just looked and sounded too exotic at first glance. Do I really want to learn Lisp? Aren’t those parentheses crazy? Shouldn’t I be programming the next million dollar idea for the iPhone?
It turns out that the answers to those questions are: Yes. No. What money?
I wanted a challenge. I wanted something that would transcend the way I’ve been thinking about programming. I wanted something that I knew would make me a better software engineer. I wanted something more expressive and powerful.
Luckily, the past two weeks I’ve had an excellent opportunity at work to apply and push my learnings of Clojure further. I thought it would be an opportune time to share the good, the bad, and whatever else in between. A reflective and honest “disclojure”.
It seems that the best way to learn a new language is finding well documented source code and blogs from likeminded people. Clojure is a “newer” burgeoning language so it goes without saying that there are far fewer comprehensive online resources to tap into. The blogosphere is actually a bit sparse on Clojure. I’d strongly recommend having a couple Clojure e-books at your fingertips for reference. Speaking of books, don’t expect to learn Clojure by reading a stack of books for 5 months and think that it will magically flow from your digits. It’ll never ever happen. The best recipe for me has been a 1:1 ratio of time spent reading and coding. If you’re in pursuit of books that might help you on your quest to learning Clojure, remember Clojure is a Lisp. The Little Schemer and The Seasoned Schemer have been priming Lisp programmers for decades. If anything reading these two books will make you hungrier for more, wink wink nudge nudge.
Github has been an invaluable asset when looking for good source code examples. One of the questions I continue to ask myself while coding, “Is the code I’m writing idiomatic?” That is important to me because one of my main objectives is to transcend my thinking and programming style. So before you add that dependency to your Leiningen project.clj have a peek at the author’s source. In reality, you might often find yourself sifting through source because of the lack of documentation or examples. Actually I read other developers source code daily for this very reason. So let me re-phrase that, while you’re trying to figure out how the heck some library or “clojar” works, be mindful of the coding style. Undoubtedly you will pick up something useful.
I had a perplexing problem the other day and for the life of me I couldn’t figure it out. I simply lacked the context necessary that other more advanced Clojure developers have. I found my answer after posting my question on the Clojure forum hosted on Google Groups. More specifically, I was using lein run to execute a simple database preparation task, but the process was hanging on well after it completed. It wasn’t until someone informed me that I should use (shutdown-agents) after the task I was running finished because of the way the agent thread pool delays shut down. See my gist here. The turn around time on the Clojure forum varies. I think it took 2 or 3 days before my post was moderated and then received a response. Most if not all of the Clojure core developers answer questions in this forum so the responses are of high quality and are super helpful.
Sometimes you’re totally stuck. Even after using your e-book as a reference and reading the source code it just doesn’t make sense. Enter Clojure IRC! During the daytime it is bustling with chatter. Lots of the folks who answer questions on the Clojure forum hang out here too. In my experiences, the community that gathers here is friendly and willing to help.
Clojure is as expressive, powerful, and succinct as advertised. The Java interop with Clojure is about the simplest and cleanest I’ve seen thus far and that includes JRuby and Scala. I’m impressed with the clean HTTP API Ring has to offer and using the Clojure JDBC API is really nice. If only JDBC were that easy in Java there would be no need for heavyweight clunky ORMs. In terms of testing, speclj is a pretty awesome testing framework that mirrors Ruby’s RSpec API. I also have enjoyed rolling my own solutions for things that aren’t available yet. And one of the most important parts…it is fast and performs very well under load. I don’t make these claims falsely either, we ran a load test today at work and we were absolutely thrilled with the response times and throughput. I look forward to posting more about Clojure in the coming weeks. Stay tuned!
Monitoring a remote JVM process
Monitoring a remote JVM process has never been easier. As part of the default JDK installation you have a utility called jconsole at your fingertips. jconsole provides a simple way to attach to a running JVM process either locally or remotely. There are basically two ways to attach to the process, either choosing to attach to the PID or through JMX. Personally, I find that attaching to a PID is the easiest either remotely or locally.
Attaching to a process remotely might seem difficult or impossible, but if you are able to use X11 window forwarding through SSH it is a nice option. Check your /etc/ssh/ssh_config and make sure that you see ForwardX11 is set to “yes”, sometimes it is X11Forwarding, check your Linux distribution to be sure.
Host * # ForwardAgent no ForwardX11 yes
Once you have made this change, restart your ssh service /etc/init.d/ssh restart or service ssh restart. Voila. Now ssh onto your remote server with the X option
ssh -X user@remote.server.com
So once you’ve SSH’d into your remote server startup jconsole, assuming it is in your PATH you should be able to just type it straight away. You should then immediately see that jconsole has started up through X11 as if it were running on your local machine. Just select the PID and you will be able to start monitoring memory, threads, garbage collection, etc.
jconsole
Generally when I am load testing an application (I use JMeter) on the JVM I will crack open jconsole to see if there are any blaring memory leaks or memory consumption problems.
Fat chunky ugly bloated “models”
I’m not referring to the runway model variety, I’m talking about code.
Observation
I’ve had the opportunity to work with many Ruby code bases the past few years. One reoccurring theme I keep seeing is bloated models. Fat chunky ugly bloated models. Literally thousands of lines, mixing in of modules, and extensions galore. All of which are convoluting the notion of two completely separate things, data access and business logic. The phrase “fat models” has been taken too literally. I have one question. Why?
I am officially declaring it an anti-pattern for the Ruby community. A “model” is supposed to be an object representation of the underlying data your application needs. So when working with a _data_ model, you ought to stay laser focused on the DML behavior. Get your data and get out. Move on to the next set of objects that are responsible for determining, strategizing, composing, decorating, and displaying (these action verbs should sound familiar) the object representations of your data.
Remarkably most Java projects I’ve seen get this one right. Typically there is DAO, there is an entity, and beyond those two objects that are focused on DML, there are other objects that responsible for doing “things” with that data. I say “things”, but I am referring to whatever your business or domain does with the data beyond creating, reading, updating, or deleting it.
Real World Example
Here’s a solid example for the sake of discussion. Every e-commerce site has some notion of a product or service. Suppose you are modeling a “Product”. Your product has some attributes regarding description, size, weight, color, etc. You get the gist. Let’s not forget pricing, every product has some type of pricing. I see pricing go sideways almost every time. Pricing always starts out simple. You might have a “base” price and “selling” price. The “base” price is your company’s cost, and the “selling” price is your customers cost. Shortly after you deliver that functionality your product development team will undoubtedly come up with all kinds of pricing strategies.
These should all sound familiar:
- site wide sales
- single use coupons
- volume discounts
- bundling discounts
- affiliate pricing tiers
- employee discounts
Wrong
This is where it goes sideways. It’s so easy, right? Just fetch the “selling” price within an instance of the “Product” model and begin applying the business rules inside your model.
Right
I hinted a couple times in that this is a perfect application for the strategy pattern. Not familiar? Each individual pricing type should have it’s own strategy. It is safe to let a “Price” object flow though the strategies. The pricing strategies classes can then apply the discounts and the appropriate pricing separately. One of the golden rules is that composition is almost always better than inheritance and that is the construct the strategy pattern embraces.
That should look and feel better on many different levels. You should observe the freedom to add or remove pricing features without the tight coupling to any particular model. In the case that your business evolves, and it does inevitably, your pricing strategies should be flexible enough to handle any sort of pricing irrespective of whether it has anything to do with a “Product” or not. It should also be easier to test. There are no worries about the creation of database objects or fixtures, simply assert the strategies of pricing with a “Pricing” object. Also as the pricing strategies grow in size and complexity it will certainly be easier to abstract away into a separate library.
Facetious Wonderment
Here is where my “wonderment” comes into action. How is it possible that so many “Rubyists” are ignoring traditional OO programming techniques and patterns when coding at the data access layer? Is it possible that the suggestive nature of the most popular and opinionated Ruby framework has created some psychological voodoo on developers making them believe that classes only come in 4 flavors (models, views, controllers, and helpers)? Is it really that scary to add new directories or categorize your classes differently? Perhaps it is the dynamic nature of the language? Can it be the power and “expressiveness” of the language? A language wherein the lack of true interfaces, strict type checking, and the magical injection of behaviors and methods has lead its developers down a path of wicked and corrupt habits. Gosh…why should I create a new class when I can just inject a few unrelated wacky ass methods to the String class? Is it laziness? Could it be that Ruby has given way and lowered the barrier to entry for people who shouldn’t be programming at all? Or perhaps it is a lack of classic OO programming training?
Conclusion
Draw your own conclusions. The major takeaway for me is that I’m sick of it; or rather I’m sickened by it. Unwavering I will remain steadfast on an ever-quest to right the wrong where ever my fingertips land regarding this matter. You should too. There are some great code katas that focus on this very problem set. Give the Gilded Rose kata a whirl. You might be surprised by the harshness of my befuddlement and/or the generalizations I’ve made about the Ruby community. Don’t get me wrong. There are many super intelligent people out there that swear by Ruby, who are doing the right thing, and you know who you are. I am just saying based on my observations and experiences there are many more people doing the wrong thing, specifically in the data access realm, that outweigh the right-doers. But hey that’s the case with most things in life, isn’t it?
Implementing Ruby’s inject method
I spent a little time tinkering with some of the “newer” RSpec 2 features while re-implementing the inject method that belongs to the Enumerable module. It is a fun and simple exercise because it reinforces the understanding of blocks and the sometimes confusing nature of the inject method. Here it is.
require 'rubygems' require 'rspec' module Enumerable def inject(*args, &block) if(args.size == 2) inject_binary_operation(args[0], args[1], self) elsif(args.size == 1 && !block_given?) inject_binary_operation(self.first, args[0], self.drop(1)) else the_memo = args[0] || self.first self.each { |item| the_memo = yield(the_memo, item) if block_given? } the_memo end end alias :reduce :inject private def inject_binary_operation(the_memo, operator, enum) raise TypeError.new("#{operator} is not a symbol") unless operator.is_a?(Symbol) enum.each { |item| the_memo = the_memo.send(operator, item) } the_memo end end shared_examples_for "an Enumerable object" do describe "inject" do describe "given there is a specified memo value" do it('should return the newly injected "memo" value if using a block') do enumerable.inject([3,4,5]) { |memo, enum_item| memo.include?(enum_item) ? memo : memo << enum_item }.should == [3, 4, 5, 1, 2] end describe "when it is a binary operation" do it('should return a TypeError because the single argument is not a Symbol') do lambda { enumerable.inject(2) }.should raise_error(TypeError) end it('should add the values') do enumerable.inject(2, :+).should == 8 end it('should multiply the values') do enumerable.inject(2, :*).should == 12 end it('should subtract the values') do enumerable.inject(2, :-).should == -4 end end end describe "given there isn't a specified memo value" do it('should return the newly injected "memo" value if using a block') do enumerable.inject { |memo, enum_item| memo > enum_item ? memo : enum_item }.should == 3 end describe "when it is a binary operation" do it('should add the values') do enumerable.inject(:+).should == 6 end it('should multiply the values') do enumerable.inject(:*).should == 6 end it('should subtract the values') do enumerable.inject(:-).should == -4 end end end end end describe Range do it_should_behave_like "an Enumerable object" do let(:enumerable) { (1..3) } end end describe Array do it_should_behave_like "an Enumerable object" do let(:enumerable) { [1,2,3] } end end
I especially like the RSpec documentation format because it provides such great readability. When appropriate the “shared_examples_for” is also nice to keep your test suite DRY.
$ rspec injectable.rb --format documentation Range it should behave like an Enumerable object inject given there is a specified memo value should return the newly injected "memo" value if using a block when it is a binary operation should return a TypeError because the single argument is not a Symbol should add the values should multiply the values should subtract the values given there isn't a specified memo value should return the newly injected "memo" value if using a block when it is a binary operation should add the values should multiply the values should subtract the values Array it should behave like an Enumerable object inject given there is a specified memo value should return the newly injected "memo" value if using a block when it is a binary operation should return a TypeError because the single argument is not a Symbol should add the values should multiply the values should subtract the values given there isn't a specified memo value should return the newly injected "memo" value if using a block when it is a binary operation should add the values should multiply the values should subtract the values Finished in 0.007 seconds 18 examples, 0 failures
And of course the gist…
https://gist.github.com/1096650
Object Oriented JavaScript
This is a presentation I gave recently on Object Oriented JavaScript. There is also source code that goes along with it that can be found here: https://github.com/techwhizbang/js_techtalk
JavaScript MVC Pattern
There are a lot of excellent JavaScript frameworks to choose from that provide some sort of MVC pattern. Of the most popular frameworks under this broad umbrella include Backbone.js, Sencha ExtJS, and SproutCore. All of these frameworks have their strengths, but I really crave simplicity. I really don’t want to have to read about a framework for a couple hours and fiddle with it before I become productive. I really don’t want to scrap all of the “legacy” JavaScript written and attempt to shoehorn the framework into my project. I really don’t need all those super fancy UI widgets. You might be thinking the same thing, but you recognize that the way you’ve been organizing and/or writing JavaScript is flawed. The biggest mistake I see again and again is that the patterns we adhere to when writing in our favorite server side language are thrown to the wind when using JavaScript. Why? Why? Why? I could speculate and throw my theories out there, but none of that really matters. The thing that matters most is that you have recognized that something is wrong and you’d like to correct it.
I’d like to offer up a pattern I follow when writing JavaScript.
Let’s take a look at the JavaScript MVC pattern I follow.
First things first. In your JavaScript directory, create the following folders.
- controllers
- models
- views
- handlers
- pages
Handlers? Pages? What? No worries. I will discuss each of these and of course you can fit and mold this pattern to your needs. My examples assume you’re using jQuery, but you could use MooTools, prototype, or none of them.
Models:
As an example lets say you have some data that represents projects. Create a file under your models directory called project.js. I like to do “Object-Oriented” JavaScript so maybe your project.js would look like this.
Project = function(projectId) { this.projectId = projectId; } Project.prototype.find = function(projectId) { $.getJSON('projects/' + this.projectId + '.json', function(data) { //do whatever with your data } }
I’d also like to point out that I often like to write the JSON or XML I need (when I can) to the page so that I don’t make superfluous network calls. In this example I created a function called find that makes an Ajax call, but it could be just as simple to read JSON from a hidden element with a custom attribute called “data” on it like so.
/* * Reads JSON from a hidden element on the page with an attribute called 'data' * Example: <input type="hidden" /> */ Project.prototype.all = function(dataSelector) { return $.parseJSON(dataSelector.attr('data')); }
My philosophy on JavaScript models is to keep them centered on the retrieval of the data they represent. Nothing more, nothing less.
Views:
This is the area where I feel most people go wrong and is generally my biggest gripe. How many times have you cracked open a .js file and found N-instances of the selector flying all over the code. Let me clarify. For example, you have a project carousel widget on your site and the selector has an id, namely projects. So everywhere, you look
$('#projects').something
is recklessly abundant all throughout the code. Good luck re-factoring or repurposing that, even _with_ tests, especially after a few months.
Now, had you captured all the selectors for your projects view in a file managed under the views directory and properly added a layer of encapsulation. It would have taken you 2 seconds to swap whatever existing selectors there were with a new one or perhaps extended your projects view. Sometimes I like to use a view factory to encapsulate my views. Again, you don’t have to use the factory pattern you could just run the prototype pattern directly. Here is an example:
ProjectViewFactory = (function() { function BaseProjectView() { this.name ="baseProjectView"; } BaseProjectView.prototype.getProjectDataSelector = function() { return $('#projects'); } function CarouselProjectView() { this.name = "carouselProjectView"; } //I hand rolled a simple inheritance mechanism, you can find it under my Github account as inheritance.js CarouselProjectView.inheritsFrom(BaseProjectView); CarouselProjectView.prototype.getProjectDataSelector = function() { return $('#carouselProjects'); } CarouselProjectView.prototype.getNextSelector = function() { return this.getProjectDataSelector.select('.next'); } CarouselProjectView.prototype.getPreviousSelector = function() { return this.getProjectDataSelector.select('.previous'); } return { getCarouselProjectView: function() { return new CarouselProjectView(); } } }();
The fundamental concept is to align your view widgets on a page to a file managed under the views directory. Anytime, you use a selector it better be in a file like this. Never leak a selector outside the views directory. It is a very simple and straightforward rule to remember.
Controllers:
I like to keep my controllers slim and trim akin to the principle used very often in the Rails community.
Your controllers should be passed the appropriate views required to activate and listen for the appropriate binding events (ie click, mouseover, etc). The controller should only bind the events and then hand the rest of the actual duties off to what I call a handler (we’ll discuss handlers next). Here is an example:
ProjectsController = (function() { return { bindEvents : function(projectsView) { var projectModel = new Project(); var projectHandler = new ProjectHandler(projectsView, projectModel); projectsView.getNextSelector.click(function() { projectHandler.updateToNext(); }); projects.getPreviousSelector.click(function() { projectHandler.updateToPrevious(); }); } } })();
Handlers:
I’d like to introduce one more letter in the MVC acronym, “H”. A handler is in charge of the dirty work bound to the event triggered. So you can imagine that with a project carousel with a next and previous button that several different things might happen like an image transition, description, numbering, etc.
ProjectHandler = function(projectView, projectModel) { this.projectView = projectView; this.projectModel = projectModel; } ProjectHandler.prototype.updateToPrevious = function() { // fill in the details here } ProjectHandler.prototype.updateToNext = function() { // fill in the details here }
Pages:
The last and final piece of the puzzle. So a page may consist of many different things including a project carousel, recommendations, comments, and whatever else you can dream up. Assuming you followed the rules given above that means you’ve developed a projects, recommendations, and comments controller along with their corresponding views, models, and handlers.
So a page might look something like this:
SomeAwesomePage = (function() { return { bindEvents: function() { //initialize the views needed here ProjectsController.bindEvents(projectsView); RecommendationsController.bindEvents(recommendationsView); CommentsController.bindEvents(commentsView); } } })();
Following this pattern lends itself to a very flexible and modular design. It is totally agnostic to other frameworks that you might already be using. The beauty part of this approach is that you can take a pure JavaScript approach without the bulkiness or dependency on yet another pre-defined framework and spin it to the needs of your project. I always encourage testing JavaScript. Recently I’ve been pairing this pattern with Jasmine.
JavaScript Testing with Jasmine
This is a presentation I gave on the use of Jasmine, a BDD testing framework for JavaScript. Examples of Jasmine specs can be found here: https://github.com/techwhizbang/js_techtalk.
Have a look at my presentation on Object Oriented JavaScript also.
MongoDB Presentation Part 2
Intro to MongoDB Presentation
Here is the second part of the presentation I gave:
http://techwhizbang.com/2011/01/mongodb-part-2/
The JVM is the future
Amongst the new stuff coming with JDK 7, I think one of the most exciting additions to come along is JSR 292. JSR 292 is all about support for dynamic languages. Wait, what? Yes, Java is finally “yielding” to other languages (pun intended). The popularity and success of JRuby, Jython, Clojure, and all the other dynamic JVM languages have become too hard to ignore.
It is sort of crazy to think that the JVM has been overlooked and misunderstood by so many for so long. Most developers think the JVM is a platform/OS agnostic place where Java byte code gets executed. Partly true. It is platform agnostic. Funny thing is the JVM doesn’t know a damn thing about the Java syntax as we developers know it, all it knows about is the specific JVM byte code format it knows how to interpret. So this means if you can “compile” a .rb or .clj file and if it outputs the expected JVM class byte code format, you’re ready to rock. This ought to dispel some of the perceived magic involved in getting other languages running on the JVM. Just stop and consider what an amazing forethought that was, or was it? In any case, brilliant.
You might be wondering why this JSR is even needed if these dynamic languages are already running on the JVM? The truth is the implementers of JRuby, Clojure, etc have gone through great lengths and round-about ways to make them work. Some would even call these solutions creative and innovative hacks. Personally, I am overjoyed that people are thinking out of the box and have pushed the JVM to new horizons and given it new life. Here is interesting article that I found that answers these types of questions.
http://java.sun.com/developer/technicalArticles/DynTypeLang/index.html
This JSR is part of a bigger and broader vision of the JVM known as the “Da Vinci Machine Project”.
http://openjdk.java.net/projects/mlvm/
Most developers know that the Java language is dated, it’s old news, but it seems many have a hard time coming to grips with it. The JVM is the future, not the Java language. But wait, what about the argument of Java’s excellent performance, what about all the great libraries and tools that have been built over the years? We need to embrace the good things, but move on to greener and more productive pastures. The beauty part of the new JVM languages is that you can intermix and mingle Java libraries with your choice of language. Check the facts, some of the “newer” languages to run on the JVM are on par with Java or close to it.
Evolve and rise above so as not to become stoic boilerplate funk. The JVM is the future where all languages may frolic about.
RMagick 2 and ImageMagick 6.6.x on CentOS
I constantly stumble on this problem when deploying an application that requires ImageMagick 6.6.x or better on CentOS RHEL 5. This is especially true when attempting to install the RMagick 2 gem. Unfortunately, I have not found a trusted yum repo to install ImageMagick 6.6.x, so I found the necessary RPM’s on the ImageMagick FTP. Here is my magick “recipe”. I ought to make this a Chef recipe, but in the meantime…
yum install freetype-devel ghostscript-devel libwmf-devel jasper-devel lcms-devel bzip2-devel librsvg2 librsvg2-devel libtool-ltdl-devel libXt-devel libtiff-devel
yum remove ImageMagick
ftp://ftp.imagemagick.org/pub/ImageMagick/linux/CentOS/x86_64/
rpm -i ImageMagick-6.6.4-8.x86_64.rpm
rpm -i ImageMagick-c++-6.6.4-8.x86_64.rpm
rpm -i ImageMagick-devel-6.6.4-8.x86_64.rpm
rpm -i ImageMagick-doc-6.6.4-8.x86_64.rpm
rpm -i ImageMagick-perl-6.6.4-8.x86_64.rpm
rpm -i ImageMagick-c++-devel-6.6.4-8.x86_64.rpm
Multiple Sinatra Apps and Cucumber
How do I test a stack of modular (multiple) Sinatra applications with Cucumber? Honestly, I was a bit baffled about how I would get my env.rb to load my config.ru the way it should be loaded. Specifically, I wanted Cucumber to run features with my Rack::Map and other middleware just the same as when my server actually starts. There’s nothing short of several thousand blog posts about Cucumber usage and setup, but I couldn’t find anything about how to do this. Here is an example of Cucumber with Capybara env.rb loading the config.ru with Rack::Builder.
ENV['RACK_ENV'] = 'test' require File.expand_path(File.dirname(__FILE__) + "/../../init") require 'capybara' require 'capybara/cucumber' require 'spec' class SinatraWorld require "selenium-webdriver" Capybara.default_driver = :selenium # use the rackup file to load the apps w/their respective URL mappings, sweet! Capybara.app = eval "Rack::Builder.new {( " + File.read(File.dirname(__FILE__) + '/../../config.ru') + "\n )}" include Capybara include Spec::Expectations include Spec::Matchers end World do SinatraWorld.new end
Since the Sinatra applications are defined in the config.ru with Rack::Map, I’m not limited to just a singular Sinatra app (as so many examples demonstrate).
Here is an example of the config.ru:
require File.expand_path(File.dirname(__FILE__) + "/init") use Rack::Static, :urls => ['/javascripts', '/stylesheets', '/images', '/html'], :root => File.join(File.dirname(__FILE__), 'public') use Rack::Lint map "/app1" do run SinatraAppOne end map "/app2" do run SinatraAppTwo end map "/app3" do run SinatraAppThree end
Silicon Valley Ruby Meetup – Sinatra
More to come…
git@github.com:techwhizbang/sinatra_slideshow_code.git
git@github.com:techwhizbang/sinatra_slideshow_code.git
Recorded audio from the presentation
Unbeknownst to me, Sinatra was once of the presenations at #RubyKaigi 2010 in Japan. What’s really amazing to me is that on the other side of the world two speakers were basically spreading the good word in a very similar way. Of the many similarities between our presentations, one was that we were reinforcing the notion of Rack being the fundamental and foundational awesomeness of Sinatra. Here is the link to Jiang Wu’s presentation http://rubykaigi.tdiary.net/20100829.html#p08.
Check out how Sinatra’s syntactic sugar is spreading in other languages. Have a look at express, it is Sinatra like framework based on node.js.
Git cheat sheet
This is my personal Git cheat sheet. It is really a mish-mosh of other resources across the web. Yep, there are a million of them. This one isn’t special, but it is useful to me. If it works for you too, awesome; if not, move on.
create a remote branch from master
git checkout -b branch_name master git push origin branch_name
delete a remote branch
git push origin :branch_nameswitching between branches on your local
git checkout branch_namemerging and tagging a release branch to master
git checkout master git merge --no-ff release_branch_name #fix any potential merge conflicts (there should be any) and commit git tag -a 2010.xxx git push --tags
merging a feature branch back into development
git checkout development git merge --no-ff myfeature_branch git push origin development # (optionally delete the feature branch, git branch -d myfeature_branch)
bug fixing and patching between branches
Depending on where you are working, commit your fix to the appropriate branch
Then use:
git log | less # Find your revision (it should be one the latest ones) git diff your_revision_hash > /tmp/my_patch (ex. git diff cfbe9041b > /tmp/my_patch) git checkout branch_you_want_to_apply patch git apply /tmp/my_patch # continue committing and pushing as you would normally
Aside from merging, sometimes you want to just pick one commit from a different branch. To apply the changes in revision rev and commit them to the current branch use:
note: try patching first, if patching isn’t working well then try cherry picking the revision(s)
git cherry-pick revision_hashmerging release branch hot-fix and bug fixes to development branch
note: Patching and cherry picking are preferred over full merging
git checkout development git merge --no-ff release_name
stash your changes for later
git stashfind out what is in your stash list
git stash listapply your stash
git stash applyclear your stash
git stash clear
notes about your .git/config
You’ll need to make changes to this as often as you contribute to the different number of branches being used for your project. Meaning if you have to commit to the development branch, latest release branch(es), and merge into the master branch – these all need to be in your .git/config file. Here is an example. Notice the rebase = true on the development branch and how it is commented out for the release branch “Alkonost”.
[branch "master"] remote = origin merge = refs/heads/master [branch "development"] remote = origin merge = refs/heads/development rebase = true [branch "ReleaseBranch"] remote = origin merge = refs/heads/ReleaseBranch rebase = true
Stored Procedure Myth Busters
The debate between developers and DBAs regarding the use of stored procedures is an old one. Most developers I know would rather not use a stored procedure unless absolutely necessary. Don’t get me wrong, when every last bit of code has been optimized and that highly transactional or trafficked portion of your application is still in need of a boost, stored procedures can provide speed and efficiency. My real disgruntlement derives from DBAs that argue for the use of stored procedures for the wrong reasons. The same reasons I hear are actually biased lines of reasoning and aren’t always true. Let me be clear that I am attempting to flush out blanket statements promoting the use of stored procedures when they aren’t needed. My underlying belief is that stored procedures are good tools, just not all the time.
“Stored procedures shield you from change in the database.”
Scenario: Adding or deleting a column
Removing or adding a column to a table can affect the stored procedure signature. A straightforward example of this would be if there was a sproc that inserted comments. Now imagine that business wanted comments, but also the locality of where the person was commenting. You would now have to provide the latitude and longitude to the sproc. This could happen easily if you are re-factoring or adding/removing columns based on new requirements. When this type of change occurs it generally cascades through the application code, the test suite, the stored procedure, QA must test it, and lastly a deployment. It is clear that a stored procedure doesn’t always shield your application from change, in fact, it added an extra step since you not only had to alter the table, but you also had to be sure that you changed the stored procedure.
Scenario: Increasing or decreasing the size of a column
This time the stored procedure signature doesn’t change, but I’ll bet your application has some server side and/or client side validation to notify users about the appropriate limits of the said data type (most likely some varchar/text). Rinse and repeat the last set of steps before a new deployment since the stored procedure hasn’t saved you from change.
Indeed the scenarios mentioned can be failures of when there is a 1:1 mapping between your sproc and table(s), a result of rapid feature development, or just good old re-factoring. Some argue that in “real databases” this doesn’t happen. Bullshit. If this is true and this never happens in your shop, you must have the world’s most robust and flexible design known to man to account for any and all requests from the business. Congratulations, you and your team must piss excellence. I can’t remember a time where I have witnessed the holy grail of database design when this hasn’t occurred in some capacity. The fact is we must agree that this does happen some of the time.
There have been other comments supporting this statement by contending that when sprocs are used properly as a logical model abstraction over the physical one, it helps shield you from change in the database. Admittedly this approach can be a valid one, but sprocs, in my opinion and I’m sure others would agree, are more about the physical model than they are the logical one. I don’t know about you, but I rather perform logical data modeling at the class and/or object level where I can leverage both the behavioral aspects and the data aspects of my domain.
Conclusion
I think the bottom line here is that this shouldn’t be the single criteria for you to make something a stored procedure. My original conclusion on this statement was that it is a myth, but I may have been a bit overzealous to prove a clever article title. It is actually neither true or false, it actually depends on your situation.
“Code related to queries and transactions belongs in the database so make it a stored procedure.”
Let’s first consider this statement from a testability perspective. Is the code and logic inside your sprocs unit tested? Chances are they’re not. Can you confidently assert that your integration test suite covers the possibilities of someone introducing a bug or defect in the code of your sproc to the extent that your CI server will go red? Maybe. I guess what I am getting at is if more of your business logic resided in the application (where it belongs) then the probability and certainty of answering yes to both of these questions would be higher.
Now let’s consider this from a troubleshooting aspect. Let’s say someone mistakenly changes a stored procedure by accidentally messing up an if/else conditional. Your CI server may or may not go red with a broken build, but where are your developers going to look for the problem? By the time someone checks the broken test(s), looks at the application code, sifts through the commit log (Does your stored procedure code get versioned? It should.), the very last place a developer is looking to solve a problem is stored procedure code. If you’re not so lucky to have a broken build, it might pop up in the error log or through some regression testing.
How about the argument that if more than one application uses the database the logic should reside in a stored procedure? I disagree. The fewer applications using the same database the better. Especially since you could have one application provide REST or SOAP services to other applications. This circumvents duplication of code and it also abstracts the notion of the database to the other applications. In fact, some of your REST or SOAP services may use a sproc or two 
Conclusion
I’m calling this a myth. Testability, troubleshooting, and multiple applications aside, I believe that modeling at the class and/or object level should be the preferred approach.
“Stored procedures are a type of SOA”
Stored procedures could be a type of SOA, however, I’m not sure that stored procedures meet all of the SOA definition criteria. Are sprocs loosely coupled? Are sprocs discoverable? Do sprocs enable reuse of logic? Are sprocs fully autonomous? I bet the answers to these questions would vary depending on who you ask. They are composable. They do optimize. They do conform to a service contract. They do encapsulate.
Conclusion
The jury for me is still out on this one. They seem like a crude type of SOA just touching on the outer edges of the definition. Gray areas are abundant.
“Stored procedures provide a good interface between the application and database.”
A good interface to the database provides developers ease and ability through good object design and abstraction where the underlying implementation details are transparent. A good interface decouples you. A decent interface and decoupling to the database would allow you to migrate from one database to the next with less pain. In my experience the more stored procedures and custom code that ends up your database translates in more work when attempting a migration. I’m sure there are plenty of shops that wish they could move away from their current database, but are afraid to because of the number of crusty stored procedures that have built up over the years.
Conclusion
Stored procedures are simply not a good interface.
“Stored procedures allow for finer more granular user permissions.”
If your DBA says it is easier for them to manage permissions on stored procedures instead of granting the appropriate permissions to the user(s) and table(s) for your application, does it make a stored procedure the right choice for a particular operation? If we are going to get into what is easier for whom, let’s discuss what makes things easier for developers (maybe some other time).
Now a couple readers have pointed out that from a security and auditing perspective stored procedures can be a better way to go. This is something I hadn’t considered before and thanks for bringing this up.
Conclusion
Stored procedures can provide finer more granular control and are more secure. But consider if this is a compelling reason to make what you’re working on a sproc? If you’re paranoid about the user doing something with table(s) that have sensitive data a stored procedure could a good way to go.
Stored procedures can shield you from an unruly schema or horrible design, but don’t always shield you from change in the database. All of the logic that could end up in a stored procedure should be kept it in your application. Stored procedures are not an interface, nor are they an abstraction to the database. Stored procedures might be a crude SOA, but consider all the better options for an SOA implementation. If your DBA keeps barking about stored procedures in excess, consider the old adage “If the only tool you have is a hammer, you tend to see every problem as a nail”. I’m curious about other stories or hair brain reasons people may have wrongly promoted the use of stored procedures, if you’ve got a good one please comment.
Infrastructure Automation with Chef
By now you’ve at least heard about Chef or perhaps you’ve thought about evaluating it. If you haven’t heard about Chef, no worries, let me explain. Chef is an open source infrastructure automation framework written in Ruby by the guys at OpsCode for developers. In my opinion, OpsCode has hit a grand slam home run on this one and I want to send them a thank you. So the big questions. Why Chef, why another framework, why should you be interested? It boils down to this, developer tooling and infrastructure automation is generally one of the most overlooked areas in software. Let’s be honest, developers never get the appropriate amount of time while working on a project to do this usually because management and stakeholders don’t understand the value it brings to a project. To the business it is totally intangible, or is it? You might have heard complaints about how they hate that it takes so long for a new developer to get setup and hit the ground running. Maybe you’ve gotten complaints about how long, unpredictable, or “manual” deployments have gotten. Perhaps you’ve been wondering how you could rapidly and consistently clone image clusters of servers. Chef makes this dead easy.
The Quick and Dirty
Chef has an organization structure of many cookbooks. Cookbooks are just folders for organization of a particular set of related recipes. Developers write recipes in a Ruby Chef DSL. You can write recipes to do just about anything. Directory creation, library installation, gem installation, file permissions, OS package management can be controlled via the Ruby DSL Chef provides. It gets better. There is probably a recipe or cookbook already written for what you want to do. There are a growing number of Chef cookbooks and recipes shared on Github. Want a Nginx and Passenger recipe? Already written. Want memcached installed? Already written. You name it, chances are there is a recipe waiting for you. Instantly velocity! Now while this is all fine and dandy for most, recipe support across the various flavors of *nix is growing. Right now there is fantastic cookbook and recipe support if you’re running a Debian, Ubuntu, CentOS, or RedHat Linux. If you’re running another flavor of *nix, it is really easy to do whatever you want. Chef bends and flexes to your needs. Anything you can do with the Ruby language can be used to your advantage while writing a recipe. That reminds me, I’ve got a sweet CentOS recipe for Nginx installation that I have to contribute. Without rambling further you can get the most up to date information on the Chef wiki.
Common Uses
Writing a recipe for developer machine setup is a breeze and you actually can use the Chef gem that comes with the chef-solo command to kick off the recipe.
Are you still using some sucky deployment tool or script written a million years ago? Maybe you’re using Capistrano. Try out the Chef “deploy” resource. The Chef deploy resource by default encapsulates all of the best practices for Rails deployment, it doesn’t get simpler than this. Capistrano was an excellent tool for it’s time, you must try deployment on Chef. It makes rollbacks a snap also. Not using Rails or have some custom deployment requirements? The deploy resource is still awesome. You can pick and choose what you want and configure whatever customizations you need.
Take deployment and rollback even a step further with a tool like Hudson or TeamCity. Configure a job for them to run and wire up the Chef deployment script. The “easy button” deployment and rollback.
Chef combined with some server virtualization management tool you could easily image and setup a whole cluster of servers.
Still not convinced? Community adoption is growing fast and some of the most popular hosting companies in the Ruby sphere of influence are already using Chef. In fact, EngineYard uses Chef for application deployment on their cloud platform.
Bright Future for Dynamic Languages
I find myself fascinated by the increasing popularity of dynamic languages. For many of us dynamic language introduction came by way of Ruby, Python, and/or Javascript. While my fascination with these languages hasn’t ended, I am more enthralled with dynamic languages whose strength is concurrency. Indeed, these are exciting times and there is a shift happening in the software community. Honestly, who hasn’t bragged on Twitter that they just finished reading the latest book on Clojure or Erlang? Clearly there is momentum and change in the air. The future of software and computing in the next 10 or 15 years is being shaped and molded with the underlying fundamentals and principles of these languages.
I’m excited about the evolution happening. There is a higher degree of maturity, professionalism, and most importantly scientific and mathematical infused creativity. We’ve all heard the anecdotal comparison of a craft like architecture versus software. Yes software as a craft is merely a fetus when compared to architecture. For centuries architecture has used equal parts science and artistry to build some of the most breathtaking sites in the world. I believe the software community has really begun using more scientific and mathematical backed artsy creativity. We must flex both sides of our brains to imagine the crazy, the impossible, and make it reality. I digress…back to the languages.
Why am I so fascinated? The bottom line is we are evolving and learning.
- Language composition and leverage
- Developer productivity
- Functional programming and concurrency
Clojure and JRuby both run on the JVM. Undoubtedly there are statistics somewhere showing that Java is one of the most popular languages of our time and it is probably running in your software shop in some capacity. As a result the barrier to entry for integrating and using these new languages has been lowered. It is comparatively easier than the transition when Java gained über popularity in the 90′s. However, this isn’t as important and _fascinating_ as how these languages integrate directly with the Java language. The number of possibilities for composing and leveraging preexisting Java libraries with these languages seems endless. This eliminates many reasons to reinvent a library or API. When you wish you could use some Java component, framework, or API, you can – easily! I’ve used this fact to my advantage using JRuby numerous times. The other really neat possibility that I haven’t messed around with yet is using Clojure in JRuby.
The next reason for my interest is directly related to why Ruby is hot and will remain so for a long time. On a side note, I feel the need to disclose that I am not a Ruby “fan boy”, I’ve spent my time in the ranks wrestling with C and Java and still have an fond appreciation for both. Back to what is important, developer productivity. Here are some of the things these dynamic languages really excel in, while keeping in mind that correlation doesn’t imply causation:
- It will make your developers happy to write less elaborate wordy boilerplate code
- Writing less boilerplate code usually results in writing more human readable and elegant code (it has been said before that code should be human readable first and just coincidentally interpreted by a machine)
- More human readable code can improve maintainability, health, and longevity of the code base
- Maintainability can result in greater agility to output more business value
Mathematicians have been joking that they’ve known that functional programming rocks for a pretty long time now. How could they not? Functions are the premise of their craft. It is true they’ve been using Lisp, Sage, and Python for years. I remember my MCS professors telling us in the late 90′s and early 2000′s that we better learn why these languages are kick ass (all the while they were teaching us the principles of OOP with this new language called Java). In fact, I noticed the other day that the new course outlines for MCS students at UIC have plenty of functional programming aspects using Python and Lisp. Sure Python isn’t a purist’s functional language, but anything with support for closures will do. Joining the bandwagon, I am trying to get good with Clojure at the moment with the trusty Pragmatic Programming Clojure book at my side. I am not going to lie, it has been tough to wrap my head around this concept (but doesn’t that generally happen with anything outside your comfort zone?). I would be remiss if I didn’t mention Erlang or Haskell in the context of popular functional languages. I suppose I could of chosen either one of those. I felt like Clojure was a better fit for me since it runs on the JVM, it’s dynamically typed (but still compiled), and the application of my learnings would be easier to integrate at work. So I have written fewer lines of code and as a result things are just concise. I am not sitting there worrying about immutability and concurrency they are just the fruits of my labor. I hope to extract some of the cool ideas out of the O’Reilly Collective Intelligence book and write them in Clojure.
I’m definitely stoked about the future with these languages. There is certainly so many possibilities and cool things to come. So the question is what will you be doing with these languages?
JRuby, ActiveRecord, JDBC to SQL Server
I recently had to figure out a good way to export data in XML form from a SQL Server database. I searched high and low for a good ActiveRecord adapter written in pure Ruby to talk to SQL Server. Unfortunately, the setup and overhead to get the right ODBC driver, DSN configs, and all those funky libraries to work properly on any *Nix-based machine were adding up way too quickly and complicating matters. So back to the drawing board.
RESTful service from .NET – no
Way too much work for such a simple task that only runs but once a week. Beside anything Microsoft is my nemesis and I am not even going to put my toe in that water.
SQL Server bcp export – no.
The bottom line – writing code in your database is always bad news. Sure the bcp export is the quick and dirty, but it might be too quick and dirty. It can make upgrades and migrations almost impossible. Just think how difficult it would be to migrate from SQL Server to say MySQL if the use of the bcp utility compounds over time. More importantly, how do you test it?
JDBC – yes!
Really? I was kind of shocked to learn that Microsoft jumped on the Java bandwagon and wrote a JDBC adapter. Woo-hoo! Wait…I don’t want to write a bunch of junky DAO’s in Java.
Enter – my good friends JRuby and ActiveRecord.
I can write sweet sugary Ruby code while utilizing the power of any or all of the existing Java libraries – which so happens to include JDBC!
So in order to glue this together it is quite simple.
- First install JRuby, it’s simple and there are 2 million blog posts about how to do this, all you need is Java pre-installed on your machine. Download it here from here.
- jruby -S gem install activerecord
- jruby -S gem install activerecord-jdbc-adapter
- Download the SQL Server JDBC driver JAR, I recommend getting version 2 or higher
- activerecord-jdbc-adapter is a nifty JRuby specific gem that acts as an adapter for ActiveRecord to speak JDBC (yet another thanks to Nick Sieger).
Once your have these things installed you can write a simple Ruby class to establish the connection and define any number of ActiveRecord classes to map to the tables you’re after. Here is a small snippet on how easy it is to connect to SQL Server using this technique.
#assuming this file is inline with your sqljdbc.jar or just put it into your $JRUBY_HOME/lib directory require 'rubygems' require 'active_record' require 'active_record/connection_adapters/jdbc_adapter' require 'sqljdbc.jar' config = {:url => "jdbc:sqlserver://localhost;databaseName=sucky_sqlserver_db", :adapter => "jdbc", :username => "user", :password => "pass"} ActiveRecord::Base.establish_connection( config ) class Person < ActiveRecord::Base; end people = Person.find(:all) ActiveRecord::Base.clear_active_connections! file = File.new("my_xml.xml", "w") file.write(people.to_xml) file.close
Approximately 10 lines of actual code and you’re done. It can’t be done any more elegant or simple than that. Here is the really beauty part, you can actually test your “script”. Use RSpec or TestUnit, but it is tested! Let cron or whatever scheduling mechanism of your choice run your script with confidence
Getting Started With Processing.js Tutorial
This past week I’ve had the opportunity to really dig in and give processing.js a work out. For those not familiar with the technology, processing.js is a JavaScript library based on the Java based processing library available for download at http://processing.org/. Processing.js uses the HTML5 Canvas element to draw shapes and designs while using a similar API to the one defined by the Java library.
I found the barrier to entry was unnecessarily frustrating due to the lack of a _good_ beginners tutorial calling out common pitfalls. But don’t let this sway you, because once you move past this, it is really fun.
I just re-visited the processing.js site and they’ve really improved the documentation and tutorials. So I am updating this post to reflect that fact since my examples are no longer really helpful.
Follow me on Twitter
My github repository
My LinkedIn Profile