A key part of building reliable software is handling the unexpected gracefully—be it invalid input, failing services, or infrastructure failures like power outages. Most bad situations can be simulated with unit tests, fault injection or integration tests, but how do you test that your system preserves data and recovers gracefully in the face of a power outage? Killing your service forcibly doesn’t really do it because your server’s drives will still have power to flush their buffers. The only way to really test how you do in a power outage is to somehow kill power to the server. But how to do so programmatically without combining Twilio and an intern?

One simple, cost-effective way is with Belkin WeMo insight switches ($59 from Amazon). Each switch exposes a series of UPnP services via WiFi that can turn the switch on and off, measure power usage, and query usage history. This means that with our simple wrapper library you can do things like simulate a power failure during a sensitive operation, examine the impact of periodic failures on a workload and measure power draw during demanding workloads.

Read on →

We have a Java application that runs on Heroku and interacts with the Salesforce API.  Salesforce enforces strict security on the usage of their API by limiting access to a set of IPs that are white-listed by an organization’s administrator¹.  This is a bit of a snag, since Heroku abstracts away the existence of servers, so the IP address from which requests emanate can and will change.

The Proximo add-on solves this problem by providing you with a proxy to send traffic through, with an IP address that won’t change. Their integration instructions suggest running your application in a wrapper that they provide in their stacklet.  Unfortunately, running our application within the wrapper mucks up the creation of the listening socket for the webserver:

Exception in thread "main" java.net.BindException: Cannot assign requested address

As an alternative to running your application within their wrapper, Proximo offers some examples of using it as an HTTP proxy, though they don’t provide an example for doing this in Java and for all my trying, I couldn’t get this to work globally through my Java application.

We looked through the aforementioned wrapper that Proximo distributes and found that it connects to the Proximo server as a SOCKS proxy.  That’s easy to do in Java, so I added this to the startup of my app:

Read on →

1. Note that this only applies to username/password based authentication. Using the preferred Oauth method requires no whitelisting. ↩

Java 8 is on its way, bringing a host of new features to the most widely-used language on the JVM.  Likely the most oft-noted feature will be lambdas, to which Scala and JRuby aficionados will release a sigh of “finally”.  Less flashy, but very important to certain classes of multithreaded applications, are the addition of LongAdder and DoubleAdder, atomic Number implementations that provide superior performance to AtomicInteger and AtomicLong when under contention from multiple threads.

Some simple benchmarking illustrates the performance difference between the two—for the following benchmarks we used an m3.2xlarge EC2 instance, which provides access to all 8 cores of an Intel Xeon E5-2670.

With a single thread, the new LongAdder is one third slower, but when threads are in contention to increment the field, LongAdder shows its value.  Note that the only thing each thread is doing is attempting to increment the counter—this is a synthetic benchmark of the most extreme kind. The contention here is higher than you’re likely to see in most real-world apps, but sometimes you do need this sort of shared counter, and LongAdder will be a big help.

Read on →

One of my favorite things about doing consulting work is the variety of projects that you get to see.  It’s certainly refreshing to be able to pick up a new project in a domain that isn’t connected to what you’ve been working on for the last 6 months.

However, what is revitalizing from an individual’s perspective can end up becoming destructive when you look at it from a step back at the team level.  When team-members are frequently moving to new projects and new clients, it can be nearly impossible to get a good sense of what anyone else is working on.  Developers begin to get siloed and you lose all the benefits of collaboration and over-the-shoulder-ness that working together conveys.  When you have to give the person next to you 15 minutes of context before you can ask what should be a simple question, it becomes hugely expensive and inefficient to get a second opinion on anything.

At Palomino, we had an influx of new projects last fall, which meant that we were more distributed than we had been in the past.  We saw a few indicators that we now think might be early warning signs:

Read on →

Amazon regularly adds new types of EC2 instances to keep up with demand and reflect the capabilities of the underlying hardware.  This week they announced medium & large instances in the M3 class.  Like the other M3 instances, these VMs are backed by Sandy Bridge or Ivy Bridge processors and SSD instance storage; more details on Amazon’s instance types page.

Last year we created a tool to make comparing the price of EC2 instances easier.  With the advent of these new instance types, we’ve updated WhichInstance.com to reflect the new options, improving the interface a bit in the process.  Give WhicInstance a look to more easily understand the costs of running an instance over time, and remember that Amazon reserved instances can save you a lot of money!

A while ago, I wrote about how to set up Guice, Jetty, Jersey, and Jackson and then how to calculate metrics about Jersey resource methods. We’ve subsequently open sourced some libraries to make it easy to use these (and other) techniques.

I’ll be describing each of our new libraries in turn, or you can skip to the end and look at a sample app to see them in action.

Jersey CORS Filter

CORS allows better cross-domain sharing of Web resources. jersey-cors-filter eases the task of adding CORS headers to Jersey resource methods. In the simple case, you can annotate a resource method with @Cors and that’s all.

Read on →

Given how ubiquitous URLs are, they seem to be surprisingly poorly understood by developers as evidenced by the plentiful questions on Stack Overflow about how to correctly build a URL. See this excellent post by Lunatech for more details about how URL syntax works.

Instead of going over URL syntax in detail (see RFC 3986, RFC 1738, the above-mentioned blog post, and W3 docs on HTML if you want the full story), I’m going to talk about how it’s been done wrong in commonly available libraries, and then finally how to do it right using url-builder, a Java library we’ve released for building correct URLs.

Read on →