What’s Being Poured at Palomino Labs

Here at Palomino Labs, there are many things that are important to our culture. If you take a tour of our office, we hope many of these passions will be apparent. While you won’t see a taqueria built into one of our conference rooms, or a smoker out on our office patio (also known as the roof of the yoga studio next door), some things might catch your eye.

In the hallway you’ll find our coffee station, with a Breville espresso machine and a Chemex, as well as a dozen or so varieties of beans from a number of our favorite roasters. On display in the main work area are our bowling trophies, including Ron’s 1st place win. And then… there’s beer.

Read on

Parse Gotchas

Backend as a Service (BaaS) companies seek to provide you, the developer, with a reliable backend solution that is quick and easy to use. With one of these services you, in theory, can save yourself the time and costs of configuring AWS servers and load balancers, or building and deploying a Ruby codebase to Heroku. However, from our experience working with Parse, one of the leading BaaS providers, we encountered a number of major gotchas that ended up making the development process take a lot longer than it would have had we built out a rails backend ourselves.
Read on

Controlling Belkin WeMo Switches from Java

A key part of building reliable software is handling the unexpected gracefully—be it invalid input, failing services, or infrastructure failures like power outages. Most bad situations can be simulated with unit tests, fault injection or integration tests, but how do you test that your system preserves data and recovers gracefully in the face of a power outage? Killing your service forcibly doesn’t really do it because your server’s drives will still have power to flush their buffers. The only way to really test how you do in a power outage is to somehow kill power to the server. But how to do so programmatically without combining Twilio and an intern?

belkin_insight_switch_resizedOne simple, cost-effective way is with Belkin WeMo insight switches ($59 from Amazon). Each switch exposes a series of UPnP services via WiFi that can turn the switch on and off, measure power usage, and query usage history. This means that with our simple wrapper library you can do things like simulate a power failure during a sensitive operation, examine the impact of periodic failures on a workload and measure power draw during demanding workloads.

Read on

Using Proximo as a SOCKS proxy in Java

We have a Java application that runs on Heroku and interacts with the Salesforce API.  Salesforce enforces strict security on the usage of their API by limiting access to a set of IPs that are white-listed by an organization’s administrator.  This is a bit of a snag, since Heroku abstracts away the existence of servers, so the IP address from which requests emanate can and will change.

The Proximo add-on solves this problem by providing you with a proxy to send traffic through, with an IP address that won’t change. Their integration instructions suggest running your application in a wrapper that they provide in their stacklet.  Unfortunately, running our application within the wrapper mucks up the creation of the listening socket for the webserver:

Exception in thread "main" java.net.BindException: Cannot assign requested address

As an alternative to running your application within their wrapper, Proximo offers some examples of using it as an HTTP proxy, though they don’t provide an example for doing this in Java and for all my trying, I couldn’t get this to work globally through my Java application.

We looked through the aforementioned wrapper that Proximo distributes and found that it connects to the Proximo server as a SOCKS proxy.  That’s easy to do in Java, so I added this to the startup of my app:

URL proximo = new URL(proximoUrl)
String userInfo = proximo.getUserInfo()
String user = userInfo.substring(0, userInfo.indexOf(':'))
String password = userInfo.substring(userInfo.indexOf(':') + 1)

System.setProperty('socksProxyHost', proximo.getHost())
System.setProperty('socksProxyPort', PROXIMO_PORT)
Authenticator.setDefault(new ProxyAuthenticator(user, password))

That ProxyAuth is defined as an inner class elsewhere:

private class ProxyAuthenticator extends Authenticator {
    private final PasswordAuthentication passwordAuthentication;

    private ProxyAuth(String user, String password) {
        passwordAuthentication = new PasswordAuthentication(user, password.toCharArray())
    }

    @Override
    protected PasswordAuthentication getPasswordAuthentication() {
        return passwordAuthentication;
    }
}

With this configured, all requests initiated by the Java application are sent via SOCKS through the Proximo proxy.  We just white list the proxy’s IP address in our Salesforce settings and everything works.

We have encapsulated this Proximo SOCKS setup code into a module, proximo-socks, that you can use in your application.  Simply add it as a dependency in your pom.xml or build.gradle and then initialize it in your app’s startup:

import com.palominolabs.heroku.Proximo
…
Proximo.setup()

See also

Java 8 Performance Improvements: LongAdder vs AtomicLong

Java 8 is on its way, bringing a host of new features to the most widely-used language on the JVM.  Likely the most oft-noted feature will be lambdas, to which Scala and JRuby aficionados will release a sigh of “finally”.  Less flashy, but very important to certain classes of multithreaded applications, are the addition of LongAdder and DoubleAdder, atomic Number implementations that provide superior performance to AtomicInteger and AtomicLong when under contention from multiple threads.

Some simple benchmarking illustrates the performance difference between the two—for the following benchmarks we used an m3.2xlarge EC2 instance, which provides access to all 8 cores of an Intel Xeon E5-2670.

With a single thread, the new LongAdder is one third slower, but when threads are in contention to increment the field, LongAdder shows its value.  Note that the only thing each thread is doing is attempting to increment the counter—this is a synthetic benchmark of the most extreme kind. The contention here is higher than you’re likely to see in most real-world apps, but sometimes you do need this sort of shared counter, and LongAdder will be a big help.

Read on

How To Put Your Distributed Development Team On The Same Page

One of my favorite things about doing consulting work is the variety of projects that you get to see.  It’s certainly refreshing to be able to pick up a new project in a domain that isn’t connected to what you’ve been working on for the last 6 months.

However, what is revitalizing from an individual’s perspective can end up becoming destructive when you look at it from a step back at the team level.  When team-members are frequently moving to new projects and new clients, it can be nearly impossible to get a good sense of what anyone else is working on.  Developers begin to get siloed and you lose all the benefits of collaboration and over-the-shoulder-ness that working together conveys.  When you have to give the person next to you 15 minutes of context before you can ask what should be a simple question, it becomes hugely expensive and inefficient to get a second opinion on anything.

At Palomino, we had an influx of new projects last fall, which meant that we were more distributed than we had been in the past.  We saw a few indicators that we now think might be early warning signs:

Read on

WhichInstance.com updated for new M3 instances

Amazon regularly adds new types of EC2 instances to keep up with demand and reflect the capabilities of the underlying hardware.  This week they announced medium & large instances in the M3 class.  Like the other M3 instances, these VMs are backed by Sandy Bridge or Ivy Bridge processors and SSD instance storage; more details on Amazon’s instance types page.

Last year we created a tool to make comparing the price of EC2 instances easier.  With the advent of these new instance types, we’ve updated WhichInstance.com to reflect the new options, improving the interface a bit in the process.  Give WhicInstance a look to more easily understand the costs of running an instance over time, and remember that Amazon reserved instances can save you a lot of money!

whichinstance

Rumrunners Kickstarter Campaign

After nearly 6 months of planning and play-testing within Palomino Labs, we’re extremely proud to announce the launch of our very first Kickstarter campaign for our new game Rumrunners. Rumrunners is a digital board game set in the Prohibition era of American history and focuses on bluffing, deception, and deduction.

rumrunnerslogo

We’re particularly excited about the digital medium that Rumrunners is going to use for gameplay. We’ve spent many of our weekly Spielnacht (Game Night) gatherings playing games in the “traitor” genre. Specific mechanics in this genre vary from game to game, but the common thread is the hidden information that’s handed out at the beginning of the game and the later discussion that’s used as an attempt to reveal that information. For example, one of our favorite games, The Resistance, has the players collectively trying to figure out who among them are spies. Throughout the game, players elect subsets of the group to go on missions. Loyal members of the resistance will help pass the mission, but if a single spy has been selected to go on the mission, they’ll cause it to fail. Whichever group can put 3 results in their favor first wins the game.

Read on