How one company improved developer productivity by 700% with reactive programming
You don't have to be a billion-dollar web company to need to figure out how to accelerate development iteration.
Take MoneySuperMarket Group, for example, the mid-sized holding company behind three of the UK's most popular comparison-shopping sites, specializing in financial services. These sites are popular enough to generate $375 million in revenue for the company in 2014, and to sustain annual investments of $23 million in creating new websites and systems.
And yet the company has IT problems that feel more like $1 billion problems.
Paul Dickinson is Head of Group Platform and Innovation at MoneySuperMarket Group. In 2012, Dickinson realized he had four pressing challenges that were critical to the business and centered in the IT function.
To survive in an increasingly competitive market and maintain and accelerate growth and profitability, he launched a project to re-evaluate the software stack that underpins the company's core "aggregation" capabilities and to define the ideal architecture that could future-proof the company for the rate of innovation demanded by the business.
To achieve this, and to help the company increase developer velocity (i.e., the rate at which new features, partner integrations, etc. are shipped), go cloud-first, outpace competitive startups, and drive revenues, Dickinson and MoneySuperMarket Group turned to reactive programming, partnering with Typesafe.
I sat down with Dickinson recently to uncover the results from his work with reactive programming.
TechRepublic: It's always a challenge at mid-sized companies to get management buy-in for serious investment in the IT function, even when that's your business. How did you make that happen?
Dickinson: The re-investment in technology is part of the company's embrace of the "startup mentality." Every week, new price comparison sites emerge, and for MoneySuperMarket Group, competitive edge is tied directly to the ability to be responsive, create and deploy new features and products quickly, and for its application infrastructure to always be one step ahead of the rate of business change.
Our ongoing investment in technology has enabled the business to re-platform a number of its core channels and will allow it to make future changes to the customer proposition in a more agile manner.
We embraced this emerging idea, called reactive, as the path to achieving our business and IT goals, championed by an internal technical architect, Sulaiman Rafiq.
TechRepublic: What was the process? Did you start with baby steps or go all-in right from the start?
Dickinson: Our first target for the re-platforming effort was the already massively popular TravelSuperMarket service, a travel industry price comparison site that helps customers "get away for less" with package holidays, city breaks, flights, hotels, and auto rental.
The service allows price comparisons across more than 650 airlines and 200,000 hotels in a matter of seconds.
The back-end of TravelSuperMarket is heavily reliant on what the company refers to as "aggregation"--when a user fills out a form to fly from LHR to JFK, for example, aggregation is the process of going out to partner APIs, pulling data back over HTTP, normalizing that data, and then pulling it back into the browser.
In addition to re-platforming to support common aggregation and supporting business logic scenarios, the team's move to reactive was tied to achieving a better baseline for the creation of new features on the user facing application.
TechRepublic: How long did the first phase take your team?
Dickinson: We built a proof of concept within six weeks with our partner Cake Systems.
The proof of concept was never considered "throw-away" code and acted as a thin slice of the most complex part of the project. By adopting this approach, the Java and Akka system was proven, and its full power for being a reactive and scalable stack was demonstrated.
Within five months, a minimum viable product was created. This had all the critical features required--including infrastructure, performance monitoring, and integration with internal systems.
TechRepublic: What kind of results did you see? What was the ROI like?
Dickinson: We saw top-line revenue growth attributable to our new platform. TravelSupermarket.com's revenues are up 28% due to ongoing investment in technology.
Also, software stack changes happen quickly using reactive. It used to take two to three weeks for engineering changes at TravelSuperMarket to propagate; today, it takes two to three days. This 700% productivity increase positively affects every aspect of feature creation through partner integration efforts, from starting a "story" to deploying to live.
We also began innovating new business channels faster. As MoneySuperMarket creates new business "channels," the team estimates that process goes about twice as fast as before, reducing the time for channel onboarding by 50%.
Finally, we got out of the tiresome business of endless capacity planning. In the past, advertising campaigns and the ensuing web traffic would require a lot of capacity planning. TravelSuperMarket's new stack allows new viral campaigns to be launched with much less thought up-front about the implications of spikes in traffic.
