The crisis with the DNC and the Iowa Caucus is another painful reminder of the true cost of cheap software. According to PolitiFact, the Iowa Democratic Party paid approximately $60,000 to a company named Shadow (you can’t make this up) for the use of the now infamous mobile application. It’s pretty staggering that the DNC thought it was appropriate to invest the same amount of money in a critical voting system as campaigns do in a single escort vehicle. This is inline with a lot of software written in recent years that has been cheap at the expense of quality and robustness.
“Well, how do you think it feels when your life depends on 150,000 parts borrowed >from the lowest bidder?” — John Glenn
There is a growing and disturbing trend in the software industry; people fundamentally believe that software products not only can be cheap to build but should be cheap to build. These people would never (knowingly) get on a cheap airplane, go to a cheap doctor or bungee-jump with an old and fraying cord. When it comes to code though, who cares, why don’t we bid it out and take the lowest option since it is all basically the same right?
No, no it is not. Like any engineering task software is complex, building scalable, reliable and mission critical systems is neither cheap, nor easy, yet somehow time and time again we see poorly thought out software solutions crumble under their own weight. After the critical failure of healthcare.gov when it was first launched hopes were high that lessons were learned.
The advent of cloud services has ushered in the massive distributed availability of computing resources. Performance testing, user testing, application resilience, and error monitoring all have myriad managed services and open source solutions. However, they all require a significant investment to configure and maintain and are invisible to the non-technical buyer. The barrier to entry for a mobile or web application that appears to work in a demo is quite low, but something that can exist in the real world with real users is a different matter.
Instead, here we are again watching another piece of poorly architected software fail to do something essential to our democracy in the spotlight. While the facts will surely come out in the following days, it appears to be an un-scalable and poorly, if at all, tested app lives at the core of the problem. Yet again we see the same culprit we did when healthcare.gov rolled out, software gets pushed into production with little to no thought and faceplants to everyone's surprise. With code at the core of so many modern companies; it is amazing to see how often it is still mismanaged.
At some point the reputation hit becomes a far larger issue than upfront dollars saved. While the company responsible rightfully tried to take responsibility the Democratic Party is left to clean up the pieces at a vital point in their journey. If you are planning on starting a new platform or building some new software, or even maintaining an old platform consider not going with the lowest bidder, or even dare I say, the second lowest bidder. In mission critical fields like politics and healthcare, risk mitigation should be scoped in the initial set of requirements. At the end of the day, you will save yourself the embarrassment of a face plant, and even worse the cost of a face plant.