Updated on February 10th, 2017

Prelude

This post is part of a series of posts designed to make you think about your own design philosophy on different topics. If you haven’t read this post yet, please do so first:

Develop Your Design Philosophy

Introduction

I want to share with you my design philosophy around the word Integrity and what it means to me from a Go perspective. Integrity is much more than just a buzzword, it is a driving principle in everything I do, both in code and in life.

This is my definition of the word integrity:

The accuracy and consistency of the code performing every read, write and the execution of every instruction. Just as important, writing less code and knowing the error handling code is the main code.

If you step back for a second and think about what the code you write actually does, it can be broken down into micro and macro level operations. Think about it, each line of code you write is either allocating, reading or writing memory (micro). Variables and statements are a driving force behind these operations. The larger amount of code you write performs data transformations (macro). We have an input and we produce an output. Our functions are a driving force behind these operations.

With this in mind there are two driving forces behind integrity.

Integrity is about every allocation, read and write of memory being accurate, consistent and efficient. The type system is critical to making sure we have this micro level of integrity.

Integrity is about every data transformation being accurate, consistent and efficient. Writing less code and error handling is critical to making sure we have this macro level of integrity.

Integrity is critical for writing software that is both resilient and reliable.

Get Serious About Reliability

If we are to become serious about reliability [1], we must become serious about integrity and therefore error handling. There was a case study that looked at a couple hundred bugs in Cassandra, HBase, HDFS, MapReduce, and Redis [2]. In the study 48 critical failures were found and this was the breakdown:

• 92% : Failures from bad error handling * 35% : Incorrect handling
  - 25% : Simply ignoring an error
  - 8% : Catching the wrong exception
  - 2% : Incomplete TODOs * 57% System specific
  - 23% : Easily detectable
  - 34% : Complex bugs
• 8% : Failures from latent human errors

The results may be surprising to you. 92% of the critical failures that occurred could have been avoided if the developers made integrity their first priority. Error handling was mishandled in several different ways. In some cases because of how exceptions were handled and in others because of developer carelessness.

Exception handling lends itself to making mistakes. It doesn’t force you to think about error handling as part of the main code but as it is titled, an exception. Something to be dealt with later, somewhere else in the code. This disconnect creates lots of issues as the postmortem discovered.

I hear all the time that error handling in Go is tedious and inconvenient but that’s the point. This is Go pushing you towards the one thing you must never ignore, integrity. We all know that for security to be effective it requires this same tedious and inconvenient nature. I don’t see error handling in Go to be any different. Integrity and security go hand in hand.

Developer Carelessness

The study also revealed an interesting trait among developers, carelessness. I believe developers operate in a matrix of carelessness vs ignorance vs intent. Anytime we discover an integrity issue we must ask ourselves how and why it happened.

I like using this matrix when trying to understand developer behavior around integrity. It is a different twist on Martin Fowler’s matrix on technical debt [3]. When an integrity issue is discovered it either comes from a place of ignorance or carelessness.

                    Non-Deliberate               Deliberate
              ------------------------------------------------------
              |                          |                         |
              |                          |                         |
   Ignorance  |  Learning / Prototyping  |    Hacking / Guessing   |
              |                          |                         |
              |                          |                         |
              |-----------------------------------------------------
              |                          |                         |
              |                          |                         |
Carelessness  |        Education         |   Dangerous Situation   |
              |                          |                         |
              |                          |                         |
              ------------------------------------------------------

Integrity issues that come from a place of ignorance either occur when we are prototyping/learning (non-deliberate) or when we are hacking our way through a solution without considering the consequences (deliberate). I am usually ok when ignorance is at play but I want to eliminate as much of the hacking and guessing as possible.

Integrity issues that come from a place of carelessness create dangerous situations. The idea that a developer would knowingly introduce and deploy code with integrity issues is beyond me. That being said, there may be critical situations where it is acceptable for very short periods of time. Non-Deliberate carelessness requires an immediate action towards education. We must eliminate carelessness from everything we do.

Write Less Code

There have been studies that have researched the number of bugs you can expect to have in your software. The industry average is around 15 to 50 bugs per 1000 lines of code [4]. One simple way to reduce the number of bugs, and increase the integrity of your software, is to write less code.

Bjarne Stroustrup stated that writing more code than you need results in Ugly, Large and Slow code [5]:

• Ugly: Leaves places for bugs to hide.
• Large: Ensures incomplete tests.
• Slow: Encourages the use of shortcuts and dirty tricks.

Conclusion

I want to conclude with my definition of integrity again:

The accuracy and consistency of the code performing every read, write and the execution of every instruction. Just as important, writing less code and knowing the error handling code is the main code.

I feel strongly that integrity must be your highest priority for everything you do and every decision you make. This means writing less code and making error handling part of your main code and not something to be dealt with as an exception. The way errors are handled in Go is proof that the language has an uncompromising attitude towards integrity and that error handling is inseparable to that cause.

Error Handling Mechanics

If you are new to Go, it might help to read these posts about error handling mechanics.

Error Handling In Go Part I
Error Handling In Go Part II
Error Training Material

References

[1] Software for infrastructure - Bjarne Stroustrup
[2] Lessons learned from reading postmortems - danluu.com
[3] Technical Debt Quadrant - Martin Fowler
[4] Ratio of bugs per line of code - Dan Mayer
[5] Masterminds of Programming - Federico Biancuzzi and Shane Warden