Prelude

This post is part of a series of posts designed to make you think about your own design philosophy on different topics. If you haven’t read this post yet, please do so first:

Develop Your Design Philosophy

After this post, read this next one:

Package Oriented Design

Introduction

In an interview given to Brian Kernighan by Mihai Budiu in the year 2000, Brian was asked the following question:

“Can you tell us about the worse features of C, from your point of view”?

This was Brian’s response:

“I think that the real problem with C is that it doesn’t give you enough mechanisms for structuring really big programs, for creating ``firewalls’’ within programs so you can keep the various pieces apart. It’s not that you can’t do all of these things, that you can’t simulate object-oriented programming or other methodology you want in C. You can simulate it, but the compiler, the language itself isn’t giving you any help.”

When I read this quote for the first time, I almost fell out of my seat! Here it was, the reason why Go has the concept of packaging. Packaging creates “firewalls” within Go programs such that: (i) the various pieces of the program can be kept apart, (ii) large teams can work on large projects, and (iii) the compiler and the language itself can provide support and help.

All that help and support from Go is great. However, you can still miss many of the benefits packaging provides if you don’t develop and employ best practices in organizing your code.

Language Mechanics

In all of the programming languages I have ever used, I was always encouraged to organize the source code inside of different folders within the source tree for a project. This idea of organizing source code via folders is the first practice we need to break free from. In Go, each folder that contains source code is considered a package, and packages provide that “firewall” of support that Brian mentioned was lacking in C. Packaging directly conflicts with how we have been taught to organize source code in other languages.

As part of the language definition, Go turns each package into an individual static library, and it’s the static library that creates the physical “firewall”. This behavior is not something you can circumvent, it’s built into the language. The creation of static libraries in other languages is optional and achieved through tooling. In those languages, it is a feature that you can choose to use or ignore.

You can think of packaging as applying the idea of microservices on a source tree. Each folder representing a self contained piece of functionality. Where in other languages, the source tree represents a monolithic application. If you want to break up the application, you need to create multiple source trees and use tooling to stitch it back together.

Go has no concept of sub-packages. All folders, regardless of their physical location, are built into these static libraries and flattened out during compilation. The physical location of a package directs the compiler (via the import statement) to the specific package that needs to be included. In Go, all packages are “first class,” and the only hierarchy is what you define in the source tree for your project.

Given that packages are standalone and their contents are “firewalled,” there needs to be a way to “open” parts of the package to the outside world. This is where the idea of exporting and unexporting comes in. Types, variables, functions and anything else you can name inside the package can be exported (“opened”) or unexported (“closed”). Start the identifier of the named entity with a capital letter, and it is exported from the package. Otherwise, it’s unexported. This naming is a brilliant use of convention over configuration and provides the concept of encapsulation in Go.

There is one more important language mechanic about Go packages. Two packages can’t cross-import each other. Imports are a one way street. There are lots of interesting reasons why this decision was made, which range from faster compile times to forcing developers to think about their dependency decisions. As a result, when two packages need to import each other, it’s a smell that the packages need to be merged or decoupled.

Design Philosophies

All design decisions start and end with the package. To be able to make better design choices, you need a set of design philosophies that will allow you to sniff out good packaging decisions from the bad ones. You also need a set of guidelines to follow that focus on purpose, usability and portability.

Purpose

Packages must be named with the intent to describe what it provides.

The purpose of a package is to provide a solution to a specific problem domain. The more focused each package’s purpose is, the more clear it should be to know what the package provides. The name of a package must describe what it provides and if the name of the package doesn’t immediately clue into this, it probably contains a fragmented set of functionality. If you are struggling to name a package, that’s a big smell the package is not focused on a single purpose. Classic examples of packages that provide are http, fmt and io. Classic examples of packages that contain are util, common and helpers.

Packages must not become a dumping ground of disparate concerns.

When you create packages that contain fragmented functionality, they become a dumping ground of disparate concerns. There is no cohesive API, and it’s a grab bag of code. What’s worse, these packages become a parasite latched on to other packages in a one-sided and unhealthy relationship. This is actually a dependency issue that creates project wide coupling. It will affect a project’s health and ability to refactor, adapt, grow and separate over time.

Usability

Packages must be intuitive and simple to use.

Packages exist to provide support for the specific problems that are being solved by application developers. A package should be intuitive and simple to use. Such packages allow application developers to focus on their concerns and develop their applications faster while maintaining high levels of integrity.

Packages must respect their impact on resources and performance.

A package will have an impact on the user’s application in terms of resources and performance. A package doesn’t have the right to be sloppy because the garbage collector works so well or that Go will be forgiving in some areas. Respect for what the application developer is going to build using the package is paramount.

Packages must protect the user’s application from cascading changes.

Decoupling is a big part of usability. The external parts of a package must protect the user’s application from cascading changes that could occur when the package itself requires internal changes. The use of interfaces to protect the application developer from these cascading changes is mandatory.

Packages must prevent the need for type assertions to the concrete.

This is why Go provides an interface for error handling. The error interface allows package developers to internally improve error handling without causing cascading changes back into the user’s application. Any time an application developer needs to perform a type assertion against the error interface value, to a concrete type value, a flag should be raised and a discussion should happen. This same check should apply to almost every type assertion to a concrete type, because the code is walking away from the decoupling and heading back into the concrete.

Packages must reduce, minimize and simplify its code base.

If an application developer has the need to decouple any concrete type declared in a separate package, they can declare the interfaces they need themselves. As an example, packages don’t need to supply interfaces for the sole purpose of allowing application developers to mock the package’s concrete types inside their applications or tests. This is thanks to Go’s ability to implicitly identify the compliance of any concrete value to any interface, regardless where the interface is declared. This helps with the package’s ability to reduce, minimize and simplify its code base.

Portability

Packages must aspire for the highest level of portability.

The more a package is decoupled from other packages, the more reusable a package becomes. The portability of a package is directly associated to the number of packages it depends on. A package that only depends on the standard library, has the highest level of portability in Go.

Packages must reduce setting policies when it’s reasonable and practical.

When packages take on dependencies, they are accepting all the policies from those packages they import. Sometimes this can prevent application developers from using a package that otherwise would have been perfect. A package that sets a policy about a particular logging framework would restrict its portability to only those application developers that want to use the same logger.

Packages must not become a single point of dependency.

When packages are created to be a single point of dependency, like a package of common types, the resulting project wide coupling is crippling. It is no longer possible to break a project up into smaller projects when the need arises. Making any change to that common package can easily break a large number of other packages, even those you might not know about.

Conclusion

You have just reviewed a set of design philosophies that are well accepted in the Go community. Things we have learned over the past several years. These thoughts and ideas establish a set of concepts to apply in our projects, but how to apply these concepts is not always obvious.

In order to apply these concepts of purpose, usability and portability, a strong project structure is needed. It’s within the project structure that teams can have constructive conversations and code reviews around packaging decisions. This is a core directive of package oriented design, to foster design conversations. In my next post, I will describe what package oriented design is, a project structure that is working well for me and the guidelines for using that structure.