Introduction

If you are using 3rd party packages, (packages that you don't own or control), you will want a way to create a reproducible build every time you build your projects. If you use 3rd party packages directly and the package authors change things, your projects could break. Even if things don't break, code changes could create inconsistent behavior and bugs.

Keith Rarick's tool godep is a great step in the right direction for managing 3rd party dependencies and creating reproducible builds. The godep tool gives you two options for managing dependencies. The first option creates a dependency file with version control information and then with some godep magic, the code is built against those versions. You can also Vendor your 3rd party packages inside your projects as well. You never need to change a single source code file and everything is accomplished in conjunction with the go tooling.

Downloading Godep

Download godep using go get and make sure your $GOPATH/bin directory is in your PATH.

Create A Project

Build your project using the 3rd party packages as you normally would. Since godep does not require you to change any import paths in the code, 'go get' the code you need and import those packages directly. To keep the post simple, I am going to use an existing program called News Search that uses one 3rd party dependency.



After 'go get' completes, I have the following code on disk inside the GOPATH.

The News Search program is using code from a different Going Go repository, which for this project is a 3rd party package. Since 'go get' was successful, the code built and is installed.

Dependency Management

Once you have a project that can build and install properly, you can use godep to create the Godeps dependency file. Change to the root location for the project and run the godep save command with the -copy=false option:


Once the save is complete, godep creates a file called Godeps. You will save this file with your project:


The Godeps file contains everything godep needs to create a reproducible build. The Godep file lists each 3rd party package and the git commit number for that version of code to use.


Now remove the 3rd party package from its original location and perform a build using the godep tool:



If you remove the original 3rd party code, you can't use 'go build' directly anymore, the imports don't exist. To build the project use 'go build' from the godep tool.

You can also use 'go install' and 'go test' as long as you run those commands through godep.

godep go build
godep go install
godep go test

To make this work, godep performs a bit of magic. It uses a working directory and manipulates the GOPATH underneath.

Run the godep path command from inside the project folder:


You should see the following output:


If you open that folder you will see the code for that version. This code is being used to build the project:


The godep tool will continue to use the code from this location to perform future builds if it exists. Calling godep go build will download the version of code specified in the Godeps file if it doesn't already exist.

If you open any of your source code files you will see the imports have not changed. The way godep works, it doesn't have to change a thing. This is one of the really awesome things about godep.

Updating Dependencies

When it is time to update one of your 3rd party libraries just 'go get' it. Then you just need to run godep save once again to update the Godeps file. Because the imports paths in the source code files are not changed, godep will find and update the dependencies.

I have changed the 3rd party package and pushed it up to GitHub:


Now I 'go get' the code changes and update the Godeps file:


If I open the Godeps file the dependencies have changed:


Now I use godep to build the code:



The godep tool downloaded the new version and built the code successfully.

Vendoring

Vendoring is the act of making your own copy of the 3rd party packages your project is using. Those copies are traditionally placed inside each project and then saved in the project repository. The godep tool supports Vendoring and will place the copies inside the project that are using them.

To Vendor code with godep, don't use any options with the save command. First clean the workspace and download a new version of the News Search program:


Now issue the godep save command again but this time without the copy option:



This time you will have a Godeps folder with a special workspace subfolder and the Godeps file. All the 3rd party packages are copied into the workspace folder under src. This is setup to work with GOPATH.

Version control files are removed and no import paths are changed in any of the source code files.
Next remove the original code for the 3rd party package and perform the build:


The build is successful and everything is ready to be pushed back into the repository.

Performing an update is as simple as downloading the new version of the 3rd party package and running godep save again.

Conclusion

The godep tool solves many of the problems that exist with creating reproducible builds. It is incredibly easy to use and sits on top of the go tooling. It doesn't change anything about go, how you write go programs or how you import 3rd party packages. The only drawback is that godep does not support Bazaar using the non-vendored option.

For the public packages your are publishing, you can include a Godeps file to provide your "stable build" configuration. Package users can choose to use it or not. That is really cool. Build the code with go directly or through godep.

In the end, godep is a tool that:

1. Supports a Vendor and Non-Vendor solution that provides a reproducible build
2. Maintains backwards compatible with all existing Go packages
3. Provides a way to publish and access the "stable build" configuration of a product
4. Easy to update package dependencies when new package versions are available