Last Updated: 2021-06-16

The Go standard library contains concise packages (http, time, io, cipher, etc) which don't require much (if any) interoperability and thus provide little guidance for how a larger Go application might fit together.

Without a guide, conventions have formed across several open source Go projects. We've selected several of those conventions and have brought them into our standard practices.

Top Level

Go project within our company have the following key packages:

|-- cmd
|-- pkg
|   |-- client
|   |-- core
|   |-- database
|-- testing

Each of these packages has a very specific intention or audience which we will look at one by one.

In Brief

cmd - This is the only location in which binary executables should exist

pkg - All production package files should live here

testing - Thing we don't intend to ship to production

Home to all executable binaries (primary and supporting) that are part of our application.

Naming Conventions

Each directory should match the name of an executable being built.

|-- cmd
    |-- cloudprint2
    |-- cp2ctl
    |-- dnshijack

The example above is from a project called cloudprint2 which (by convention) is also the name of the primary binary. Thus we can see the corresponding /cmd/cloudprint2 directory.

We also have a "cloudprint2 control" tool (cp2ctl) which is able to make GRPC calls to our administrative endpoints. In addition we wrote a small DNS Hijacking tool (dnshijack) that is often useful for our testing.

Code

The code in cmd is very thin. Generally just enough to parse command line options, initialize any dependencies, and call into pkg logic.

Accepting Reality

Sometimes we break these rules. DNS Hijacking (dnshijack), for example, is a convenient testing tool that is not part of our core ownership. The entire code for that binary lives within /cmd/dnshijack. That is to say, it is not extremely thin and does not use anything from our pkg folders.

Cobra

If you are curious, we tend to use spf13/cobra for command line parsing.

This is where we place the majority of our code, both public (client) and private (databases, core, etc.).

Naming Conventions

|-- pkg
    |-- client
    |-- core
        |-- model
    |-- databases
    |   |-- crdb
    |   |-- memdb
    |-- server
    |-- service
    |-- util
    |-- etc.
    |-- etc.

Each package has an intended audience.

Client

The top-level client package is intended to be the only package an external team needs to integrate with our service. It generally contains (1) an interface and corresponding implementation of thin wrappers to the RPC endpoints and (2) public types (which are sometimes transitive references to the core/model package.)

Database

At the top-level of database we create interface files which other packages can depend upon. In the nested packages (crdb, memdb) we place our implementation specific logic.

At the top-level database we also place useful shims/wrappers around our database that are implementation (crdb vs memdb) agnostic such as a cache.go or instrumentation.go.

Core

The top-level core package contains objects that are shared across our implementation layer. For example metrics.go might contain metrics objects we emit, messages.go might contain Kafka messages we emit, etc.

In a fair amount of our code, we tend to describe types using protos. These protos are persisted in our database, used across our GRPC endpoints, and as our in-memory representations. In this type of project, we might see a simple types.go file with a few non-core types.

In core/model we place a go file to help us wrap and unwrap each proto that we use. Our representation is generally a thin struct with a pointer to the protobuf along with friendlier methods for accessing the various internal fields. This is essentially wrapping the protobuf.

Accepting Reality

Accepting Reality

At our company, we also see several technology specific directories such as helm (deployment), istio (service discovery), and proto (defines schemas of structured data).

Don't hesitate to place these into your structure as well.

|-- cmd
|-- helm  // deployment
|-- istio // service discovery
|-- pkg
|   |-- client
|   |-- database
|   |-- core
|       |-- model
|-- proto // schemas of structured data
|-- testing