Speeding up Go builds with go 1.10 build cache in Docker containers

I have a Go project with a large vendor/ directory which almost never changes.

I am trying to use the new go 1.10 build cache feature to speed up my builds in Docker engine locally.

Avoiding recompilation of my vendor/ directory would be enough optimization. So I'm trying to do Go equivalent of this common Dockerfile pattern for Python:

FROM python COPY requirements.txt .              # <-- copy your dependency list RUN pip install -r requirements.txt  # <-- install dependencies COPY ./src ...                       # <-- your actual code (everything above is cached) 

Similarly I tried:

FROM golang:1.10-alpine COPY ./vendor ./src/myproject/vendor RUN go build -v myproject/vendor/... # <-- pre-build & cache "vendor/" COPY . ./src/myproject 

However this is giving "cannot find package" error (likely because you cannot build stuff in vendor/ directly normally either).

Has anyone been able to figure this out?

2 Answers

Answers 1

Here's something that works for me:

FROM golang:1.10-alpine WORKDIR /usr/local/go/src/github.com/myorg/myproject/ COPY vendor vendor RUN find vendor -maxdepth 2 -mindepth 2 -type d -exec sh -c 'go install -i github.com/myorg/myproject/{}/... || true' \; COPY main.go . RUN go build main.go 

It makes sure the vendored libraries are installed first. As long as you don't change a library, you're good.

Answers 2

Just use go install -i ./vendor/....

Consider the following Dockerfile:

FROM    golang:1.10-alpine  ARG     APP ENV     PTH $GOPATH/src/$APP WORKDIR $PTH  # Pre-compile vendors. COPY    vendor/ $PTH/vendor/ RUN     go install -i ./vendor/...  ADD     . $PTH/  # Display time taken and the list of the packages being compiled. RUN     time go build -v 

You can test it doing something like:

docker build -t test --build-arg APP=$(go list .) . 

On the project I am working on, without pre-compile, it takes ~12sec with 90+ package each time, after, it take ~1.2s with only 3 (only the local ones).

If you still have "cannot find package", it means there are missing vendors. Re-run dep ensure should fix it.

An other tip, unrelated to Go is to have your .dockerignore start with *. i.e. ignore everything and then whitelist what you need.

If You Enjoyed This, Take 5 Seconds To Share It

• Facebook
• Twitter
• Google+
• StumbleUpon
• Digg
• Delicious
• LinkedIn
• Reddit
• Technorati