Prepare for dependency vendoring.

- Add helper script to simplify glide usage.
- Add validation script for unwanted changes to vendoring.
- Relax/tighten up .{git,docker}ignore to cover vendored files properly.
- .validate: Protect from unbound variable in case of nounset setting.
- Install more recent hg version in the build container.
- Remove glide installation steps from Dockerfile.
- Update documentation.
This commit is contained in:
Timo Reimann 2017-02-02 00:49:32 +01:00
parent dae28f7f17
commit 49a09ab7dd
13 changed files with 196 additions and 42 deletions

View file

@ -1,5 +1,3 @@
dist/ /dist/
vendor/ !/dist/traefik
!dist/traefik /site/
site/
**/*.test

View file

@ -2,7 +2,7 @@
### Building ### Building
You need either [Docker](https://github.com/docker/docker) and `make` (Method 1), or `go` and `glide` (Method 2) in order to build traefik. You need either [Docker](https://github.com/docker/docker) and `make` (Method 1), or `go` (Method 2) in order to build traefik. For changes to its dependencies, the `glide` dependency management tool and `glide-vc` plugin are required.
#### Method 1: Using `Docker` and `Makefile` #### Method 1: Using `Docker` and `Makefile`
@ -26,7 +26,7 @@ $ ls dist/
traefik* traefik*
``` ```
#### Method 2: Using `go` and `glide` #### Method 2: Using `go`
###### Setting up your `go` environment ###### Setting up your `go` environment
@ -42,20 +42,25 @@ This can be verified via `$ go env`
- You will want to add those 2 export lines to your `.bashrc` or `.bash_profile` - You will want to add those 2 export lines to your `.bashrc` or `.bash_profile`
- You need `go-bindata` to be able to use `go generate` command (needed to build) : `$ go get github.com/jteeuwen/go-bindata/...` (Please note, the ellipses are required) - You need `go-bindata` to be able to use `go generate` command (needed to build) : `$ go get github.com/jteeuwen/go-bindata/...` (Please note, the ellipses are required)
###### Setting up your `glide` environment #### Setting up `glide` and `glide-vc` for dependency management
- Glide is not required for building; however, it is necessary to modify dependencies (i.e., add, update, or remove third-party packages)
- Glide can be installed either via homebrew: `$ brew install glide` or via the official glide script: `$ curl https://glide.sh/get | sh` - Glide can be installed either via homebrew: `$ brew install glide` or via the official glide script: `$ curl https://glide.sh/get | sh`
- The glide plugin `glide-vc` must be installed from source: `go get github.com/sgotti/glide-vc`
The idea behind `glide` is the following : If you want to add a dependency, use `$ glide get` to have glide put it into the vendor folder and update the glide manifest/lock files (`glide.yaml` and `glide.lock`, respectively). A following `glide-vc` run should be triggered to trim down the size of the vendor folder. The final result must be committed into VCS.
- when checkout(ing) a project, run `$ glide install -v` from the cloned directory to install Dependencies for the integration tests in the `integration` folder are managed in a separate `integration/glide.yaml` file using the same toolset.
(`go get …`) the dependencies in your `GOPATH`.
- if you need another dependency, import and use it in Care must be taken to choose the right arguments to `glide` when dealing with either main or integration test dependencies, or otherwise risk ending up with a broken build. For that reason, the helper script `script/glide.sh` encapsulates the gory details and conveniently calls `glide-vc` as well. Call it without parameters for basic usage instructions.
the source, and run `$ glide get github.com/Masterminds/cookoo` to save it in
`vendor` and add it to your `glide.yaml`. Here's a full example:
```bash ```bash
$ glide install --strip-vendor # install the new main dependency github.com/foo/bar and minimize vendor size
$ ./script/glide.sh get github.com/foo/bar
# install another dependency, this time for the integration tests
$ ( cd integration && ../script/glide.sh get github.com/baz/quuz )
# generate (Only required to integrate other components such as web dashboard) # generate (Only required to integrate other components such as web dashboard)
$ go generate $ go generate
# Standard go build # Standard go build
@ -105,14 +110,12 @@ TESTFLAGS="-check.f MyTestSuite.*Test" make test-integration
More: https://labix.org/gocheck More: https://labix.org/gocheck
##### Method 2: `go` and `glide` ##### Method 2: `go`
- Tests can be run from the cloned directory, by `$ go test ./...` which should return `ok` similar to: - Tests can be run from the cloned directory, by `$ go test ./...` which should return `ok` similar to:
``` ```
ok _/home/vincent/src/github/vdemeester/traefik 0.004s ok _/home/vincent/src/github/vdemeester/traefik 0.004s
``` ```
- Note that `$ go test ./...` will run all tests (including the ones in the vendor directory for the dependencies that glide have fetched). If you only want to run the tests for traefik use `$ go test $(glide novendor)` instead.
### Documentation ### Documentation

12
.gitignore vendored
View file

@ -1,15 +1,13 @@
/dist /dist
gen.go /autogen/gen.go
.idea .idea
.intellij .intellij
*.iml *.iml
traefik /traefik
traefik.toml /traefik.toml
*.test /static/
vendor/
static/
.vscode/ .vscode/
site/ /site/
*.log *.log
*.exe *.exe
.DS_Store .DS_Store

View file

@ -45,7 +45,7 @@ test-integration: build ## run the integration tests
$(DOCKER_RUN_TRAEFIK) ./script/make.sh generate binary test-integration $(DOCKER_RUN_TRAEFIK) ./script/make.sh generate binary test-integration
validate: build ## validate gofmt, golint and go vet validate: build ## validate gofmt, golint and go vet
$(DOCKER_RUN_TRAEFIK) ./script/make.sh validate-glide validate-gofmt validate-govet validate-golint validate-misspell $(DOCKER_RUN_TRAEFIK) ./script/make.sh validate-glide validate-gofmt validate-govet validate-golint validate-misspell validate-vendor
build: dist build: dist
docker build $(DOCKER_BUILD_ARGS) -t "$(TRAEFIK_DEV_IMAGE)" -f build.Dockerfile . docker build $(DOCKER_BUILD_ARGS) -t "$(TRAEFIK_DEV_IMAGE)" -f build.Dockerfile .

View file

@ -1,10 +1,18 @@
FROM golang:1.7 FROM golang:1.7
# Install a more recent version of mercurial to avoid mismatching results
# between glide run on a decently updated host system and the build container.
RUN awk '$1 ~ "^deb" { $3 = $3 "-backports"; print; exit }' /etc/apt/sources.list > /etc/apt/sources.list.d/backports.list && \
DEBIAN_FRONTEND=noninteractive apt-get update && \
DEBIAN_FRONTEND=noninteractive apt-get install -t jessie-backports --yes --no-install-recommends mercurial=3.9.1-1~bpo8+1 && \
rm -fr /var/lib/apt/lists/
RUN go get github.com/jteeuwen/go-bindata/... \ RUN go get github.com/jteeuwen/go-bindata/... \
&& go get github.com/golang/lint/golint \ && go get github.com/golang/lint/golint \
&& go get github.com/kisielk/errcheck \ && go get github.com/kisielk/errcheck \
&& go get github.com/client9/misspell/cmd/misspell \ && go get github.com/client9/misspell/cmd/misspell \
&& go get github.com/mattfarina/glide-hash && go get github.com/mattfarina/glide-hash \
&& go get github.com/sgotti/glide-vc
# Which docker version to test on # Which docker version to test on
ARG DOCKER_VERSION=1.10.3 ARG DOCKER_VERSION=1.10.3
@ -24,13 +32,4 @@ RUN mkdir -p /usr/local/bin \
| tar -xzC /usr/local/bin --transform 's#^.+/##x' | tar -xzC /usr/local/bin --transform 's#^.+/##x'
WORKDIR /go/src/github.com/containous/traefik WORKDIR /go/src/github.com/containous/traefik
COPY glide.yaml glide.yaml
COPY glide.lock glide.lock
RUN glide install -v
COPY integration/glide.yaml integration/glide.yaml
COPY integration/glide.lock integration/glide.lock
RUN cd integration && glide install
COPY . /go/src/github.com/containous/traefik COPY . /go/src/github.com/containous/traefik

View file

@ -1,6 +1,6 @@
#!/bin/bash #!/bin/bash
if [ -z "$VALIDATE_UPSTREAM" ]; then if [ -z "${VALIDATE_UPSTREAM:-}" ]; then
# this is kind of an expensive check, so let's not do this twice if we # this is kind of an expensive check, so let's not do this twice if we
# are running more than one validate bundlescript # are running more than one validate bundlescript

119
script/glide.sh Executable file
View file

@ -0,0 +1,119 @@
#!/bin/bash
set -o errexit
set -o pipefail
set -o nounset
####
### Helper script for glide[-vc] to handle specifics for the Traefik repo.
##
# In particular, the 'integration/' directory contains its own set of
# glide-managed dependencies which must not have its nested vendor folder
# stripped. Depending on where the script is called from, it will do the Right
# Thing.
#
CWD="$(pwd)"; readonly CWD
GLIDE_ARGS=()
GLIDE_VC_ARGS=(
'--use-lock-file' # `glide list` seems to miss test dependencies, e.g., github.com/mattn/go-shellwords
'--only-code'
'--no-tests'
)
usage() {
echo "usage: $(basename "$0") install | update | get <package> | trim
install: Install all dependencies and trim the vendor folder afterwards (alternative command: i).
update: Update all dependencies and trim the vendor folder afterwards (alternative command: up).
get: Add a dependency and trim the vendor folder afterwards.
trim: Trim the vendor folder only, do not install or update dependencies.
The current working directory must contain a glide.yaml file." >&2
}
is_integration_dir() {
[[ "$(basename ${CWD})" = 'integration' ]]
}
if ! is_integration_dir; then
GLIDE_ARGS+=('--strip-vendor' '--skip-test')
fi
if ! type glide > /dev/null 2>&1; then
echo "glide not found in PATH." >&2
exit 1
fi
if ! type glide-vc > /dev/null 2>&1; then
echo "glide-vc not found in PATH." >&2
exit 1
fi
if [[ ! -e "${CWD}/glide.yaml" ]]; then
echo "no glide.yaml file found in the current working directory" >&2
exit 1
fi
if [[ $# -lt 1 ]]; then
echo "missing command" >&2
usage
exit 1
fi
readonly glide_command="$1"
shift
skip_glide_command=
case "${glide_command}" in
'install' | 'i')
if [[ $# -ne 0 ]]; then
echo "surplus parameters given" >&2
usage
exit 1
fi
;;
'update' | 'up')
if [[ $# -ne 0 ]]; then
echo "surplus parameters given" >&2
usage
exit 1
fi
;;
'get')
if [[ $# -ne 1 ]]; then
echo 'insufficient/surplus arguments given for "get" command' >&2
usage
exit 1
fi
GLIDE_ARGS=("$1" "${GLIDE_ARGS[@]}")
shift
;;
'trim')
if [[ $# -ne 0 ]]; then
echo "surplus parameters given" >&2
usage
exit 1
fi
skip_glide_command=yes
;;
*)
echo "unknown command: ${glide_command}" >&2
usage
exit 1
esac
readonly skip_glide_command
if [[ -z "${skip_glide_command}" ]]; then
# Use parameter substitution to account for an empty glide arguments array
# that would otherwise lead to an "unbound variable" error due to the nounset
# option.
GLIDE_ARGS=("${GLIDE_ARGS+"${GLIDE_ARGS[@]}"}")
echo "running: glide ${glide_command} ${GLIDE_ARGS[*]}"
glide ${glide_command} ${GLIDE_ARGS[*]}
fi
echo "trimming vendor folder using: glide-vc ${GLIDE_VC_ARGS[*]}"
glide-vc ${GLIDE_VC_ARGS[*]}

View file

@ -6,7 +6,7 @@ if grep -q "$(glide-hash)" glide.lock; then
echo 'Congratulations! glide.lock is unchanged.' echo 'Congratulations! glide.lock is unchanged.'
else else
{ {
echo "Error: glide.lock has been manually changed. Don't do this. Use glide up instead." echo "Error: glide.lock has been manually changed. Don't do this. Use script/glide.sh up instead."
echo echo
} >&2 } >&2
false false

View file

@ -3,7 +3,7 @@
source "$(dirname "$BASH_SOURCE")/.validate" source "$(dirname "$BASH_SOURCE")/.validate"
IFS=$'\n' IFS=$'\n'
files=( $(validate_diff --diff-filter=ACMR --name-only -- '*.go' | grep -v '^vendor' || true) ) files=( $(validate_diff --diff-filter=ACMR --name-only -- '*.go' | grep -v '^\(integration/\)\?vendor/' || true) )
unset IFS unset IFS
badFiles=() badFiles=()

View file

@ -3,7 +3,7 @@
source "$(dirname "$BASH_SOURCE")/.validate" source "$(dirname "$BASH_SOURCE")/.validate"
IFS=$'\n' IFS=$'\n'
files=( $(validate_diff --diff-filter=ACMR --name-only -- '*.go' | grep -v '^vendor/\|autogen' || true) ) files=( $(validate_diff --diff-filter=ACMR --name-only -- '*.go' | grep -v '^\(integration/\)\?vendor/\|autogen' || true) )
unset IFS unset IFS
errors=() errors=()

View file

@ -3,7 +3,7 @@
source "$(dirname "$BASH_SOURCE")/.validate" source "$(dirname "$BASH_SOURCE")/.validate"
IFS=$'\n' IFS=$'\n'
files=( $(validate_diff --diff-filter=ACMR --name-only -- '*.go' | grep -v '^vendor/' || true) ) files=( $(validate_diff --diff-filter=ACMR --name-only -- '*.go' | grep -v '^\(integration/\)\?vendor/' || true) )
unset IFS unset IFS
errors=() errors=()

View file

@ -3,7 +3,7 @@
source "$(dirname "$BASH_SOURCE")/.validate" source "$(dirname "$BASH_SOURCE")/.validate"
IFS=$'\n' IFS=$'\n'
src=( $(validate_diff --diff-filter=ACMR --name-only -- '*.go' | grep -v '^vendor/\|autogen' || true) ) src=( $(validate_diff --diff-filter=ACMR --name-only -- '*.go' | grep -v '^\(integration/\)\?vendor/\|autogen' || true) )
docs=( $(validate_diff --diff-filter=ACMR --name-only -- 'docs/*.md') ) docs=( $(validate_diff --diff-filter=ACMR --name-only -- 'docs/*.md') )
unset IFS unset IFS
files=("${src[@]}" "${docs[@]}") files=("${src[@]}" "${docs[@]}")

37
script/validate-vendor Executable file
View file

@ -0,0 +1,37 @@
#!/bin/bash
set -o errexit
set -o pipefail
set -o nounset
SCRIPTDIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"; export SCRIPTDIR
source "${SCRIPTDIR}/.validate"
# Iterate over all directories containing vendor folders.
for dir in . integration; do
vendor_dir="${dir}/vendor/"
IFS=$'\n' files=( $(validate_diff --diff-filter=ACMR --name-only -- ${vendor_dir} || true) )
if [[ ${#files[@]} -gt 0 ]]; then
# We run glide install to and see if we have a diff afterwards
echo "checking ${vendor_dir} for unintentional changes..."
(
cd ${dir}
"${SCRIPTDIR}/glide.sh" install
)
# Let see if the working directory is clean
diffs="$(git status --porcelain -- ${vendor_dir} 2>/dev/null)"
if [[ "$diffs" ]]; then
{
echo "The result of 'glide install' for vendor directory '${dir}' differs"
echo
echo "$diffs"
echo
echo 'Please vendor your package(s) with script/glide.sh.'
echo
} >&2
exit 2
fi
fi
done
echo 'Congratulations! All vendoring changes are done the right way.'