Rebase Git Flow

Git flow used on cloud projects with fast releases, and only one deployed version. Variant of git-flow, putting emphasis on clean git history. Two infinite branches, develop+master, and three short-lived branch types: hotfix, release, feature*. This workflow is mapped to our deploy spaces in following way:

master branch → canary space deployed after promoting - completely identical to production develop branch → staging space for testing hotfix* && release* branch → release space

Hotfix and release can be on same space, as we very very rarely have a situation with active hotfix and release branch.

All merges between all 4 named branches should be done by rebase, to evade non-feature merge commits polluting our git history, and messing with further syncs of master - develop.

All branches need to be up to date before merging. Conflicts needs to be solved in non-master branches via git pull --rebase

Overview:

Examples:

New Feature:

Create new branch from latest develop branch

  git fetch origin
  git checkout develop
  git checkout -b feature/my_awesome_feature

Do some commits, and keep your branch updated with latest changes from develop branch:

  git add -u; git commit -m "My great commit"
  git pull origin develop --rebase
  git push -u origin feature/my_awesome_feature

Due to possible commit order getting updated, you might need to force push, I recommend to use --force-with-lease to verify that upstream branch is in state that you expect, nobody pushed anything, and you fetched latest changes.

  git push origin feature/my_awesome_feature --force-with-lease

After your feature branch is ready, create Pull request, rebase/squash/merge here is up to developer,

I suggest to merge larger features, so they can be reverted easily via merge commit revert,

Squash if you didn’t clean your feature commit history, and Rebase for small things.

Hotfix

Create hotfix branch from master branch:

  git fetch origin
  git checkout master
  git checkout -b hotfix-1

Do your fix, and push hotfix branch back to origin.(keep your branch up to date)

  git add -u; git commit -m "My great commit"
  git pull origin master --rebase
  git push -u origin hotfix-1

After push to hotfix*, deployment should be done to release space, after QA you can proceed with PR to master Merge to master is done with rebase.

Release

Create release branch from develop branch, and push to origin:

  git fetch origin
  git checkout develop
  git checkout -b release-1
  git push origin release-1

Wait for release space to get updated, do final smoke tests, optionally push extra commits.

After QA verification, you can proceed with PR to master

Merge to master is done with rebase.

Master → Develop sync

Due to translations/security fixes/hotfixes, you occasionally have to sync master back to develop.

This updates commit history, and so it has to be force pushed back to develop.

  git fetch origin
  git checkout develop
  git pull origin master --rebase
  git push origin develop --force-with-lease

All Devs have to update their local develop branches:

  git fetch origin
  git checkout develop
  git reset --hard origin/develop

Final Notes:

• All Merging between named branches is done by rebase, to eliminate merge commits, that are impossible to keep in sync between branches.
• Feature branches can be merged by any means necessary.
• Master branch is the only branch protected against force-push.
• Force pushing is done with --force-with-lease

Pros:

• Clean history, nothing gets duplicated

Cons:

• Majority of this flow can’t be easily required in github, therefore it is dependent on developers discipline

Linear History can be enforced by branch protection rules in public github:

https://help.github.com/en/github/administering-a-repository/requiring-a-linear-commit-history

Not available in enterprise github.

Generating X-Hub-Signature

Today will be a quick one, because I could not google this exact question.

Lets say, you have a microservice that accepts github webhooks, and you want to test it with set secret. Github client libraries have ValidatePayload/ValidateSignatur: https://github.com/google/go-github/blob/master/github/messages.go#L201 but no generate signature.

You can use crypto packages to create signature for any payload like so: https://www.digitalocean.com/community/tutorials/how-to-use-node-js-and-github-webhooks-to-keep-remote-projects-in-sync

But if you just want to create a single signature for single payload, to unit test your endpoints, that seems like a overkill.

Fortunately IT swiss army knife CyberChef exists.

And it contains HMAC recipe

Go relative imports, and building

This post was only made because, because docker docs example wasn’t working for me:

https://docs.docker.com/develop/develop-images/multistage-build/

It is reasonable to use multi-stage build for containers with go binaries, because image to build go is pretty large:

REPOSITORY               SIZE
go-scratch-binary        6.24MB
go-alpine-binary         14.5MB
go-builder               807MB

“Problem” is, our go microservice is living inside corporate github, and it’s packages can’t be accessed from golang image without installing certificates, or adding our microservice into GOPATH, without these, you will end up with cannot find module providing package <corp.github.url>/package even though the package is in the same folder.

Modules to the rescue

Fortunately Go 1.11 introduced Modules

with modules, relative imports inside go source code works like a charm:

start with go mod init <package_name> inside folder with your go main.go

├── main.go
├── controllers 
│   ├── some_controller.go
│   ├── something.go

this creates go.mod, and go.sum file:

go.mod

module <package_name>

  go 1.12

  require (
          github....

main.go

package main

import (
  "<package>/controllers"
)

controllers/some_controller

package controllers

import (
  ...
)

with this structure, you can build your go module anywhere with something like this:

Dockerfile

FROM golang:alpine

# Go mod needs git
RUN apk update && apk add --no-cache git

RUN mkdir /app
ADD . /app/
WORKDIR /app
RUN go build
CMD ["/app/<package>"]

docker build . -t go-build:latest && docker run -it go-build => running binary

2019/07/17 08:54:47 @@@Hello world!!!@@@
2019/07/17 08:54:47 Listening...

But this results in ~300MB image =>

Docker multistage build to the rescue

Dockerfile

FROM golang:alpine AS builder

# Go mod needs git
RUN apk update && apk add --no-cache git

RUN mkdir /app
ADD . /app/
WORKDIR /app
RUN go build
CMD ["/app/<package>"]

FROM alpine:latest AS alpine
COPY --from=builder /app/<package> /app/<package>
CMD ["/app/<package>"]

docker build --target=alpine -t go-alpine:latest

=> ~10 MB image(depends on Go dependencies)

We can go further, and build statically linked binary, and use scratch docker image:

if you try to use standard binary with scratch image you’ll end with

standard_init_linux.go:207: exec user process caused "no such file or directory"

Dockerfile

FROM golang:alpine AS builder

# Go mod needs git
RUN apk update && apk add --no-cache git

RUN mkdir /app
ADD . /app/
WORKDIR /app

RUN CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -ldflags="-w -s"
CMD ["/app/<package>"]

FROM alpine:latest AS alpine
COPY --from=builder /app/<package> /app/<package>
CMD ["/app/<package>"]

FROM scratch AS binary
COPY --from=builder /app/<package> /app/<package>
CMD ["/app/<package>"]

docker build --target=binary -t go-binary:latest

=> ~5MB image(depends on dependencies)