Workflow

In general we should follow the Github workflow but starting from dev.

Create a new issue

On Github:

head to https://github.com/nestauk/<reponame>/issues;
create a new issue by pressing the green button New issue;
in the page that you get (https://github.com/nestauk/<reponame>/issues/new):
- fill the title (let's say the new feature is titled Add the new /users API),
- describe the bug or the feature,
- eventually select labels, projects, and a milestone;
after pressing the button Submit new issue you should get the new issue page (https://github.com/nestauk/<reponame>/issues/<number>);
please take note of the issue number.

Checkout and update `dev`

Switch to the dev branch, making sure it's up to date with main or staging.

git checkout dev
git pull

Let's assume that by issuing git log you get:

commit 7b5b295120324db1a77acf1ca5b35a7b717b9a19
Author: John Foo <john_foo@users.noreply.github.com>
Date:   Thu May 16 14:34:16 2019 +0100

    Initial commit, added a README

Branch off a feature branch from `dev`

Create a new branch.

The branch name should be the combination of the issue number and some free text for you to easily identify the branch should you work on many at once:

<number of the issue it refers to>_<free text with underscores>

Tip: using underscores will make it easier to select the branch name in the terminal just by double clicking on it.

Assuming you decided to name it 05_my_feature, create that branch and immediately switch to it with:

git checkout -b 05_my_feature

Add your commits

As usual, add or modify code and push to the feature branch and commit. Assuming you're in the branch 05_my_feature, let's say you make 3 commits like these:

git add file1.py README.md
git commit -m "work in progress"

git move file3.py file2.py
git commit -m "file renaming"

git remove file4.py
git commit -m "removing an unnecessary file"

Note: it's considered good practise that you check that you're only adding changes relevant to the feature you are working on, so in order to avoid accidentally committing credentials or polluting the repo with temporary/large files: - please don't use git commit -a - please don't use git add <directory>

When you feel the feature or the fix is ready for the review, you should push the branch:

git push

If this is the first time you push this branch, git will ask to you to be a bit more specific because the 05_my_feature branch is unknown to origin (the default name of your remote, your repo on Github):

git push --set-upstream origin 05_my_feature

Open a pull request

Head to https://github.com/nestauk/<reponame>/pulls and click on New pull request.

Choose dev as a base and 05_my_feature as a target branch (if 05_my_feature is a sub-feature branch, choose the feature branch instead). Then click on Create pull request.

Select one or more reviewers.

It's a good habit to start working in a new branch by referencing the relative issue(s) in the commit message with Closes/Fixes clauses, so that Github will cross reference the issue and the PR, making navigation easier for everyone.

Here's an issue referencing a PR:

Here's a PR referencing an issue:

Draft pull requests

You'll have the option to open a draft PR which is usually preferable to start the conversation about your work on the upcoming PR as soon as possible.

Before asking for a review, please mark the PR ready for review by clicking on the button Ready for review at the bottom of the draft PR:

Discuss the pull request review

At this point you can wait for the assignees to review you code, but you can still add/remove/edit code.

Let's say you push this commit:

git add file1.py
git commit -m "added a comment and fixed a typo"
git push

Now a reviewer might ask for some changes, so you might have to add more commits. Let's say you need a couple of new commits:

git add file1.py
git commit -m "implement review changes"
git push

git add file1.py
git add file1b.py
git commit -m "actually, split file1 in 2 modules"
git push

Notify the reviewer(s) that you need a new review

At some point, you need to let the reviewer(s) know that you need a new review from them: to do so, click on the little "reload" icon near the name of the reviewer in the top-right corner of the PR page, it should turn into a yellow dot.

Depending on their Github settings, they should be notified via email, desktop/mobile notification.

:warning: Although this step might seem minor, it's actually quite important to speed up the process and to avoid confusion: if we never notify our reviewers about the PR status, they won't know if it's too soon to review so they might just wait for a long time before eventually having to ask you if the PR is ready, or they might just start a review when actually you're still working on it or planning to do some more adjustments.

Pull request deployment (optional)

In some case your code might be built and deployed on some platform. For example the repository might be a web application that is setup to be deployed on Netlify, in order to actually see a web application running.

By testing the application you or the reviewers might spot some problems, so you might need to push some new commit.

git add file2.py
git commit -m "the /users API now filters out invalid items"
git push

Merge the feature branch

At some point you get the approval to merge from all of your reviewers, yay!

By issuing git log now you'll see all your commits, say (formatted for simplicity here):

ff8d26f Initial commit, added a README
6d5d610 work in progress
363fae5 file renaming
664b7d9 removing an unnecessary file
7fd87a8 added a comment and fixed a typo
fb577e8 implement review changes
c063d3e actually, split file1 in 2 modules
672b05c the /users API now filters out invalid items

On this branch there are now the original commit that we had on dev plus all of the new commits.

Remember, though, that the original issue was titled "Add the new /users API". In the future we will want to come back to when we added this feature and read about it in a single commit: we don't need to know about typos or little changes we did along the way.

So, instead of merging this branch into dev as it is, we want to clean our history a bit and "squash" all of the 7 new commits into one, getting a history like:

ff8d26f Initial commit, added a README
newhash Add the new /users API

Squash the branch before merging

You can do so on the PR page on Github.

Head to the button used to merge the PR: you should see a dropdown to choose how to actually perform the merge: choose Squash and merge:

.

Note that some repositories might have squashing as the only permitted option, so you might find that the drop-down already set to Squash and merge.

At this point you can also edit the commit title and the additional content:

and finally click on Confirm squash and merge.

Also see Squashing via terminal below if you need a bit more control over this process.

Merging `dev` to `main`/`staging`

When it comes to merging dev to main or staging, we do:

git checkout main
git merge dev
git push

To ensure interoperability, rebasing should always happen on (sub)feature branches: we need to make sure to never squash or rebase dev, main or staging because rebasing changes commits hashes, which would make these branches incompatible with the ones we shared on Github (see the example below), which would then cause troubles to others in the team to merge their content in dev.

For this reason, if we are collaborating with others on a repository, we should never find ourselves in a situation where we need to force-push these 3 branches: if we have to, we need to discuss our intention with our collaborators on that repo.

Delete the merged branch(es)

Delete it on the remote

After we merged a branch, it is good practise to remove it from the remote so to: (1) reduce size of cloning/fetching the remote; (2) make for a cleaner output of git log --all --decorate --oneline --graph, (3) reduce the clutter when we're looking for a branch. On Github, the branches menu can become very long making it difficult to find the branch we need to select; on the terminal, issuing git branch can return an unnecessarily long list of old branches.

You can click on the delete icon near the branch you intend to delete in https://github.com/nestauk/YOUR_REPO/branches:

Or use the terminal:

$ git push -d <remote_name> <branch_name>

In our case, this will usually be:

$ git push -d origin <branch_name>

Delete it locally

If you are sure you won't need the branch anymore, in your terminal you can do:

$ git branch -d <branch_name>

Note that your local repository doesn't necessarily know that you have merged this branch on Github, so git might complain that this branch hasn't been fully merged. If you're sure it has been merged, you can use -D which force-deletes the branch irrespective of its merged status:

$ git branch -D <branch_name>

Or you could just fetch and merge the destination branch then delete the branch with -d:

$ git checkout dev
$ git pull
$ git branch -d <branch_name>

Delete multiple stale branches

If you have deleted some branches on Github manually, you can use:

$ git fetch --prune

or just

$ git fetch -p

to remove any remote-tracking references that no longer exist on the remote.

This is especially useful if you're working on multiple machines, to avoid having to run git branch -d <branch_name> for all the branches you might have checked out on those machines.

Appendix

Squashing via terminal

(not recommended for beginners)

To squash via terminal, you count the new commits (7) and run this command:

git rebase -i HEAD~7

or you can just use the hash of the commit you want to start rebasing from. In our case it's ff8d26f Initial commit, added a README, the one we had in dev:

git rebase -i ff8d26f

At this point git will show a message to choose what to do of these commits.

pick 6d5d610 work in progress
pick 363fae5 file renaming
pick 664b7d9 removing an unnecessary file
pick 7fd87a8 added a comment and fixed a typo
pick fb577e8 implement review changes
pick c063d3e actually, split file1 in 2 modules
pick 672b05c the /users API now filters out invalid items

# Rebase ff8d26f..672b05c onto ff8d26f (7 commands)
#
# Commands:
# p, pick <commit> = use commit
# r, reword <commit> = use commit, but edit the commit message
# e, edit <commit> = use commit, but stop for amending
# s, squash <commit> = use commit, but meld into previous commit
# f, fixup <commit> = like "squash", but discard this commit's log message
# x, exec <command> = run command (the rest of the line) using shell
# d, drop <commit> = remove commit
# l, label <label> = label current HEAD with a name
# t, reset <label> = reset HEAD to a label
# m, merge [-C <commit> | -c <commit>] <label> [# <oneline>]
# .       create a merge commit using the original merge commit's
# .       message (or the oneline, if no original merge commit was
# .       specified). Use -c <commit> to reword the commit message.
#
# These lines can be re-ordered; they are executed from top to bottom.
#
# If you remove a line here THAT COMMIT WILL BE LOST.
#
#       However, if you remove everything, the rebase will be aborted.
#
#
# Note that empty commits are commented out

You need to edit this message so to have r (reword) for the first one, and f (fixup) for all the others, getting to something like this:

r 6d5d610 work in progress
f 363fae5 file renaming
f 664b7d9 removing an unnecessary file
f 7fd87a8 added a comment and fixed a typo
f fb577e8 implement review changes
f c063d3e actually, split file1 in 2 modules
f 672b05c the /users API now filters out invalid items

# ...

Then save.

If you have setup an editor, this will present an editor where you can turn the first commit into a multiline commit message like, say:

Add the new /users API.

`GET /users`
=>
{
  amount: 34,
  items: [
    {name: 'a', city: 'London'},
    {name: 'b', city: 'Paris'},
    ...
  ]
}

Note that this filters out invalid items (like, `{name: '', city: 'London'}`).

Closes #5

Save and issue git log to check that you have just 2 commits:

ff8d26f Initial commit, added a README
f83de17 Add the new /users API.

Note that if we setup dev as the main branch by adding Closes #5 we mean to close issue #5 automatically.

If you issue a git status now, git will tell you that this branch has diverged from the same branch on Github (because effectively the two branches have now different histories), so you'll have to "force push".

If you're absolutely sure no one is collaborating with you pushing commits on this branch, you can use:

git push -f

Otherwise, please use:

git push --force-with-lease

(man git-push to learn more)

At this point, to close the pull request you'll need to move to dev, merge the feature branch and push:

git checkout dev
git merge 05_my_feature
git push

Since dev and 05_my_feature will now be equal on Github, the PR will be closed automatically and since we added "Closes #5" the issue number 5 will also be closed automatically by Github.

Example of conflict caused by rebase

Imagine we have dev with 3 commits:

q345saw <- zsvd98u <- spsafd8
                             ^ HEAD (dev)

we branch off featurebranch:

q345saw <- zsvd98u <- spsafd8
                             ^ HEAD (dev, featurebranch)

and add some commits

q345saw <- zsvd98u <- spsafd8 <- s98sd98 <- z34sx3w
                             ^ dev                 ^ HEAD (featurebranch)

Now we go back to dev and squash the last 2 commits (zsvd98u, spsafd8), getting this:

q345saw <- a4s3dwa
                  ^ HEAD (dev)

We now want to merge featurebranch, but now dev and featurebranch have q345saw in common while the content in a4s3dwa and the content in the 2 commits zsvd98u <- spsafd8 are probably conflicting.

Now every time we will want to merge branches that are originating from a commit in dev before the squash (zsvd98u or spsafd8) we'll have to fix those conflicts.

Too many branches, Terminal

$ git branch
08_github
10_daps_dumps
11_data_sources
13_schema_current
14_graph
14_safe
14_safe2
26_imt_related_repos
* 41_workflow_doc
dev
main