Getting Started With Git

Getting Started With Git

Git is a modern (SCM) source code management system written by Linus Torvalds. Like all his software, he names it after himself.

(This page is intended for UCSC Genome Browser developers and staff.)
We are currently in the process of migrating from CVS to Git.

Setting Up Your Own Personal Git Kent Repository

To create your personal git repository of the kent source, please use the following simple directions:

 cd $HOME
 
 git config --global user.name "Your Name Here"
 git config --global user.email yourlogin@soe.ucsc.edu

If you like colors:

 git config --global color.diff auto
 git config --global color.status auto
 git config --global color.branch auto

If you are security-conscious:

 chmod 644 ~/.gitconfig 

If you have an existing old test local git repo ~/kentgit, please remove it before proceeding.

 mv kent kent-cvs    # move your old kent directory out of the way
 git clone yourlogin@hgwdev.cse.ucsc.edu:/data/git/kent.git/ kent
 cd kent

/data/git/kent.git is our shared kent repository. We access it via SSH.

Some simple real-world git usage

Here is an example of some git commands used to modify a file, check on it, diff it, check it in, push it up to shared repo:

 vi doSomethingCool.csh
 git diff --help
 git diff  # show detailed diff between working dir and staged
 git diff --stat  # show condensed view of the names of files that changed
 git diff --name-status  # show only the names of files that changed

 git diff --cached # show detailed diff between staged and HEAD of current branch
 git diff HEAD # show detailed diff between working dir and HEAD of current branch
 git diff master origin/master # show detailed diff between local master and central master

 git status     # another way to see info about changes
 git add doSomethingCool.csh   # notice that this adds it our list of things
                           # to be committed together in the next commit.
                           # Whether it is a new file, or just a change, you do a git add.
                           # Several related things can and should be committed at the same time as a unit.
 git commit -m 'made some useful changes ...'  # Commits to your local repo only.
                           # Make the comments useful.
 git diff       # check that the changes got committed
 git status     # another way to see info about changes and so on

 git diff origin   # but what about diffing to the shared-repo?
 git push origin master    # push my change up to be shared with all
 git diff origin   # verify that there are no more outstanding changes
 git status  # check info again

Warning:

 git commit -a      # DO NOT USE. Commits ALL changes in your local-repo (dangerous!)
 

-a automatically adds ANY and ALL changes in your local-repo to the "cache/commit index" list of things to commit for existing tracked files. Unlike CVS, it is not influenced by your current directory location. Any tracked files that you have modified, for instance common.mk will get checked in.

 instead use git add . -u # this gets all tracked changes in current dir and subdirs.
                          # do not forget the -u or it will add ALL files including untracked files.

Sharing Changes With Others

Git is a distributed SCM so it works a bit differently from CVS. Each user has their own local project repository which includes the full history of all changes ever made. This allows one to work offline and had other advantages besides. However, for simplicity there is a shared repository that people push changes to from their local repository. More complex configurations are possible, such as hierarchical. Probably the shared repository approach is fine for our group.

 git pull # equivalent to cvs up -dP, this pulls in changes by others.

 git push # equivalent to cvs commit, only do this when your changes are ready to share with others.

origin refers to the shared repository from which your local repo was cloned.

Stash

Git stash is handy when you are not keeping your sandbox clean with other methods such as using development branches, and you are doing something potentially dangerous such as pulling, merging, or switching branches. Git refuses to lose your stuff due to failures in these operations, so you may be required to use git stash to save away those changes you are not ready to commit yet.

Git stash supports several operations:

git stash # save to stash stack
git stash list  # list all the stashes you have
git stash show <name>  # show --stat level details about stash 
git stash show -p <name>  # show diff details about stash
git stash save <name>  # save a named stash
git stash apply <name> # apply a named stash, i.e. merge it in
git stash drop <name>  # delete the stash
git stash pop          # apply the stash and clear it from the stack
git stash clear        # be careful, wipes out all of your stashes

If when pulling or merging you get an error saying "not uptodate", this is usually because you have changes in your working directory that are in danger of being lost because they are not checked in anywhere.

Should recovery from a failed merge get ugly, you might lose your edits and git refuses to be responsible for that.

Newer versions of git have an improved error message that advises you to commit or stash your working dir changes.

If this error occurs, here are your options:

1. Check it in. If it's ready, don't be afraid of commitment.

git add myfile
git commit -m 'new great thing'
git pull
# note that you may now see a regular merge conflict to be resolved.

2. Use stash.

git stash
git pull 
git stash pop
# note that the pop is a merge and you see a conflict to be resolved.

3. Check it in on another development branch.

git checkout -b brandNewTopic
git add myfile
git commit -m 'new great thing'
git checkout master
# now I can pull or merge
# the myfile changes are no longer in the working dir.
git pull
 

4. Abandon the changes.

# I really didn't want those changes anyway
git checkout myfile  # over-write myfile with last committed version on HEAD.
# can alternatively use git reset commands with caution, see the section on reset.
git pull

5. Copy it aside and restore with checkout or reset(yuck).

# Not recommended.  You can't simply do a unix mv,
# because git will think you deleted the file and that 
# this deletion is just another unchecked-in change.
cp myfile myfileX  # If you try to restore later, BEWARE losing of other peoples changes!!!
git checkout myfile  # over-write myfile with last committed version on HEAD.
# can alternatively use git reset commands with caution, see the section on reset.
git pull

When git makes a stash it is really a commit. The git stash command is primarily a convenience feature. The stash works roughly equivalent to:

git checkout -b mystash0
git add -u    # add all tracked but dirty changes 
git commit -m 'my stash 0'
git checkout master  # return to master branch, or whatever branch you were on.
   # note that there are no more dirty files on either branch

The real stashes are stored in a separate namespace from tags and branches.

Git stash apply is roughly like:

git merge mystash0
git reset HEAD^

CVS Equivalents

More equivalents here

 cvs add somefile ==> git add somefile
 
 cvs commit -m 'comment' somefile ==> git add somefile; git commit -m 'comment'; git push
 
 cvs log somefile  ==> git log somefile
 You may find it useful to pipe some outputs into the "tig" utility.
 
 cvs ann somefile ==> git blame somefile
 You can see who did what when.
 
 cvs up -dP ==>  git pull  
 (git fetch only pulls in new objects but does no merging)
 
 cvsup ==> git status  
 (but nicely git status does not need to update 
 and thereby mess with your working dir to do it)
 
 cvs rm somefile ==> git rm somefile
 
 git mv somepath newpath (cvs has not real equivalent)

git diff

 git diff does a lot of things.
 You can see just names or full details.
 You can diff between different specific commits,
 between branches, between repositories,
 between your sandbox and your commit-list,
 between your commit-list and the head, etc.

Ignoring files

Configuring git to ignore certain files here

Terminology

We say sandbox or working-dir interchangeably here. We also say commit-list, stage, cached, or index, but they all mean the same thing. This is the changes you have said you want in the next commit, but that commit has not been frozen in yet.

Comments

Please use good comments. This is often all people have to go on when looking through the git log commit history. Try to make it a meaninful one-liner if you can. It's worth taking a moment to get it right. And with git, you can fix the comment that is messed up before pushing to central repo and shared history.

 git commit --amend

Branches

Your default branch in your own repository is called master. Because we imported cvs history, we all have a lot of branches already. There is also a master or head branch on the shared-repository.

You can and should easily create additional branches in your local repository. This requires NO TAGGING, and it's fast and convenient. You can switch back and forth between the master branch for a quick fix and some more involved detailed development branch, or make a quick branch to test some idea, or another friends code. It's cheap to leave these local branches, they don't clog up the shared repository, and you can also clean up ones that you no longer need. Merging stuff between branches is usually pretty easy and smooth. Note that if you have outstanding changes that would be lost when switching branches, you can tuck them away with the git stash command. Then you should be able to switch branches. But you need to later use your stash and delete it to tidy up.

Tree-ish

(Do not be alarmed. You are not experiencing perl hell. Breathe deeply.)

 You can use a hashId from a commit.
 You can use a symbol such as a tag or a branch-tag.
 You can use tree-ish commands:
 Remember that merge commits have two parents,
 the first parent is the mainline branch master^1,
 the second parent is the other merged-in branch master^2.
 
 master^2^1 means the first parent of the second parent of master.
 master^1 == master^
 master^^^ == master~3
 master^  == master~ == master^1 == master~1
 
 You can use this to choose ancestors relative
 to a branch-tag or other symbol or SHA1-id.
 Use git rev-parse <some-tree-expression> 
 to actually resolve a tree-ish expression into a specific hash-id.

Git Diff and Git Log

 git diff and git log (and other commands too) use tree-ish.
 But they use it in different ways.
 
 git diff x..y means give me the diff from x to y (but not including x itself).
 It simply does a diff between those two commit-endpoints.
 It does not consider the exact history between them.
 Notice that git diff y..x is the inverse of x..y
 so that insertion becomes deletion etc.
 
 Unlike git diff, git log is concerned with all 
 the history between the two points.
 git log x..y means git log ^x y.
 ^x means not in x (^==NOT==EXCLUSION)
 Notice that the caret(^) is on the LEFT.
 So git log ^x y means y and its ancestors
 not including x and its ancestors.
 But git log y..x is nothing like git log x..y
 git log y..x means git log ^y x which means
 x and its ancestors not including y and its ancestors.
 
 So x..y means very different things to diff and log.
 
 There is a x...y (triple dots) also for both diff and log.
 In the case of git log, x...y means all things
 that are in x and its ancestors 
 and y and its ancestors, but not in any of their common ancestors.
 
 git diff x...y means find common ancestor of x and y,
 and then diff from there to y.

 Here is a handy diff if you have been working on a branch,
 and you do not want to have to do a git pull from master,
 you can use this method to see only the changes ON YOUR BRANCH,
 and ignore changes on master since your branch started:

 git diff --stat origin/master...HEAD
 git log --stat ^origin/master HEAD

Re-basing

Don't go crazy with re-basing. It's not usually necessary.

Another important rule is, do not change shared history.

Re-basing is a special technique used in your local repo before you push your changes to the shared repo. It allows you make the history appear tidier and more linear. For instance, you may have been working and checked in 3 small closely related changes. You want to just have them all be one single commit with a good comment. Re-basing is one way.

Re-basing can also be used to linearize the history, which is sometimes helpful for making a nice change list. It basically takes your changes since your branch was forked off, updates all your changes and re-writes them as if they had been patched in after the current shared state. This creates a simpler merge and makes reading the shared history easier. There is some fluidity to changes in your own repo, but once it gets pushed up to the shared repo, it's not so easy to change, in part because everyone's history would have to be modified at that point.

Quick Repo Updates

CVS update was getting very slow because the source had grown to thousands of files and CVS update has to check each one for changes. With git, a change log is kept and only new changes need to be processed. This is usually very fast.

No undetected corruption

Git is also big on making sure digital corruption does not creep in, and it will detect it automatically if it happens. Everytime that any object is accessed in git, it runs the SHA1 has on the extracted content and compares it to its ID. If any corruption has occurred, either accidentally or intentionally, you will be informed right away.

No more hanging locks

No more concerns about hanging CVS locks, for example:

 cvs log somefile | more

You are working in your own repo, and have full access to all the history. You are not holding up anybody else.

Mirror site access to git repository

Mirror sites will read-only access here:

 git://genome-source.soe.ucsc.edu/kent.git

If you have firewall issues, this will also be provided:

 http://genome-source.soe.ucsc.edu/kent.git

External git Documentation

The official git home page, documentation, and tutorial. Another interesting tutorial.

Installing git on a Mac OS

See also: Installing git for other operating systems.

$ sudo port install git-core

That seems pretty straight forward, but the problem is you may not have the 'port' command installed yet. That is a little more involved. To get this "MacPorts' system installed, please follow their installation procedures at: Guide MacPorts. This could be an extensive procedure depending upon what you may not yet have installed on your Mac since it needs the X11 system and the Xcode tools installed to get MacPorts installed. But once you have this 'port' command, you can install a vast array of interesting software quite easily.

Genome Browser Mirror Sites

Move your existing cvs kent directory out of the way:

mv kent kent.cvs

Start a new kent git repository. This clone command will establish a new kent directory:

git clone git://genome-source.cse.ucsc.edu/kent.git

If you have a firewall that interferes with that operation, use http:

git clone http://genome-source.cse.ucsc.edu/kent.git

Then mark your repository with the beta tag so it will track along with our beta releases:

git checkout -t -b beta origin/beta