Git shared repository: Difference between revisions
(Created page with "Category:git Category:git shared repository The git shared repositories used by the genome browser staff and genecats is located at hgwdev:/data/git Inside there are va...") |
No edit summary |
||
(9 intermediate revisions by 3 users not shown) | |||
Line 5: | Line 5: | ||
is located at hgwdev:/data/git | is located at hgwdev:/data/git | ||
Inside there are various bare repositories including kent.git. | Inside there are various bare repositories including kent.git. See this [http://genomewiki.ucsc.edu/index.php/Getting_Started_With_Git Getting Started With Git] page for more info. | ||
There are cron jobs that update various copies | There are cron jobs that update various copies | ||
Line 16: | Line 16: | ||
From Galt's crontab: | From Galt's crontab: | ||
<pre> | |||
# update git public repo | # update git public repo | ||
0,15,30,45 * * * * /data/git/public/refresh.csh | 0,15,30,45 * * * * /data/git/public/refresh.csh | ||
0 * * * * /data/git/backup/refresh.csh | 0 * * * * /data/git/backup/refresh.csh | ||
</pre> | |||
Periodically (at least every 6 months, but maybe more often) | Periodically (at least every 6 months, but maybe more often) | ||
Line 28: | Line 31: | ||
And then git regular clone was only 3 seconds! | And then git regular clone was only 3 seconds! | ||
<pre> | |||
# Send out announcement to genecats and browser-staff | # Send out announcement to genecats and browser-staff | ||
# that we will be doing git maintenance for 5 minutes. | # that we will be doing git maintenance for 5 minutes. | ||
Line 43: | Line 47: | ||
cd .. | cd .. | ||
mv kent-real.git kent.git | mv kent-real.git kent.git | ||
</pre> | |||
That's it! | That's it! | ||
Line 53: | Line 58: | ||
It is preferable to perform this garbage collection late at night | It is preferable to perform this garbage collection late at night | ||
when few people are working. | when few people are working. | ||
---- | |||
Garbage Collection Done: | |||
* 2012-07-23 | |||
* 2013-03-26 | |||
* 2013-07-29 | |||
* 2014-08-08 | |||
* 2015-04-03 | |||
* 2016-01-04 | |||
* 2016-09-07 | |||
* 2018-04-16 | |||
(I did it but did not get logged here) | |||
* 2020-05-27 |
Latest revision as of 01:42, 28 May 2020
The git shared repositories used by the genome browser staff and genecats
is located at hgwdev:/data/git
Inside there are various bare repositories including kent.git. See this Getting Started With Git page for more info.
There are cron jobs that update various copies of kent.git such as the hourly copies that are refreshed with git pull, and a copy which is updated every 15 minutes that the general public fetches from, both directly with git, and with web-git. In addition the admins keep daily, weekly, monthly, and yearly backups of hgwdev:/data/git.
From Galt's crontab:
# update git public repo 0,15,30,45 * * * * /data/git/public/refresh.csh 0 * * * * /data/git/backup/refresh.csh
Periodically (at least every 6 months, but maybe more often)
git needs to garbage-collect the shared bare repository.
Not ONLY to save space, it ends up saving time too.
It had grown to 2GB! After gc, it was only 288MB.
Also, it had taken git bare clone took 13 minutes to run.
After gc, the git bare clone took less than 1 second.
And then git regular clone was only 3 seconds!
# Send out announcement to genecats and browser-staff # that we will be doing git maintenance for 5 minutes. ssh hgwdev cd /data/git # move the real repo aside so nobody can use it. mv kent.git kent-real.git # get rid of the old copy if any rm -fr kent.git.copy # make a new backup copy just to be safe rsync -a kent-real.git kent.git.copy cd kent-real.git # only takes 5 minutes git --bare gc cd .. mv kent-real.git kent.git
That's it! It doesn't seem that the other git repos in /data/git/ get enough commits to even bother with gc.
Apparently, when we run a git on a non-bare clone such as your personal ~/kent, it does gc automatically.
It is preferable to perform this garbage collection late at night when few people are working.
Garbage Collection Done:
- 2012-07-23
- 2013-03-26
- 2013-07-29
- 2014-08-08
- 2015-04-03
- 2016-01-04
- 2016-09-07
- 2018-04-16
(I did it but did not get logged here)
- 2020-05-27