Gbib updates

The Gbib contains a script /root/updateBrowser.sh. It updates itself, the directories gbdb, cgi-bin, htdocs, mysql databases and a few other files across the gbib's filesystem (gbib "push" directory).

To aim of this update process is to keep the CGIs, htdocs, gbdb and mysql files current with the version on hgdownload or with the versions prepared on hgwdev if run with parameter "alpha" or "beta".

It is run from various places:

• manually, via the script /home/browser/updateBrowser, which uses sudo without a password prompt. You can specify a parameter here, "beta" or "alpha" (see below)
• upon boot, with the output directed to the main terminal, without a parameter.
• every 4 minutes via a cronjob, without a parameter. The cron job has a delay of 120 seconds, to make sure that the invocation on the terminal makes it first. The cronjob also has a delay of a random number of seconds < 30 to spread out the hits to our rsync server.

The script goes through these steps:

1. It first checks if it's run as root. It stops if not.
2. It then checks if it's already running. If it seems to be already running, it stops.
3. The script then compares the time of http://hgdownload.cse.ucsc.edu/gbib/lastUpdate via a HTTP HEAD request and checks if the URL is more recent than the time of the file /root/lastUpdateTime.flag. If not, it stops. After a successful update, it sets the time of /root/lastUpdateTime.flag to the current time.
4. It will then download a new copy of itself from the source server. This server is hgdownload by default, or hgwdev if run with parameter "alpha" or "beta" and run this updated copy.
5. It checks and stops if a hgMirror job is running.
6. It updates the CGIs and htdocs either via rsync from hgdownload or by downloading/extracting a tarball from hgwdev.
7. It updates the /gbdb files
   1. note: /gbdb, mysql and hgcentral always come from hgdownload because cluster-admon doesn't want to have an open rsyncd on hgwdev and QA might be working off campus
8. It updates the mysql files
9. It updates hgcentral
10. It copies all other files (gbib.css, gbib icons, updated config files, if needed: new ubuntu packages, if needed: other gbib specific patches) from hgdownload or hgwdev from their "gbib/push" directories into the "/" directory of gbib
11. it modifies the main and the left side menu (remove visiGene, neandertal, galaxy, etc), and rewrites the main page (e.g. replace UCSC Genome browser with "UCSC Genome Browser in a Box")
12. it modifies the contacts page to have a link to UCSC, not to the gbib
13. it makes sure the tableList is not in our local copy of hgcentral
14. it hides the conservation and retroAli tracks in hg19 trackDb
15. it fixes all mysql tables in hg19, hgFixed and hgcentral, as they are often in a crashed state on hgdownload (as the server's tables might not be flushed when the copy is made)
16. it runs hgMirror to let it do "postRsyncUpdates" (these are part of hgMirror, because they also have to be run after every hgMirror download)
    1. To improve performance of search, hide some tracks from hgTracks feature searches (e.g. the dozens of outdated gencode tracks)
    2. To improve performance, hide some tracks (e.g. intronEst)
    3. update the local tableList with protected tracks that exist only in gbib, but not on hgdownload. At the moment, this is only HGMD and OMIM.
17. it touches /root/lastUpdateTime.flag