Genome Browser in a Box config
The Gbib is a Ubuntu 14 (=supported until late 2019) VirtualBox VM with apache, mysql and the genome browser (see Preparing_VirtualBox_images). It has a special hg.conf, uses the tableList tables in all organism DBs and loads most files from hgdownload on the fly.
For QA release instructions, see Gbib release
Development for the Gbib is a different business and described in Gbib development.
Administrators of the Gbib image might find the page Preparing_VirtualBox_images worth a look.
The Gbib update process is described in detail on a separate page Gbib updates
Updating a box
We do not make changes to our GBIB master image. Changes find their way the GBIBs out there through changes to the push directories on hgdownload or - usually - through changes to the updateBrowser.sh script which updates a GBIB to our current GBIB state. since updateBrowser.sh is also run on the buildmeister's GBIB, our own GBIB master image is always current.
On the box, the update is started by /root/updateBrowser.sh.
By default, it updates the box to "release" state, which is the one on hgdownload, but can also update to "devbox", "alpha", or "beta" states, these can be prepared by developers or QA during testing, see Gbib release
The script updates various directories on the box:
- cgi-bin: from hgdownload (or from hgwdev, if alpha/beta/devbox state)
- htdocs: from hgdownload (or from hgwdev, if alpha/beta/devbox state)
- mysql: always from hgdownload
- additional files from a "push" directory on our servers: e.g. additional icons, the tableListAdd.*.tab files in /root and a special hgNear script. Depending on state, from hgdownload or hgwdev
- html patches: Remove menu items, change some text. Static code.
- some mysql tables (via hgMirror): remove some tracks from search, hide some tracks, load /root/tableListAdd.* etc. Calls hgMirror for this.
protected tracks (see below) are not updated.
Protected tracks
protected tracks are tracks that are not on hgdownload but are available from the box. Currently these are OMIM and hgmd, but will hopefully include LOVD and decipher one day.
These tracks are missing from the tableList table in hg19, so their tableList info is in /root/tableListAdd.hg19.tab and is added to tableList after each rsync. This is done by hgMirror, (is called from updateBrowser.sh), as hgMirror includes all local post-rsync database changes.
The track tables are added manually to the box once by developers and are not changed by updateBrowser.sh.
OMIM is a copy of the OMIM tables (copied in Apr 2014).
The HGMD table is a pointer to a bigbed file on hgdownload. Our binaries include code to add a password when this file is loaded via https. This password is only compiled into the code when the compilation is done on a system with access to the hive and the password is not part of the source tree. See /hive/groups/browser/gbib/. This track is the only one that is even shown when GBIB is in off-line mode.
Auto updates
The updateBrowser.sh script is called by cron every five minutes. This can be changed with autoUpdatesOff and autoUpdatesOn as user "browser" (see ~browser/.bashrc).
The updateBrowser.sh script checks if a flagfile on hgdownload has changed relative to a local file. If it has not been changed, then the script just stops.
To switch auto update off before you even start up the GBiB virtual machine, run this command:
VBoxManage guestproperty set browserbox gbibAutoUpdateOff yes
Brian: It makes me think of the movie Inception. It's as if we're reaching into our sleeping GBiB VM and planting ideas there to make it do different things when it wakes up. :-)
Pushing files into the box
If a CGI release requires other files, outside the cgi-bin directory, or special software on the box, these files have to be pushed into the box.
To push a file into the box, on hgwdev, copy it into /usr/local/apache/gbib/push/ in many cases you will have to create the right subdirectory. Note that /usr/local/apache/gbib/push/ is the "root" of the box. After each update, the box will put the files and subdirs from /usr/local/apache/gbib/push/ into /.
You can test if this works by running as qateam on hgwdev
boxPrepare beta
and then on the box
updateBrowser beta
If it works, tell the pushers to push the push directory onto hgdownload. On the next update of the box (usually Sunday), all boxes will pull the push directory into their root directory.
If an update requires an ubuntu package, you will have to add "apt-get install" commands to updateBrowser.sh and put the new updateBrowser.sh script onto hgdownload again.
