Genome Browser in a box config

Virtual machines

The Gbib is a ubuntu 13 linux with apache, mysql and the default tools. See Preparing_VirtualBox_images

Hgwdev has virtualbox installed and two virtual machines (VMs) added: browserbox and browserbox-dev. Browserbox-dev can be used for development of box-specific patches, it is not relevant for QA. browserbox is the "master" image of the gbib and used by QA.

qateam has aliases to start and stop the VMs: boxStart / boxStop

Once started, browserbox listens at port 1234 for web requests and at 1235 for ssh

SSH

qateam has these entries in their .ssh/config:

 Host devbox
 Hostname localhost
 Port 1237
 User browser

 Host box
 Hostname localhost
 Port 1235
 User browser

So you can do "ssh box" or "ssh devbox" on hgwdev to connect to the boxes.

There is no password, because the qateam ssh key was added to both boxes (see qateam's "scpdsa" command)

Connecting to the boxes from your desktop machine

browserbox and browserbox-dev both accept only connections from hgwdev, not from outside. You cannot connect from your desktop machine to any box. So ssh on your desktop has to be told to forward a few ports to hgwdev. Once you start ssh, it will listen to some ports on your machine, all data gets tunneled through the ssh connection and appears on hgwdev on the localhost interface, where hgwdev will forward it to the browserbox. To make this happen, add this to your file ~/.ssh/config on your own OSX or linux machine:

 Host hgwdevtunnel
 Hostname hgwdev.sdsc.edu
 User <YOURHGWDEVUSER e.g. rhead>
 LocalForward 127.0.0.1:1234 127.0.0.1:1234
 LocalForward 127.0.0.1:1236 127.0.0.1:1236

You can then do a "ssh devtunnel" to get connected to hgwdev and have the ports on your machine 1234 (or 1236) forwarded to the boxes. Port 1234 is a http port, so you can then work the browserbox by going to http://127.0.0.1:1234

Box release cycle

• for beta testing, a special update target on the box can pull in the beta CGIs that were compiled by the buildmeister
• for public release, the box in this beta state is cleaned and zipped. This zip can be tested on Windows or OSX. This zip is then pushed to hgdownload.

Development can happen on the devbox, a development version of the browserbox with git, gcc etc. For testing during development, a special command on the box can pull in the CGIs that were compiled on the devbox.

Beta testing

Copy cgi-bin-beta from hgwdev into the beta distribution directory of the box:

 ssh box updateBrowser beta

Setup the tunnel:

 ssh hgwdevtunnel

Now you can test the beta CGIs via http://127.0.0.1:1234.

Release

To release the box:

 boxRelease

This will ssh into the box, clean it (remove temp files, sessions, etc), stop it, zip it into a zipfile and place it onto hgwdev on http://hgwdev.soe.ucsc.edu/browserbox/browserbox.zip This zipfile can then be pushed to hgdownload.

development on browserbox-dev with testing on browserbox-dev or the browserbox

This section is for developers. It's not relevant for QA.

You can start and stop browserbox-dev with the aliases devStart and devStop. Browserbox-dev listens at 1236 for web requests and at 1237 for ssh.

All virtual machines live in qateam/VirtualBox VMs/. The central virtualbox xml config file where they are registered is ~/.config/VirtualBox/VirtualBox.xml

start browserbox-dev:

 devStart

connect + git pull + compile:

 ssh devbox
 compileCgis

You can now test the CGIs on port 1236 of hgwdev. You can also hack on the /home/browser/kent directory and e.g. run makes in there.

To do a more realistic test, the alpha binaries can also be put into the final browserbox.

To copy the alpha binaries into the browserbox: Copy them from devbox to alpha distribution rsync server directory (this command is defined in qateam's .bashrc) by running this command as qateam:

 boxUpdateAlpha

Now start the box and pull the alpha binaries from the rsync server:

 boxStart
 ssh box
 updateBrowser alpha

You can now use the alpha CGIs on port 1234 of hgwdev.

When you're done: stop the development box:

 devStop