Genome Browser in a box config

Internal config

The Gbib is a ubuntu 13 linux with apache, mysql and the default tools. See Preparing_VirtualBox_images

Hgwdev has virtualbox installed and two virtual machines (VMs) added: browserbox and browserbox-dev. Browserbox-dev can be used for development of box-specific patches. browserbox is the "master" image of the gbib.

The virtual machines live in qateam/VirtualBox VMs/. The central virtualbox xml config file where they are registered is ~/.config/VirtualBox/VirtualBox.xml

qateam has aliases to start and stop the VMs: boxStart / boxStop and devStart / devStop.

Once started, browserbox listens at port 1234 for web requests and at 1235 for ssh. browserbox-dev listens at 1236 for web requests and at 1237 for ssh.

SSH

qateam has these entries in their .ssh/config:

 Host devbox
 Hostname localhost
 Port 1237
 User browser

 Host box
 Hostname localhost
 Port 1235
 User browser

So you can do "ssh box" or "ssh devbox" on hgwdev to connect to the boxes.

There is no password, because the qateam ssh key was added to both boxes (see qateam's "scpdsa" command)

Connecting to the boxes from your desktop machine

browserbox and browserbox-dev both accept only connections from hgwdev, not from outside. You cannot connect from your desktop machine to any box. So ssh on your desktop has to be told to forward a few ports to hgwdev. Once you start ssh, it will listen to some ports on your machine, all data gets tunneled through the ssh connection and appears on hgwdev on the localhost interface, where hgwdev will forward it to the browserbox. To make this happen, add this to your file ~/.ssh/config on your own OSX or linux machine:

 Host hgwdevtunnel
 Hostname hgwdev.sdsc.edu
 User <YOURHGWDEVUSER e.g. rhead>
 LocalForward 127.0.0.1:1234 127.0.0.1:1234
 LocalForward 127.0.0.1:1236 127.0.0.1:1236

You can then do a "ssh devtunnel" to get connected to hgwdev and have the ports on your machine 1234 (or 1236) forwarded to the boxes. Port 1234 is a http port, so you can then work the browserbox by going to http://127.0.0.1:1234

Box release cycle

• for beta testing, a special update target on the box can pull in the beta CGIs that were compiled by the buildmeister
• for public release, the box in this beta state is cleaned and zipped. This zip can be tested on Windows or OSX. This zip is then pushed to hgdownload.

Development can happen on the devbox, a development version of the browserbox with git, gcc etc. For testing during development, a special command on the box can pull in the CGIs that were compiled on the devbox.

Beta testing

Copy cgi-bin-beta from hgwdev into the beta distribution directory of the box:

 ssh box updateBrowser beta
 

Now you can test the beta CGIs via port 1234 on hgwdev.

Release

To release the box:

 boxRelease

This will ssh into the box, clean it (remove temp files, sessions, etc), stop it, zip it into a zipfile and place it onto hgwdev on http://hgwdev.soe.ucsc.edu/browserbox/browserbox.zip This zipfile can then be pushed to hgdownload.

Test binaries in browserbox

On your desktop machine:

 ssh hgwdevtunnel

on hgwdev, start browserbox:

 boxStart

wait a few seconds, then:

 ssh box
 updateBrowser beta

this will put the cgis and htdocs that were rsync'ed to hgwdev into the browserbox.

You can now connect from your desktop to http://127.0.0.1:1234 and test the CGIs.

If they are OK:

 sudo /root/cleanVm
 logout
 boxStop

development on browserbox-dev with testing on browserbox-dev or the browserbox

start browserbox-dev:

 devStart

connect + git pull + compile:

 ssh devbox compileCgis

You can now test the CGIs on port 1236 of hgwdev. You can also hack on the /home/browser/kent directory and e.g. run makes in there. To do a more realistic test, this can also be tested on the final browserbox.

Copy from devbox to alpha distribution rsync server directory (this command is defined in qateam's .bashrc):

 boxUpdateAlpha

Now start the box and pull in the alpha binaries from the rsync server:

 boxStart
 ssh box updateBrowser alpha

You can now test the alpha CGIs on port 1234 of hgwdev.

When you're done: you can stop the development box:

 devStop