Genome Browser in a box config: Difference between revisions

From genomewiki
Jump to navigationJump to search
Line 40: Line 40:
You can then do a "ssh devtunnel" to get connected to hgwdev and have the ports on your machine 1234 and 1236 forwarded to the boxes.
You can then do a "ssh devtunnel" to get connected to hgwdev and have the ports on your machine 1234 and 1236 forwarded to the boxes.


== Compile binaries in browserbox-dev ==
== Box release cycle ==
 
We have the usual three stage release cycle for the box:
* development happens on the devbox, a development version of the browserbox with git, gcc etc.
* for alpha testing, a special command on the master box can pull in the CGIs that were compiled on the dev box
* for beta testing, a special update target on the box can pull CGIs that were compiled by the buildmeister
* for public release, the box in this beta state is cleaned and zipped. This zip can be tested on Windows or OSX. This zip is then pushed to hgdownload.
 
== development on browserbox-dev with testing on browserbox-dev or the browserbox ==


We have a three 1/2 stage release cycle: devbox correspond to hgwdev, browserbox corresponds to beta, browserboxBeta.zip is a version of beta that can be tested on a OSX machine, the final zip file corresponds to the RR.
start browserbox-dev:
start browserbox-dev:
   devStart
   devStart
Line 50: Line 56:
   ssh devbox compileCgis
   ssh devbox compileCgis


copy from devbox to beta distribution rsync server directory (this is defined in qateam's .bashrc):
You can now test the CGIs on port 1236 of hgwdev. You can also hack on the /home/browser/kent directory and e.g. run makes in there. To do a more realistic test, this can also be tested on the final browserbox.
   devUpdateRsync
 
Copy from devbox to alpha distribution rsync server directory (this command is defined in qateam's .bashrc):
   boxUpdateAlpha
 
Now start the box and pull in the alpha binaries from the rsync server:
  boxStart
  ssh box updateBrowser alpha
 
You can now test the alpha CGIs on port 1234 of hgwdev.
 
When you're done: you can stop the development box:
  devStop
 
== beta testing on the box ==
 
To test the current beta CGIs on the box:
  boxUpdateBeta
 
This will copy the cgi-bin-beta from hgwdev into the beta distribution directory.
 
  ssh box updateBrowser beta
 
Now you can test the beta CGIs via port 1234 on hgwdev.
 
== Release ==
 
To release the box:
  boxRelease


stop the box:
This will ssh into the box, clean it (remove temp files, sessions, etc), stop it, zip it into a zipfile and place it onto hgwdev on http://hgwdev.soe.ucsc.edu/browserbox/browserbox.zip
  boxStop
This zipfile can then be pushed to hgdownload.


== Test binaries in browserbox ==
== Test binaries in browserbox ==

Revision as of 21:24, 9 April 2014

Internal config

The Gbib is a ubuntu 13 linux with apache, mysql and the default tools. See Preparing_VirtualBox_images

Hgwdev has virtualbox installed and two virtual machines (VMs) added: browserbox and browserbox-dev. Browserbox-dev can be used for development of box-specific patches. browserbox is the "master" image of the gbib.

The virtual machines live in qateam/VirtualBox VMs/. The central virtualbox xml config file where they are registered is ~/.config/VirtualBox/VirtualBox.xml

qateam has aliases to start and stop the VMs: boxStart / boxStop and devStart / devStop.

Once started, browserbox listens at port 1234 for web requests and at 1235 for ssh. browserbox-dev listens at 1236 for web requests and at 1237 for ssh.

SSH

qateam has these entries in their .ssh/config:

 Host devbox
 Hostname localhost
 Port 1237
 User browser
 Host box
 Hostname localhost
 Port 1235
 User browser

So you can do "ssh box" or "ssh devbox" on hgwdev to connect to the boxes.

There is no password, because the qateam ssh key was added to both boxes (see qateam's "scpdsa" command)

Connecting to the boxes from your desktop machine

browserbox and browserbox-dev both accept only connections from hgwdev, not from outside. You cannot connect from your desktop machine to any box. So ssh on your desktop has to be told to forward a few ports to hgwdev. Once you start ssh, it will listen to some ports on your machine, all data gets tunneled through the ssh connection and appears on hgwdev on the localhost interface, where hgwdev will forward it to the browserbox. To make this happen, add this to your file ~/.ssh/config on your own OSX or linux machine:

 Host hgwdevtunnel
 Hostname hgwdev.sdsc.edu
 User <YOURHGWDEVUSER e.g. rhead>
 LocalForward 127.0.0.1:1234 127.0.0.1:1234
 LocalForward 127.0.0.1:1236 127.0.0.1:1236

You can then do a "ssh devtunnel" to get connected to hgwdev and have the ports on your machine 1234 and 1236 forwarded to the boxes.

Box release cycle

We have the usual three stage release cycle for the box:

  • development happens on the devbox, a development version of the browserbox with git, gcc etc.
  • for alpha testing, a special command on the master box can pull in the CGIs that were compiled on the dev box
  • for beta testing, a special update target on the box can pull CGIs that were compiled by the buildmeister
  • for public release, the box in this beta state is cleaned and zipped. This zip can be tested on Windows or OSX. This zip is then pushed to hgdownload.

development on browserbox-dev with testing on browserbox-dev or the browserbox

start browserbox-dev:

 devStart

connect + git pull + compile:

 ssh devbox compileCgis

You can now test the CGIs on port 1236 of hgwdev. You can also hack on the /home/browser/kent directory and e.g. run makes in there. To do a more realistic test, this can also be tested on the final browserbox.

Copy from devbox to alpha distribution rsync server directory (this command is defined in qateam's .bashrc):

 boxUpdateAlpha

Now start the box and pull in the alpha binaries from the rsync server:

 boxStart
 ssh box updateBrowser alpha

You can now test the alpha CGIs on port 1234 of hgwdev.

When you're done: you can stop the development box:

 devStop

beta testing on the box

To test the current beta CGIs on the box:

 boxUpdateBeta

This will copy the cgi-bin-beta from hgwdev into the beta distribution directory.

 ssh box updateBrowser beta
 

Now you can test the beta CGIs via port 1234 on hgwdev.

Release

To release the box:

 boxRelease

This will ssh into the box, clean it (remove temp files, sessions, etc), stop it, zip it into a zipfile and place it onto hgwdev on http://hgwdev.soe.ucsc.edu/browserbox/browserbox.zip This zipfile can then be pushed to hgdownload.

Test binaries in browserbox

On your desktop machine:

 ssh hgwdevtunnel

on hgwdev, start browserbox:

 boxStart

wait a few seconds, then:

 ssh box
 updateBrowser beta

this will put the cgis and htdocs that were rsync'ed to hgwdev into the browserbox.

You can now connect from your desktop to http://127.0.0.1:1234 and test the CGIs.

If they are OK:

 sudo /root/cleanVm
 logout
 boxStop