Gbib development

This page is for developers, it's not relevant for QA. The QA page is Genome_Browser_in_a_box_config.

The devbox listens to http on 1236 and for ssh on 1237.

Box-specific development can happen on the devbox, a development version of the browserbox with git, gcc etc. For testing during development, a special command on the box can pull in the CGIs that were compiled on the devbox.

(sidenote: All virtual machines live in qateam/VirtualBox VMs/. The central virtualbox xml config file where they are registered is ~/.config/VirtualBox/VirtualBox.xml )

To start the devbox, you need to login as qateam onto hgwdev.

From hgwdev, you can then connect to the box with "ssh devbox". This works, because of the .ssh/config file:

 Host devbox
 Hostname localhost
 Port 1237
 User browser

You can start and stop browserbox-dev with the aliases devStart and devStop.

Setup your laptop to connect to the devbox

add this to .ssh/config on your laptop:

 Host devtunnel
 Hostname hgwdev.soe.ucsc.edu
 User max
 # local forward to gbib running on hgwdev, using port 4321 to avoid clashing with the gbib running on your own machine
 LocalForward 127.0.0.1:4321 127.0.0.1:1234
 # local forward to devbox running on hgwdev
 LocalForward 127.0.0.1:1236 127.0.0.1:1236

Commented walkthrough of a GBib specific bugfix using the devbox

start browserbox-dev as qa:

 devStart

connect from hgwdev + git pull + compile with coredumps:

 ssh devbox
 cd kent/src
 git remote set-url origin ssh://YOURUSERNAME@hgwdev.cse.ucsc.edu/data/git/kent.git/
 git pull
 make docs
 make cgi -j8 COREDUMP=1

setup the tunnel on port 1236 to hgwdev from your laptop (do this in a 2nd terminal):

 ssh devtunnel

You can now test the CGIs on the devbox by going to this URL

 http://127.0.0.1:1236

You can browse around on this site. It's slightly less configured than gbib (colors, icons etc) but the CGIs are the same. As soon as you go to the URL that fails in your browser e.g.:

 http://127.0.0.1:1236/cgi-bin/hgFileUi?db=hg19&g=wgEncodeAwgSegmentation

.. there will be no error message, but the CGI crashed with "abort()" and it will core dump.

This should produce a new core file on the devbox:

 ll /data/trash/core/core
 -rw------- 1 www-data www-data 73338880 Oct  3 08:07 /data/trash/core/core

Run gdb with the core file (the sudo password is "browser"):

 sudo gdb /usr/local/apache/cgi-bin/hgFileUi /data/trash/core/core

A typical solution is to replace cloneString with hReplaceGbdb at places where files in /gbdb are opened. Fix the bug and test after recompile.

Then don't forget to checkout the master branch and commit there (!):

 git checkout master
 git add hg/lib/fileUi.c
 git commit
 git push
 

Now you can file a build patch with this commit id.

testing of alpha binaries on a real gbib

To do a more realistic test, the alpha binaries can also be put into the final browserbox.

To copy the alpha binaries into the browserbox: Copy them from the devbox to alpha distribution rsync server directory (this command is defined in qateam's .bashrc) by running this command as qateam on hgwdev while the devbox is running (this will ssh into the devbox and then copy the binaries onto hgwdev):

 boxUpdateAlpha

Now start the gbib running on hgwdev and pull the alpha binaries from the rsync server:

 boxStart # will show an error message if already running
 ssh box
 updateBrowser alpha

Or you can run on the gbib on your own machine and type:

 updateBrowser alpha

You can then test the alpha CGIs in the "real" gbib environment via 127.0.0.1:1234. Don't forget to start the ssh tunnel to hgwdev if you're using the gbib on hgwdev.

Pushing your changes to hgwdev from the devbox

Direct - easier: set username for kent repo:

 cd kent
 git remote set-url origin ssh://YOURUSERNAME@hgwdev.cse.ucsc.edu/data/git/kent.git/
 git push
 # You can always set it back to the public repo
 git remote set-url origin git://genome-source.cse.ucsc.edu/kent.git

Indirect: simply rsync or scp to copy into your own homedir on hgwdev and commit there:

 rsync file USER@hgwdev:kent/PATH/FILE
 logout