Installing and testing previous CGI versions

Setting up a mirror with an archived CGI version is relatively easy and consists of four main steps:

1. get and setup linux VM
2. download and install a mirror with the GBiC (to setup mysql/apache/etc)
3. download the source code
4. compile the code.

Step 1: Get a Virtual Machine Running.

Step 1 is getting a Linux Virtual machine running in some form. I like to run CentOS VMs cause that's what the RR/dev/beta run on. There's a guide for setting up Vagrant here, but you can do it however you want (hint: vagrant up and vagrant ssh where you have a VagrantFile). Once I have the VM running I like to install a few tools and setup a basic .bashrc to make things more convenient. For the case of testing the TrackCheck tool against the old CGI's, I installed the following tools:

sudo yum install -y wget vim rsync libuuid-devel R httpd mysql ant libpng gcc gcc-c++ httpunit jtidy rhino mysql-connector-java libpng-devel mariadb-devel git

Then I added the line "set -o vi" to my .bashrc (totally optional I just like vi mode on the command line). The ant, jtidy, httpunit, and mysql-connector-java programs are all specific to TrackCheck and are not normally required.

Step 2: Install a basic mirror.

After your VM is running and you have all the necessary dependencies installed, the next step is to use the GBiC to install the current CGIs and to get a free Apache/MySQL setup. Only one command is needed (see GBiC testing about /cluster/home/chmalee/gbicLocation):

sudo bash browserSetup.sh -b install

And you should be set. We're gonna overwrite the CGIs in step 4 so no need to worry about them.

Step 3: Download the source.

Download the source code. A zip file of the source is available here:
http://hgdownload.soe.ucsc.edu/admin/

And is installed with the following commands:

wget http://hgdownload.soe.ucsc.edu/admin/jksrc.v361.zip
unzip -q jksrc.v361.zip

Step 4: Compile

Compilation is "easy", but will vary depending on the system. First make libs:

cd kent/src
make libs

If this fails it's probably because a dependency is missing, just read the error message and google for the right library to install.

After a successful make libs (which can take a while), make the CGI directory writable and then make alpha:

sudo chmod -R 777 /usr/local/apache/
cd kent/src
make alpha

Don't worry about making CGI's readable by the world since hopefully this mirror is confined to localhost and will be deleted when you're done testing anyways right? After this step you should have a successfully working mirror, go to hgTracks and check the Title string for the right version number.

TrackCheck specific Steps

After getting a working mirror running, if you want to run TrackCheck you'll need to complete a few extra steps. The first thing is to configure the kent/java build to your system. You'll want to modify the kent/java/build.xml file to point to the correct httpunit, jtidy, mysql-connector and rhino libraries, modify TrackCheck.java itself to query hgcentral instead of hgcentralbeta, and tell ant where the JAVA_HOME variable should go:

1. Change lines 28-31 of build.xml to point to the right locations (probably /usr/share/java/className)
2. Change line 63 of kent/java/src/edu/ucsc/genome/qa/cgiCheck/TrackCheck.java to say hgcentral instead of hgcentralbeta
3. Add the JAVA_HOME env variable to your .bashrc: export JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.161-0.b14.el7_4.x86_64/

Now you should be able to compile TrackCheck:

cd kent/java
./build

This will place a working TrackCheck binary in ~HOME/bin. Which you can then run just as you would on hgwdev. TrackCheck requires an .hg.conf, a local database with a chromInfo table, and a props file to run correctly.

To make the local db:

sudo mysql -e "create database mm10"
sudo rsync hgdownload.soe.ucsc.edu::mysql/mm10/chromInfo.* /var/lib/mysql/mm10/
sudo mkdir -p /gbdb/mm10 # not sure if this or the next line is needed but just in case
sudo rsync hgdownload.soe.ucsc.edu::gbdb/mm10/mm10.2bit /gbdb/mm10/mm10.2bit

An example .hg.conf:

db.host=localhost
db.user=readonly
db.password=access
db.socket=/var/run/mysqld/mysqld.sock
central.db=hgcentral
central.host=localhost
#central.socket=/var/run/mysqld/mysqld.sock

# Be sure this user has UPDATE AND INSERT privs for hgcentral
central.user=readwrite
central.password=update
gbdbLoc1=/gbdb/
gbdbLoc2=http://hgdownload.soe.ucsc.edu/gbdb/
slow-db.host=genome-mysql.soe.ucsc.edu
slow-db.user=genomep
slow-db.password=password

And an example props file for testing mm10:

machine localhost
server localhost
quick false
dbSpec mm10
table all
zoomCount 3

Quick Steps Overview

• Note: this is with Vagrantfile having config.vm.box = "centos/7"
• Also be sure to uncomment the 8080 line to view your mirror at http://127.0.0.1:8080

   1  vagrant up / vagrant ssh
   2  sudo yum install -y wget vim rsync libuuid-devel R httpd mysql ant libpng gcc gcc-c++ httpunit jtidy rhino mysql-connector-java libpng-devel mariadb-devel git
   3  echo "set -o vi" >> .bashrc
   4  source .bashrc
   5  wget https://URL -see /cluster/home/chmalee/gbicLocation- /browserSetup.sh
   6  sudo bash browserSetup.sh -b install
   7  wget http://hgdownload.soe.ucsc.edu/admin/jksrc.v361.zip  (or other v3##.zip) 
   8  unzip -q jksrc.v361.zip  (or the other v##.zip you got)
   9  cd kent/src
  10  make libs &> make.out  
  11  tail make.out  (can check for errors)
  12  sudo chmod -R 777 /usr/local/apache/
  13  make alpha
  14  exit