Make your own virtual space

So you want to have your own copies of the binaries and your own hgwdev-user.cse.ucsc.edu browser? Here is a walkthrough:

Request a "virtual space" from the cluster-admins:

This includes, among other things, adding a block to the file /usr/local/apache/conf/httpd.conf

<VirtualHost 128.114.xx.yyy>
 ServerName hgwdev-myusername.cse.ucsc.edu
 ServerAlias hgwdev-myusername
 ScriptAlias /cgi-bin/ "/usr/local/apache/cgi-bin-myusername/"
</VirtualHost>

Populate my /usr/local/apache/cgi-bin-myusername/ directory for the first time by doing the following:

• Obtain a copy of the file .bashrc from another QA engineer. Copy this file into your home directory at /cluster/home/myusername

• Set up your own personal Kent repository by following the instructions here: http://genomewiki.ucsc.edu/index.php/Getting_Started_With_Git#Setting_Up_Your_Own_Personal_Git_Kent_Repository

• Set your environment variables by typing the following:

[myusername@hgwdev ~]$ MYSQLINC=/usr/include/mysql
[myusername@hgwdev ~]$ MYSQLLIBS="/usr/lib64/mysql/libmysqlclient.a -lz" 
[myusername@hgwdev ~]$ MACHTYPE=x86_64
[myusername@hgwdev ~]$ HGCGI=/usr/local/apache/cgi-bin
[myusername@hgwdev ~]$ export MYSQLINC MYSQLLIBS MACHTYPE HGCGI

(For more on environment variables, see the article at http://genomewiki.ucsc.edu/index.php/Build_Environment_Variables)

• From your home directory on hgwdev /cluster/home/myusername type the following things:

[myusername@hgwdev ~]$ cd kent
[myusername@hgwdev ~/kent]$ git pull
[myusername@hgwdev ~/kent]$ cd src
[myusername@hgwdev src]$ make libs
[myusername@hgwdev src]$ cd hg
[myusername@hgwdev hg]$ make cgi
[myusername@hgwdev hg]$ cd makeDb/trackDb
[myusername@hgwdev trackDb]$ make update

You now have your own compiled version of the CGIs.

• Please note, in order for your browser to be fully functional, there needs to be a symlink to trash and to the javascript in your htdocs-myusername:

[myusername@hgwdev ~]$ cd /usr/local/apache/htdocs-myusername
[myusername@hgwdev htdocs-myusername]$ ln -s ../trash
[myusername@hgwdev htdocs-myusername]$ cd js
[myusername@hgwdev js]$ ln -s ../../htdocs/js/myusername

Create /usr/local/apache/cgi-bin-myusername/hg.conf:

• Create a hg.conf file in '/usr/local/apache/cgi-bin-username/' with these lines using your user name:

include ../cgi-bin/hg.conf
db.trackDb=trackDb_myusername

# Angie recommends that you do not add these lines unless you are using
# your own central database instead of hgcentraltest.
# Assuming you are using the default central.db (hgcentraltest):
# * If you use the default central.* settings from ../cgi-bin/hg.conf, then you can
#   seamlessly switch between genome-test and hgwdev-$USER -- your cookie session
#   will be the same on both.
# * If you change the central.* options, you will have separate cookies between
#   genome-test and hgwdev-$USER, which means your settings from one won't carry
#   over to the other unless you use the same hgsid when switching hosts.
central.host=localhost
central.domain=hgwdev-myusername.cse.ucsc.edu
central.cookie=hguid.hgwdev-myusername
backupcentral.domain=hgwdev-myusername.ucsc.edu 

• You can define your own background image by putting it in ~/public_html/ and adding

browser.background=http://hgwdev.cse.ucsc.edu/~myusername/yourimage.jpg  

• Here is a generic [background image] file you can download and edit.

Keep it current:

Here is a script (makeBinaries.csh) that you can copy and paste into your ~myusername/bin directory to make your binaries every night. Make sure to change the permissions (chmod 775) and if the script doesn't run check that the export PATH is set in your .bashrc file:

#!/bin/tcsh

if ($HOST != hgwdev) then
  echo "\n  this must be run from hgwdev!\n"
endif

cd ~/kent/src
make clean > /dev/null
cd ~/kent
git pull | mail -s 'Git pull report on kent tree' $USER
cd ~/kent/src
make libs
cd ~/kent/src/hg
make cgi
cd ~/kent/src/hg/makeDb/trackDb
make update

Automate it:

Create a crontab file. Note that you may have numerous crontab files. Each machine that you want cron jobs running on will need its own crontab file (assuming the jobs are different). Example, name your crontab file: hgwdev.cron for your hgwdev crontab. With your editor, add a line to your crontab file to run the command at the time you desire. For example, to run your daily build at 04:42 every weekday, the line would read:

42 04 * * 1-5 bin/makeBinaries.csh

See also: 'man 5 crontab' for a description of the fields used in these crontab lines.

When your crontab file is set correctly, hand it off to the cron system with the command:

$ crontab <yourCrontabFile>

Example: 'crontab hgwdev.cron'

To see what you have submitted to the cron system, ask it to display what it knows about your cron jobs with the -l argument:

$ crontab -l

Cron job tips: It is better to not run your jobs exactly at the top of the hour. They would be competing with system cron jobs that are running at times like the top of the hour. Better to run your job at odd minutes during the hour. Don't use the example time listed here, otherwise everyone's job will be running at 04:42 every day and competing with each other.

Any output to stdout or stderr by your cron job will be emailed to you via the cron system as it runs your job. If you would rather not see that output in email, construct your cron job shell script in such a way that it takes care of all of its stdout and stderr output. For example, it could create an organized by date hierarchy of log files for its output.