Build Environment Variables
See also: in the source tree src/product/scripts/browserEnvironment.txt
The kent/src build makes use of environment variables to configure include and lib paths used when compiling. The most important are MYSQLLIBS and MYSQLINC -- see step 3 of kent/src/product/README.building.source. This page focus on several optional environment variables, with defaults provided by kent/src/inc/common.mk.
External library configuration (USE_*)
Several browser features require the installation of additional external software packages. They are disabled unless their corresponding environment variables are set to 1. If you are compiling on hgwdev, then you can use any or none of these as you wish (but you may get an annoying track handler warning if you don't set USE_BAM to 1, and then visit hg18 in hgTracks). To set a variable USE_XXX to 1, if you have a ~/.cshrc or ~/.tcshrc variable, add this:
# Enable UCSC XXX features setenv USE_XXX 1
and if you have a ~/.bashrc, add this:
# Enable UCSC XXX features export USE_XXX=1
After editing, source the file that you edited (e.g. "source ~/.cshrc"). Then, do a clean build of your tree, to make sure that both libraries and CGIs/utils are compiled with consistent settings.
cd kent/src make clean make
USE_BAM (+KNETFILE_HOOKS)
BAM is the compressed binary form of the SAM/BAM file format for short read alignments. The Genome Browser can display BAM data tracks when this is enabled. The samtools C library must be installed first. Then, in addition to setting USE_BAM to 1, you must define another variable or two depending on how you have installed samtools locally. If you have done the default build, which places libbam.a in the top-level samtools directory along with the .c and .h files, then set the environment variable SAMDIR to that directory. If you have manually installed libbam.a and .h files in separate directories, then set SAMINC to the directory that contains the .h files, and SAMLIB to <libDirectory>/libbam.a .
UCSC has an extension to samtools that allows its network module (knetfile) to be configured as a wrapper on a different module, so we can configure it to use UCSC's network code. This speeds up BAM display by caching parts of the data file that have already been accessed, and supports https and http with authentication. If you obtain the extension from UCSC KNETFILE_HOOKS (or if it is someday incorporated into the main distribution of samtools as we hope) and recompile samtools with it, then you can set KNETFILE_HOOKS to 1 to make the UCSC code utilize the extension.
USE_TABIX (+KNETFILE_HOOKS)
Tabix is a compression and indexing utility that works on any tab-separated file that is sorted by genomic position (sequence name, start offset). As of Mar. 16, 2011, when USE_TABIX is set to 1, the kent/src tree has library support for reading tabix-compressed files, and development of VCF+tabix track display is underway. The tabix C library, which is distributed from the same site as samtools, must be installed first. Then, in addition to USE_TABIX=1, one or more variables pointing to the local installation must be defined. If you have done the default build, which places libtabix.a in the top-level tabix directory along with the .c and .h files, then set the environment variable TABIXDIR to that directory. If you have manually installed libtabix.a and .h files in separate directories, then set TABIXINC to the directory that contains the .h files, and TABIXLIB to <libDirectory>/libtabix.a .
UCSC has an extension to tabix that allows its network module (knetfile) to be configured as a wrapper on some other module, so we can configure it to use UCSC's network code. This speeds up fetching of remote data by aching parts of the data file that have already been accessed, and supports https and http with authentication. If you obtain the extension from UCSC KNETFILE_HOOKS (or if it is someday incorporated into the main distribution of tabix as we hope) and recompile samtools with it, then you can set KNETFILE_HOOKS to 1 to make the UCSC code utilize the extension.
Note: both samtools and tabix libraries include bgzf_* functions for compressing and decompressing data. The functions are almost identical, but not completely. Therefore, linking with both libraries can fail unless the function names are uniquified. Included in UCSC's KNETFILE_HOOKS patch for tabix is a change of bgzf_* function names to ti_bgzf_*, which prevents the clash. If KNETFILE_HOOKS is set to 1, both samtools and tabix must be patched using their respective KNETFILE_HOOKS patch files.
USE_PNG
This is no longer an option as of the middle of 2010. You must set USE_PNG=1 in your shell environment to build the source tree and have the browser function. You may need to install the libpng-devel package if it is not already. To check your installed packages on linux:
rpm -qa | grep -i png
libpng-devel often installs its include files in places such as:
/usr/include/libpng10
In which case you may need to define:
PNGINCL="-I/usr/include/libpng10"
in your build environment to direct the compiler to that directory to find png.h
PNG (Portable Networks Graphics) is a widely used image file format, supported by all major browsers, that generally produces smaller image file sizes than the Genome Browser's default format, GIF. The libpng library must be installed before setting USE_PNG to 1; if on a linux system, use your favorite package manager (rpm, yum, apt-get etc.). A caveat: since kent/src/inc/common.h includes setjmp.h, libpng's pngconf.h must be edited to remove the syntax error that is intentionally generated when setjmp.h has already been included.
USE_SSL
SSL (Secure Socket Layer) provides encrypted transfer of data. In the Genome Browser, URLs that begin with "https://" or "ftps://" are supported only when this is enabled. The libssl library (development version w/headers) must be installed first; if on a linux system, use your favorite package manager (rpm, yum, apt-get etc.).
Compilation environment variables
COPT
Default: -O -g (optimize but keep debugger symbols). When debugging gets gnarly due to optimizations, recompile with COPT=-ggdb (or if not using gcc/gdb, just -g).
HG_WARN
By default, we use -Wall and add in a few others on specific platforms. When OSTYPE does not include "darwin" or "solaris", -Werror is added. This is appended to common.mk's CFLAGS.
OS
If OS is Windows_NT (as it is in the Cygwin environment), then necessary file suffixes are tweaked for a.out and executable names.
STRIP
Default: STRIP=strip is applied to executables after compilation, stripping their symbol tables and making them nice and small but not debugger-friendly. To bypass strip, set STRIP=true and make will run 'true exec' instead of 'strip exec'.
Installation environment variables
BINDIR
Installation directory for utility programs compiled from C.
CGI_BIN
Installation directory for CGI programs (hgTracks, hgc etc)
DOCUMENTROOT
/usr/local/apache/htdocs by default; if you use a different web server or htdocs root for Genome Browser files, change this. CGI programs expect to read some files from this directory.
SCRIPTS
Installation directory for utility programs written in an interpreted scripting language (bash, tcsh, perl etc.).
cygwin compile issues
MySQL libraries
There seems to be a problem of finding a compatible libmysql system to work on Windows. The source tree would build and link but end up with a SIGSEGV in any call into the MySQL functions.
common.mk
This problem may get fixed in the code, but as of July 2010 the following changes were needed in src/inc/common.mk to get the libraries to compile on cygwin with gcc-4.3.4: remove the -Werror at line 112 defining HG_WARN
HG_WARN = -Wall -Wformat -Wimplicit -Wreturn-type
and to use the openssl libraries, add a line 27 following the -lssl definition to include -lcrypto:
L+=-lssl L+=-lcrypto
samtools build
Attempting to build the samtools library on cygwin required a couple of lines changed in the Makefile, as found in this note samtools on cygwin
add -D_WIN32 to the DFLAGS definition and add an LFLAGS: LFLAGS= -lws2_32 -lgcc -lcygwin -lbz2 -lz -lstdc++
and optionally change the linker command for the samtools executable build:
$(CC) $(CFLAGS) -o $@ $(AOBJS) -Xlinker --enable-auto-import -lm libbam.a -lm $(LIBPATH) $(LIBCURSES) -lz -L. -lbam $(LFLAGS)
You do not need this executable to build the kent libraries since they only use libbam.a
Set SAMDIR SAMINC and SAMLIB in your environment. If you build libbam.a in .../samtools-0-1.8/ the settings are:
SAMDIR=/path/to/samtools-0-1.8
SAMINC=${SAMDIR}
SAMLIB=${SAMDIR}/libbam.a
