|
|
| (146 intermediate revisions by 8 users not shown) |
| Line 1: |
Line 1: |
| This page explains the process we use for building and releasing our CGIs. This is done on a two-week [[CGI_Build_Schedule | schedule]]. | | This page is no longer maintained. |
| | |
| The builds are built into sandboxes which are located here:
| |
| hgwbeta:/data/tmp/releaseBuild
| |
| titan:/scratch/releaseBuild
| |
| | |
| ==Setting Up the Environment for the Build==
| |
| Before starting as buildmeister, you will need to set up the following in your log in file:
| |
| * go to your home directory
| |
| hgwdev> cd
| |
| | |
| * edit your .tcshrc file
| |
| hgwdev> vi .tcshrc file
| |
| | |
| * add the following lines to set environment variables ($BOX32, $WEEKLYBLD, $BUILDDIR and $BRANCHNN are set in one of the build scripts - buildEnv.csh):
| |
| alias wb 'cd $WEEKLYBLD'
| |
|
| |
| if (( "$HOST" == "$BOX32" ) || ( "$HOST" == "hgwbeta" )) then
| |
| setenv hg $BUILDDIR/v${BRANCHNN}_branch/kent/src/hg
| |
| endif
| |
| | |
| * To be able to run the Java robot programs, add the following to the top of your path setting:
| |
| set path = ( /usr/java/jdk1.5.0_10/bin \
| |
| $path \
| |
| .
| |
| .
| |
| .
| |
| )
| |
| | |
| * source your .tcshrc to check the variables:
| |
| hgwdev> tcsh
| |
| | |
| * Set up autologin to hgwdev and titan:
| |
| | |
| ==Preview Day Build==
| |
| '''This is day 1 in the [[CGI_Build_Schedule | schedule]].'''
| |
| | |
| ===Run CVS Reports===
| |
| * go to /cluster/bin/build/scripts/oldsettings
| |
| hgwbeta> cd $WEEKLYBLD
| |
|
| |
| * edit buildEnv.csh: change the 5th line then the 4th line
| |
| hgwbeta> vi buildEnv.csh
| |
| | |
| line 5 (OLD): setenv LASTREVIEWDAY 2006-03-20 # preview of v110
| |
| line 5 (NEW): // change it to look like the date and comment from line 4
| |
| | |
| line 4 (OLD): setenv REVIEWDAY 2006-04-03 # preview of v111
| |
| line 4 (NEW): setenv REVIEWDAY 2006-today # preview of v112
| |
| | |
| * copy this edited buildEnv.csh file to the oldsettings directory to a new file called buildEnv.csh.vXXX[+ 1].preview
| |
| hgwbeta> cp buildEnv.csh oldsettings/buildEnv.csh.v112.preview
| |
| | |
| * source your .tcshrc
| |
| hgwbeta> tcsh
| |
| | |
| * check that the vars are correct
| |
| hgwbeta> env
| |
| | |
| * run doNewReview.csh as a test to see what it will do (review the variablies)
| |
| hgwbeta> ./doNewReview.csh
| |
| | |
| * run for real (this takes about 20 minutes - it does the run by ssh'ing to hgwdev) and direct output to a log file
| |
| hgwbeta> ./doNewReview.csh real >& doNewRev.log
| |
| | |
| * review the output file as it is written to
| |
| hgwbeta> tail -f doNewRev.log
| |
| | |
| ===Check the reports===
| |
| * The reports are automatically built by the script into this [http://hgwdev.cse.ucsc.edu/cvs-reports location].
| |
| * Briefly review the reports quickly as a sanity check.
| |
| | |
| ===Generate summary and review pairings=== (Donna takes care of this)
| |
| * Summarize the code changes that were committed during the past week.
| |
| * Update this [http://hgwdev.cse.ucsc.edu/builds/versions.html page] with the summary.
| |
| * Assign code-review partners.
| |
| * Send an email to browser-staff with the summary and code-review pairings.
| |
| | |
| ===Test on hgwbeta===
| |
| * Wait to hear from QA about how their CGIs look on hgwbeta. Each member of the QA team has [http://hgwdev.cse.ucsc.edu/qa/cgiTesting.html testing responsiblities].
| |
| | |
| ==Make changes to code base as necessary==
| |
| '''This happens on days 9, 10, and 11 in the [[CGI_Build_Schedule | schedule]].'''
| |
| * If there are problems with the build a developer will fix the code. This fix needs to be patched into the build on hgwbeta. This [http://genome-test.cse.ucsc.edu/qa/test-protocols/movingBranchTags.html document] explains how to move a Branch Tag on hgwbeta.
| |
| | |
| ==Final Build==
| |
| '''This is day 8 in the [[CGI_Build_Schedule | schedule]].'''
| |
| | |
| ===Do the Build===
| |
| | |
| * go to /cluster/bin/build/scripts/oldsettings
| |
| hgwbeta> cd $WEEKLYBLD
| |
| | |
| * edit the buildEnv.csh file
| |
| hgwbeta> vi buildEnv.csh
| |
| setenv BRANCHNN 131
| |
| setenv TODAY 2006-04-10 # v131 final
| |
| setenv LASTWEEK 2006-03-27 # v130 final
| |
| | |
| * edit file: add one to the BRANCHNN
| |
| setenv BRANCHNN 132
| |
| | |
| * edit file: change the 3rd line then the 2nd line
| |
| | |
| line 3 (OLD): setenv LASTWEEK 2006-03-27 # v130 final
| |
| line 3 (NEW): // change it to look like the date and comment from line 2
| |
| | |
| line 2 (OLD): setenv TODAY 2006-04-10 # v131 final
| |
| line 2 (NEW): setenv TODAY (today) # v132 final
| |
| | |
| * copy this edited buildEnv.csh file to the oldsettings directory to a new file called buildEnv.csh.vXXX.final
| |
| hgwbeta> cp buildEnv.csh oldsettings/buildEnv.csh.v112.final
| |
| | |
| * source your .tcshrc
| |
| hgwbeta> tcsh
| |
| | |
| * check that the vars are correct
| |
| hgwbeta> env
| |
| | |
| * run doNewBranch.csh as a test to see what it will do (review the variables)
| |
| hgwbeta> ./doNewBranch.csh
| |
| | |
| * run for real and send the output to a file
| |
| hgwbeta> ./doNewBranch.csh real >& doNew.log
| |
| | |
| * review the output file as it is written to
| |
| hgwbeta> tail -f doNew.log
| |
| | |
| * look for files that tell you it was successful
| |
| hgwbeta> /cluster/bin/build/scripts/32bitUtils.ok
| |
| hgwbeta> /cluster/bin/build/scripts/CvsReports.ok
| |
| | |
| | |
| * What the doNewBranch.csh script does:
| |
| | |
| # edits the versionInfo.h file
| |
| # makes tags (takes several minutes)
| |
| # builds CVS reports (takes 5-10 minutes)
| |
| # does build (takes 5-10 minutes)
| |
| ## builds CGIs (most important)
| |
| ## builds utils (of secondary importance)
| |
| # runs robots one at a time
| |
| ## hgNear (20 min)
| |
| ## TB (1 hour)
| |
| ## TrackCheck (several hours)
| |
| ## LiftOverTest (quick)
| |
| | |
| * Review the error logs for the three robots:
| |
| /build/scripts/logs/ (error logs)
| |
|
| |
| hgNear -- sends email with results
| |
| hgTables -- send email with results
| |
| hgTrackCheck -- must check by hand+
| |
| LiftOverTest -- must check by hand+
| |
| + located here: hgwbeta:/cluster/bin/build/scripts/logs
| |
| | |
| ==Push the CGIs==
| |
| '''This is day 12 in the [[CGI_Build_Schedule | schedule]].'''
| |
| | |
| ===Push to hgwN only===
| |
| * hgwN is one of the RR machines, hgw1-8. Each build, rotate to the next machine in numeric order i.e. hgw1 then hgw2 etc. so that one machine is not being worked more than the others.
| |
| * Wait to hear that all pair-reviews are complete.
| |
| * Ask for push of CGIs from hgwbeta to hgwN only. Make sure you add a note to the cluster-admins to make sure they know which RR machine to push to first. If there is a NEW CGI going out this week, be sure to make a prominent note of it in your push request. The admins push from a script, and they will need to add your new CGI to the script.
| |
| * Run TrackCheck.java on hgwN. This is the responsibility of the QA person who tests hgTracks.
| |
| | |
| # start here: [hgwdev:/scratch/javaBuild/src/edu/ucsc/genome/qa/cgiCheck>
| |
| # make a props file which specifies the machine/db to check. If you are in a hurry, set zoomCount=0 and it will only check the default position for each assembly (much quicker!).
| |
| # run it: [hgwdev:/scratch/javaBuild/src/edu/ucsc/genome/qa/cgiCheck> nohup TrackCheck hgw1.props > & $WEEKLYBLD/logs/hgw1.07-13-2006
| |
| # grep the output for 'error'.
| |
| | |
| * Monitor Apache Error Log
| |
| hgwN:/usr/local/apache/logs/error_log
| |
| | |
| * Wait to hear from QA about how their CGIs look on hgwN. Each member of the QA team has [http://hgwdev.cse.ucsc.edu/qa/cgiTesting.html testing responsiblities]. Check also that TrackCheck ran successfully.
| |
| | |
| ===Push to the rest of the RR===
| |
| * Ask for push from hgwbeta to hgw2-8 & MGC.
| |
| | |
| ==Final Build Wrap-up==
| |
| '''This is day 12 in the [[CGI_Build_Schedule | schedule]].'''
| |
| | |
| * zip the source code
| |
| hgwbeta> cd /cluster/bin/build/scripts
| |
| hgwbeta> tcsh
| |
| hgwbeta> env
| |
| hgwbeta> doZip.csh (this is automatically pushed to hgdownload)
| |
| | |
| * build and push hgcentral
| |
| hgwdev> ./buildHgCentralSql.csh
| |
| hgwdev> ./buildHgCentralSql.csh real
| |
| ** request push of hgcentral to hgdownload
| |
| Please push from hgwdev --> hgdownload
| |
| /usr/local/apache/htdocs/admin/hgcentral.sql
| |
| This includes settings for the mirror sites.
| |
| | |
| * build 32bit build and push to hgdownload
| |
| hgwbeta> cd /cluster/bin/build/scripts
| |
| hgwbeta> doBuildCgi32.csh
| |
| ** the push to hgdownload happens automatically to here:
| |
| hgdownload:/mirrordata/apache/cgi-bin-i386/
| |
| | |
| | |
| * zip liftOver utils on hgwbeta and scp them to hgdownload
| |
| hgwbeta> cd /cluster/bin/build/scripts
| |
| hgwbeta> doHgDownloadUtils.csh
| |
| | |
| * send email to genome-mirror. Include link to latest source: http://hgdownload.cse.ucsc.edu/admin/jksrc.zip.
| |
| | |
| [[Category:Browser QA]]
| |
