CGI Build Process: Difference between revisions

From genomewiki
Jump to navigationJump to search
No edit summary
(Replacing page with 'This page is no longer maintained.')
 
(151 intermediate revisions by 8 users not shown)
Line 1: Line 1:
This page explains the process we use for building and releasing our CGIs.  This is done on a two-week [[CGI_Build_Schedule | schedule]].
This page is no longer maintained.
 
The builds are built into sandboxes which are located here:
hgwbeta:/data/tmp/releaseBuild
titan:/scratch/releaseBuild
 
==Preview Day Build==
'''This is day 1 in the [[CGI_Build_Schedule | schedule]].'''
 
===Run CVS Reports===
* go to /cluster/bin/build/scripts/oldsettings
hgwbeta> cd $WEEKLYBLD
* edit buildEnv.csh: change the 5th line then the 4th line
hgwbeta> vi buildEnv.csh
 
  line 5 (OLD): setenv LASTREVIEWDAY 2006-03-20 # preview of v110
  line 5 (NEW): // change it to look like the date and comment from line 4
 
  line 4 (OLD): setenv REVIEWDAY 2006-04-03  # preview of v111
  line 4 (NEW): setenv REVIEWDAY 2006-today  # preview of v112
 
* copy this edited buildEnv.csh file to the oldsettings directory to a new file called buildEnv.csh.vXXX[+ 1].preview
hgwbeta> cp buildEnv.csh oldsettings/buildEnv.csh.v112.preview
 
* source your .tcshrc
hgwbeta> tcsh
 
* check that the vars are correct
hgwbeta> env
 
* run doNewReview.csh as a test to see what it will do (review the variablies)
hgwbeta> ./doNewReview.csh
 
* run for real (this takes about 20 minutes - it does the run by ssh'ing to hgwdev) and direct output to a log file
hgwbeta> ./doNewReview.csh real >& doNewRev.log
 
* review the output file as it is written to
hgwbeta> tail -f doNewRev.log
 
===Check the reports===
* The reports are automatically built by the script into this [http://hgwdev.cse.ucsc.edu/cvs-reports location].
* Briefly review the reports quickly as a sanity check.
 
===Generate summary and review pairings=== (Donna takes care of this)
* Summarize the code changes that were committed during the past week.
* Update this [http://hgwdev.cse.ucsc.edu/builds/versions.html page] with the summary.
* Assign code-review partners.
* Send an email to browser-staff with the summary and code-review pairings.
 
===Test on hgwbeta===
* Wait to hear from QA about how their CGIs look on hgwbeta. Each member of the QA team has [http://hgwdev.cse.ucsc.edu/qa/cgiTesting.html testing responsiblities].
 
==Make changes to code base as necessary==
'''This happens on days 9, 10, and 11 in the [[CGI_Build_Schedule | schedule]].'''
* If there are problems with the build a developer will fix the code.  This fix needs to be patched into the build on hgwbeta.  This [http://genome-test.cse.ucsc.edu/qa/test-protocols/movingBranchTags.html document] explains how to move a Branch Tag on hgwbeta.
 
==Final Build==
'''This is day 8 in the [[CGI_Build_Schedule | schedule]].'''
 
===Do the Build===
 
* go to /cluster/bin/build/scripts/oldsettings
hgwbeta> cd $WEEKLYBLD
 
* edit the buildEnv.csh file
hgwbeta> vi buildEnv.csh
setenv BRANCHNN 131
setenv TODAY 2006-04-10    # v131 final
setenv LASTWEEK 2006-03-27  # v130 final
 
* edit file: add one to the BRANCHNN
setenv BRANCHNN 132
 
* edit file: change the 3rd line then the 2nd line
 
line 3 (OLD): setenv LASTWEEK 2006-03-27  # v130 final
line 3 (NEW): // change it to look like the date and comment from line 2
 
line 2 (OLD): setenv TODAY 2006-04-10    # v131 final
line 2 (NEW): setenv TODAY (today)        # v132 final
 
* copy this edited buildEnv.csh file to the oldsettings directory to a new file called buildEnv.csh.vXXX.final
hgwbeta> cp buildEnv.csh oldsettings/buildEnv.csh.v112.final
 
* source your .tcshrc
hgwbeta> tcsh
 
* check that the vars are correct
hgwbeta> env
 
* run doNewBranch.csh  as a test to see what it will do (review the variables)
hgwbeta> ./doNewBranch.csh
 
* run for real and send the output to a file
hgwbeta> ./doNewBranch.csh real >& doNew.log
 
* review the output file as it is written to
hgwbeta> tail -f doNew.log
 
* look for files that tell you it was successful
hgwbeta> /cluster/bin/build/scripts/32bitUtils.ok 
hgwbeta> /cluster/bin/build/scripts/CvsReports.ok
 
 
* What the doNewBranch.csh script does:
 
# edits the versionInfo.h file
# makes tags (takes several minutes)
# builds CVS reports (takes 5-10 minutes)
# does build (takes 5-10 minutes)
## builds CGIs (most important)
## builds utils (of secondary importance)
# runs robots one at a time
## hgNear (20 min)
## TB (1 hour)
## TrackCheck (several hours)
## LiftOverTest (quick)
 
* Review the error logs for the three robots:
/build/scripts/logs/ (error logs)
hgNear -- sends email with results
hgTables -- send email with results
hgTrackCheck -- must check by hand+
LiftOverTest -- must check by hand+
+ located here: hgwbeta:/cluster/bin/build/scripts/logs
 
==Push the CGIs==
'''This is day 12 in the [[CGI_Build_Schedule | schedule]].'''
 
===Push to hgw1 only===
* Wait to hear that all pair-reviews are complete.
* Ask for push of CGIs from hgwbeta to hgw1 only.  If there is a NEW CGI going out this week, be sure to make a prominent note of it in your push request.  The admins push from a script, and they will need to add your new CGI to the script.
* Run TrackCheck.java on hgw1. This is the responsibility of the QA person who tests hgTracks.
 
# start here: [hgwdev:/scratch/javaBuild/src/edu/ucsc/genome/qa/cgiCheck>
# make a props file which specifies the machine/db to check.  If you are in a hurry, set zoomCount=0 and it will only check the default position for each assembly (much quicker!).
# run it: [hgwdev:/scratch/javaBuild/src/edu/ucsc/genome/qa/cgiCheck> nohup TrackCheck hgw1.props > &  $WEEKLYBLD/logs/hgw1.07-13-2006
# grep the output for 'error'.
 
* Monitor Apache Error Log
hgw1:/usr/local/apache/logs/error_log
 
* Wait to hear from QA about how their CGIs look on hgw1. Each member of the QA team has [http://hgwdev.cse.ucsc.edu/qa/cgiTesting.html testing responsiblities]. Check also that TrackCheck ran successfully.
 
===Push to the rest of the RR===
* Ask for push from hgwbeta to hgw2-8 & MGC.
 
==Final Build Wrap-up==
'''This is day 12 in the [[CGI_Build_Schedule | schedule]].'''
 
* zip the source code
hgwbeta> cd /cluster/bin/build/scripts
hgwbeta> tcsh
hgwbeta> env
hgwbeta> doZip.csh  (this is automatically pushed to hgdownload)
 
* build and push hgcentral
hgwdev> ./buildHgCentralSql.csh
hgwdev> ./buildHgCentralSql.csh real
** request push of hgcentral to hgdownload
  Please push from hgwdev --> hgdownload
  /usr/local/apache/htdocs/admin/hgcentral.sql
  This includes settings for the mirror sites.
 
* build 32bit build and push to hgdownload
hgwbeta> cd /cluster/bin/build/scripts
hgwbeta> doBuildCgi32.csh
** the push to hgdownload happens automatically to here:
hgdownload:/mirrordata/apache/cgi-bin-i386/
 
 
* zip liftOver utils on hgwbeta and scp them to hgdownload
hgwbeta> cd /cluster/bin/build/scripts
hgwbeta> doHgDownloadUtils.csh
 
* send email to genome-mirror.  Include link to latest source: http://hgdownload.cse.ucsc.edu/admin/jksrc.zip.
 
[[Category:Browser QA]]

Latest revision as of 18:39, 10 March 2011

This page is no longer maintained.