Patching a Branch: Difference between revisions

From Genecats
Jump to navigationJump to search
No edit summary
No edit summary
 
(29 intermediate revisions by 3 users not shown)
Line 6: Line 6:
* ('''Developer'''): Create a [http://redmine.soe.ucsc.edu/projects/genomebrowser/wiki/Build_Patch| Build Patch] issue in redmine. Fix the error (and get a second opinion on your fix from another developer).  Let the QA person know that it is fixed.  
* ('''Developer'''): Create a [http://redmine.soe.ucsc.edu/projects/genomebrowser/wiki/Build_Patch| Build Patch] issue in redmine. Fix the error (and get a second opinion on your fix from another developer).  Let the QA person know that it is fixed.  
* ('''QA'''): Verify the developer's fix on hgwdev.  Send email to Build-meister (cc'ing browser-qa and developer) with revision number of the fix and patch request (including an explanation of the problem and the fix and the CGIs that will be affected).
* ('''QA'''): Verify the developer's fix on hgwdev.  Send email to Build-meister (cc'ing browser-qa and developer) with revision number of the fix and patch request (including an explanation of the problem and the fix and the CGIs that will be affected).
* ('''Build-Meister'''): Check the Git log to make sure that you are making the correct change.
* ('''Build-Meister'''): You should try to review the commit id to be cherry-picked.
  git show <commitid>
or
  git show --stat <commitid>
  The cherry-pick script will show you a summary anyways.
NOTE FROM GALT: this paragraph seems confused:
Check the Git log to make sure that you are making the correct change.
  hgwdev> cd $BUILDDIR/v${BRANCHNN}_branch/kent/src
  hgwdev> cd $BUILDDIR/v${BRANCHNN}_branch/kent/src
  hgwdev> git log hg/someCgi/someFile.c
  hgwdev> git log hg/someCgi/someFile.c
  hgwdev> ssh -X build@hgwbeta         # the optional '-X' allows X-windows support     
  hgwdev> ssh -X build@hgwdev         # the optional '-X' allows X-windows support     
  <build@hgwbeta> cd $BUILDDIR/v${BRANCHNN}_branch/kent/src
  <build@hgwdev> cd $BUILDDIR/v${BRANCHNN}_branch/kent/src
  <build@hgwbeta> git log hg/someCgi/someFile.c
  <build@hgwdev> git log hg/someCgi/someFile.c
* ('''Build-Meister'''): Tell the program which file and version you want to change to.
 
  <build@hgwbeta> cd $WEEKLYBLD
* ('''Build-Meister'''): Add the commit(s) to be cherrypicked onto the release branch.
  <build@hgwbeta> edit CherryPickCommits.conf
hgwdev> ssh -X build@hgwdev          # the optional '-X' allows X-windows support   
** This is a list and you can have more than one line if you need.
  <build@hgwdev> cd $WEEKLYBLD
** Edit this file to include the git commit hash id like so:
  <build@hgwdev> vi CherryPickCommits.conf  # Edit CherryPickCommits.conf
* Remove all the lines from the last time it was used. These commit ids have been logged into a file already.
* This is a list and you can have more than one line if you need.
* Each commit to be patched goes on its own line.
* Edit this file to include the git commit hash id like so:
96b0dc9826d589e09daf2bd1a36ef513c91255b1
96b0dc9826d589e09daf2bd1a36ef513c91255b1
* ('''Build-Meister'''): Do a test run and verify that everything is set up correctly:
* ('''Build-Meister'''): Do a test run and verify that everything is set up correctly:
  <build@hgwbeta> ./cherryPickCommits.csh
  <build@hgwdev> ./cherryPickCommits.csh >& logs/v$BRANCHNN.patch1.log  # use patch2, patch3 etc as needed.
* ('''Build-Meister'''): If it is correct, run it for real:
* ('''Build-Meister'''): Examine the patch output log.
  <build@hgwbeta> ./cherryPickCommits.csh real
<build@hgwdev> cat logs/v$BRANCHNN.patch1.log
* If it is correct, run it for real:
  <build@hgwdev> ./cherryPickCommits.csh real >>& logs/v$BRANCHNN.patch1.log  # use patch2, patch3 etc as needed.
* ('''Build-Meister'''): Examine the patch output log.
<build@hgwdev> cat logs/v$BRANCHNN.patch1.log
* ('''Build-Meister'''): Determine which CGIs are affected by this file change:
* ('''Build-Meister'''): Determine which CGIs are affected by this file change:
  Actually this is obsolete, the dependences program no longer works in git,
  so you will have to figure it out in other ways.
<build@hgwbeta> cd /usr/local/apache/cgi-bin
<build@hgwbeta:/usr/local/apache/cgi-bin> ./dependencies fileName.c
* ('''Build-Meister'''): Go to the current build:
* ('''Build-Meister'''): Go to the current build:
  <build@hgwbeta> cd ${BUILDDIR}/v${BRANCHNN}_branch
  <build@hgwdev> cd ${BUILDDIR}/v${BRANCHNN}_branch
* ('''Build-Meister'''): Do a 'make' in the library directory, if necessary. If the file change was a lib file, then do a make in the corresponding lib directory:
* ('''Build-Meister'''): Do a 'make' in the library directory, if necessary. If .h files were changed use make clean, either at CGI, or lib level. If the file change was a lib file, then do a make in the corresponding lib directory:
  <build@hgwbeta> cd ${BUILDDIR}/v${BRANCHNN}_branch/kent/src/lib>
  <build@hgwdev> cd ${BUILDDIR}/v${BRANCHNN}_branch/kent/src/lib
  <build@hgwbeta> make
  <build@hgwdev> make
** -or-  
** -or-  
  <build@hgwbeta> cd ${BUILDDIR}/v${BRANCHNN}_branch/kent/src/hg/lib>
  <build@hgwdev> cd ${BUILDDIR}/v${BRANCHNN}_branch/kent/src/hg/lib
  <build@hgwbeta> make
  <build@hgwdev> make
* ('''Build-Meister'''): If only one or two programs are impacted by the change, do a 'make alpha' in their CGI directories:
* ('''Build-Meister'''): If only one or two programs are impacted by the change, do a 'make beta' in their CGI directories:
  <build@hgwbeta> cd ${BUILDDIR}/v${BRANCHNN}_branch/kent/src/hg
  <build@hgwdev> cd ${BUILDDIR}/v${BRANCHNN}_branch/kent/src/hg
  <build@hgwbeta> cd path/to/c/file  # e.g.: cd hgTracks
  <build@hgwdev> cd path/to/c/file  # e.g.: cd hgTracks
  <build@hgwbeta> make alpha
  <build@hgwdev> make beta
** -or-  
** -or-  
* ('''Build-Meister'''): If all or many are impacted, do a 'make alpha' in hg/:
* ('''Build-Meister'''): If all or many are impacted, do a 'make beta' in hg/:
  <build@hgwbeta> cd ${BUILDDIR}/v${BRANCHNN}_branch/kent/src/hg
  <build@hgwdev> cd ${BUILDDIR}/v${BRANCHNN}_branch/kent/src/hg
  <build@hgwbeta> make alpha
  <build@hgwdev> make beta
* ('''Build-Meister'''): Update the redmine ticket that the build has been patched and binaries made on hgwbeta.
 
* ('''Build-Meister'''): Check CGIs were updated as expected:
<build@hgwdev> ls -ltr /usr/local/apache/cgi-bin-beta
 
Rsync binaries to hgwbeta machine. BE CAREFUL not to over-write hg.conf* files on hgwbeta:
 
<build@hgwdev> rsync -a -P --exclude=hg.conf --exclude=hg.conf.private --exclude=hgPhyloPlaceData \
  /usr/local/apache/cgi-bin-beta/ qateam@hgwbeta:/data/apache/cgi-bin/
 
Angie's automated system updates both hgwdev:cgi-bin-beta and hgwbeta:cgi-bin
so there is no need to update hgPhyloPlaceData.
 
Note if you did a full make beta in hg, you may need to do these too,
which update htdocs, js, and style dirs:
 
<build@hgwdev> rsync -a -P          /usr/local/apache/htdocs-beta/      qateam@hgwbeta:/data/apache/htdocs/
<build@hgwdev> rsync -a -P --delete /usr/local/apache/htdocs-beta/js/    qateam@hgwbeta:/data/apache/htdocs/js/
<build@hgwdev> rsync -a -P --delete /usr/local/apache/htdocs-beta/style/ qateam@hgwbeta:/data/apache/htdocs/style/
 
* ('''Build-Meister'''): Update the redmine ticket that the build has been patched and binaries made on hgwdev and synced to hgwbeta.
* ('''Build-Meister'''): If utils changed, do wb to $weeklybuild, run .doUtils.csh will update /cluster/bin using symtrick.csh
* ('''Build-Meister'''):      so make a copy of .doUtils.csh comment out the symtrick.csh in it (leave unsymtrick) and run it, because that will update your own executables in build@hgwdev $HOME which robots use. Run robots again if needed.
* ('''Build-Meister'''): Update or re-do gbib build steps.
* ('''Build-Meister'''): Update or re-do gbib build steps.
* ('''Build-Meister'''): If the build-wrapup has already been done, you will need to re-do the relevant parts  (minimally, the beta branch and release tag and source.zip would have to be re-done.
* ('''Build-Meister'''): If the final build-wrapup has already been done, you will need to re-do the relevant parts  (minimally, the beta branch and release tag (vNNN_branch.1 ->.2) and source.zip would have to be re-done.
* ('''QA'''): Test the change on hgwbeta.
* ('''QA'''): Test the change on hgwbeta.
* ('''QA'''): Reply to the email about the efficacy of the change.
* ('''QA'''): Reply to the email about the efficacy of the change.
----
'''Here is an example narrative about the process from a patch in v329:'''
In a typical case, an engineer's fix would be committed
and pushed to a shared repo already on the main master branch
(the one that most of us use). Hopefully, after that, the developer will also have remembered
to recompile the relevant CGI (hgTables or hgTracks here?) on hgwdev,
so that you can test it there.
This helps you confirm that the fix works. The next step after that is the patching.
The buildmeister (Hiram) has already made a special new
branch called v329_branch which represents our official version 329 release.
We make it so that we can isolate the release from other ongoing
development on master. This allows us to test and fix things on
the branch without having to merge in other work happening on master. In any case, the buildmeister has a little script that grabs the
commitId for the patch from the master branch and applies it to
a special sandbox with branch v329_branch so that we can get the
correction onto the release. No other changes come with it,
just the commit being patched. After buildmeister patches, he will recompile some or all of the
CGIs, and then they get rsynced to hgwbeta.
At that point we can verify by testing on hgwbeta
that the fix works there too, just to double-check the patching
worked -- which it nearly always does.


[[Category:Browser QA]]
[[Category:Browser QA]]
[[Category:Browser QA CGI]]
[[Category:Browser QA CGI]]

Latest revision as of 22:57, 9 November 2023

Edited to use Git and cherry-picking.

How? ("Who" in parenthesis)

  • (QA): Find an error in the binaries, determine if it needs to be patched, and alert the Developer.
  • (Developer): Create a Build Patch issue in redmine. Fix the error (and get a second opinion on your fix from another developer). Let the QA person know that it is fixed.
  • (QA): Verify the developer's fix on hgwdev. Send email to Build-meister (cc'ing browser-qa and developer) with revision number of the fix and patch request (including an explanation of the problem and the fix and the CGIs that will be affected).
  • (Build-Meister): You should try to review the commit id to be cherry-picked.
  git show <commitid>
or
  git show --stat <commitid>
 The cherry-pick script will show you a summary anyways.
NOTE FROM GALT: this paragraph seems confused:
Check the Git log to make sure that you are making the correct change.
hgwdev> cd $BUILDDIR/v${BRANCHNN}_branch/kent/src
hgwdev> git log hg/someCgi/someFile.c
hgwdev> ssh -X build@hgwdev          # the optional '-X' allows X-windows support    
<build@hgwdev> cd $BUILDDIR/v${BRANCHNN}_branch/kent/src
<build@hgwdev> git log hg/someCgi/someFile.c
  • (Build-Meister): Add the commit(s) to be cherrypicked onto the release branch.
hgwdev> ssh -X build@hgwdev          # the optional '-X' allows X-windows support    
<build@hgwdev> cd $WEEKLYBLD
<build@hgwdev> vi CherryPickCommits.conf  # Edit CherryPickCommits.conf
  • Remove all the lines from the last time it was used. These commit ids have been logged into a file already.
  • This is a list and you can have more than one line if you need.
  • Each commit to be patched goes on its own line.
  • Edit this file to include the git commit hash id like so:

96b0dc9826d589e09daf2bd1a36ef513c91255b1

  • (Build-Meister): Do a test run and verify that everything is set up correctly:
<build@hgwdev> ./cherryPickCommits.csh >& logs/v$BRANCHNN.patch1.log  # use patch2, patch3 etc as needed.
  • (Build-Meister): Examine the patch output log.
<build@hgwdev> cat logs/v$BRANCHNN.patch1.log
  • If it is correct, run it for real:
<build@hgwdev> ./cherryPickCommits.csh real >>& logs/v$BRANCHNN.patch1.log  # use patch2, patch3 etc as needed.
  • (Build-Meister): Examine the patch output log.
<build@hgwdev> cat logs/v$BRANCHNN.patch1.log
  • (Build-Meister): Determine which CGIs are affected by this file change:
  • (Build-Meister): Go to the current build:
<build@hgwdev> cd ${BUILDDIR}/v${BRANCHNN}_branch
  • (Build-Meister): Do a 'make' in the library directory, if necessary. If .h files were changed use make clean, either at CGI, or lib level. If the file change was a lib file, then do a make in the corresponding lib directory:
<build@hgwdev> cd ${BUILDDIR}/v${BRANCHNN}_branch/kent/src/lib
<build@hgwdev> make
    • -or-
<build@hgwdev> cd ${BUILDDIR}/v${BRANCHNN}_branch/kent/src/hg/lib
<build@hgwdev> make
  • (Build-Meister): If only one or two programs are impacted by the change, do a 'make beta' in their CGI directories:
<build@hgwdev> cd ${BUILDDIR}/v${BRANCHNN}_branch/kent/src/hg
<build@hgwdev> cd path/to/c/file   # e.g.: cd hgTracks
<build@hgwdev> make beta
    • -or-
  • (Build-Meister): If all or many are impacted, do a 'make beta' in hg/:
<build@hgwdev> cd ${BUILDDIR}/v${BRANCHNN}_branch/kent/src/hg
<build@hgwdev> make beta
  • (Build-Meister): Check CGIs were updated as expected:
<build@hgwdev> ls -ltr /usr/local/apache/cgi-bin-beta

Rsync binaries to hgwbeta machine. BE CAREFUL not to over-write hg.conf* files on hgwbeta:

<build@hgwdev> rsync -a -P --exclude=hg.conf --exclude=hg.conf.private --exclude=hgPhyloPlaceData \
 /usr/local/apache/cgi-bin-beta/ qateam@hgwbeta:/data/apache/cgi-bin/

Angie's automated system updates both hgwdev:cgi-bin-beta and hgwbeta:cgi-bin so there is no need to update hgPhyloPlaceData.

Note if you did a full make beta in hg, you may need to do these too, which update htdocs, js, and style dirs:

<build@hgwdev> rsync -a -P          /usr/local/apache/htdocs-beta/       qateam@hgwbeta:/data/apache/htdocs/
<build@hgwdev> rsync -a -P --delete /usr/local/apache/htdocs-beta/js/    qateam@hgwbeta:/data/apache/htdocs/js/
<build@hgwdev> rsync -a -P --delete /usr/local/apache/htdocs-beta/style/ qateam@hgwbeta:/data/apache/htdocs/style/
  • (Build-Meister): Update the redmine ticket that the build has been patched and binaries made on hgwdev and synced to hgwbeta.
  • (Build-Meister): If utils changed, do wb to $weeklybuild, run .doUtils.csh will update /cluster/bin using symtrick.csh
  • (Build-Meister): so make a copy of .doUtils.csh comment out the symtrick.csh in it (leave unsymtrick) and run it, because that will update your own executables in build@hgwdev $HOME which robots use. Run robots again if needed.
  • (Build-Meister): Update or re-do gbib build steps.
  • (Build-Meister): If the final build-wrapup has already been done, you will need to re-do the relevant parts (minimally, the beta branch and release tag (vNNN_branch.1 ->.2) and source.zip would have to be re-done.
  • (QA): Test the change on hgwbeta.
  • (QA): Reply to the email about the efficacy of the change.

Here is an example narrative about the process from a patch in v329: In a typical case, an engineer's fix would be committed and pushed to a shared repo already on the main master branch (the one that most of us use). Hopefully, after that, the developer will also have remembered to recompile the relevant CGI (hgTables or hgTracks here?) on hgwdev, so that you can test it there. This helps you confirm that the fix works. The next step after that is the patching. The buildmeister (Hiram) has already made a special new branch called v329_branch which represents our official version 329 release. We make it so that we can isolate the release from other ongoing development on master. This allows us to test and fix things on the branch without having to merge in other work happening on master. In any case, the buildmeister has a little script that grabs the commitId for the patch from the master branch and applies it to a special sandbox with branch v329_branch so that we can get the correction onto the release. No other changes come with it, just the commit being patched. After buildmeister patches, he will recompile some or all of the CGIs, and then they get rsynced to hgwbeta. At that point we can verify by testing on hgwbeta that the fix works there too, just to double-check the patching worked -- which it nearly always does.