Changes to cv.ra: Difference between revisions

From Genecats
Jump to navigationJump to search
(Undo revision 4734 by Daniel (talk))
 
(16 intermediate revisions by 2 users not shown)
Line 10: Line 10:
Each of the directories were a convenient way to keep cv.ra changes staged.  One person would edit the cv.ra in the alpha directory and see changes with a "make alpha DBS", the QA person would periodically copy the alpha/cv.ra to beta/cv.ra, creating a snap shot to QA.  When ready for release, they QA person would copy the final beta/cv.ra version to public/cv.ra, thus again creating a third public snapshot.
Each of the directories were a convenient way to keep cv.ra changes staged.  One person would edit the cv.ra in the alpha directory and see changes with a "make alpha DBS", the QA person would periodically copy the alpha/cv.ra to beta/cv.ra, creating a snap shot to QA.  When ready for release, they QA person would copy the final beta/cv.ra version to public/cv.ra, thus again creating a third public snapshot.


Previously the process of "make public DBS=hg##" would use the copy of cv.ra in the public directory to populate beta-public. Then a request to push trackDb and friends took beta-public cv.ra and brought it to the RR.  That changed in 2013 when we moved machines to San Diego.  While the request to push trackDb and friends to the RR in 2013 would still take the cv.ra from beta-public, getting that cv.ra to beta-public is was different in 2013.
Previously the process of "make public DBS=$db" would use the copy of cv.ra in the public directory to populate beta-public. Then a request to push trackDb and friends took beta-public cv.ra and brought it to the RR.  That changed in 2013 when we moved machines to San Diego.  While the request to push trackDb and friends to the RR in 2013 would still take the cv.ra from beta-public, getting that cv.ra to beta-public is was different in 2013.


It all changed once again in 2014 when it cv.ra was axed completely from the trackDb make and push process.
It all changed once again in 2014 when it cv.ra was axed completely from the trackDb make and push process.  For almost all purposes cv.ra is frozen, the metadata we control is not changing, but the links can go stale that point to antibody and cell suppliers.
 
Note, during genome-asia installation a file ''/data/apache/cgi-bin/encode/cv.ra'' was required in a push.


==Current Release Process==
==Current Release Process==
Now, one must request a push of the copy of cv.ra in beta to the beta-public machine (and for housekeeping purposes, also copy the beta/cv.ra version to public/cv.ra).
Now, one must uncomment lines in the trackDb makefile to populate cv.ra changes on alpha and beta. Then push it everywhere. 
 
Momentarily return lines in alpha and beta:
<pre>
#      rm -f ${CGI_BIN}/encode/cv.ra
#      cp -p cv/alpha/cv.ra ${CGI_BIN}/encode
 
 
#      rsync -a cv/beta/cv.ra ${CGI_BIN}-beta/encode/cv.ra
#      rsync -a cv/beta/cv.ra ${BETA_MACHINE}:${CGI_BIN}/encode/cv.ra
</pre>
Then run make, example: "make alpha DBS=$db"  and "make beta DBS=$db".  (any $db will work, a smaller one will go faster)
Check your changes on alpha/beta.
Remember to return your makefile to normal by doing a "git checkout makefile".
 
 
Subject: Tri-push of cv.ra from beta machine to beta-public, RR, and hgdownload '''be sure to include genome-asia, genome-euro''' in request
<pre>
Dear Pushers,
 
This is an unusual push for the older cv.ra file that was used during the ENCODE process.
 
Please tri-push cv.ra form beta machine to beta-public, RR, and hgdownload machines:
 
For beta-public
 
/usr/local/apache/cgi-bin/encode/cv.ra  
      hgwbeta --> to hgwbeta-public at:
/usr/local/apache/cgi-bin-public/encode/cv.ra
 
AND RR/Euro/Asia
 
/usr/local/apache/cgi-bin/encode/cv.ra   
    hgwbeta --> to the RR at:
/usr/local/apache/cgi-bin/encode/cv.ra
 
AND hgdownload
 
/usr/local/apache/cgi-bin/encode/cv.ra   
    hgwbeta --> to hgdownload at:
/usr/local/apache/htdocs/goldenPath/encodeDCC/cv.ra
 
Reason: Infrequent cv.ra changes for public release.


Thanks!
Brian
</pre>


Please push cv.ra form beta machine to public machine:
/usr/local/apache/cgi-bin/encode/cv.ra   
      hgwbeta --> hgwbeta-public
/usr/local/apache/cgi-bin-public/encode/cv.ra
Reason: cv.ra changes ready for public release.


Overview:
Overview:
*1) Original edits start at /kent/src/hg/makeDb/trackDb/cv/alpha/cv.ra,  do "make alpha DBS=hg##" to see changes.  
*1) Original edits start at /kent/src/hg/makeDb/trackDb/cv/alpha/cv.ra,  do "make alpha DBS=$db" to see changes (once reinstated makefile lines).  
*2)  FInal version of alpha copied from alpha/cv.ra to beta/cv.ra and public/cv.ra  
*2)  FInal version of alpha copied from alpha/cv.ra to beta/cv.ra and public/cv.ra Make sure you commit and push changes when you are satisfied.
*3)  Do "make beta DBS=hg##" to double check changes on beta. (If changes needed, start edits on alpha and copy to beta &public when done).
*3)  Do "make beta DBS=$db" to double check changes on beta. (If changes needed, start edits on alpha and copy to beta &public when done -note you must have reinstated the makelines).
*4)  Request a push  /usr/local/apache/cgi-bin/encode/cv.ra from       hgwbeta --> hgwbeta-public   /usr/local/apache/cgi-bin-public/encode/cv.ra
*4)  Request a push  /usr/local/apache/cgi-bin/encode/cv.ra from   (send above email) 
*5)  Do "make public DBS=hg##" and check that cv.ra changes are on beta-public
    hgwbeta --> hgwbeta-public at  /usr/local/apache/cgi-bin-public/encode/cv.ra
*6) Request a push of trackDb and friends for the related $db.
as well as to
*7)  Check RR to see that cv.ra change is now public. 
    hgwbeta --> to the RR at  /usr/local/apache/cgi-bin-public/encode/cv.ra
*8)  Housecleaning:  do a diff alpha/cv.ra beta/cv.ra public/cv.ra  (in the least beta and public should be the same, alpha only different if you have more changes coming).  Also you may want to navigate and download and check to see that your cv.ra changes ended up here:  http://hgdownload.soe.ucsc.edu/goldenPath/encodeDCC/cv.ra,  that is part of the regular push of beta-public to RR for cv.ra process.
as well as to
    hgwbeta --> to hgdownload at /usr/local/apache/htdocs/goldenPath/encodeDCC/cv.ra  
*5)  Spot check that cv.ra changes are on beta-public and the RR
*6)  Housecleaning:  do a diff alpha/cv.ra beta/cv.ra public/cv.ra  (in the least beta and public should be the same, alpha only different if you have more changes coming since you started the process).  Also you may want to navigate and download and check to see that your cv.ra changes ended up here:  http://hgdownload.soe.ucsc.edu/goldenPath/encodeDCC/cv.ra,  (that previously was part of the regular push of beta-public to RR for cv.ra process, now it must be requested as described in the tri-push above).

Latest revision as of 16:55, 29 August 2018

Background

cv.ra stands for controlled vocabulary, it was a way to be sure everyone was talking about the same cell or antibody or lab when discussing metada (data that describes data).

History

The controlled vocabulary file, cv.ra, exists at the trackDb level, in a directory called cv, with three sub directories, alpha, beta, and public.

/kent/src/hg/makeDb/trackDb/cv 
                           |_/alpha
                           |_/beta
                           |_/public

Each of the directories were a convenient way to keep cv.ra changes staged. One person would edit the cv.ra in the alpha directory and see changes with a "make alpha DBS", the QA person would periodically copy the alpha/cv.ra to beta/cv.ra, creating a snap shot to QA. When ready for release, they QA person would copy the final beta/cv.ra version to public/cv.ra, thus again creating a third public snapshot.

Previously the process of "make public DBS=$db" would use the copy of cv.ra in the public directory to populate beta-public. Then a request to push trackDb and friends took beta-public cv.ra and brought it to the RR. That changed in 2013 when we moved machines to San Diego. While the request to push trackDb and friends to the RR in 2013 would still take the cv.ra from beta-public, getting that cv.ra to beta-public is was different in 2013.

It all changed once again in 2014 when it cv.ra was axed completely from the trackDb make and push process. For almost all purposes cv.ra is frozen, the metadata we control is not changing, but the links can go stale that point to antibody and cell suppliers.

Note, during genome-asia installation a file /data/apache/cgi-bin/encode/cv.ra was required in a push.

Current Release Process

Now, one must uncomment lines in the trackDb makefile to populate cv.ra changes on alpha and beta. Then push it everywhere.

Momentarily return lines in alpha and beta:

#       rm -f ${CGI_BIN}/encode/cv.ra
#       cp -p cv/alpha/cv.ra ${CGI_BIN}/encode


#       rsync -a cv/beta/cv.ra ${CGI_BIN}-beta/encode/cv.ra
#       rsync -a cv/beta/cv.ra ${BETA_MACHINE}:${CGI_BIN}/encode/cv.ra

Then run make, example: "make alpha DBS=$db" and "make beta DBS=$db". (any $db will work, a smaller one will go faster) Check your changes on alpha/beta. Remember to return your makefile to normal by doing a "git checkout makefile".


Subject: Tri-push of cv.ra from beta machine to beta-public, RR, and hgdownload be sure to include genome-asia, genome-euro in request

Dear Pushers, 

This is an unusual push for the older cv.ra file that was used during the ENCODE process.

Please tri-push cv.ra form beta machine to beta-public, RR, and hgdownload machines: 

For beta-public

/usr/local/apache/cgi-bin/encode/cv.ra    
      hgwbeta --> to hgwbeta-public at:
/usr/local/apache/cgi-bin-public/encode/cv.ra

AND RR/Euro/Asia

/usr/local/apache/cgi-bin/encode/cv.ra    
     hgwbeta --> to the RR at:
/usr/local/apache/cgi-bin/encode/cv.ra

AND hgdownload

/usr/local/apache/cgi-bin/encode/cv.ra    
     hgwbeta --> to hgdownload at:
/usr/local/apache/htdocs/goldenPath/encodeDCC/cv.ra 

Reason: Infrequent cv.ra changes for public release.

Thanks!
Brian


Overview:

  • 1) Original edits start at /kent/src/hg/makeDb/trackDb/cv/alpha/cv.ra, do "make alpha DBS=$db" to see changes (once reinstated makefile lines).
  • 2) FInal version of alpha copied from alpha/cv.ra to beta/cv.ra and public/cv.ra Make sure you commit and push changes when you are satisfied.
  • 3) Do "make beta DBS=$db" to double check changes on beta. (If changes needed, start edits on alpha and copy to beta &public when done -note you must have reinstated the makelines).
  • 4) Request a push /usr/local/apache/cgi-bin/encode/cv.ra from (send above email)
    hgwbeta --> hgwbeta-public  at  /usr/local/apache/cgi-bin-public/encode/cv.ra

as well as to

    hgwbeta --> to the RR at  /usr/local/apache/cgi-bin-public/encode/cv.ra

as well as to

    hgwbeta --> to hgdownload at /usr/local/apache/htdocs/goldenPath/encodeDCC/cv.ra 
  • 5) Spot check that cv.ra changes are on beta-public and the RR
  • 6) Housecleaning: do a diff alpha/cv.ra beta/cv.ra public/cv.ra (in the least beta and public should be the same, alpha only different if you have more changes coming since you started the process). Also you may want to navigate and download and check to see that your cv.ra changes ended up here: http://hgdownload.soe.ucsc.edu/goldenPath/encodeDCC/cv.ra, (that previously was part of the regular push of beta-public to RR for cv.ra process, now it must be requested as described in the tri-push above).