Changes to cv.ra: Difference between revisions
No edit summary |
|||
(20 intermediate revisions by 2 users not shown) | |||
Line 1: | Line 1: | ||
==Background== | |||
cv.ra stands for controlled vocabulary, it was a way to be sure everyone was talking about the same cell or antibody or lab when discussing metada (data that describes data). | |||
==History== | |||
The controlled vocabulary file, cv.ra, exists at the trackDb level, in a directory called cv, with three sub directories, alpha, beta, and public. | The controlled vocabulary file, cv.ra, exists at the trackDb level, in a directory called cv, with three sub directories, alpha, beta, and public. | ||
/kent/src/hg/makeDb/trackDb/cv | /kent/src/hg/makeDb/trackDb/cv | ||
Line 7: | Line 10: | ||
Each of the directories were a convenient way to keep cv.ra changes staged. One person would edit the cv.ra in the alpha directory and see changes with a "make alpha DBS", the QA person would periodically copy the alpha/cv.ra to beta/cv.ra, creating a snap shot to QA. When ready for release, they QA person would copy the final beta/cv.ra version to public/cv.ra, thus again creating a third public snapshot. | Each of the directories were a convenient way to keep cv.ra changes staged. One person would edit the cv.ra in the alpha directory and see changes with a "make alpha DBS", the QA person would periodically copy the alpha/cv.ra to beta/cv.ra, creating a snap shot to QA. When ready for release, they QA person would copy the final beta/cv.ra version to public/cv.ra, thus again creating a third public snapshot. | ||
Previously the process of "make public DBS= | Previously the process of "make public DBS=$db" would use the copy of cv.ra in the public directory to populate beta-public. Then a request to push trackDb and friends took beta-public cv.ra and brought it to the RR. That changed in 2013 when we moved machines to San Diego. While the request to push trackDb and friends to the RR in 2013 would still take the cv.ra from beta-public, getting that cv.ra to beta-public is was different in 2013. | ||
It all changed once again in 2014 when it cv.ra was axed completely from the trackDb make and push process. For almost all purposes cv.ra is frozen, the metadata we control is not changing, but the links can go stale that point to antibody and cell suppliers. | |||
Note, during genome-asia installation a file ''/data/apache/cgi-bin/encode/cv.ra'' was required in a push. | |||
==Current Release Process== | |||
Now, one must uncomment lines in the trackDb makefile to populate cv.ra changes on alpha and beta. Then push it everywhere. | |||
Momentarily return lines in alpha and beta: | |||
<pre> | |||
# rm -f ${CGI_BIN}/encode/cv.ra | |||
# cp -p cv/alpha/cv.ra ${CGI_BIN}/encode | |||
# rsync -a cv/beta/cv.ra ${CGI_BIN}-beta/encode/cv.ra | |||
# rsync -a cv/beta/cv.ra ${BETA_MACHINE}:${CGI_BIN}/encode/cv.ra | |||
</pre> | |||
Then run make, example: "make alpha DBS=$db" and "make beta DBS=$db". (any $db will work, a smaller one will go faster) | |||
Check your changes on alpha/beta. | |||
Remember to return your makefile to normal by doing a "git checkout makefile". | |||
Subject: Tri-push of cv.ra from beta machine to beta-public, RR, and hgdownload '''be sure to include genome-asia, genome-euro''' in request | |||
<pre> | |||
Dear Pushers, | |||
This is an unusual push for the older cv.ra file that was used during the ENCODE process. | |||
Please tri-push cv.ra form beta machine to beta-public, RR, and hgdownload machines: | |||
For beta-public | |||
/usr/local/apache/cgi-bin/encode/cv.ra | |||
hgwbeta --> to hgwbeta-public at: | |||
/usr/local/apache/cgi-bin-public/encode/cv.ra | |||
AND RR/Euro/Asia | |||
/usr/local/apache/cgi-bin/encode/cv.ra | |||
hgwbeta --> to the RR at: | |||
/usr/local/apache/cgi-bin/encode/cv.ra | |||
AND hgdownload | |||
/usr/local/apache/cgi-bin/encode/cv.ra | |||
hgwbeta --> to hgdownload at: | |||
/usr/local/apache/htdocs/goldenPath/encodeDCC/cv.ra | |||
Reason: Infrequent cv.ra changes for public release. | |||
Thanks! | |||
Brian | |||
</pre> | |||
Overview: | Overview: | ||
*1) Original edits start at /kent/src/hg/makeDb/trackDb/cv/alpha/cv.ra, do "make alpha DBS= | *1) Original edits start at /kent/src/hg/makeDb/trackDb/cv/alpha/cv.ra, do "make alpha DBS=$db" to see changes (once reinstated makefile lines). | ||
*2) FInal version of alpha copied from alpha/cv.ra to beta/cv.ra and public/cv.ra | *2) FInal version of alpha copied from alpha/cv.ra to beta/cv.ra and public/cv.ra Make sure you commit and push changes when you are satisfied. | ||
*3) Do "make beta DBS= | *3) Do "make beta DBS=$db" to double check changes on beta. (If changes needed, start edits on alpha and copy to beta &public when done -note you must have reinstated the makelines). | ||
*4) Request a push /usr/local/apache/cgi-bin/encode/cv.ra from | *4) Request a push /usr/local/apache/cgi-bin/encode/cv.ra from (send above email) | ||
*5) | hgwbeta --> hgwbeta-public at /usr/local/apache/cgi-bin-public/encode/cv.ra | ||
as well as to | |||
hgwbeta --> to the RR at /usr/local/apache/cgi-bin-public/encode/cv.ra | |||
* | as well as to | ||
hgwbeta --> to hgdownload at /usr/local/apache/htdocs/goldenPath/encodeDCC/cv.ra | |||
*5) Spot check that cv.ra changes are on beta-public and the RR | |||
*6) Housecleaning: do a diff alpha/cv.ra beta/cv.ra public/cv.ra (in the least beta and public should be the same, alpha only different if you have more changes coming since you started the process). Also you may want to navigate and download and check to see that your cv.ra changes ended up here: http://hgdownload.soe.ucsc.edu/goldenPath/encodeDCC/cv.ra, (that previously was part of the regular push of beta-public to RR for cv.ra process, now it must be requested as described in the tri-push above). |
Latest revision as of 16:55, 29 August 2018
Background
cv.ra stands for controlled vocabulary, it was a way to be sure everyone was talking about the same cell or antibody or lab when discussing metada (data that describes data).
History
The controlled vocabulary file, cv.ra, exists at the trackDb level, in a directory called cv, with three sub directories, alpha, beta, and public.
/kent/src/hg/makeDb/trackDb/cv |_/alpha |_/beta |_/public
Each of the directories were a convenient way to keep cv.ra changes staged. One person would edit the cv.ra in the alpha directory and see changes with a "make alpha DBS", the QA person would periodically copy the alpha/cv.ra to beta/cv.ra, creating a snap shot to QA. When ready for release, they QA person would copy the final beta/cv.ra version to public/cv.ra, thus again creating a third public snapshot.
Previously the process of "make public DBS=$db" would use the copy of cv.ra in the public directory to populate beta-public. Then a request to push trackDb and friends took beta-public cv.ra and brought it to the RR. That changed in 2013 when we moved machines to San Diego. While the request to push trackDb and friends to the RR in 2013 would still take the cv.ra from beta-public, getting that cv.ra to beta-public is was different in 2013.
It all changed once again in 2014 when it cv.ra was axed completely from the trackDb make and push process. For almost all purposes cv.ra is frozen, the metadata we control is not changing, but the links can go stale that point to antibody and cell suppliers.
Note, during genome-asia installation a file /data/apache/cgi-bin/encode/cv.ra was required in a push.
Current Release Process
Now, one must uncomment lines in the trackDb makefile to populate cv.ra changes on alpha and beta. Then push it everywhere.
Momentarily return lines in alpha and beta:
# rm -f ${CGI_BIN}/encode/cv.ra # cp -p cv/alpha/cv.ra ${CGI_BIN}/encode # rsync -a cv/beta/cv.ra ${CGI_BIN}-beta/encode/cv.ra # rsync -a cv/beta/cv.ra ${BETA_MACHINE}:${CGI_BIN}/encode/cv.ra
Then run make, example: "make alpha DBS=$db" and "make beta DBS=$db". (any $db will work, a smaller one will go faster) Check your changes on alpha/beta. Remember to return your makefile to normal by doing a "git checkout makefile".
Subject: Tri-push of cv.ra from beta machine to beta-public, RR, and hgdownload be sure to include genome-asia, genome-euro in request
Dear Pushers, This is an unusual push for the older cv.ra file that was used during the ENCODE process. Please tri-push cv.ra form beta machine to beta-public, RR, and hgdownload machines: For beta-public /usr/local/apache/cgi-bin/encode/cv.ra hgwbeta --> to hgwbeta-public at: /usr/local/apache/cgi-bin-public/encode/cv.ra AND RR/Euro/Asia /usr/local/apache/cgi-bin/encode/cv.ra hgwbeta --> to the RR at: /usr/local/apache/cgi-bin/encode/cv.ra AND hgdownload /usr/local/apache/cgi-bin/encode/cv.ra hgwbeta --> to hgdownload at: /usr/local/apache/htdocs/goldenPath/encodeDCC/cv.ra Reason: Infrequent cv.ra changes for public release. Thanks! Brian
Overview:
- 1) Original edits start at /kent/src/hg/makeDb/trackDb/cv/alpha/cv.ra, do "make alpha DBS=$db" to see changes (once reinstated makefile lines).
- 2) FInal version of alpha copied from alpha/cv.ra to beta/cv.ra and public/cv.ra Make sure you commit and push changes when you are satisfied.
- 3) Do "make beta DBS=$db" to double check changes on beta. (If changes needed, start edits on alpha and copy to beta &public when done -note you must have reinstated the makelines).
- 4) Request a push /usr/local/apache/cgi-bin/encode/cv.ra from (send above email)
hgwbeta --> hgwbeta-public at /usr/local/apache/cgi-bin-public/encode/cv.ra
as well as to
hgwbeta --> to the RR at /usr/local/apache/cgi-bin-public/encode/cv.ra
as well as to
hgwbeta --> to hgdownload at /usr/local/apache/htdocs/goldenPath/encodeDCC/cv.ra
- 5) Spot check that cv.ra changes are on beta-public and the RR
- 6) Housecleaning: do a diff alpha/cv.ra beta/cv.ra public/cv.ra (in the least beta and public should be the same, alpha only different if you have more changes coming since you started the process). Also you may want to navigate and download and check to see that your cv.ra changes ended up here: http://hgdownload.soe.ucsc.edu/goldenPath/encodeDCC/cv.ra, (that previously was part of the regular push of beta-public to RR for cv.ra process, now it must be requested as described in the tri-push above).