Genome-preview machine: Difference between revisions
(→Updating Genome-Preview with pushDevToPreview: correcting style list location) |
|||
| (22 intermediate revisions by 4 users not shown) | |||
| Line 14: | Line 14: | ||
The genome-preview machine is updated on a weekly basis like so: | The genome-preview machine is updated on a weekly basis like so: | ||
On Friday Mornings (5:27 am) a build crontab makes the CGIs for genome preview. | |||
On Friday nights (11:33 pm), the following stuff is copied from hgwdev: | On Friday nights (11:33 pm), the following stuff is copied from hgwdev: | ||
* mysql tables (excluding mysql, customTrash, and other things) | * mysql tables (excluding mysql, customTrash, and other things) | ||
| Line 29: | Line 29: | ||
==Updating Genome-Preview with pushDevToPreview== | ==Updating Genome-Preview with pushDevToPreview== | ||
If | '''PREPARATION from your hgwdev kent repository clean your tree and update your libraries. | ||
If your libraries are out of date or your repository is not clean you can run into errors when making your CGIs. | |||
Be sure your branch is current. If you are seeing errors on the build process ''please ask for help.'' | |||
'''NOW from your hgwdev kent repository''' complete the below steps to update CGIs/js/style files. | |||
Please make sure that you do not have stuff that is not checked in | |||
and your git repo working directories are clean so you will not | |||
get merge conflicts etc. | |||
'''Automated Method''' | |||
Run one script to do it all. | |||
Be sure to have the genecats repo installed in your home dir. | |||
Update it first. | |||
<pre> | <pre> | ||
pushd ~/genecats | |||
git pull | |||
popd | |||
~/genecats/qa/testTools/previewUpdate | |||
</pre> | |||
'''Manual Method''' | |||
<pre> | |||
cd ~/kent/src | |||
git pull | |||
make clean > /dev/null | |||
cd ~/kent/src/hg/lib | |||
make | |||
</pre> | </pre> | ||
| Line 50: | Line 67: | ||
<pre> | <pre> | ||
cd ~/kent/src/hg/js | cd ~/kent/src/hg/js | ||
make USER=preview | make USER=preview | ||
</pre> | </pre> | ||
| Line 57: | Line 73: | ||
<pre> | <pre> | ||
cd ~/kent/src/hg/htdocs/style | cd ~/kent/src/hg/htdocs/style | ||
make alpha | make alpha | ||
</pre> | </pre> | ||
| Line 64: | Line 79: | ||
<pre> | <pre> | ||
cd ~/kent/src | cd ~/kent/src | ||
USER=preview make cgi | USER=preview make cgi | ||
</pre> | </pre> | ||
Be sure to login as qateam on hgwdev before starting any of these operations. | |||
<pre> | |||
ssh qateam@hgwdev | |||
</pre> | |||
If there is a need to sync the CGIs on hgwdev to genome-preview the '''pushDevToPreview script''' | |||
<pre> | |||
pushDevToPreview -h | |||
Usage: pushDevToPreview [-chiqvy] | |||
-c | --check : Don't do the push, just say what would be pushed. | |||
-h | --help : Print this message. | |||
-i : Interactive mode. Ask me before doing any command. | |||
-q | --quiet : Quiet mode. Don't print anything unless necessary. | |||
-v | --verbose : Increase verbosity. | |||
-y | --noprompt : Just do it, don't ask me. | |||
</pre> | |||
Once all your files are updated, you should run the command with the -c --check option to see what will be updated: '''pushDevToPreview -c''' | |||
If things look pretty good, then you are ready to run the command, with the -i interactive mode so you can see what commands are being run: | If things look pretty good, then you are ready to run the command, with the -i interactive mode so you can see what commands are being run: | ||
Latest revision as of 00:24, 26 January 2022
In early 2011, we created a new machine called genome-preview, which is a mirror of our development machine: genome-test.
It is here: http://genome-preview.soe.ucsc.edu/
It has these warnings on the home page:
WARNING: This is our preview site. This website is a weekly mirror of our internal development server for public access. Data and tools here are under construction, have not been quality reviewed, and are subject to change at any time. We provide this site for early access, with the warning that it is less available and stable than our public site. For high-quality reviewed annotations on our production server, visit http://genome.ucsc.edu.
In general, we do not push updates/files/tables/patches to this machine by hand. It is generally understood that it will sometimes be out of commission, as it is a mirror of a development machine.
The impetus behind this machine was to allow early access to the ENCODE data by ENCODE consortia members.
Automated Updates
The genome-preview machine is updated on a weekly basis like so: On Friday Mornings (5:27 am) a build crontab makes the CGIs for genome preview. On Friday nights (11:33 pm), the following stuff is copied from hgwdev:
- mysql tables (excluding mysql, customTrash, and other things)
- hgcentraltest tables (excluding userDb, sessionDb, namedSessionDb and hubStatus)
- CGIs
- htdocs
Most of the stuff on /gbdb comes from backups of the /hive/data and /hive/groups.
- /hive/data is backed up on Fridays at 5:22 am
- /hive/groups is backed up on Wednesdays at 5:22 am.
Note that this may cause some inconsistencies if new data is added on Thursday to /hive/groups and accessed by new CGIs, since the data will not be available on the backups until the following Wednesday.
Updating Genome-Preview with pushDevToPreview
PREPARATION from your hgwdev kent repository clean your tree and update your libraries.
If your libraries are out of date or your repository is not clean you can run into errors when making your CGIs. Be sure your branch is current. If you are seeing errors on the build process please ask for help.
NOW from your hgwdev kent repository complete the below steps to update CGIs/js/style files.
Please make sure that you do not have stuff that is not checked in and your git repo working directories are clean so you will not get merge conflicts etc.
Automated Method
Run one script to do it all. Be sure to have the genecats repo installed in your home dir. Update it first.
pushd ~/genecats git pull popd ~/genecats/qa/testTools/previewUpdate
Manual Method
cd ~/kent/src git pull make clean > /dev/null cd ~/kent/src/hg/lib make
Note: you'll first need to update the js files for preview at:
ls -l /usr/local/apache/htdocs/js/preview/
cd ~/kent/src/hg/js make USER=preview
Note: you'll first need to update the style files for hgwdev at: ls -lrt /usr/local/apache/htdocs/style
cd ~/kent/src/hg/htdocs/style make alpha
Note: you'll first need to update the CGIs for preview at: ls -lrt /usr/local/apache/cgi-bin-preview/
cd ~/kent/src USER=preview make cgi
Be sure to login as qateam on hgwdev before starting any of these operations.
ssh qateam@hgwdev
If there is a need to sync the CGIs on hgwdev to genome-preview the pushDevToPreview script
pushDevToPreview -h Usage: pushDevToPreview [-chiqvy] -c | --check : Don't do the push, just say what would be pushed. -h | --help : Print this message. -i : Interactive mode. Ask me before doing any command. -q | --quiet : Quiet mode. Don't print anything unless necessary. -v | --verbose : Increase verbosity. -y | --noprompt : Just do it, don't ask me.
Once all your files are updated, you should run the command with the -c --check option to see what will be updated: pushDevToPreview -c
If things look pretty good, then you are ready to run the command, with the -i interactive mode so you can see what commands are being run:
pushDevToPreview -i Pushing files from hgwdev to genome-preview Do you wish to proceed? (y/[n]): y Doing directory /usr/local/apache/cgi-bin-preview/ between hgwdev and genome-preview The command to be executed is /usr/bin/rsync -rlptDi --delete --chown=qateam:protein --exclude=hg.conf* /usr/local/apache/cgi-bin-preview/ genome-preview:/usr/local/apache/cgi-bin-preview/ ... ...
Finally you'll check on genome-preview if the changes are updated.
