Updating QA crontabs: Difference between revisions

From Genecats
Jump to navigationJump to search
No edit summary
 
(13 intermediate revisions by 2 users not shown)
Line 1: Line 1:
This page is about changing the nightly cron jobs that run on hgwdev to a controlled fashion.  
This page is about changing the nightly cron jobs that run on hgwdev to a controlled fashion. Here is a wikipage of the [http://genomewiki.ucsc.edu/genecats/index.php/Monitoring_Tasks monitoring tasks].
 
Per crontab man page:
 
''Each user can have their own crontab, and though
these are files in /var/spool/cron/crontabs, they are not
intended to be edited directly.''


''DO NOT EDIT THE QATEAM hgwdev CRONTAB FILE DIRECTLY''
''DO NOT EDIT THE QATEAM hgwdev CRONTAB FILE DIRECTLY''


===Summary===
===Summary===
* Edit (in your home dir): genecats/qa/crontabs/hgwdev.crontab.  
* Edit (in your home dir): genecats/qa/crontabs/hgwdev.crontab.  
* Make a file with the contents of QA Team's hgwdev crontab file, and do a diff with YOUR genecats file you just edited.
* From qateam@hgwdev, make a backup copy of the crontab file, and do a diff with YOUR genecats file you just edited.
* Run a command to update QA Team's hgwdev crontab file so that it reflects your changes.  
* From qateam@hgwdev, run a command to update, or "install" your changes (from ~/genecats/qa/crontabs/hgwdev.crontab) into the crontab on qateam@hgwdev.
 
 


===Steps===
===Steps===
Line 25: Line 34:
'''Step 4.'''  [OPTIONAL STEP, only needed for a new job] If you have a new cronjob, test your changes from qateam, for example in the above run <tt>/cluster/bin/x86_64/hubPublicCheck hubPublic  -udcDir=/data/tmp/qa</tt> and your other commands to see if they will work as expected from the home directory.
'''Step 4.'''  [OPTIONAL STEP, only needed for a new job] If you have a new cronjob, test your changes from qateam, for example in the above run <tt>/cluster/bin/x86_64/hubPublicCheck hubPublic  -udcDir=/data/tmp/qa</tt> and your other commands to see if they will work as expected from the home directory.


'''Step 5.'''  Once you are happy with your edits, commit & push your changes. From your own ~/genecats directory, do a git status, add hgwdev.crontab, commit, and push.  
'''Step 5.'''  Once you are happy with your edits, '''commit & push your changes.''' From your own ~/genecats directory, do a git status, add hgwdev.crontab, commit, and push.  


'''Step 6.'''  Ok, now YOUR genecats crontab has been updated, but it now needs to be sync'd to the crontab on qateam@hgwdev. This step provides instructions on how to compare YOUR genecats crontab with QA Team's hgwdev crontab.  
'''Step 6.'''  Ok, now YOUR genecats crontab has been updated, but it now needs to be sync'd to the crontab on qateam@hgwdev. This step provides instructions on how to compare YOUR genecats crontab with QA Team's hgwdev crontab.  
#From qateam@hgwdev, output the current crontab with "crontab -l", which essentially creates a copy of the crontab. Note that this file does not have your edits yet, because you made your edits in YOUR genecats repo, not to the crontab on QA Team's hgwdev.   
::From qateam@hgwdev, output the current crontab with "crontab -l", which essentially creates a copy of the crontab. Note that this file does not have your edits yet, because you made your edits in YOUR genecats repo, not to the crontab on QA Team's hgwdev.   
#You can put this in a temp file such as <tt>crontab -l > tmp/deleteMe/genecats.hgwdev.crontab.[today'sDate]</tt>, but it really doesn't matter where you put this file. You could even put it in your home directory, such as <tt>~/temp/crontab.hgwdev</tt>
::You can put this in a temp file such as <tt>crontab -l > /data/tmp/deleteMeFile</tt>
::If you put this file elsewhere, be sure to delete it.
 
'''Step 7.'''  Now that you have your comparison file, you can compare the two to be sure your changes are all you are updating (where <tt>brianlee</tt> is changed to your directory). 
::<tt>diff  /data/tmp/deleteMeFile ~brianlee/genecats/qa/crontabs/hgwdev.crontab</tt>
::Once you do the diff, you should only see the line/s that you added or edited.
'''Step 8.''' From qateam@hgwdev, you now need to update the crontab on qateam@hgwdev by "installing" your revised genecats version:
:<tt>crontab ~brianlee/genecats/qa/crontabs/hgwdev.crontab </tt>
Now your crontab from your home dir (in the genecats repository) will match the crontab for QA Team's hgwdev, and you are done!
 
===Example Commands===
* [mspeir@hgwdev]$ ssh qateam@hgwdev
* [qateam@hgwdev /cluster/home/qateam] crontab -l > ~mspeir/temp/qateam.dev.crontab
* [qateam@hgwdev /cluster/home/qateam] diff ~mspeir/temp/qateam.dev.crontab ~mspeir/genecats/qa/crontabs/hgwdev.crontab 
* [qateam@hgwdev /cluster/home/qateam] crontab ~mspeir/genecats/qa/crontabs/hgwdev.crontab


'''Step 7.'''  Now that you have your comparison file, you can compare the two to be sure your changes are all you are updating (where <tt>brianlee</tt> is changed to your directory).  <tt>diff /data/tmp/deleteMe ~brianlee/genecats/qa/crontabs/hgwdev.crontab</tt>. Once you do the diff, you should only see the line/s that you added or edited.
# From qateam@hgwdev, you now need to update QA Team's hgwdev current crontab by "running" your revised genecats version: <tt>crontab ~brianlee/genecats/qa/crontabs/hgwdev.crontab </tt>. Now your crontab from your home dir (in the genecats repository) will match the crontab for QA Team's hgwdev.


===Resources===
===Resources===
Test your cron setting: [http://cron.schlitt.info/ http://cron.schlitt.info/]
Test your cron setting: [http://cron.schlitt.info/ http://cron.schlitt.info/]
http://askubuntu.com/questions/216692/where-is-the-user-crontab-stored
===Other Machines===
Note that genome-asia and genome-euro have their own cronjobs for qateam.
When you log in to genome-asia, you can assume the qateam identity with  <code>sudo /bin/su - qateam</code>  and then leave with <code>exit</code>.


[[Category:Browser QA]]
[[Category:Browser QA]]

Latest revision as of 22:34, 14 December 2017

This page is about changing the nightly cron jobs that run on hgwdev to a controlled fashion. Here is a wikipage of the monitoring tasks.

Per crontab man page:

Each user can have their own crontab, and though these are files in /var/spool/cron/crontabs, they are not intended to be edited directly.

DO NOT EDIT THE QATEAM hgwdev CRONTAB FILE DIRECTLY


Summary

  • Edit (in your home dir): genecats/qa/crontabs/hgwdev.crontab.
  • From qateam@hgwdev, make a backup copy of the crontab file, and do a diff with YOUR genecats file you just edited.
  • From qateam@hgwdev, run a command to update, or "install" your changes (from ~/genecats/qa/crontabs/hgwdev.crontab) into the crontab on qateam@hgwdev.


Steps

Step 1. Go to your genecats repository and do a git pull to bring it up to date.

Step 2. Edit your genecats/qa/crontabs/hgwdev.crontab (add a new cronjob, change an exisiting cronjob, etc.). For example:

MAILTO=braney@soe.ucsc.edu,brianlee@soe.ucsc.edu
HGDB_CONF=.hg.conf.beta

15 03 * * * /cluster/bin/x86_64/hubPublicCheck hubPublic  -udcDir=/data/tmp/qa
16 02 * * * /cluster/bin/x86_64/hgsql -e "select hubUrl from hubPublic" hgcentralbeta | tail -n +2 | while read url; do /cluster/bin/x86_64/hubCheck  -udcDir=/data/tmp/qa $url; done

(Reminder of first numbers: Minute then Hour DOM Month DOW command)

Step 3. In another terminal you can ssh qateam@hgwdev

Step 4. [OPTIONAL STEP, only needed for a new job] If you have a new cronjob, test your changes from qateam, for example in the above run /cluster/bin/x86_64/hubPublicCheck hubPublic -udcDir=/data/tmp/qa and your other commands to see if they will work as expected from the home directory.

Step 5. Once you are happy with your edits, commit & push your changes. From your own ~/genecats directory, do a git status, add hgwdev.crontab, commit, and push.

Step 6. Ok, now YOUR genecats crontab has been updated, but it now needs to be sync'd to the crontab on qateam@hgwdev. This step provides instructions on how to compare YOUR genecats crontab with QA Team's hgwdev crontab.

From qateam@hgwdev, output the current crontab with "crontab -l", which essentially creates a copy of the crontab. Note that this file does not have your edits yet, because you made your edits in YOUR genecats repo, not to the crontab on QA Team's hgwdev.
You can put this in a temp file such as crontab -l > /data/tmp/deleteMeFile
If you put this file elsewhere, be sure to delete it.

Step 7. Now that you have your comparison file, you can compare the two to be sure your changes are all you are updating (where brianlee is changed to your directory).

diff /data/tmp/deleteMeFile ~brianlee/genecats/qa/crontabs/hgwdev.crontab
Once you do the diff, you should only see the line/s that you added or edited.

Step 8. From qateam@hgwdev, you now need to update the crontab on qateam@hgwdev by "installing" your revised genecats version:

crontab ~brianlee/genecats/qa/crontabs/hgwdev.crontab

Now your crontab from your home dir (in the genecats repository) will match the crontab for QA Team's hgwdev, and you are done!

Example Commands

  • [mspeir@hgwdev]$ ssh qateam@hgwdev
  • [qateam@hgwdev /cluster/home/qateam] crontab -l > ~mspeir/temp/qateam.dev.crontab
  • [qateam@hgwdev /cluster/home/qateam] diff ~mspeir/temp/qateam.dev.crontab ~mspeir/genecats/qa/crontabs/hgwdev.crontab
  • [qateam@hgwdev /cluster/home/qateam] crontab ~mspeir/genecats/qa/crontabs/hgwdev.crontab


Resources

Test your cron setting: http://cron.schlitt.info/ http://askubuntu.com/questions/216692/where-is-the-user-crontab-stored

Other Machines

Note that genome-asia and genome-euro have their own cronjobs for qateam.

When you log in to genome-asia, you can assume the qateam identity with sudo /bin/su - qateam and then leave with exit.