Monitoring Tasks: Difference between revisions

From Genecats
Jump to navigationJump to search
No edit summary
(Adding column for updates.)
Line 4: Line 4:
<th>task</th>
<th>task</th>
<th>frequency</th>
<th>frequency</th>
<th>who NOW (starting Aug '14)</th>
<th>who NOW (starting Sept '15)</th>
<th>who Aug '14 - Aug '15</th>
<th>who Aug '13 - Jul '14</th>
<th>who Aug '13 - Jul '14</th>
<th>who before</th>
<th>who before</th>
Line 10: Line 11:


<tr>
<tr>
<td colspan="5" style="background-color:gray"><b>from qateam cron</b></td>
<td colspan="6" style="background-color:gray"><b>from qateam cron</b></td>
</tr>
</tr>


Line 16: Line 17:
<td>ask Galt/Jim about blat updates</td>
<td>ask Galt/Jim about blat updates</td>
<td>3x per year</td>
<td>3x per year</td>
<td> </td>
<td>Steve, Jonathan</td>
<td>Steve, Jonathan</td>
<td>Pauline, Steve, Jonathan</td>
<td>Pauline, Steve, Jonathan</td>
Line 24: Line 26:
<td>Respond to items on the broken links report: Check to make sure that broken links reported are actually broken.  If so, fix the link if it is clear what the new link should be and push to the RR.  If not, contact the appropriate person at the web site in question to find out what the new link should be.</td>
<td>Respond to items on the broken links report: Check to make sure that broken links reported are actually broken.  If so, fix the link if it is clear what the new link should be and push to the RR.  If not, contact the appropriate person at the web site in question to find out what the new link should be.</td>
<td>daily</td>
<td>daily</td>
<td> </td>
<td>Matt</td>
<td>Matt</td>
<td>Matt</td>
<td>Matt</td>
Line 32: Line 35:
<td>Respond to the trackDbGlobal report: Make public and request a push of trackDb and friends for any assemblies that show up as changed in the trackDbGlobal report.</td>
<td>Respond to the trackDbGlobal report: Make public and request a push of trackDb and friends for any assemblies that show up as changed in the trackDbGlobal report.</td>
<td>weekly</td>
<td>weekly</td>
<td> </td>
<td>Matt</td>
<td>Matt</td>
<td>Luvina</td>
<td>Luvina</td>
Line 40: Line 44:
<td>receive output from hgTracksRandom cron; alert admins to problems</td>
<td>receive output from hgTracksRandom cron; alert admins to problems</td>
<td>as needed</td>
<td>as needed</td>
<td> </td>
<td>many: kuhn, katrina, brianlee, braney, luvina, gary, ann, steve, jcasper, matt</td>
<td>many: kuhn, katrina, brianlee, braney, luvina, gary, ann, steve, jcasper, matt</td>
<td>many: kuhn, pauline, katrina, brianlee, braney, luvina, gary, ann, steve, jcasper, matt</td>
<td>many: kuhn, pauline, katrina, brianlee, braney, luvina, gary, ann, steve, jcasper, matt</td>
Line 48: Line 53:
<td>receive output from gbLoaded cron job and alert braney if anything seems very outdated</td>
<td>receive output from gbLoaded cron job and alert braney if anything seems very outdated</td>
<td>daily, problems every 3-6 months</td>
<td>daily, problems every 3-6 months</td>
<td> </td>
<td>Steve, Jonathan</td>
<td>Steve, Jonathan</td>
<td>Steve, Jonathan</td>
<td>Steve, Jonathan</td>
Line 56: Line 62:
<td>receive output from showProcesslist.csh</td>
<td>receive output from showProcesslist.csh</td>
<td>daily, no action</td>
<td>daily, no action</td>
<td> </td>
<td>Steve, Jonathan</td>
<td>Steve, Jonathan</td>
<td>Steve, Jonathan</td>
<td>Steve, Jonathan</td>
Line 64: Line 71:
<td>get output from checkTableStatus.csh on Mondays (this is a report on whether the genbank table dumps are up-to-date...if we could get rid of our dependence on those, we could get rid of this cron job)</td>
<td>get output from checkTableStatus.csh on Mondays (this is a report on whether the genbank table dumps are up-to-date...if we could get rid of our dependence on those, we could get rid of this cron job)</td>
<td>weekly</td>
<td>weekly</td>
<td> </td>
<td>Steve, Jonathan</td>
<td>Steve, Jonathan</td>
<td>Steve, Jonathan</td>
<td>Steve, Jonathan</td>
Line 72: Line 80:
<td>get an email from backupCentral.csh each month (there's a redmine ticket on whether we should change this one: http://redmine.soe.ucsc.edu/issues/7834)</td>
<td>get an email from backupCentral.csh each month (there's a redmine ticket on whether we should change this one: http://redmine.soe.ucsc.edu/issues/7834)</td>
<td>monthly</td>
<td>monthly</td>
<td> </td>
<td>Steve, Jonathan</td>
<td>Steve, Jonathan</td>
<td>Steve, Jonathan</td>
<td>Steve, Jonathan</td>
Line 80: Line 89:
<td>receive an email on additions and subtractions to files in /gbdb on hgnfs1</td>
<td>receive an email on additions and subtractions to files in /gbdb on hgnfs1</td>
<td>daily</td>
<td>daily</td>
<td> </td>
<td>Steve, Jonathan</td>
<td>Steve, Jonathan</td>
<td>Steve, Jonathan</td>
<td>Steve, Jonathan</td>
Line 88: Line 98:
<td>receive an email from cron if qateam DailySandboxUpdate fails</td>
<td>receive an email from cron if qateam DailySandboxUpdate fails</td>
<td>daily</td>
<td>daily</td>
<td> </td>
<td>Steve, Jonathan</td>
<td>Steve, Jonathan</td>
<td>Steve, Jonathan</td>
<td>Steve, Jonathan</td>
Line 96: Line 107:
<td>receive output from checkMetaAday.csh; alert (steve, jcasper) if anything is out of sync (compares hgcentralbeta to hgcentral for db of the day)</td>
<td>receive output from checkMetaAday.csh; alert (steve, jcasper) if anything is out of sync (compares hgcentralbeta to hgcentral for db of the day)</td>
<td>daily</td>
<td>daily</td>
<td> </td>
<td>Matt</td>
<td>Matt</td>
<td>Matt</td>
<td>Matt</td>
Line 104: Line 116:
<td>receive output from syncAday.csh; alert (steve, jcasper) if anything is out of sync (checks that tables are in sync between beta and RR for database of the day) </td>
<td>receive output from syncAday.csh; alert (steve, jcasper) if anything is out of sync (checks that tables are in sync between beta and RR for database of the day) </td>
<td>daily</td>
<td>daily</td>
<td> </td>
<td>Brian</td>
<td>Brian</td>
<td>Brian</td>
<td>Brian</td>
Line 112: Line 125:
<td>get output from checkGenomeMysql</td>
<td>get output from checkGenomeMysql</td>
<td>daily</td>
<td>daily</td>
<td> </td>
<td>Steve, Jonathan</td>
<td>Steve, Jonathan</td>
<td>Steve, Jonathan</td>
<td>Steve, Jonathan</td>
Line 120: Line 134:
<td>hubPublicCheck complains when the hubPublic table doesn't have the same strings as the actual hub.txt file. '' (NOTE: To run only on Beta hubs requires these declarations: export HGDB_CONF=$HOME/.hg.conf.beta and SHELL=/bin/sh )''</td>
<td>hubPublicCheck complains when the hubPublic table doesn't have the same strings as the actual hub.txt file. '' (NOTE: To run only on Beta hubs requires these declarations: export HGDB_CONF=$HOME/.hg.conf.beta and SHELL=/bin/sh )''</td>
<td>runs daily, complains rarely?</td>
<td>runs daily, complains rarely?</td>
<td> </td>
<td>Brian Lee</td>
<td>Brian Lee</td>
<td>Brian Lee</td>
<td>Brian Lee</td>
Line 128: Line 143:
<td>omimWrapper.sh checks for new OMIM data and downloads and loads it into hgwdev if so</td>
<td>omimWrapper.sh checks for new OMIM data and downloads and loads it into hgwdev if so</td>
<td>runs weekly, complains bi-weekly</td>
<td>runs weekly, complains bi-weekly</td>
<td> </td>
<td>Brian Lee</td>
<td>Brian Lee</td>
<td>braney</td>
<td>braney</td>
Line 136: Line 152:
<td>omimUploadWrapper.sh uploads the OMIM tables to OMIM</td>
<td>omimUploadWrapper.sh uploads the OMIM tables to OMIM</td>
<td>runs weekly, complains rarely</td>
<td>runs weekly, complains rarely</td>
<td> </td>
<td>Brian Lee</td>
<td>Brian Lee</td>
<td>braney</td>
<td>braney</td>
Line 144: Line 161:
<td>decipherWrapper.sh checks for new DECIPHER data and downloads and loads it into hgwdev if so</td>
<td>decipherWrapper.sh checks for new DECIPHER data and downloads and loads it into hgwdev if so</td>
<td>runs weekly, complains monthly</td>
<td>runs weekly, complains monthly</td>
<td> </td>
<td>Luvina</td>
<td>Luvina</td>
<td>braney</td>
<td>braney</td>
Line 152: Line 170:
<td>iscoWrapper.sh checks for new ISCA data and downloads and loads it into hgwdev if so</td>
<td>iscoWrapper.sh checks for new ISCA data and downloads and loads it into hgwdev if so</td>
<td>runs weekly, complains monthly</td>
<td>runs weekly, complains monthly</td>
<td> </td>
<td>Steve</td>
<td>Steve</td>
<td>braney</td>
<td>braney</td>
Line 160: Line 179:
<td>gwasWrapper.sh checks for new GWAS data and downloads and loads it into hgwdev if so</td>
<td>gwasWrapper.sh checks for new GWAS data and downloads and loads it into hgwdev if so</td>
<td>runs weekly, complains quarterly</td>
<td>runs weekly, complains quarterly</td>
<td> </td>
<td>Jonathan</td>
<td>Jonathan</td>
<td>braney</td>
<td>braney</td>
Line 168: Line 188:
<td>geneReviewsWrapper.sh checks for new GeneReviews data and downloads and loads it into hgwdev if so</td>
<td>geneReviewsWrapper.sh checks for new GeneReviews data and downloads and loads it into hgwdev if so</td>
<td>runs weekly, complains quarterly</td>
<td>runs weekly, complains quarterly</td>
<td> </td>
<td>Matt</td>
<td>Matt</td>
<td>braney</td>
<td>braney</td>
Line 176: Line 197:
<td>sends a reminder to check the list of folks who have mirror sites. This list is created by a script of Hiram's and located here: http://genome-test.cse.ucsc.edu/admin/tools/useCount.html. The information comes from the phoneHome program in the CGIs. Sometimes companies set up mirrors, which is not OK.</td>
<td>sends a reminder to check the list of folks who have mirror sites. This list is created by a script of Hiram's and located here: http://genome-test.cse.ucsc.edu/admin/tools/useCount.html. The information comes from the phoneHome program in the CGIs. Sometimes companies set up mirrors, which is not OK.</td>
<td>monthly</td>
<td>monthly</td>
<td> </td>
<td>Matt</td>
<td>Matt</td>
<td>Pauline</td>
<td>Pauline</td>
Line 184: Line 206:
<td>checkGbibMd5.sh compares the md5sum of gbibBeta.zip on dev with the sum for gbib.zip on genome-store, complaining if they don't match (and it's not during final build)</td>
<td>checkGbibMd5.sh compares the md5sum of gbibBeta.zip on dev with the sum for gbib.zip on genome-store, complaining if they don't match (and it's not during final build)</td>
<td>runs weekly on Thursday</td>
<td>runs weekly on Thursday</td>
<td> </td>
<td>Brian, Steve, Jonathan</td>
<td>Brian, Steve, Jonathan</td>
<td>-</td>
<td>-</td>
Line 192: Line 215:
<td>Curl of http://genomewiki.ucsc.edu/index.php/Assembly_Hubs to test Public Wiki is accessible, output only if no response.</td>
<td>Curl of http://genomewiki.ucsc.edu/index.php/Assembly_Hubs to test Public Wiki is accessible, output only if no response.</td>
<td>hourly</td>
<td>hourly</td>
<td> </td>
<td>Brian</td>
<td>Brian</td>
<td>-</td>
<td>-</td>
Line 198: Line 222:


<tr>
<tr>
<td colspan="5" style="background-color:gray"><b>from build cron</b></td>
<td colspan="6" style="background-color:gray"><b>from build cron</b></td>
</tr>
</tr>


Line 205: Line 229:
<td>receive cron output for tableDescriptions build errors</td>
<td>receive cron output for tableDescriptions build errors</td>
<td>rare</td>
<td>rare</td>
<td> </td>
<td>Steve, Luvina</td>
<td>Steve, Luvina</td>
<td>Pauline, Matt</td>
<td>Pauline, Matt</td>
Line 211: Line 236:


<tr>
<tr>
<td colspan="5" style="background-color:gray"><b>from galt's cron</b></td>
<td colspan="6" style="background-color:gray"><b>from galt's cron</b></td>
</tr>
</tr>


Line 217: Line 242:
<td>run ReleaseLog script (this requires a copy of the kent repository)</td>
<td>run ReleaseLog script (this requires a copy of the kent repository)</td>
<td>daily</td>
<td>daily</td>
<td> </td>
<td>Galt</td>
<td>Galt</td>
<td>Galt (if QA takes this over, Pauline will do it)</td>
<td>Galt (if QA takes this over, Pauline will do it)</td>
Line 225: Line 251:
<td>back up the qapushq</td>
<td>back up the qapushq</td>
<td>every 30 minutes during the work day/week</td>
<td>every 30 minutes during the work day/week</td>
<td> </td>
<td>Galt</td>
<td>Galt</td>
<td>Galt</td>
<td>Galt</td>
Line 231: Line 258:


<tr>
<tr>
<td colspan="5" style="background-color:gray"><b>non-cron jobs</b></td>
<td colspan="6" style="background-color:gray"><b>non-cron jobs</b></td>
</tr>
</tr>


Line 237: Line 264:
<td>Check old forum (http://redmine.soe.ucsc.edu/forum/index.php) for MLQs that may have slipped through the cracks: Check to make sure that the last response for every ticket came from one of us.  If not, look it up in redmine to ensure that A) there is indeed a redmine ticket for it, and B) it has been responded to.  If not, notify the appropriate MLM with as much information about the MLQ as possible.</td>
<td>Check old forum (http://redmine.soe.ucsc.edu/forum/index.php) for MLQs that may have slipped through the cracks: Check to make sure that the last response for every ticket came from one of us.  If not, look it up in redmine to ensure that A) there is indeed a redmine ticket for it, and B) it has been responded to.  If not, notify the appropriate MLM with as much information about the MLQ as possible.</td>
<td>weekly</td>
<td>weekly</td>
<td> </td>
<td>Luvina</td>
<td>Luvina</td>
<td>Luvina</td>
<td>Luvina</td>
Line 245: Line 273:
<td>update sla.html whenever we have RR downtime</td>
<td>update sla.html whenever we have RR downtime</td>
<td>hopefully, not much!</td>
<td>hopefully, not much!</td>
<td> </td>
<td>Steve, Jonathan</td>
<td>Steve, Jonathan</td>
<td>Steve, Jonathan</td>
<td>Steve, Jonathan</td>
Line 253: Line 282:
<td>update crontab in genecats/qa/crontabs/*.crontab (and [http://genomewiki.cse.ucsc.edu/genecats/index.php/Updating_QA_crontabs log in as qateam and restart the qateam crontab] with updates)</td>
<td>update crontab in genecats/qa/crontabs/*.crontab (and [http://genomewiki.cse.ucsc.edu/genecats/index.php/Updating_QA_crontabs log in as qateam and restart the qateam crontab] with updates)</td>
<td>as needed</td>
<td>as needed</td>
<td> </td>
<td>Jonathan</td>
<td>Jonathan</td>
<td>Pauline, Jonathan</td>
<td>Pauline, Jonathan</td>
Line 261: Line 291:
<td>update UCSC Entrez LinkOut when necessary. Entrez LinkOut sends requests for changes (and statistics) to the browser-qa email address</td>
<td>update UCSC Entrez LinkOut when necessary. Entrez LinkOut sends requests for changes (and statistics) to the browser-qa email address</td>
<td>rarely</td>
<td>rarely</td>
<td> </td>
<td>Matt</td>
<td>Matt</td>
<td>Bob</td>
<td>Bob</td>
Line 267: Line 298:


<tr>
<tr>
<td colspan="5" style="background-color:gray"><b>ENCODE</b></td>
<td colspan="6" style="background-color:gray"><b>ENCODE</b></td>
</tr>
</tr>


Line 273: Line 304:
<td>Check genome-preview with Selenium test, ensures various big ENCODE tracks are accessible, and that machine is still running</td>
<td>Check genome-preview with Selenium test, ensures various big ENCODE tracks are accessible, and that machine is still running</td>
<td>weekly</td>
<td>weekly</td>
<td> </td>
<td>Brian</td>
<td>Brian</td>
<td>Brian</td>
<td>Brian</td>
Line 281: Line 313:
<td>Cronjob reports checkUrlsInTable hg38 wgEncodeCell (likely infrequent results) </td>
<td>Cronjob reports checkUrlsInTable hg38 wgEncodeCell (likely infrequent results) </td>
<td>weekly</td>
<td>weekly</td>
<td> </td>
<td>Brian</td>
<td>Brian</td>
<td>Brian</td>
<td>Brian</td>

Revision as of 21:51, 3 September 2015

task frequency who NOW (starting Sept '15) who Aug '14 - Aug '15 who Aug '13 - Jul '14 who before
from qateam cron
ask Galt/Jim about blat updates 3x per year Steve, Jonathan Pauline, Steve, Jonathan Pauline, Steve, Jonathan
Respond to items on the broken links report: Check to make sure that broken links reported are actually broken. If so, fix the link if it is clear what the new link should be and push to the RR. If not, contact the appropriate person at the web site in question to find out what the new link should be. daily Matt Matt Steve
Respond to the trackDbGlobal report: Make public and request a push of trackDb and friends for any assemblies that show up as changed in the trackDbGlobal report. weekly Matt Luvina Steve
receive output from hgTracksRandom cron; alert admins to problems as needed many: kuhn, katrina, brianlee, braney, luvina, gary, ann, steve, jcasper, matt many: kuhn, pauline, katrina, brianlee, braney, luvina, gary, ann, steve, jcasper, matt many: kuhn, rhead, pauline, katrina, brianlee, braney, luvina, gary, ann, steve, jcasper
receive output from gbLoaded cron job and alert braney if anything seems very outdated daily, problems every 3-6 months Steve, Jonathan Steve, Jonathan Brooke, Steve, Jonathan
receive output from showProcesslist.csh daily, no action Steve, Jonathan Steve, Jonathan Brooke, Steve, Jonathan
get output from checkTableStatus.csh on Mondays (this is a report on whether the genbank table dumps are up-to-date...if we could get rid of our dependence on those, we could get rid of this cron job) weekly Steve, Jonathan Steve, Jonathan Brooke, Steve, Jonathan
get an email from backupCentral.csh each month (there's a redmine ticket on whether we should change this one: http://redmine.soe.ucsc.edu/issues/7834) monthly Steve, Jonathan Steve, Jonathan Brooke, Steve, Jonathan
receive an email on additions and subtractions to files in /gbdb on hgnfs1 daily Steve, Jonathan Steve, Jonathan Brooke, Steve, Jonathan
receive an email from cron if qateam DailySandboxUpdate fails daily Steve, Jonathan Steve, Jonathan Brooke, Steve, Jonathan
receive output from checkMetaAday.csh; alert (steve, jcasper) if anything is out of sync (compares hgcentralbeta to hgcentral for db of the day) daily Matt Matt Luvina
receive output from syncAday.csh; alert (steve, jcasper) if anything is out of sync (checks that tables are in sync between beta and RR for database of the day) daily Brian Brian Luvina
get output from checkGenomeMysql daily Steve, Jonathan Steve, Jonathan Brooke, Steve, Jonathan
hubPublicCheck complains when the hubPublic table doesn't have the same strings as the actual hub.txt file. (NOTE: To run only on Beta hubs requires these declarations: export HGDB_CONF=$HOME/.hg.conf.beta and SHELL=/bin/sh ) runs daily, complains rarely? Brian Lee Brian Lee braney
omimWrapper.sh checks for new OMIM data and downloads and loads it into hgwdev if so runs weekly, complains bi-weekly Brian Lee braney braney
omimUploadWrapper.sh uploads the OMIM tables to OMIM runs weekly, complains rarely Brian Lee braney braney
decipherWrapper.sh checks for new DECIPHER data and downloads and loads it into hgwdev if so runs weekly, complains monthly Luvina braney braney
iscoWrapper.sh checks for new ISCA data and downloads and loads it into hgwdev if so runs weekly, complains monthly Steve braney braney
gwasWrapper.sh checks for new GWAS data and downloads and loads it into hgwdev if so runs weekly, complains quarterly Jonathan braney braney
geneReviewsWrapper.sh checks for new GeneReviews data and downloads and loads it into hgwdev if so runs weekly, complains quarterly Matt braney braney
sends a reminder to check the list of folks who have mirror sites. This list is created by a script of Hiram's and located here: http://genome-test.cse.ucsc.edu/admin/tools/useCount.html. The information comes from the phoneHome program in the CGIs. Sometimes companies set up mirrors, which is not OK. monthly Matt Pauline Ann
checkGbibMd5.sh compares the md5sum of gbibBeta.zip on dev with the sum for gbib.zip on genome-store, complaining if they don't match (and it's not during final build) runs weekly on Thursday Brian, Steve, Jonathan - -
Curl of http://genomewiki.ucsc.edu/index.php/Assembly_Hubs to test Public Wiki is accessible, output only if no response. hourly Brian - -
from build cron
receive cron output for tableDescriptions build errors rare Steve, Luvina Pauline, Matt Brooke, Pauline
from galt's cron
run ReleaseLog script (this requires a copy of the kent repository) daily Galt Galt (if QA takes this over, Pauline will do it) Galt
back up the qapushq every 30 minutes during the work day/week Galt Galt Galt
non-cron jobs
Check old forum (http://redmine.soe.ucsc.edu/forum/index.php) for MLQs that may have slipped through the cracks: Check to make sure that the last response for every ticket came from one of us. If not, look it up in redmine to ensure that A) there is indeed a redmine ticket for it, and B) it has been responded to. If not, notify the appropriate MLM with as much information about the MLQ as possible. weekly Luvina Luvina Steve
update sla.html whenever we have RR downtime hopefully, not much! Steve, Jonathan Steve, Jonathan Brooke, Pauline
update crontab in genecats/qa/crontabs/*.crontab (and log in as qateam and restart the qateam crontab with updates) as needed Jonathan Pauline, Jonathan Brooke, Pauline
update UCSC Entrez LinkOut when necessary. Entrez LinkOut sends requests for changes (and statistics) to the browser-qa email address rarely Matt Bob Ann
ENCODE
Check genome-preview with Selenium test, ensures various big ENCODE tracks are accessible, and that machine is still running weekly Brian Brian Brian
Cronjob reports checkUrlsInTable hg38 wgEncodeCell (likely infrequent results) weekly Brian Brian Brian
Retrieved from "https://genomewiki.ucsc.edu/genecats/index.php?title=Monitoring_Tasks&oldid=2284"