Monitoring Tasks

task	frequency	who NOW	who Aug '14 - Aug '15-Apr'16	who Aug '13 - Jul '14	before
from hgwdev qateam cron
ask Galt/Jim about blat updates	3x per year	Cath, Brian	Steve, Jonathan	Pauline, Steve, Jonathan	Pauline, Steve, Jonathan
Respond to items on the broken links report: Check to make sure that broken links reported are actually broken. If so, fix the link if it is clear what the new link should be and push to the RR. If not, contact the appropriate person at the web site in question to find out what the new link should be.	daily	Cath, Matt	Matt	Matt	Steve
Respond to the trackDbGlobal report: Make public and request a push of trackDb and friends for any assemblies that show up as changed in the trackDbGlobal report.	weekly	Cath	Matt	Luvina	Steve
receive output from hgTracksRandom cron; alert admins to problems	as needed	many: cath, christopher, kuhn, brianlee, braney, luvina, gary, ann,, jcasper, matt	many: kuhn, katrina, brianlee, braney, luvina, gary, ann, steve, jcasper, matt	many: kuhn, pauline, katrina, brianlee, braney, luvina, gary, ann, steve, jcasper, matt	many: kuhn, rhead, pauline, katrina, brianlee, braney, luvina, gary, ann, steve, jcasper
receive output from gbLoaded cron job and alert braney if anything seems very outdated	daily, problems every 3-6 months	Cath, Luvina	Steve, Jonathan	Steve, Jonathan	Brooke, Steve, Jonathan
receive output from showProcesslist.csh	daily, no action	Brian, Matt	Steve, Jonathan	Steve, Jonathan	Brooke, Steve, Jonathan
get output from checkTableStatus.csh on Mondays (this is a report on whether the genbank table dumps are up-to-date...if we could get rid of our dependence on those, we could get rid of this cron job)	weekly	Brian, Jonathan	Steve, Jonathan	Steve, Jonathan	Brooke, Steve, Jonathan
get an email from backupCentral.csh each month that keeps a dump of hgcentral tables (there's a redmine ticket on whether we should change this one: http://redmine.soe.ucsc.edu/issues/7834)	monthly	Matt, Brian	Steve, Jonathan	Steve, Jonathan	Brooke, Steve, Jonathan
receive an email on additions and subtractions to files in /gbdb on hgnfs1	daily	Christopher, Luvina	Steve, Jonathan	Steve, Jonathan	Brooke, Steve, Jonathan
receive an email from cron if qateam DailySandboxUpdate fails	daily	Matt, Brian	Steve, Jonathan	Steve, Jonathan	Brooke, Steve, Jonathan
receive output from checkMetaAday.csh; alert; if anything is out of sync (compares hgcentralbeta to hgcentral for db of the day) Instructions & Example	daily	Cath, Matt	Matt	Matt	Luvina
receive output from syncAday.csh; alert; if anything is out of sync (checks that tables are in sync between beta and RR for database of the day)	daily	Brian	Brian	Brian	Luvina
increment databaseAday, /cluster/bin/scripts/databaseAday.csh	problems only	Brian, Jonathan	Steve, Jonathan	Steve, Jonathan	Brooke, Steve, Jonathan
checkHgFindSpec -checkTermRegex, /cluster/bin/scripts/runCheckHgFindSpec go	problems only	Brian, Jonathan	Steve, Jonathan	Steve, Jonathan	Brooke, Steve, Jonathan
check to see if there are new files to go with the CGI push /cluster/home/qateam/bin/scripts/findCgiFiles.csh go	problems only	Brian, Jonathan	Steve, Jonathan	Steve, Jonathan	Brooke, Steve, Jonathan
get output from checkGenomeMysql	daily	Christopher, Matt	Steve, Jonathan	Steve, Jonathan	Brooke, Steve, Jonathan
Check for unigene update for human SAGE track	daily	Luvina, Braney	Steve, Braney
look for broken symlinks in /gbdb:	daily	Browser Staff	Browser Staff
look for broken symlinks in in (very large) visiGene directory	twice yearly	Cath, Galt	Steve, Galt
look for broken symlinks in htdocs-hgdownload/goldenpath	daily	Browser Staff	Browser Staff
Check the source tree for duplicate case-independent filenames. These wreck havoc with git.	daily	Browser Staff	Browser Staff
Check to see what kind of browsers and o/s people are using to access the public website	weekly	Ann, Christopher, Brian	Ann, Steve
run /cluster/bin/scripts/sessionStats.csh	monthly	Angie, Brian, Bob	Angie, Brian, Bob, Steve
check that the ga4gh beacon server is running	daily	Cath	Ann	Ann
check that blat servers are running ok	ongoing	Ann, Brian, Donna, Galt, Luvina, Matt, Gary,	Ann, Brian, Donna, Galt, Luvina, Steve, Gary
check that beta builds alright	3x week	Brian, Christopher,Braney	Steve, Jonathan, Braney
hubPublicCheck complains when the hubPublic table doesn't have the same strings as the actual hub.txt file. (NOTE: To run only on Beta hubs requires these declarations: export HGDB_CONF=$HOME/.hg.conf.beta and SHELL=/bin/sh).	runs daily, complains rarely	Matt	Brian	Brian	braney
email request to build doPublicCrawl public.ix and public.ixx hub search files and push them.	runs quarterly	Christopher
Cronjob reports checkUrlsInTable hg38 wgEncodeCell and cv.ra (likely infrequent results)	weekly	Christopher	Brian	Brian	Brian
sends a reminder to check the list of folks who have mirror sites. This list is created by a script of Hiram's and located here: http://genome-test.cse.ucsc.edu/admin/tools/useCount.html. The information comes from the phoneHome program in the CGIs. Sometimes companies set up mirrors, which is not OK.	monthly	Brian, Luvina	Matt	Pauline	Ann
checkGbibMd5.sh compares the md5sum of gbibBeta.zip on dev with the sum for gbib.zip on genome-store, complaining if they don't match (and it's not during final build)	runs weekly on Thursday	Brian, Cath, Jonathan	Brian, Steve, Jonathan	-	-
Curl of http://genomewiki.ucsc.edu/index.php/Assembly_Hubs to test Public Wiki is accessible, output only if no response.	hourly	Brian	Brian	-	-
from build cron
receive cron output for tableDescriptions build errors	rare	Luvina	Steve, Luvina	Pauline, Matt	Brooke, Pauline
from galt's cron
run ReleaseLog script (this requires a copy of the kent repository)	daily	Galt	Galt	Galt	Galt
back up the qapushq	every 30 minutes M-F	Galt	Galt	Galt	Galt
non-cron jobs
Check old forum (http://redmine.soe.ucsc.edu/forum/index.php) for MLQs that may have slipped through the cracks: Check to make sure that the last response for every ticket came from one of us. If not, look it up in redmine to ensure that A) there is indeed a redmine ticket for it, and B) it has been responded to. If not, notify the appropriate MLM with as much information about the MLQ as possible.	weekly	Luvina	Luvina	Luvina	Steve
update sla.html whenever we have RR downtime. See wiki procedures	hopefully, not much!	Cath, Luvina	Steve, Jonathan	Steve, Jonathan	Brooke, Pauline
update crontab in genecats/qa/crontabs/*.crontab (and log in as qateam and restart the qateam crontab with updates)	as needed	Brian	Jonathan	Pauline, Jonathan	Brooke, Pauline
update UCSC Entrez LinkOut when necessary. Entrez LinkOut sends requests for changes (and statistics) to the browser-qa email address	rarely	Matt	Matt	Bob	Ann
Check genome-preview machine is still running	biweekly	Brian	Brian	Brian	Brian
Updated Facebook with Announcements	As needed	Brian	Brian
Update Twitter with Announcements	As needed	Brian	Ann
otto cron jobs (/hive/data/outside/otto)
omimWrapper.sh checks for new OMIM data and downloads and loads it into hgwdev if so	runs weekly, complains bi-weekly		braney	braney	braney
omimUploadWrapper.sh uploads the OMIM tables to OMIM	runs weekly, complains rarely		braney	braney	braney
decipherWrapper.sh checks for new DECIPHER data and downloads and loads it into hgwdev if so	runs weekly, complains monthly		braney	braney	braney
iscaWrapper.sh checks for new ISCA data and downloads and loads it into hgwdev if so	runs weekly, complains monthly	jcasper	braney	braney	braney
gwasWrapper.sh checks for new GWAS data and downloads and loads it into hgwdev if so	runs weekly, complains quarterly		braney	braney	braney
geneReviewsWrapper.sh checks for new GeneReviews data and downloads and loads it into hgwdev if so	runs weekly, complains quarterly		braney	braney	braney