GenbankServiceLog

This page is a record of service on the genbank process.

2012-09-11 braney duplicate lines in refGene table

Symptom: rn4.refGene table had 11 exactly duplicated records

Solution: do uniq on cat's of PSL's from align blats which use overlapping regions of the genome so sometimes find the same gene twice. commit 3c7a5e8e0482663aac77bce798cd2caf0bf4d06c

2012-08-25 braney CCDS load failed with differing number of fields

Symptom: CCDS import failed due to new fields being added to the GroupVersions table

Solution: grab new SQL files from import, check them into the tree, convert them to mySQL, check those into the tree, rebuild server. Follow instructions in kent/src/hg/makeDb/genbank/src/ccds/README

2012-08-25 braney grepIndex overflowing hgwbeta disks

Symptom: the load process on hgwbeta was building grepIndex into /data/tmp/grepIndex which was filling up the disk

Solution: discovered beta wasn't using grepIndex on /data/tmp anyway, so I stopped building it there. beta currently uses /gbdb/genbank/grepIndex, which Mark says will mean that searches on genbank on beta will be broken, but our testing has uncovered the fact that it's actually broken everywhere! lol.

2012-08-19 braney upstream download files are empty on ce6 and ce10

Symptom: compressed FASTA files on hgdownload were empty

Solution: discovered that refGene on worm doesn't have any genes with annotated 5' and 3' UTRs which is a requirement for them to be used in the upstream* files. Wrote to NCBI help desk to notify them of the problem. Changed ce6 and ce10 to use ensGene instead of refGene to make the upstream files.

2012-08-17 braney ran out of disk space on hgwbeta while loading

Symptom: ran out of disk space on hgwbeta while loading

Solution: asked Tim to move some of the old builds out of /data/releaseBuild.

2012-07-06 braney CCDS load failed with differing number of fields

Symptom: CCDS import failed due to new fields being added to the Build table

Solution: grab new SQL files from import, check them into the tree, convert them to mySQL, check those into the tree, rebuild server. Follow instructions in kent/src/hg/makeDb/genbank/src/ccds/README

2012-06-25 braney load failed with allowLargeDeletesProblem

Symptom: load failed on dev due to missing "full" directories for genbank.190.0 in ce2, sacCer1, sacCer2, sacCer3, xenTro1

Solution: the problem has to do with migrating to a new release when there are no new alignments to do, so the Finish process is not called, and therefore the "full" directory is not built. In the interest of expediency, I just re-did the initial alignments for these five assemblies. This means this problem will probably reappear with genbank.191.0, but by that time I should have fixed the code.

2012-06-25 braney build failed on /data/tmp/ remove

Symptom: build failed on inability to remove a directory in /data/tmp that was owned by Mark

Solution: asked mark to remove the directories in /data/tmp that he owned.

2012-06-11 braney non-existent table found in mkdownloads

Symptom: mkdownload fails on message about a gene model table not found

hgwdev 2012.06.11-20:07:57 mkdownload: creating /hive/data/outside/genbank/data/ftp/canFam3/bigZips/refMrna.fa.gz table ensGene not found for any chroms Error: 1 input table(s)/file(s) do not exist for any of the chroms specified command failed: featureBits -fa=stdout canFam3 ensGene:upstream:1000 | gzip -c > /hive/data/outside/genbank/data/ftp/canFam3/bigZips/upstream1000.fa.gz.tmp.gz at /hive/data/outside/genbank/bin/../lib/gbCommon.pm line 272. at /hive/data/outside/genbank/bin/../lib/gbCommon.pm line 272.

Solution: fixed etc/genbank.conf to list an existing table for upstreamGeneTbl

canFam3.upstreamGeneTbl = refGene

2012-06-09 braney load failed due to malformed genbank entry

Symptom: dbload of refseq.53 update from 2012-06-08 failed with message ' invalid signed integer: ">1632" ' Problem was '<' character in mdiff field of GB record:

">" in http://www.ncbi.nlm.nih.gov/nuccore/NM_001267664

Solution: fixed parser to ignore '<' characters in mdiff field.

2012-06-06 braney hgnfs copy script failed

Symptom: genbank-10 was turned off but the hgnfs1 copy script still referenced it.

Solution: change genbank-10 reference to hgwdev

2012-06-06 braney gbSanity failed on rn5

Symptom: lots of error messages of the form "Error: rn5: NR_051977: not in gbCdnaInfo table, referenced in gbIndex"

Solution: re-did the initial load of rn5 on dev.

2012-06-06 braney build failed on /data/tmp/ remove

Symptom: build failed on inability to remove a directory in /data/tmp that was owned by Hiram

Solution: asked mark and hiram to remove the directories in /data/tmp that they owned.

2012-05-31 braney xenoMrna change

Symptom: dbload fails with allowLargeDeletes message. Hiram had changed the loading of xenoMrna in the genbank.conf file for several databases without dropping the xenoMrna table.

Solution: changed genbank.conf to continue to update xenoMrna if it had already been loaded.

2012-05-25 braney allowLargeDeletes problem

Symptom: dbload fails with "Stoping due to maxShrinkage limit being exceeded in one or more partitions. Investigate and rerun with -allowLargeDeletes."

Solution: discovered four assemblies that were missing their full directory in either refSeq.53/<db>/ or genbank.189.0/<db>. No explanation was found for the missing directories. To restore these four assemblies (sacCer1,sacCer2,sacCer3,ce2) I built the alignments from scratch using bin/gbDbAlignStep -initial <db>