Updating blat servers: Difference between revisions

From Genecats
Jump to navigationJump to search
(BLAT machines as distinguished by Gary Morro)
No edit summary
 
(6 intermediate revisions by 5 users not shown)
Line 1: Line 1:
See also: '''Emergency backup blat''' servers for the top 5 assemblies here -- [[Emergency Backup BLAT Servers]]
This below page is not about emergencies.
----
There are two places where blat servers are located. Assemblies that are important (eg. human or mouse) and are expected to get a lot of hits are hosted here at UCSC and will have a blat host name like blat#$ (where # is a number and $ is a letter, for example: blat4a). Assemblies that are going to stay on hgwdev (never to be released publicly) will get a blat server also at UCSC, but on a less important server. They will have a blat host name of blatx. Wherever the blat server is hosted, the steps below will allow you to transfer the pointers to a new machine.  
There are two places where blat servers are located. Assemblies that are important (eg. human or mouse) and are expected to get a lot of hits are hosted here at UCSC and will have a blat host name like blat#$ (where # is a number and $ is a letter, for example: blat4a). Assemblies that are going to stay on hgwdev (never to be released publicly) will get a blat server also at UCSC, but on a less important server. They will have a blat host name of blatx. Wherever the blat server is hosted, the steps below will allow you to transfer the pointers to a new machine.  


Line 19: Line 23:
:DNA
:DNA
:protein
:protein
* Open 6 windows: BLAT DNA, BLAT protein, PCR for dev and the RR (we don't check on beta since it doesn't really matter if blat is working there or not). Make sure that all 6 windows function without errors.
* Open 9 windows: BLAT DNA, BLAT protein, PCR for hgwdev, the RR, and genome-euro (we don't check on beta since it doesn't really matter if blat is working there or not). Make sure that all 9 windows function without errors.
<pre>
<pre>
Getting into hgcentral database on dev, beta, RR:
Getting into hgcentral database on dev, beta, RR:
Line 42: Line 46:
* Go back to the browser, hit back for the 3 windows open on dev, and then click re-submit. Check to see if broken. If not, go on; if yes, troubleshoot until solution is found.
* Go back to the browser, hit back for the 3 windows open on dev, and then click re-submit. Check to see if broken. If not, go on; if yes, troubleshoot until solution is found.
* Repeat step 4 for beta and 4-5 for the RR.
* Repeat step 4 for beta and 4-5 for the RR.
* Ask the buildmeister to rebuild the hgCentral.sql file for mirror sites.  The file at http://hgdownload.cse.ucsc.edu/admin/ should be updated.  Optional: Some mirrors like to get hgcentral tables via ftp or rsync from ftp://hgdownload.cse.ucsc.edu/mysql/hgcentral/.  You can request a push from hgnfs1 --> hgdownload if you want to make the hgcentral info available there immediately (otherwise it will be copied there by a weekly rsync).
* hgcentral changes are synced from the RR to genome-euro via the pullHgcentral script.  See the [http://genomewiki.ucsc.edu/genecats/index.php/Releasing_an_assembly#Verify_that_everything_is_working_on_genome-euro new assembly release page] for more info.
* Notify the mirrors (on genome-mirror@soe.ucsc.edu) that we have updated our blatServers.  An example email is here: https://lists.soe.ucsc.edu/pipermail/genome-mirror/2009-March/001189.html.  If you didn't ask for a push of the mysql tables, you can just say that they will be updated over the weekend.
* Ask the buildmeister to rebuild the hgCentral.sql file for mirror sites.  The [http://hgdownload.soe.ucsc.edu/admin/hgcentral.sql file] at http://hgdownload.soe.ucsc.edu/admin/ should be updated.  Optional: Some mirrors like to get hgcentral tables via ftp or rsync from ftp://hgdownload.soe.ucsc.edu/mysql/hgcentral/.  You can request a push from hgnfs1 --> hgdownload if you want to make the hgcentral info available there immediately (otherwise it will be copied there by a weekly rsync).
* Notify the mirrors (on genome-mirror@soe.ucsc.edu) that we have updated our blatServers.  An example genome-mirror ''new hgcentral.sql (blatServers table update)'' email is here: https://groups.google.com/a/soe.ucsc.edu/d/msg/genome-mirror/xxMcXbuXq9Y/lGW5fJ95NzMJ  If you didn't ask for a push of the mysql tables, you can just say that they will be updated over the weekend.
 
Your changes to the table blatServers will sync to euro (and asia, I think) hourly. To push them sooner, run a script:
See: http://genomewiki.ucsc.edu/genecats/index.php/Releasing_an_assembly#Verify_that_everything_is_working_on_genome-euro


[[Category:Browser QA]]
[[Category:Browser QA]]

Latest revision as of 21:45, 25 September 2020

See also: Emergency backup blat servers for the top 5 assemblies here -- Emergency Backup BLAT Servers This below page is not about emergencies.


There are two places where blat servers are located. Assemblies that are important (eg. human or mouse) and are expected to get a lot of hits are hosted here at UCSC and will have a blat host name like blat#$ (where # is a number and $ is a letter, for example: blat4a). Assemblies that are going to stay on hgwdev (never to be released publicly) will get a blat server also at UCSC, but on a less important server. They will have a blat host name of blatx. Wherever the blat server is hosted, the steps below will allow you to transfer the pointers to a new machine.

  • Do a select statement on dev to make sure that everything is where you expect it to be:
hgwdev: hgsql hgcentraltest
mysql> select * from blatServers where host="blat#" ORDER BY port;
+---------+--------+-------+---------+--------+
| db      | host   | port  | isTrans | canPcr |
+---------+--------+-------+---------+--------+
| apiMel1 | blat12 | 17778 |       1 |      0 |
| apiMel1 | blat12 | 17779 |       0 |      1 |
| monDom1 | blat12 | 17786 |       1 |      0 |
| monDom1 | blat12 | 17787 |       0 |      0 |
| caeRei0 | blat12 | 17790 |       1 |      0 |
| caeRei0 | blat12 | 17791 |       0 |      1 |
+---------+--------+-------+---------+--------+
  • Get DNA and protein sequences for a gene for given assembly and paste below for future reference:
DNA
protein
  • Open 9 windows: BLAT DNA, BLAT protein, PCR for hgwdev, the RR, and genome-euro (we don't check on beta since it doesn't really matter if blat is working there or not). Make sure that all 9 windows function without errors.
Getting into hgcentral database on dev, beta, RR:

DEV: hgwdev: hgsql hgcentraltest
BETA: hgwbeta: hgsql hgcentralbeta
RR: hgwdev: hgsql -h genome-centdb hgcentral 
          
Statements for changing pointers:
        
update blatServers set host='blat#', port=177## where db='xxxXxx#' and isTrans=1;
update blatServers set host='blat#', port=177## where db='xxxXxx#' and isTrans=0;

**Note that the admins will simiply say:
bosTau3
trans port 17786
untrans port 17787

This means put the trans port number for isTrans=1 and the untrans port number for isTrans=0
  • Go to hgcentral on dev and change the pointers for the assembly of interest.
  • Go back to the browser, hit back for the 3 windows open on dev, and then click re-submit. Check to see if broken. If not, go on; if yes, troubleshoot until solution is found.
  • Repeat step 4 for beta and 4-5 for the RR.
  • hgcentral changes are synced from the RR to genome-euro via the pullHgcentral script. See the new assembly release page for more info.
  • Ask the buildmeister to rebuild the hgCentral.sql file for mirror sites. The file at http://hgdownload.soe.ucsc.edu/admin/ should be updated. Optional: Some mirrors like to get hgcentral tables via ftp or rsync from ftp://hgdownload.soe.ucsc.edu/mysql/hgcentral/. You can request a push from hgnfs1 --> hgdownload if you want to make the hgcentral info available there immediately (otherwise it will be copied there by a weekly rsync).
  • Notify the mirrors (on genome-mirror@soe.ucsc.edu) that we have updated our blatServers. An example genome-mirror new hgcentral.sql (blatServers table update) email is here: https://groups.google.com/a/soe.ucsc.edu/d/msg/genome-mirror/xxMcXbuXq9Y/lGW5fJ95NzMJ If you didn't ask for a push of the mysql tables, you can just say that they will be updated over the weekend.

Your changes to the table blatServers will sync to euro (and asia, I think) hourly. To push them sooner, run a script: See: http://genomewiki.ucsc.edu/genecats/index.php/Releasing_an_assembly#Verify_that_everything_is_working_on_genome-euro