Requests for liftOver files

Overview

The MLM FAQ on LiftOver files shares some guidelines:

liftOver files for assemblies with no browser

We do not provide liftOver files for assemblies for which we do not have at least a minimal browser. If a user requests such a file, please point them to this wiki page and wish them luck!

liftOver file requests for assemblies with a browser

When a specific liftOver file is requested from a user for an existing browser, we should definitely consider making it. It should go through the usual process (make a redmine, assign to an engineer, place in pushQ, QA and release). QA can do a minimal QA job on it (does it work? does it show up?) before pushing it to hgdownload and enabling it on the RR. There is no need to create companion net/chain tracks (unless the user specifically asked for that). The MLM who received the question is not responsible for QAing the liftOver file (it should just go into the pushQ like all data). After it is released, whoever QAd it should contact the user to let them know that it's available.

Here is a quick review of the process

push the files in /gbdb/ on hgwdev to hgnfs1

These files have the same name and live in the same place as hgdownload-test, but are what is used by the CGIs to allow lifting.

edit hgcentral on beta and the RR once gbdb goes through

The lines on dev in hgcentraltest that allow the /gbdb/ files to load in the CGIs need to added to hgcentralbeta and hgcentral.

push the files from hgdownload-test to hgdownload

To make the files available for download, a push of the files to hgdownload is also needed.

push the files in /gbdb/ on hgwdev to hgnfs1

Here is an example push email where mm10 <---> danRer10 liftOver files and rn6 <---> danRer10 liftOver files and hg19 <---> danRer10 files are being released.

Hi Pushers,

Please push:

/gbdb/mm10/liftOver/mm10ToDanRer10.over.chain.gz
/gbdb/danRer10/liftOver/danRer10ToMm10.over.chain.gz
/gbdb/rn6/liftOver/rn6ToDanRer10.over.chain.gz
/gbdb/danRer10/liftOver/danRer10ToRn6.over.chain.gz
/gbdb/hg19/liftOver/hg19ToDanRer10.over.chain.gz
/gbdb/danRer10/liftOver/danRer10ToHg19.over.chain.gz

      from hgwdev --> hgnfs1/genome-euro/-asia

Note that items that are symlinked on hgwdev should become real files on hgnfs1.

Reason: Releasing LiftOvers as part of MLQ ######

Thanks,

This push will make the files available on both hgwbeta and the RR as hgnfs1 is shared by both of them.

Edit hgcentral on beta and the RR once gbdb push goes through

Note that these examples must be edited to the appropriate databases and also be updated in both directions. In the below example danRer6 <---> mm9 is beign release, but first in the danRer6 direction, it will need be repeated a second time.

Acquire the hgwdev entry

 hgsql -Ne "select * from liftOverChain where fromDb='danRer6' and toDb='mm9';" hgcentraltest > liftOver.dev

This will save the danRer6 --> mm9 lines to a file called liftOver.dev

Check that the lines do not exist on Beta

hgsql  -h mysqlbeta  -Ne "select * from liftOverChain where fromDb='danRer6' and toDb='mm9';" hgcentralbeta

This will query beta and should come back empty indicating there are no lines like the ones. We will next populate the lines using what we took from dev.

Provided lines don't exist, load lines from dev on Beta

hgsql -h mysqlbeta -e "load data local infile 'liftOver.dev' into table liftOverChain" hgcentralbeta

Check behavior on Beta

At this point you should be able to go to hgLiftover and hgConvert and see things in operation on hgwbeta, where in this example danRer6 will have an option to lift coordinates to mm9

Check that lines do not exit on RR (before loading)

hgsql  -h genome-centdb -Ne "select * from liftOverChain where fromDb='danRer6' and toDb='mm9';" hgcentral

This will query beta and should come back empty indicating there are no lines like the ones. We will next populate the lines using what we took from dev.

Provided lines don't exit, load lines on the RR

hgsql -h genome-centdb -e "load data local infile 'liftOver.dev' into table liftOverChain;" hgcentral

Check behavior on the RR

At this point you should be able to go to hgLiftover and hgConvert and see things in operation on hgwbeta, where in this example danRer6 will have an option to lift coordinates to mm9.

Repeat the process in the reverse direction

Now reverse the names and repeat the process. I suggest removing liftOver.dev before starting.

For reference here are the steps (note that everywhere danRer6 has been replaced by mm9 and vice-versa and the load steps are the same). Acquire:

hgsql -Ne "select * from liftOverChain where fromDb='mm9' and toDb='danRer6';" hgcentraltest > liftOver.dev

Check it doesn't exist on beta

hgsql  -h mysqlbeta  -Ne "select * from liftOverChain where fromDb='mm9' and toDb='danRer6';" hgcentralbeta

Load on Beta (provided it doesn't exist)

hgsql -h mysqlbeta -e "load data local infile 'liftOver.dev' into table liftOverChain" hgcentralbeta

Check it doesn't exist on RR

hgsql  -h genome-centdb -Ne "select * from liftOverChain where fromDb='mm9' and toDb='danRer6';" hgcentral

Load on RR (provided it doesn't exist):

hgsql -h genome-centdb -e "load data local infile 'liftOver.dev' into table liftOverChain;" hgcentral