View source for Chromosome name conversion

Hiram's wisdom on conversion of chromosomes names in data files

Max made a command to change names in our standard file types:

    chromToUcsc

Brian is in the process of fixing our system so that any name
in a big* file that can be found in our alias system just works
without any translation.

All assembly hubs have IGV compatible alias name files, for
example:
 
    https://hgdownload.soe.ucsc.edu/hubs/GCF/000/001/405/GCF_000001405.39/GCF_000001405.39.chromAlias.txt

All of our database browsers have the same type of alias name file,
for example:
    https://hgdownload.soe.ucsc.edu/goldenPath/xenTro9/bigZips/xenTro9.chromAlias.txt

as well as an equivalent database table:
   hgsql -e 'desc chromAlias;' xenTro9
   +--------+--------------+------+-----+---------+-------+
   | Field  | Type         | Null | Key | Default | Extra |
   +--------+--------------+------+-----+---------+-------+
   | alias  | varchar(255) | NO   | PRI | NULL    |       |
   | chrom  | varchar(255) | NO   | MUL | NULL    |       |
   | source | varchar(255) | NO   | MUL | NULL    |       |
   +--------+--------------+------+-----+---------+-------+

BED custom tracks submitted to the browser already understand how to take
advantage of the alias system.


In this case, you may need to make your own translation set
from the genome sequence itself.  I probably don't have
those contig names anywhere.

Get the 2bit file of the genome to translate
somewhere to work on in a directory, then:

   doIdKeys.pl -buildDir=`pwd` -twoBit=`pwd`/yourGenome.2bit yourGenome

This will create the files:

   yourGenome.idKeys.txt
   yourGenome.keySignature.txt

That idKeys.txt file can be joined to existing database idKey files:

    /hive/data/genomes/<db>/bed/idKeys/<db>.idKeys.txt

To get the name to name correspondence.

I've probably already done this if I came anywhere near these genomes.
It is practically the first step I have to do anytime I encounter
any of that one-off work.

For example, for the cactus241 alignment, I checked all those genomes
to find out which ones matched any of our database genomes and did the
name translation thereby.  hg38 to 'Homo_sapiens' happens to come out
with the same names since Joel used our sequence, but it also proved
the Homo_sapiens sequences was identical to hg38:

   join \
  /hive/data/genomes/hg38/bed/idKeys.p13/hg38.p13.idKeys.txt \
     /hive/data/genomes/hg38/bed/cactus242way/idKeys/Homo_sapiens/Homo_sapiens.idKeys.txt

The keySignature.txt file is a single MD5 sum representing the entire genome, in which
case if you expect the entire genome to be completely identical, just match it to
any of the existing keySignature files.

And of course, such idKey files also exist for all assembly hubs so you can find
a matching genome there too if it isn't one of our database genomes.  I have
a specialized 'find' command that can rapidly locate all the assembly hub idKeys.
For example, here are all the keySignatures for all the UCSC database genomes,
refseq, genbank assembly hubs, and Ensembl genomes:

   /hive/data/inside/assemblyEquivalence/2021-05-11/ucsc/ucsc.keySignatures.txt
   /hive/data/inside/assemblyEquivalence/2021-05-11/refseq/refseq.keySignatures.txt
   /hive/data/inside/assemblyEquivalence/2021-05-11/genbank/genbank.keySignatures.txt
   /hive/data/inside/assemblyEquivalence/2021-05-11/ensembl/ensembl.keySignatures.txt

A simple join on any of those lists immediately identifies identical genomes.
This business creates the hgFixed table asmEquivalent

   hgsql -e 'desc asmEquivalent;' hgFixed
   +----------------------+-------------------------------------------+------+-----+---------+
   | Field                | Type                                      | Null | Key | Default |
   +----------------------+-------------------------------------------+------+-----+---------+
   | source               | varchar(255)                              | NO   | MUL | NULL    |
   | destination          | varchar(255)                              | NO   | MUL | NULL    |
   | sourceAuthority      | enum('ensembl','ucsc','genbank','refseq') | NO   |     | NULL    |
   | destinationAuthority | enum('ensembl','ucsc','genbank','refseq') | NO   |     | NULL    |
   | matchCount           | bigint(20)                                | NO   |     | NULL    |
   | sourceCount          | bigint(20)                                | NO   |     | NULL    |
   | destinationCount     | bigint(20)                                | NO   |     | NULL    |
   +----------------------+-------------------------------------------+------+-----+---------+

The idKeys.txt files are used to find nearly identical genomes.