HgTracks multi-region changes

This page provides an overview of the changes made to hgTracks and the position cart variable by Galt when implementing multi-region (a.k.a. "exon-mostly") display mode. It began as part of code review #16514 by Angie. Instead of adding these notes to the Redmine ticket, I am putting them in the wiki so that Galt can correct any of my misunderstandings, and hopefully this will serve as a rough guide for any developers who will need to interact with this code in the future.

The big merge of code changes for the initial implementation of multi-region display is 8c908f948b; see also http://genecats.cse.ucsc.edu/git-reports-history/v327/review/user/galt/index.html

hgTracks has always operated on the assumption that the user is viewing a single region of some reference assembly sequence. There are global variables for that region and global variables for layout parameters of a single region image. These global variables are referenced throughout all tracks' drawing code. Track loaders fetch data from a single region. Galt implemented multi-region display in a way that barely touched the bulk of hgTracks code -- quite an accomplishment! -- partly by introducing new structures and loops, and partly by carefully overwriting the pre-existing global variables. Developers working on loading and drawing code in the future can mostly ignore those changes and continue using the old globals in the same way as before. However, there may be cases in which track handlers will have to make use of data from all regions (for example, wiggle auto-scale). Developers who add a new global variable will have to decide whether it needs to be updated every time the current region changes. But hopefully they'll not add any more globals.  :)

Multi-region display UI

There is a new button underneath the browser image, labeled 'multi-region'. Clicking it produces a pop-up with options for multi-region display. In the initial version there is a radio button set as shown below, followed by a checkbox that controls whether the regions are visually separated by thin red lines (the default) or highlighting alternating regions.

(*) Show single-chromosome view (default)

( ) Show exons or ( ) genes using GENCODE v22. Use padding of: [6] bases.

( ) Custom regions from BED URL: [ ]

( ) Show one alternate haplotype, placed on its chromosome, using ID: [ chr1_KI270762v1_alt ]

[ ] Highlight alternating regions in multi-region view

The 'Show exons or genes' line appears only when a suitable gene table can be identified. The 'Show one alternate haplotype' option appears only when there is an altLocations sql table.

Within the four main options, there are additional parameters:

• a fifth mode option to omit regions between genes instead of between exons; default unselected
• the padding option for exon or gene mode; default 6
• the URL for a BED file (with optional multi-region-specific settings) provided by the user; default empty
• an alternate haplotype ID; default is "chr6_cox_hap2" or if not applicable, the first item in the altLocations table

There are also two new keyboard bindings: 'e v' changes the mode to 'Show exons' and 'd v' changes the mode to single-region.

The code includes several addition modes not shown above, mostly "demo" modes for testing during development. Since those are not reachable by users, I won't describe those.

Changes to the position cart variable (and hence other CGIs)

Before multi-region, the position cart variable contained either seq:start-end or a search term that must be resolved into seq:start-end. seq was always a chromosome/scaffold name that could be found in the current db's chromInfo table, and had start and end coordinates relative to that chromosome/scaffold. Several other CGIs (such as the Table Browser, Gene Sorter, DI, VAI) assumed this formatting of position. Now, when viewing custom regions from the user or an alt haplotype, position's value begins with virt and its start and end coordinates are for a "virtual chromosome" constructed by joining all regions.

The virt:start-end form does not (yet) make sense to any other CGI besides hgTracks, so some code has been added to the beginning of each CGI that uses the position variable to detect virt: at the beginning; if found, it swaps in the value of a new cart variable, nonVirtPosition.

Code changes

Needless to say, changing a paradigm in hgTracks involves adding and changing a lot of code. This is a high-level overview, not a comprehensive list.

New cart variables

virtMode

Boolean: true if we are not in single-region mode.

virtModeType

One of {"default", "exonMostly", "geneMostly", "customUrl", "singleAltHaplo" and some others not accessible by UI}. This stores the user's choice of mode in the multi-region pop-up.

multiRegionsBedUrl

The user's URL for custom regions (user sets in dialog)

singleAltHaploId

The name of the alternate haplotype sequence used in alt-haplo mode (user sets in dialog)

lastVirtModeType

Stored in cart with the same value as virtModeType, for detecting changes to virtModeType.

lastVirtModeExtraState

CGI-encoded sequence of several mode-specific parameters for detecting changes:

• the name of the alt haplotype when in alt-haplo mode (i.e. singleAltHaploId)
• the name and file modification date of the user's URL for custom regions (i.e. multiRegionsBedUrl)

virtModeShortDescr

Typically one word like "exons" or "genes"; this appears in the ideogram. This is inferred from the mode except when in customUrl mode if the user specifies one.

nonVirtPosition

When in multi-region mode, the start and end of all regions on the first region's chromosome. For use by other CGIs when position begins with "virt".

oldPosition

For detecting changes in position that somehow are not caught by pre-existing cart var lastPosition.

position.db

Not just a position -- this stores pretty much all of the above, CGI-encoded, in case we change db and then return to this db.

Changes to struct track

nextWindow

New data structures in hgTracks.h

struct virtRegion

This maps a particular region to its place in the reference assembly.

struct virtRegion
/* virtual chromosome structure */
   {
   struct virtRegion *next;
   char *chrom;
   int start;
   int end;
   char strand[2];	/* + or - for strand */
   };

struct virtChromRegionPos

This is one member of an array; the members of the array contain successive offsets within the "virtual chromosome" for each region.

struct virtChromRegionPos
/* virtual chromosome region position*/
   {
   long virtPos;
   struct virtRegion *virtRegion;
   };

struct positionMatch

This is bed3 without the chrom, i.e. a start-end range.

struct positionMatch
/* virtual chroom position that matches or overlaps search query chrom,start,end */
{
struct positionMatch *next;
long virtStart;
long virtEnd;
};

New global variables

Many new global variables were added, following the convention of declaring in hgTracks.h and initializing in simpleTracks.c. However, several of these are used only within hgTracks.c.