Pushing trackDb

2022 UPDATE: Install an alias to run from hgwbeta: alias trackDbPush='ssh qateam@hgwbeta sudo /root/trackDbPush'

a.k.a. "trackDb and friends"

Starting with v243 CGIs and the release of the Track Search feature, every time we push the trackDb table, we need to also push the index files that go with it (.ix and .ixx), the cv.ra file, and the metaDb table. With the [ThreeStateTrackDb] already making things complicated, and with files needing a storage location that is a special case on hgwbeta, we need a standard way to push "everything that needs to be pushed when trackDb goes out". We'll include the hgFindSpec table, as it is often forgotten (it has nothing to do with Track Search). See related page Push-Request_Etiquette.

Tables

These should be pushed from hgwbeta --> mysqlrr:

• trackDb_public (and renamed trackDb)
• metaDb_public (and renamed metaDb) -- only need to push this if it exists in the db
• hgFindSpec_public (and renamed hgFindSpec)

Files

The two index files are normally kept in /gbdb/$db/. Since gbdb/ is a shared location between hgwbeta and the RR, the files have to have a special location on hgwbeta.

• /data/trix/$db_trackDb_public.ix on hgwbeta should become /gbdb/$db/trackDb.ix on hgnfs1.
• /data/trix/$db_trackDb_public.ixx on hgwbeta should become /gbdb/$db/trackDb.ixx on hgnfs1.

The cv.ra file is not database-specific. It should be pushed from cgi-bin-public on hgwbeta --> RR (to be used by the CGIs) and to hgdownload (for users to download):

• /usr/local/apache/cgi-bin-public/encode/cv.ra on hgwbeta should become /usr/local/apache/cgi-bin/encode/cv.ra on the RR.
• /usr/local/apache/cgi-bin-public/encode/cv.ra on hgwbeta should also become /usr/local/apache/htdocs/goldenPath/encodeDCC/cv.ra on hgdownload.

The cv.ra needs to be pushed into place on hgwbeta-public before trackDb and friends pushes it out and to hgdownload:

/usr/local/apache/cgi-bin/encode/cv.ra    
       hgwbeta --> hgwbeta-public
/usr/local/apache/cgi-bin-public/encode/cv.ra

• The person updating cv.ra also first checks in a copy of src/hg/makeDb/trackDb/cv/alpha/cv.ra-> ../cv/beta/cv.ra & ../cv/public/cv.ra If you are doing this, please see Changes_to_cv.ra

More details for anyone interested

The trix files move like this:

On hgwdev:
make alpha in trackDb puts the two trix files in /hive/data/inside/trix/trackDb.ix[x].

/gbdb/$db/trackDb.ix[x] contains symlinks to these /hive files. We do it this way because the hive is supposed to be all symlinks and no actual data (data in gbdb makes the admins angry).

On hgwbeta:
make beta in trackDb puts the two trix files in /data/trix/$db_trackDb.ix[x].
make public in trackDb puts the two trix files in /data/trix/$db_trackDb_public.ix[x].

Both hgwbeta and hgwbeta-public use the "browser.trixPath" hg.conf setting to tell the CGIs to look for the trix files in this weird location. We do it this way because hgwbeta, hgwbeta-public, and the RR have a common /gbdb directory (hosted on hgnfs1).

On the RR:
The files exist in their normal location in /gbdb on the RR (well, on hgnfs1):
/gbdb/$db/trackDb.ix[x]

Our push request asks for the /data/trix/$db_trackDb_public.ix[x] files on hgwbeta to be pushed to hgnfs1, and to be named /gbdb/$db/trackDb.ix[x]. There are no special symlinks or hg.conf settings.