CGI Testing

Also see: version notes and CGI_testing_responsibilities.

As of v255 CGIs, we are focusing testing on hgTables and hgTracks. Other CGIs no longer receive systematic testing every CGI release; only bug fixes, new features and changes need to be tested. There should be redmine tickets associated with changes, but also check the version notes for changes related to your CGIs.

Testing on hgwbeta needs to be done in IE on Windows machine.

While testing, open the error console on your web browser and watch for errors. In FireFox, go to Tools -> Error Console. You should explicitly choose the "Errors" tab if you are using FF, because the warnings tab shows almost exclusively false positives (e.g. CSS "Errors" that we and almost all other sites generate)." Errors in IE are flagged by a little error icon in the lower left hand corner of the web browser window (usually a message like "Done, with errors on page"). If you click on that icon, you see the error. This page tells you how to look at the console under IE (useful if there are multiple errors on the page).

Larry says that many non-obvious bugs leave behind evidence in the web browser error log, so it can be very useful.

hgTracks

TrackCheck

The TrackCheck robot checks the first 4 links at the default position for every track in every assembly (except for UCSC Genes). If zoomcount is set to something higher than 1, it also zooms out and checks the first 4 links there, and repeats this until zoomcount is reached. TrackCheck is 1 of 4 test robots that are run automatically on hgwbeta as part of the build process (the hgTracks tester runs it on hgw0).

• Check the TrackCheck output (located in hgwdev:/cluster/bin/build/scripts/logs/) for the current build version for any errors found hgwbeta (it runs automatically on hgwbeta as part of the build process, currently with zoomCount=3).
• If there are errors found on hgwbeta, look at the error logs on hgwbeta for clues about what is causing it, since the error reported by TrackCheck is often just something like "error getting zoom button," with no other information. The error logs are located at hgwbeta:/usr/local/apache/logs/. The file called "error_log" is the latest one, and those with a number appended (error_log.1, etc.) are older. Look for errors that say they are coming from hgwdev's IP address (128.114.50.189) around the time the robots were running.
• Even if there are no errors reported on hgwbeta, it is useful to scan the error logs for unreported errors triggered by TrackCheck. (This whole process is admittedly lame. A better robot would just report the errors directly.)
• On CGI push day, run TrackCheck on hgw0. It is not required that the hgTracks tester watch the error logs on hgw0 while TrackCheck is running, but it can be helpful. To get to the apache logs, do:

$ ssh qateam@hgw0
$ cd logs

main page testing

• Manually check anything that has been an issue during the past release cycle.
• Set position to a region of about 20bp and turn on all of the tracks in hg18 to dense (using configure page). You may need to do this in several chunks (as of the v244 CGIs, if you get too many tracks on at once, you will see an error that says "Image is over 32,000 pixels high (32013 pix) at the following track which is now hidden:"). Make sure all tracks load okay. -- addendum: chr22:39,957,028-39,957,030 on hg19 seems to load all tracks in one fell swoop.
• Check that drag/reorder of tracks works
• Check that reverse works
• Check that resize works
• Check the various options on the configure page
• Try a few of the right click functions

track search

• Try a search that will bring up the 3 different kinds of tracks, which are are shaded with different colors:
  • tan: super-track (has a folder/wrench icon)
  • light orange: composite (has a folder/wrench icon)
  • light yellow: an actual data track (not a container)

(note that ENCODE Regulation track on hg18 is a special kind of super-track)

• Make sure you can configure tracks via the folder/wrench icons as well as by adjusting the "view" drop-downs (after hitting submit on a configure page, you should be returned to the track search page)
• Try the different sort types ("Alphabetically" sorts on the longLabel)
• Check that the track description pops up when you click a blue track name
• Check that metadata appears when you click the down arrow "(v)" after a track name
  • Check that the links within the metadata work
• Try a couple of advanced searches
  • Make sure searching with "ENCODE terms" is working

Information on how visibilities work can be found here

hgTables

Notes from Jim about what hgTablesTest robot does: the hgTables robot goes through the first two tables in the first four tracks in each group, and brings up the schema and summary page on a 5 megabase window, and then does the various outputs, checking that there are the right number of items. It does not test filters or intersections.

The output of the hgTablesTest robot is in /cluster/bin/build/scripts/logs/.

1. check that all of the ways to limit output work: position, "define regions," "paste identifiers," filter, filter on related tables
2. check all output formats; for "sequence," make sure the different sequence types work
3. check output for track types that are not checked by hgTablesTest, such as: microarray, (please add more!)
4. do an intersection with some different options and output formats selected
5. do a subtrack merge (select a table from a composite track to get the option)
6. create a custom track from the TB
7. import a track hub
8. do a correlation
9. try sending output to Galaxy and GREAT
10. make sure summary/stats button works
11. try different combinations of the above

hgc

check the hgc details of one track per group on hg18, try a few links on the details page for a few, and try at least one composite to try composite specific things like metadata link. Also test hgApi as it pertains to hgc (see no. 2 in hgApi).

hgTrackUi

There are a couple of tracks to always check since they are very popular.

• ENCODE
  • test the correspondence between the matrix and the list of tracks when you turn on or off different tracks, including the graying out of boxes in the matrix
  • test different views for multiple tracks while testing correspondence between matrix and list of tracks
  • test one track that has 3 dimensional matrix
  • try sorting list of tracks by different columns
  • test links at the top of the page to downloads, subtracks and description
  • test that the buttons allowing users to go to super-tracks from child-tracks at the top of the page work
  • test hgApi as it pertains to hgTrackUi (see no. 1 in hgApi)
  • test filterBy settings (use Gencode)
• Conservation (a "full" track like the one found on hg19)
  • test all aspects and buttons of the multiz track, including codon frames and the mini-wiggles visible when both the subtrack and parent track are on "full"
  • use this as an opportunity to test all aspects of a wiggle track
  • try filtering by score
• SNPs
  • test that adding different gene prediction tracks do change the details page for individual SNPs
  • test one of the Filtering Options (e.g. Minimum Average Heterozygosity, Maximum Weight, etc.)
  • test the filtering by attribute
  • test color specification, including changing the default color for a specification
• mRNAs
  • test filtering by one and more than one search term, as well as different filtering options
  • test that the coloring by codons/alignment function as expected

Finally, test a few random tracks from a few different assemblies (good candidates with non-standard trackUis include Chain/Nets, TransMap, Human Proteins, Restriction Enzymes). A fun one to test is HapMap LD Phased on hg18 since it has a radically different hgTrackUi.

Also test the "configure" button for one assembly.

top

Information on how visibilities work can be found here

hgGene

Pick a UCSC Gene / Known Gene on one of the organisms with a knownGene track (Human, Mouse, Rat, D. melanogaster, C. elegans, or S. cerevisiae). Then check:

• Under "Sequence and Links to Tools and Databases"
  • the internal links (in green boxes)
  • the external links (in blue boxes). Make an effort to see that these link to gene-specific pages. There should be a "User Annotations" link (to the wiki track) should be available for the later human and mouse assemblies.
• Check a representative link from each section.
• Make sure you can change the expression ratio to blue/yellow.
• In the "Orthologous Genes in Other Species" section, make sure that if there is a link in any particular column, all entries of that column have links (except for Zebrafish -- it doesn't have an hgGene page to link back to).
• Make sure the documentation links at the bottom of the page work.
• Check LS-SNP and Chimera links
• Check that can collapse and expand sections

hgNear

Notes from Jim about what the hgNear robot does: The hgNear robot goes turns on each column and makes sure that it doesn't crash, and has the appropriate number of items. It tests lightly the gene filters, just working on the filters on the gene column itself. It also tests each sort, making sure it doesn't crash and still you get the right number of items.

1. To test on this click on "Gene Sorter" from the blue navigation bar.
2. Type a gene into the search text box
3. Pick a gene that has a long list of results.
4. Test sort by.
5. Test configure (make sure new columns show up and test the order feature)
   • Expression colors (change them to see if they are displaying)
6. Click the column headers to make sure that the descriptions show up
7. Click on a couple of entires to make sure they display in the browser.
8. Click on "sequence" from the sorter
9. Click on "filter" and try filtering from a specific column. Don't for get to put a "*" next to the search term or it most likely won't return any results.

hgCustom

1. Test all three methods of entering a Custom Track (CT): typed in, uploaded by file, URL.
   1. Test the detail pages for each custom track uploaded by clicking on the browser image for that track.
2. Test editing, deleting, adding, updating, HTML docs, etc.
3. Test adding mutiple tracks at once (multiple tracks in one file, multiple URLs, and pasting in multiple tracks)
4. Test CTs in relation to the Table Browser.

• See also examples page here: Custom Track Examples
• Can also see Hiram's page of nearly all custom track types: http://genome-test.cse.ucsc.edu/~hiram/ctDb/hg19.chr21.ct.txt

hgVisiGene

1. test the search box by entering a gene name
2. check the zoom buttons
3. check that the "Gene" link opens the correct gene details page, and that that "visiGene" link in that gene details page retrieves the correct images in visiGene
4. pick an image and check all of the links for that page
5. for images composed of several smaller images, check that the pane descriptions are displaying correctly (not all panes will have an associated pane description)

(Note: the images from Mahoney are a subset of the MGI/Jax images. The Mahoney images should list two sources and should show two sets of acknowledgements.)

hgPal

There are 3 ways to get to the hgPal CGI:

1. check a "Protein FASTA" click-through from a UCSC Gene details page
2. check a "CDS FASTA" click-through from a RefSeq Genes details page
3. using the Table Browser, choose "CDS FASTA" as the output format (this should work for any genePred track)

Check that different settings give the expected results.

hgBlat

1. perform both a nucleotide and a protein search with default settings
2. make sure colors listed in description section are right
3. zoom in on an alignment and test "View details of parts of alignment within browser window"
4. try different sorts and output types
5. make sure all the buttons work, including uploading a file
6. try uploading a file with too many bases or too many queries and verify error message

top

hgPcr

1. test some perfectly matching primers, including a pair on the negative strand
2. find a pair of primers that shouldn't match UCSC Genes and test that they don't
3. vary settings and input primers for several assemblies and see if results make sense
4. for human and mouse browsers, test the "UCSC Genes" target. Check that UI functions as expected
5. check that the UI functions as expected for regular primers as well

hgLiftOver, hgConvert

1. Choose an assembly and go to hgTracks
2. Hit "Convert" in the blue bar at the top of the page
3. Compare the output to the same conversion using liftOver:
4. LiftOver is at Home -> Utilities -> Batch Coordinate Conversion (liftOver)
5. Test both position and BED format
6. Test a variety of settings
7. Try converting multiple positions at once
8. Try uploading a file

pbGateway, pbGlobal, pbTracks

1. general testing; click around
2. enter a protein symbol
3. review results page
4. click into pbGlobal and check display

hgSession

1. check that a new session can be saved
2. check that old sessions are still there
3. delete a session
4. check browser & email button; click the title of session & make sure you can save changes to the description
5. try loading a session via a file and via a URL
6. logout and try to load a session that can be shared and one that can't be shared

hgGenome

1. find it by going Home -> Genome Graphs
2. upload a dataset from a file
3. upload a data set from a URL
4. import data from a track
5. change some configurations
6. check that "browse regions" & "sort genes" work
7. check the correlate button

Some good genome graph data for testing: http://hgwdev.cse.ucsc.edu/~rhead/genomeGraphsWithColumns

cartDump, cartReset

1. check cart
2. reset cart
3. check cart again

See also: cart test protocol

hgApi

This CGI is responsible for the metadata "..." links on the hgTrackUi page and the "metadata" links on the hgc details pages of composite tracks with metadata.

1. click on a "..." link and check that it opens up - no need to check the content. You only need to test one "..." link - if hgApi is broken, it will break all "..." links.
2. This CGI can also be tested by clicking on a "metadata" link in hgc details and check that it opens up - no need to check the content.

Probably doing 1 or 2 is sufficient, but it is a good habit to have the testers of hgTrackUi and hgc check 1 and 2 respectively, that way we are double covered in case one person forgets.

hgEncodeVocab

1. Go to an ENCODE track, most matrix headers should be links. Click on them, should take you to a page with a row of info about the term.
2. Go to a couple of these pages and check that the sections are displaying (these pages use hgEncodeVocab to display the info on them):
   • Registered Variables
   • Antibodies
   • Human Cell Types
   • Mouse Cell Types
3. Check a few links in the metadata from hgc details (click "metadata" link to view other links) or hgTrackUi (click "..." link to view other links)

hgFileUi

1. Check that the Downloads links on hgTrackUi page of a track that is known to have fileSortOrder goes to hgFileUi, e.g. Broad Histone hg19 (beta | hgw0)
2. Manipulate the URL to see a composite without fileSortOrder in hgFileUi (beta | hgw0)
3. Make sure download button prompts a download (and doesn't take you to an error page)
4. Check the File Filter multi-select drop-downs
5. Check the sorting of columns
6. Check the "Track Settings" link takes you back to the track's hgTrackUi page
7. Check the "File Search" link takes you to File Search

top

More info about this CGI:

Only tracks with a fileSortOrder line in their trackDb blocks will have links to the hgFileUi page, however, all composites with downloads and metadata should have a functioning hgFileUi page. If a track has a fileSortOrder line, then the "Downloads" links in the top-most light blue bar and under the subtrack list will link to the appropriate hgFileUi page. If not, then the same Download links will go to the index.html on hgdownload (or hgdownload-test), but you can still get to the hgFileUi page for a track of this kind by manipulating the URL: <machine>.ucsc.edu/cgi-bin/hgFileUi?g=<trackname> (for example: genome.ucsc.edu/cgi-bin/hgFileUi?g=wgEncodeCaltechRnaSeq). The first track with the fileSortOrder and therefore links that go to hgFileUi is ENCODE Broad Histone on hg19

hgFileSearch

(aka File Search)

About:

• Each ENCODE assembly has File Search, e.g. hg19 File Search.
• From File Search, users can search all the ENCODE download files in an assembly (across all composite tracks).
• Navigate to this CGI via the Search links on the ENCODE portal, then click one of the File Search buttons.

Testing:

1. Try searching on the different fields and drop-downs and different combinations
2. Try sorting the columns
3. Check the folder icon links back to hgFileUi for that track

hgHubConnect

This is new as of v254 CGIs, which coincides with Jim's suggestion to not systematically test all CGIs.

top