CGI Testing

Also see: version notes and CGI_testing_responsibilities.

As of v255 CGIs, we are focusing testing on hgTables and hgTracks (and as of v317, also hgIntegrator). Other CGIs no longer receive systematic testing every CGI release; only bug fixes, new features and changes need to be tested. There should be redmine tickets associated with changes, but also check the version notes for changes related to your CGIs. Link to see the CURRENT Build Calendar

Testing on hgwbeta needs to be done in IE on Windows machine or VM.

While testing, open the error console on your web browser and watch for errors. In FireFox, go to Tools -> Error Console. You should explicitly choose the "Errors" tab if you are using FF, because the warnings tab shows almost exclusively false positives (e.g. CSS "Errors" that we and almost all other sites generate)." Errors in IE are flagged by a little error icon in the lower left hand corner of the web browser window (usually a message like "Done, with errors on page"). If you click on that icon, you see the error. This page tells you how to look at the console under IE (useful if there are multiple errors on the page). Larry says that many non-obvious bugs leave behind evidence in the web browser error log, so it can be very useful.

Sometimes when we upgrade machines we can forget supporting files, see a section below about tools and files.

hgApi

This CGI is responsible for the metadata "down arrow" links (formerly "..." links) on the hgTrackUi page and the "metadata" links on the hgc details pages of composite tracks with metadata.

1. click on a down arrow link and check that it opens up - no need to check the content. You only need to test one down arrow link - if hgApi is broken, it will break all down arrow links.
2. This CGI can also be tested by clicking on a "metadata" link in hgc details and check that it opens up - no need to check the content.

Probably doing 1 or 2 is sufficient, but it is a good habit to have the testers of hgTrackUi and hgc check 1 and 2 respectively, that way we are double covered in case one person forgets.

Information on how visibilities work can be found here

hgBlat

1. perform both a nucleotide and a protein search with default settings
2. make sure colors listed in description section are right
3. zoom in on an alignment and test "View details of parts of alignment within browser window" (DNA blat)
4. try different sorts and output types
5. make sure all the buttons work, including uploading a file
6. try uploading a file with too many bases or too many queries and verify error message
7. Try blatting against all organisms (new feature in v373).
8. Try outputting a custom track and saving the custom track in a session.

Some example input:

TTTCAGAAGACTCAAGTTCACCAGAAAGAACTTCACCACCAGATTCTTC

FQKTQVHQKELHHQILQ

hgc

This CGI displays the detail page for individual items in a track. For wiggles, it displays the details about the range of data displayed by the track in the current browser window.

Testing:

• details are displayed
• links on the details page work
• check bigBed extra field support. For example, open a CRISPR track on beta and look for the extra fields of data.
• composite tracks have specific things like metadata (Previously this use to be a drop down link, see no. 2 in hgApi, now it is all displayed. A good sample hub is the BluePrint Hub or load an ENCODE track on hg19).

cartDump, cartReset

1. check cart
2. reset cart
3. check cart again

See also: cart test protocol

hgCollection

GENERAL TESTING

1. Make a new collection
2. Add some tracks using both methods:
   • click the green plus icon
   • rt-click --> add
3. Double-click the collection and change the name and description
4. Click "GO" to see the collection in hgTracks
   • Tip, h+a keyboard shortcut will hide all tracks except the collection.
5. Turn on GC Percent:
   • rt-click from hgTracks and add it to the collection from step-4.
6. Below hgTracks, in the category for the collection, click edit.
7. Change trackUi settings:
   • Update the track height and verify that all subtracks in the collection have changed.
   • Switch between bar and points graph types.
   • Test autoscale
8. Go back to hgTrackUi and then click on the link to get back to hgCollection, Go to Track Collection Builder
9. Double-click the collection, now change the color.
   • Verify that the color has changed on hgTracks.
10. Check that the different merge methods are working:
    • transparent
    • solid
    • stacked
11. Check how missing data is treated with the following merge methods:
    • add
    • subtract
12. Go back to CGI and remove tracks using both methods:
    • Click the red minus icon
    • rt-click --> remove
13. Click discard changes to check that the last move was undone.
14. Delete the entire collection
    • Click GO to save the removal and make sure it's gone from hgTracks.

hgCustom

1. Test all three methods of entering a Custom Track (CT): pasted/typed in text, uploaded by file, URL (see Custom Track Examples or Brian's links). dev beta hgw0
2. Test the detail pages for custom tracks uploaded by clicking on the browser image for tracks. (Ex. bedDetail displays info -Brian's Rachel set displays BC###)
3. Test editing, deleting, adding, updating, HTML docs, etc.
4. Test adding multiple tracks at once (multiple tracks in one file, multiple URLs, and pasting in multiple tracks)
5. Test custom track output in relation to the Table Browser.
6. With a custom track set loaded on the Browser display (Ex: dev beta hgw0 ), turn on the multiregion with 'e v' and then zoom out 10x.

• See also examples page here: Custom Track Examples Hiram's page: http://genome-test.gi.ucsc.edu/~hiram/ctDb/hg19.chr21.ct.txt, with BrianL's additions: http://hgwdev.gi.ucsc.edu/~brianlee/customTracks/examples.WITHOUT.FTPS.txt
• It is useful giving BAMS and VCFs extra attention since they involve an additional indexed file (bam.bai with .bam and vcf.gz.tbi with .vcf.gz).

hgEncodeApi

This CGI is currently used internally by these pages:

• Experiment Matrix on human & mouse
• ChIP-seq Experiement Matrix on human & mouse
• Experiment Summary on human & mouse

These pages are part of the ENCODE portal. This CGI obtains the experiments from the hgFixed.encodeExp table on the respective machine (gets pushed every Friday from hgwdev -> hgwbeta -> mysqlrr) and obtains the dataTypes, cellTypes, and antibodies from the controlled vocabulary. Testing:

1. Make sure the matrix loads on the Experiment Matrix pages: human (beta | hgw0) & mouse (beta | hgw0).
2. Make sure the matrix loads on the ChIP-seq Experiment Matrix pages: human (beta | hgw0) & mouse (beta | hgw0).
3. Make sure the summary tables load on the Experiment Summary pages: human (beta | hgw0) & mouse (beta | hgw0).
4. Make sure that clicking on rows in the Experiment Summary and the boxes in the Experiment Matrix and ChIP-seq Experiment Matrix pages take you to results in track/file search (depending on radio button selection).
5. hgEncodeVocab description should be pulled up when clicking on the cell type matrix headings (y-axis). For the Assays, they aren't clickable but there should be a tool-tip with the full title of the assay when you hover over.
6. Leaving and returning from the Experiment Matrix pages (human and mouse) it should remember if the "Overview" was closed or open.

hgEncodeVocab

1. Go to an ENCODE track, most matrix headers should be links. Click on them, should take you to a page with a row of info about the term.
2. Go to a couple of these pages and check that the sections are displaying (these pages use hgEncodeVocab to display the info on them):
   • Registered Variables: beta, hgw0
   • Antibodies: beta, hgw0
   • Human Cell Types: beta, hgw0
   • Mouse Cell Types: beta, hgw0
3. Check a few links in the metadata from hgc details (click "metadata" link to view other links) or hgTrackUi (click arrow in subTrack list below matrix to open metadata for each subTrack with vocab terms)

hgFileSearch

(aka File Search)

About:

• Each ENCODE assembly has File Search, e.g. hg19 File Search.
• From File Search, users can search all the ENCODE download files in an assembly (across all composite tracks).
• Navigate to this CGI via the Search links on the ENCODE portal, then click one of the File Search buttons.

Testing:

1. Try searching on the different fields and drop-downs and different combinations
2. Try sorting the columns
3. Check the folder icon links back to hgFileUi for that track
4. Check the clear and cancel buttons work
5. Try clicking "Download" button to see if you can download a file
6. Encode terms: plus and minus buttons, drop down check-box lists, term specific help links

hgFileUi

1. Check that the Downloads links (below matrix) on hgTrackUi page of a track that is known to have fileSortOrder goes to hgFileUi, e.g. Broad Histone hg19 (beta | hgw0)
2. Manipulate a URL to directly access hgFileUi (beta | hgw0)
3. Make sure download button prompts a download (and doesn't take you to an error page)
4. Check the File Filter multi-select drop-downs, look for (interdependence) greyed out options
5. File accounting
6. Check the sorting of columns
7. Check the "Track Settings" link takes you back to the track's hgTrackUi page
8. Check the "File Search" link takes you to File Search

top

More info about this CGI:

Only tracks with a fileSortOrder line in their trackDb blocks will have links to the hgFileUi page, however, all composites with downloads and metadata should have a functioning hgFileUi page. If a track has a fileSortOrder line, then the "Downloads" links in the top-most light blue bar and under the subtrack list will link to the appropriate hgFileUi page. If not, then the same Download links will go to the index.html on hgdownload (or hgdownload-test), but you can still get to the hgFileUi page for a track of this kind by manipulating the URL: <machine>.ucsc.edu/cgi-bin/hgFileUi?g=<trackname> (for example: genome.ucsc.edu/cgi-bin/hgFileUi?g=wgEncodeCaltechRnaSeq). The first track with the fileSortOrder and therefore links that go to hgFileUi is ENCODE Broad Histone on hg19

hgGateway

• Click the Popular Species links and see the appropriate assembly loads
• Click within the REPRESENTED SPECIES section and see the appropriate assembly loads
• Search within the search box and see a result (Ex: panda). Click (or hit return twice) and see the appropriate assembly loads
• Search within the search box and see a result (Ex: panda) and ensure an assembly hub loads (Ex. GCF_002007445.1 = Panda).
• Hover the mouse over the phylogeny tree and see the name of the branch (Hominoidea -near human).
• Click View Sequences (try it on an assembly hub -you can then click through to hgTracks on the Sequence/Chrom name).

hgGene

Pick a UCSC Gene / Known Gene on one of the organisms with a knownGene track (Human, Mouse, Rat, D. melanogaster, C. elegans, or S. cerevisiae). For example h19.SIRT-beta hg19.SIRT1-hgw0 Then check:

• Under "Sequence and Links to Tools and Databases"
  • the internal links (in green boxes)
  • the external links (in blue boxes). Make an effort to see that these link to gene-specific pages. There should be a "User Annotations" link (to the wiki track) should be available for the later human and mouse assemblies.
• Check a representative link from each section.
• Make sure you can change the expression ratio to blue/yellow.
• In the "Orthologous Genes in Other Species" section, make sure that if there is a link in any particular column, all entries of that column have links (except for Zebrafish -- it doesn't have an hgGene page to link back to).
• Make sure the documentation links at the bottom of the page work.
• Check LS-SNP and Chimera links Example.SOD1-hgw0
• Check that can collapse and expand sections
• Check in Gene Alleles, all buttons, table sort, variant link
• Check BACE1 Beta-uc010rxh.2 hgw0-uc010rxh.2 to ensure entire page loads. (Unique variant issue, RM 11894)

hgGeneGraph

This CGI displays interactions between the selected gene and many others. There are two ways to get to this CGI: 1) through the "Tools" menu or 2) through the "Gene Interactions" track on hg19 and hg38 under the "Phenotype and Literature" group.

Here are some things you should check when testing this CGI:

• Check links to hgGene and to hgTracks
• Check that search works:
  • Enter a gene symbol (e.g. DKC1 or SURF6) into the search box and ensure that you're taken to the correct page
  • Enter junk into the search box and ensure that you see the message about the gene symbol not being found.
• Filter the displayed interactions based on their support (all, databases + curated, curated-only)
• Hide interactions outside of those with your selected gene
• Display a low amount of interactions and ensure that genes that were in the image before now show up in the "Extra Interactions" section
• Display a lot of interactions and ensure that those that were filtered before now show up in the graph
• Annotate the genes with all of the different options and check that information relevant to the annotation method is displayed on the gene mouseover
• Click a gene and ensure that hgGeneGraph is recentered around this gene
• Click a line connecting two genes and ensure the information displayed matches the line type
  • Ensure that the line "Back to ..." at the top of the page works
• Hover over a line connecting two genes and ensure that information is displayed correctly
• Hover a gene and ensure gene-relevant information shows up
• Below the interaction graph in the "Extra Interactions" section:
  • Hover over a gene name to ensure relevant info shows up
  • Click on a gene name to ensure that the supporting info is displayed
  • Click on the little arrow next to a gene and see that hgGeneGraph is recentered around that gene
• Ensure the various links to the help page work correctly

hgGenome

1. To test, select "Genome Graphs"from the "Tools" menu
2. upload a dataset from a file (can paste data too, these are small dots)

chr1 40004000 10
chr2 40004000 10
chr3 40004000 10
chr4 40004000 10
chr5 40004000 10
chrX 40004000 10
chrY 40004000 10

http://hgwdev.gi.ucsc.edu/goldenPath/help/examples/hgGenome_example1.txt

rs10218492 0.384 0.882

1. upload a data set from a URL Ex: http://hgwdev.gi.ucsc.edu/~brianlee/examples/miscFiles/hgGenomeChrBase
   1. (This was from for j in {1..22} X Y; do for i in 10001000 20002000 30003000 40004000  ; do echo chr$j $i 10 >> test; done ; done )
2. import data from a track Ex: knownGene
3. change some configurations
4. check that "browse regions" & "sort genes" work
5. check the correlate button (recommend importing two gene tracks, data should be sort of similar)

hgGtexTrackSettings

This CGI is for the "Body Map" track settings for GTEx, currently only for hg38 and hg19. It is the only CGI specifically for a track settings page.

Background Information

• CGI = hgGtexTrackSettings
• Engineer/Developer: Kate
• Urls to Body Map

hg38: http://genome.ucsc.edu/cgi-bin/hgGtexTrackSettings?db=hg38&g=gtexGene

Mouse Rollover Functionality

A. Body Map

Rolling your mouse over the actual image of the tissue (e.g., the brain image, not the labels), should

• 1. highlight that image with a bold blue or white outline around the organ or tissue,
• 2. should also highlight the associated label in the body map (and the dotted line leading the image to its label), and
• 3. should also highlight the corresponding tissue in the table (bold line around checkbox, and bold checkbox label).

B. Tissue Table

Rolling your mouse over the checkbox or checkbox label should

• 1. highlight in bold the checkbox and the checkbox label, and should
• 2. highlight the corresponding image, image label, and dotted "label to image" connector line in the body map.

Clicking Buttons Overview

A. Tissue Table

1. Tissues: "set all" - should color in all checkboxes in the tissue table. This "sets" all the tissues to be visible in the graphs.

• After 'setting all', check that all tissues are set in the track.

2. Tissues: "clear all" - should clear the fill color in all checkboxes in the tissue table.

• After 'clearing all', check that all tissues are cleared in the Tissue Table. Going to hgTracks after "clearing all" should reset to displaying all (since it's pointing to see none).

3. Try checking a few tissues in the table, such as all of the 'yellow' brain tissues checkboxes.

• After setting the brain subset, then go to the browser to see only yellow brain tissue in the graphs.

B. Body Map

• 1. Repeat step 3 above, but instead of using the checkbox in the Tissue Table, use the image labels to set the checkboxes.

C. Information button in top gold bar

• 1. Click the small blue circle with an "i" in it (information), you should move to the track description.

D. "Page Up" buttons

• 1. Scroll down to the section labeled "Track Description" or "Data Information." To the right of the blue horizontal heading bar if a small "up" arrow. Clicking this should bring you to the top of the page.

E. Go Buttons

• 1. Test the main 'Go' button in the gold bar, upper right.
• 2. Test the other 'Go' button in the Configuration blue bar.

F. Gray mini-button

• 1. In hgTracks, click on the grey mini-button to go to GTEx track settings.

G. "Links"

• 1. Check link "View table schema" in "Data Information" section.
• 2. Check all links in Track Description.

Configuration

• 1. Test view modes by toggling the drop-down from various states (full, squish, etc.) and seeing the corresponding view after going to the track.
• 2. Test the 3 different "Label" settings radio buttons.
• 3. Compare views for a checked and unchecked "Log10 transform" to see that graphs change in size, bar height should be different.
• 4. Test the "Limit to protein coding genes" checkbox. In hgTracks, hide all except GTEx, then turn on the track, "Non-coding RNA" to "show." Toggle the GTEx settings to see related charts appear and disappear. Example region: chr1:13,500-31,443. hg38 rr session
• 5. Test "Show GTEx gene model" Checking this will show Ensembl genes with chevrons, you can click on them to go into hgc.
• 6. Test "View limits maximum" ____ RPKM (default is 300). Hover over a single bar of a GTEx graph and note the score. Now change the score limit to hide it.
• 7. Test "Limit to genes scored at or above". With "Show GTEx gene model" ON, click on it to see its score in hgc, then change the limit to hide that gene.

hgHubConnect

Started with v254 CGIs, which coincides with Jim's suggestion to not systematically test all CGIs.

• Load a hub with a URL (http://genome.ucsc.edu/goldenPath/help/examples/hubDirectory/hub.txt ) dev beta hgw0

( Or use public DNA Methylation hub: http://smithlab.usc.edu/trackdata/methylation/hub.txt for multiple assemblies,mm9,hg18,hg19+) dev beta hgw0

• Check to be sure it displays properly for assembly and click items and make changes to track settings.
• Check disconnect button. Disconnect from a hub from hgTracks, the public hubs page, and the my hubs page.
• Go the table browser, test getting some output from the hub.
• From the table browser, try changing the genome away from the hub's assembly too, and test getting some output.
• After getting output from a hub in the Table Browser,then change species, and get output from something else, then a hub output from another species (say mm9 from the above methylation hub, or clade:DNA methylation, checkout arabidopsis).
• Try loading a dynamically created hub from an external site like ENCODE (look for "Visualize" button -will go to RR or click below beta link)

http://hgwbeta.soe.ucsc.edu/cgi-bin/hgTracks?db=mm10&hubUrl=https://www.encodeproject.org/experiments/ENCSR736GVO/@@hub/hub.txt

(If ENCODE changes this hub, this link may stop working, but searching their site should bring other hubs to test. Here is also another dynamic hub from http://statehub.org that may or may not change in the future).

Selenium_HubTest1 script available for automated FireFox Selenium PlugIn testing, defunct with v301 release.

hgIntegrator

This CGI takes up to 4 tracks and outputs selected information from these tracks based on their intersections with a primary track. For more information see the user guide: http://genome.ucsc.edu/goldenPath/help/hgIntegratorHelp.html

• Test output for different selections from the "region to annotate" menu
  • For "position or search term", test searching IDs from different tracks (e.g., UCSC Genes, Assembly, New HGVS Terms (ALK 1494, BRCA1 Ala744Trp, NP_002993.1:p.Asp92Glu etc.)
  • Similarly, test searching for an ID that has multiple hits (e.g., gene symbol, ENSG* IDs, etc.)
  • Paste in some regions and get output for these, clear these regions and then upload a file of regions and get output for these
• Test adding and removing data sources
  • Add 5 data sources/tracks and ensure that you can't add a 6th or add the same track twice
  • Ensure that track hubs and custom tracks show in the data sources
  • Rearrange your data sources, and check that rearrangement is reflected in output
  • Ensure that BAM/MAF (bigMaf OK) based tracks don't show up in the drop-down menus
• Select and deselect different fields in different tracks. Test "Clear all" and "Set all" buttons
• Check that you can add related tables for tracks that have linked tables
  • Check that you cannot add more than 4 related tables for a single track
• Ensure that your deselected fields don't show up in output and vice versa
• Test different output formats (viewed in web browser or downloaded as a file)
• Test getting output for custom tracks and tracks hubs (a good choice is to use one with unusual big* tracks like the Mouse strains hub).

hgLiftOver, hgConvert

1. Choose an assembly and go to hgTracks
2. Under the "View" drop down menu, click the "In Other Genomes (Convert)" link
3. Compare the output to the same conversion using liftOver:
4. LiftOver is under the "Tools" drop-down menu
5. Test both position and BED format
6. Test a variety of settings
7. Try converting multiple positions at once
8. Try uploading a file

hgLinkIn

Coming in v351 allows uniProt to have direct links.

hgLinkIn is not interactive; it's designed for automated redirects only. The invocation is straightforward: http://genome.ucsc.edu/cgi-bin/hgLinkIn?id=<id>&resource=<resource>. The CGI takes the identifier, looks for it in the specified resource, and attempts to find an assembly and position to redirect the user to in hgTracks. If no resource is supplied, the CGI attempts to locate the id in all resources (right now, June 2017, this is just UniProtKB, identified as resource=uniProt). If multiple possible locations are identified, only the first one found is used.

hgLogin

Note that genome.ucsc.edu is hard coded in to several hgLogin links, so you cannot test many components on hgw0 only, you need to wait till it hits all RR machines. Testing on beta works as expected.

You access the hgLogin CGI from the hgSession CGI (http://hgwbeta.soe.ucsc.edu/cgi-bin/hgSession)

1. Create a new account (click "Create Account" from hgSession):
   1. make sure it catches you if:
      • email addresses don't match
      • passwords entered do not match
   2. make sure you get confirmation email
   3. click confirmation email link and log into account
   4. sign out and then back in to account
2. Change account password (it should not log you out during this process):
   • Log out and back in with new password
3. Test retrieving a "lost" user name (must start signed out):
   1. At login screen click "Can't Access Your Account?" and follow prompts:
      • Testing an unregistered email address: Enter an email address that you know does NOT have an account, but you have access to and can check (e.g. a person email acct)
        • You should not get an email to an address that does not have an account (see redmine)
      • Testing a registered email address: Enter the email address you used to make the new account:
        • You should get an email with the correct username(s) associated with the account)
          • Note that more than one user name can be associated with an account, and if so, the email should list all associated user names
4. Test retrieving a "lost" password:
   1. At the login screen click "Can't Access Your Account?" and follow prompts
      • You should receive an email with a new password
      • Log in with the new (temporary) password
        • You will now be asked to change the temp password provided in the email
        • After changing password, you should STILL be logged in
5. Note: hg.conf variables control hgLogin, look to Redmine #22244 and others to learn more background.

hgNear

hgNearTest robot:
Notes from Jim about what the hgNear robot does: The hgNear robot goes turns on each column and makes sure that it doesn't crash, and has the appropriate number of items. It tests lightly the gene filters, just working on the filters on the gene column itself. It also tests each sort, making sure it doesn't crash and still you get the right number of items.

1. To test, select "Gene Sorter" from the Tools menu.
2. Type a gene into the search text box
3. Pick a gene that has a long list of results.
4. Test sort by.
5. Test configure (make sure new columns show up and test the order feature)
   • Expression colors (change them to see if they are displaying)
6. Click the column headers to make sure that the descriptions show up
7. Click on a couple of entires to make sure they display in the browser.
8. Click on "sequence" from the sorter
9. Click on "filter" and try filtering from a specific column. Don't forget to put a "*" next to the search term or it most likely won't return any results.

hgPal

There are ways to get to the hgPal CGI:

1. check the "Other Species Fasta" click-through link from a UCSC Gene details page, eg "MTOR"
2. check a "CDS FASTA" click-through from a RefSeq Genes details page (Probably outdated)
3. using the Table Browser, choose "CDS FASTA" as the output format (this should work for any genePred track) (Probably Outdated)

Check that the different settings give the expected alignment results.

hgPcr

1. test some perfectly matching primers, including a pair on the negative strand
2. find a pair of primers that shouldn't match UCSC Genes and test that they don't
3. vary settings and input primers for several assemblies and see if results make sense
4. for human and mouse browsers, test the "UCSC Genes" target. Check that UI functions as expected
5. check that the UI functions as expected for regular primers as well

hgPhyloPlace

1. Angie added this CGI to support the wuhCor1 browser. See RM#26516 for note on how it was added to regular pushes.

hgPublicSessions

Tool for sharing sessions with the world. Available in the "My Data" drop-down menu as "Public Sessions".

• Ensure that you can add and remove sessions to the Public Sessions page
  • Add sessions by checking the box under "Add to public listing" on hgSession
  • Remove sessions by unchecking that box
• Test that session sorting works properly
  • Can be sorted in ascending/descending order by popularity/time
• Check that the thumbnails are loading properly
  • If not, you should see a message stating "Thumbnail not available"
  • Thumbnails should always be available for public sessions on our servers
• Check that session name and description changes (from hgSession) show up on the Public Sessions page
• Ensure that sessions that not marked to not be loaded by others can't be marked as public.
• Test searching/filtering of sessions (enter search terms in the box in the upper right-hand corner of the page)
• Test displaying varying number of sessions on the page (change between the different numbers, click arrows at the bottom to see other pages)

Note: See Sessions section for various QA accounts (QAtester,QAtester2,QAtester3) that exist on different machines.

For adding new permanent Public Sessions an account PublicSessions has been created with an email to genecats.ucsc@gmail.com.

hgSession

1. check that a new session can be saved (note account QAtester:sessions and QAtester2:sessions QAtester3:sessions exists on hgwdev, hgwbeta, hgw0) session named hg38 exists at all three hgw0 hgwbeta hgwdev (these sessions won't show up on GBiB or other mirrors, they are are all artificially similar to streamline testing; the data isn't shared across the different machines)
2. check that old sessions are still there
3. try logging-in out of genome-preview sessions since problems with links are not visible on hgwbeta. (this is a good way to get a feel of possible genome-euro/genome-asia behavior)
4. delete a session
5. check browser & email button (may require email client)
6. click the title of session & make sure you can save changes to the description
7. try loading a session via a file and via a URL dev beta hgw0 genome-asia genome-euro
   1. There have been issues with loading assembly hubs sessions, specifically one with custom tracks and from a file. hgw0 assembly hub with custom track example that should work. hgwbeta
   2. Another potential group of sessions with pasted multi-region on an assembly hub and custom track "myTrack" (bottom) dev beta hgw0 asia euro
8. logout and try to load a session that can be shared and one that can't be shared hgw0 hgwbeta BROKEN LINK hgwdev
9. Try logging-in as case-insensitive via hgLogin "Login" link, for example using qatester instead of QAtester #17396 for details.
10. Try logging-in and out with an unusual character set (userName: AnaïsTest pwd: sessions ) this account has a session called hg38 on dev,beta,RR,asia,euro Link to load session Asia euro hgw0 beta BROKEN LINK dev (Note: OK that it will display weird AnaïsTest on dev/asia/euro)
11. Make a Public Session and ensure the image shows up on hgPublicSession (upon clicking Public Session creation on hgSession a thumbnail image is generated). Remember to remove the test Public Sesssion if made on the RR.
12. Test the ability to download custom tracks from a session (new in 2019, not required to test the reupload).
13. Not required, but you can examine #17327 to see Developer Tools level of session testing.

hgTables

hgTablesTest robot:
This script automates some testing of hgTables on hgwdev and hgwbeta.
By default, it takes the first 2 tables of the first 4 tracks for all groups.
It brings up the schema and summary page on a 5 megabase window from the middle of the first chrom in the chromInfo table.
It does the various outputs, checking that there are the right number of items.
It does not test filters or intersections.

The output of the hgTablesTest robot is in:

/hive/groups/browser/newBuild/kent/src/utils/qa/weeklybld/logs/

During Preview2 week, the log file will be named something like v4##.preview2.hgTables.log.

See also what hgTablesTest does HgTablesTest_details which also mentions HgTablesTest_details#Errors_you_can_ignore.

QA should test hgTables:

1. Check that all of the ways to limit output work: position, "define regions," "paste identifiers," filter, filter on related tables.
   1. Note that position can now take input like ALK G1494E or like NM_198056.2(SCN5A):c.1654G>T
2. Check all output formats; for "sequence," make sure the different sequence types work
3. Check output for track types that are not checked by hgTablesTest, such as: microarray names (available on GNF Atlas 2 track), MAF (available on multiz tracks), . . . please add more!
4. Do an intersection with some different options and output formats selected.
5. Do a subtrack merge (select a table from a composite track to get the option).
6. Create a custom track from the TB.
7. Import a track hub.
8. Do a correlation.
9. Try sending output to Galaxy, and GREAT.
10. Try downloading the output as a file
11. Make sure summary/stats button works. Compare values between hgwdev, hgwbeta, and the RR
12. Try different combinations of the above.

hgTrackUi

This CGI displays the track controls (trackDb .ra file settings determine the settings) and description page (determined by the trackDb .html file) of each track. A couple ways to get to hgTrackUi are clicking on the gray mini-button on the left-hand side of the browser image or by clicking on the title of the track's pull-down menu on the hgTracks page.

Here are some tracks that are popular or have complex hgTrackUis:

• ENCODE
  • test the correspondence between the matrix and the list of tracks when you turn on or off different tracks, including the graying out of boxes in the matrix
  • test different views for multiple tracks while testing correspondence between matrix and list of tracks
  • test one track that has 3 dimensional matrix beta RNA-seq, RNA-extract dimension
  • try sorting list of tracks by different columns
  • test links at the top of the page to downloads, subtracks and description
  • test that the buttons allowing users to go to super-tracks from child-tracks at the top of the page work
  • test hgApi as it pertains to hgTrackUi (see no. 1 in hgApi)
  • test filterBy settings (use Gencode)
• Conservation (a "full" track like the one found on hg19)
  • test all aspects and buttons of the multiz track, including codon frames and the mini-wiggles visible when both the subtrack and parent track are on "full"
  • use this as an opportunity to test all aspects of a wiggle track
  • try filtering by score
• SNPs
  • test that adding different gene prediction tracks do change the details page for individual SNPs
  • test one of the Filtering Options (e.g. Minimum Average Heterozygosity, Maximum Weight, etc.)
  • test the filtering by attribute
  • test color specification, including changing the default color for a specification
• mRNAs
  • test filtering by one and more than one search term, as well as different filtering options
  • test that the coloring by codons/alignment function as expected

Other non-standard trackUis include Chain/Nets, TransMap, Human Proteins, Restriction Enzymes. A fun one to test is HapMap LD Phased on hg18 since it has a radically different hgTrackUi.

Information on how visibilities on composite tracks work can be found here

hgTracks

TrackCheck

During the beta build a log is created automatically that you can view here ls -rlt /hive/groups/browser/newBuild/kent/src/utils/qa/weeklybld/logs/ , you can ignore the "breaking out early" messages. The TrackCheck robot checks the first 4 links at the default position for every track in every assembly (except for UCSC Genes). If zoomcount is set to something higher than 1, it also zooms out and checks the first 4 links there, and repeats this until zoomcount is reached. TrackCheck is 1 of 4 test robots that are run automatically on hgwbeta as part of the build process (the hgTracks tester runs it on hgw0).

• Check the TrackCheck output (located in hgwdev: /cluster/bin/build/scripts/logs/ same data as above) for the current build version for any errors found hgwbeta (it runs automatically on hgwbeta as part of the build process, currently with zoomCount=3).
• If there are errors found on hgwbeta, look at the error logs on hgwbeta by doing ssh qateam@hgwbeta for clues about what is causing it, since the error reported by TrackCheck is often just something like "error getting zoom button," with no other information. The error logs are located at hgwbeta:/usr/local/apache/logs/. The file called "error_log" is the latest one, and those with a number appended (error_log.1, etc.) are older. Look for errors that say they are coming from hgwdev's IP address (was 128.114.50.189, now 132.249.245.79 and now 128.114.198.32) around the time the robots were running.

trackCheckLog

Use the above alias on beta or the full command below.

cat /home/qateam/logs/error_log | grep -v 'CGI_TIME\|trackLog\|hidden' | grep 128.114.198.32 | less

• Even if there are no errors reported on hgwbeta, it is useful to scan the error logs for unreported errors triggered by TrackCheck. (This whole process is admittedly lame. A better robot would just report the errors directly.) If you see an error in the TrackCheck log like, Error getting zoom button this can be a clue that data didn't load on the assembly and looking into the error_log can provide hints (such as the /gbdb/ file is missing, if the data is being staged on hgwbeta).
• Quickly check the time the robot took with the stat command: stat /hive/groups/browser/newBuild/kent/src/utils/qa/weeklybld/logs/v3##.TrackCheck.log Compare the time difference between Access: and the final date of the file, or last Change: which should be under 5 1/2 hrs for the beta robot. (since Feb 2018 this has increased to several days for no clear reason).
• On CGI push day, run TrackCheck on hgw0. It is not required that the hgTracks tester watch the error logs on hgw0 while TrackCheck is running, but it can be helpful. To get to the apache logs, do:

$ ssh qateam@hgw0
$ cd logs
$ less -S error_log  (Once you are viewing the log, typing SHIFT-F will allow you to follow incoming errors.  To exit the log, type CTRL-C and then Q)

• To follow the error_log from hgw0 without the CGI_TIME or trackLog updates use the following instead:

$   tail -f  /usr/local/apache/logs/error_log | grep -v "CGI_TIME\|trackLog"  

• Note that any number of search terms can be grepped in this manner
• Until October 2012, trackCheck was run on hgw0 on release day with zoomCount=1 for all assemblies. Because running trackCheck with zoomCount=1 is not very informative and because running trackCheck with zoomCount=3 for all assemblies takes too long, as of October 2012, trackCheck will be run differently:
  • For most releases, trackCheck will be run on hgw0 on release day with zoomCount=3 for hg19, hg38, mm9 and mm10 only.
  • For all releases ending in 0 or 5 (e.g., v280, v285, etc.), trackCheck will be run on hgw0 with zoomCount=3 for all assemblies. Before March 2017, the CGIs were pushed to hgw0 on Friday afternoon so trackCheck could run over the weekend. However, the Track Check procedures have been updated: the early Friday push of all CGIs to hgw0 is no longer needed; on all releases ending in 0 or 5, the usual Track Check will be run for the usual 4 assembles (the 2 most recent human and mouse assemblies) during the release while everyone is on Skype and before the CGIs are pushed to the RR. After the release, the full Track Check for all assemblies will be run on hgw0, after the new CGIs are on the RR. Because Track Check rarely find problems, the procedure changed to reduce burden on the admins and push shepherd such that the early Fri push is no longer needed.

If problems are found on the full Track Check, they should be reported in Redmine#19112

• • A full trackCheck will still be run on beta as a normal part of the release cycle.

Track Check errors to ignore in /hive/groups/browser/newBuild/kent/src/utils/qa/weeklybld/logs:

1. Breaking out early
2. svg errors, e.g.,

• line 22 column 13 - Error: <svg> is not recognized!
• line 23 column 3 - Error: path is not recognized!
• line 23 column 23,224 - Error: circle is not recognized!
• line 23 column 23,276 - Error: path is not recognized!
• line 23 column 23,661 - Error: circle is not recognized!
• line 23 column 23,737 - Error: circle is not recognized!
• line 23 column 23,789 - Error: <g> is not recognized!
• This document has errors that must be fixed before
• using HTML Tidy to generate a tidied up version
• Can't find cartVersion in track database $DBS chromosome chr*

main page testing

• Manually check anything that has been an issue during the past release cycle.
• Set position to a region of about 20bp

  chr1:5000-5015

  and turn on all of the tracks in hg38 to dense (using configure page). You will need to do this in several chunks (as of the v386 CGIs, if you get too many tracks on at once, you will see an error that says "Image is over 32,000 pixels high (32013 pix) at the following track which is now hidden:"). Make sure all tracks load okay. -- On hg38, you can turn on all track sets, then hide the last 4, then turn on all but Conservation, then do just Conservation. --- TBD on hg19 ---
• Turn off centerLabels and leftLabels, zoom in or out and drag the display around and make sure everything is functioning correctly.
• Check that drag/reorder of tracks works
• Check that reverse works
• Check that resize works
• Check View -> External Tools sends DNA to external sites (recommend going to less than 100 bases)
• Try a few of the right click functions
  • Check the various options on pop-up right-click configure pages (Ex. genes track/mRNAs/GC percent, others will take you to hgTrackUi)
• Go to the beginning of a chromosome, turn on exon view "e v", changing padding, zoom out (make sure genes don't disappear from start of chromosome #16708)
• Turn on gene view through multi-region, drag and drop screen
• Turn on haplotype view in multi-region, and highlight feature
• Turn on multi-region custom track, try to zoom out (should stick to regions). Example BED12 hg38: http://hgwbeta.cse.ucsc.edu/goldenPath/help/examples/bed12formultregion.txt
• As part of GBiB testing, turn on GbibSlowNet and open hgTracks to just the defaults. Click into hgGene (report anything unusually slow). See GBiB Testing for more info.
  • You can ssh to the identity of qateam@hgwdev and then use 'ssh boxBeta' with the default GBiB password of "browser" and have access to the command line to run gbibSlowNet and then when you are done return it to normal with gbibFastNet .
• Check for speed of loading during beta week on hg38. Reset and add &measureTiming=1 beta link and navigate to 'chr1' timing should be about Overall total time: 40479 millis , if the timing seems supsicious, repeat (this is to catch an issue like ORegAnno #17452).

track search

• Always be thinking of new track types (GTEx barChart, ?other new types?) to check if they result in searches.
  • QA can help remind engineers to include these new types within different browser resources such as Track Search (?other areas?).
• Try a search that will bring up the 3 different kinds of tracks, which are are shaded with different colors:
  • tan: super-track (has a folder/wrench icon)
  • light orange: composite (has a folder/wrench icon)
  • light yellow: an actual data track (not a container)

(note that ENCODE Regulation track on hg18 is a special kind of super-track)

• Make sure you can configure tracks via the folder/wrench icons as well as by adjusting the "view" drop-downs (after hitting submit on a configure page, you should be returned to the track search page)
• Try the different sort types ("Alphabetically" sorts on the longLabel)
• Check that the track description pops up when you click a blue track name
• Check that metadata appears when you click the down arrow "(v)" after a track name
  • Check that the links within the metadata work
• Try a couple of advanced searches
  • Make sure searching with "ENCODE terms" is working

hgUserSuggestion

This is new. For URLs supplied below, don't forget to adjust the server and the "txtInput" field (the simple captcha code) as required for the page you're submitting from.

• Make a legitimate suggestion and see if it actually goes through.
• Try mismatched email addresses
• Try adding multiple email addresses
• Try a URL like the following BROKEN http://hgwdev.gi.ucsc.edu/cgi-bin/hgUserSuggestion?suggestName=a&suggestEmail=jcasper@soe.ucsc.edu&suggestCfmEmail=jcasper@soe.ucsc.edu&suggestCategory=CRAZY&suggestSummary=test&suggestDetails=testing&txtInput=73147&do.suggestSendMail=1 to see if you can submit to a random category.
• Try a URL like BROKENhttp://hgwdev.gi.ucsc.edu/cgi-bin/hgUserSuggestion?suggestName=a&suggestEmail=jcasper@soe.ucsc.edu&suggestCfmEmail=jcasper@soe.ucsc.edu&suggestCategory=Tracks&suggestSummary=test&suggestDetails=testing&txtInput=73147&suggestWebsite=http://www.junk.com&do.suggestSendMail=1 to make sure the hidden field filter works.

An additional new responsibility for this feature includes following up with the suggestion emails, which can result in creating and answering a MLQ ticket if there is a normal looking question, or following-up with appropriate people if the suggestion is more complicated.

hgVai

hgVai will generate a set of data to work with if the Artificial Variants dataset is chosen, but it will also work on a pgSNP or VCF custom track, a VCF track that is part of a data hub, or on a list of rs IDs. This data hub has a VCF file in it: BROKEN http://hgwdev.gi.ucsc.edu/~rhead/hubs/hub.txt.

Once some sample data is loaded via a custom track and/or a track hub (it's good to test both), navigate to the VAI from the Tools menu or by clicking the "go to variant annotation integrator" button in hgCustom. Then:

• Try selecting different gene sets for annotations. You should get different effect predictions in places where the gene tracks differ. Note that the effect predictions are not optional; they will appear in the output no matter what.

• Try adding optional annotations: dbNSFP scores, the rsID from dbSNP, conserved elements, conservation scores. Note that the optional annotations (at least for now) depend only on the positions of the variants in the user data; they are not affected by the gene set selected.

• Try filtering the results in different ways.

• Make sure the different output formats work.

• Make sure the "maximum number of variants to be processed" drop-down (near the top of the page) is working.

• Make sure you can change "region to annotate" to a position and output is limited to that postion.

• Since this page uses javascript, it's important to test any changes on Windows/IE.

hgVisiGene

1. test the search box by entering a gene name
2. check the zoom buttons
3. check that the "Gene" link opens the correct gene details page, and that that "visiGene" link in that gene details page retrieves the correct images in visiGene
4. pick an image and check all of the links for that page
5. for images composed of several smaller images, check that the pane descriptions are displaying correctly (not all panes will have an associated pane description)

(Note: the images from Mahoney are a subset of the MGI/Jax images. The Mahoney images should list two sources and should show two sets of acknowledgements.)

Other

hubApi

hubApi testing is split into two parts. Testing on dev, which is more robust and includes Hiram's test script, and then beta/rr testing.

For testing on dev (preview1/preview2):

First you will want to run Hiram's test script. This is particularly useful when new features are added as it looks at 100+ different inputs. It also checks for specific outputs, which aren't always static on dev, such as the number of tracks in hg19. For this reason, some errors will be found (2 in 8/2019), however it is clear that they are both due to updated data (related to a different itemCount).

Run the tests with the commands below, and check for any additional errors. This is only necessary during preview1/preview2 as the URL used in the script is hard-coded to genome-test/Hiram's sandbox. The -k flag tells the program to continue despite errors.

cd ~/kent/src/hg/hubApi/tests
make test -k

The second test script is a python script created by QA that will perform most of the queries in the help page (http://genome.ucsc.edu/goldenPath/help/api.html). The script should error out if any of the commands do not function, however, some of them also have a sanity check output. The script can be run with no arguments to see a help message.

cd ~/genecats/qa/testTools/hubApiTest
./hubApiTest.py -m dev

Additionally if there are new features, additional examples should be tested. Function templates can always be found in the help page (http://genome.ucsc.edu/goldenPath/help/api.html). If new features are being added, remember to update the help page as well.

For testing on beta/rr:

For beta and the rr the python script should also be used with the beta and then the rr designation. Likewise, new features should be tested with additional functions. If additions have been made to the help page, don't forget to push the new page out after the CGI release.

• Note: The QA script can be expanded at any time to cover additional cases or features. See the readme in the directory: ~/genecats/qa/testTools/hubApiTest

If you want manually visit the API on beta and watch the logs you can do something like the following:

• https://hgwbeta.soe.ucsc.edu/cgi-bin/hubApi/ ...
• https://hgwbeta.soe.ucsc.edu/cgi-bin/hubApi/list/ucscGenomes
• https://hgwbeta.soe.ucsc.edu/cgi-bin/hubApi/list/tracks?genome=hg38
• https://hgwbeta.soe.ucsc.edu/cgi-bin/hubApi/list/chromosomes?genome=hg38;track=gold
• https://hgwbeta.soe.ucsc.edu/cgi-bin/hubApi/getData/sequence?genome=hg38;chrom=chrM;start=4321;end=5678
• https://hgwbeta.soe.ucsc.edu/cgi-bin/hubApi/getData/track?genome=hg38;track=gold;chrom=chr1;start=47000;end=48000
• https://hgwbeta.soe.ucsc.edu/cgi-bin/hubApi/list/schema?genome=hg38;track=gold
• https://hgwbeta.soe.ucsc.edu/cgi-bin/hubApi/list/hubGenomes?hubUrl=http://hgdownload.soe.ucsc.edu/hubs/mouseStrains/hub.txt

das

This CGI is rarely tested, but here is an example query, it returns all the records in the refGene table for the chromosome position chr1:1-100000 on the hg38 assembly in XML:

http://genome.ucsc.edu/cgi-bin/das/hg38/features?segment=1:1,100000;type=refGene

Here is info from our FAQ and here is external site info about Distributed Annotation System (DAS).

genome-euro genome-asia

Our official mirrors genome-euro and genome-asia are separate machines. Once the CGI testing process is complete and the CGIs get pushed out to the RR, genome-euro/asia should receive some basic testing to ensure that nothing obvious is broken.

You can shell into genome-euro by using the command: ssh qateam@genome-euro (genome-asia requires an account, ask admin if you don't have one).

The spirit of this testing responsibility is to ensure that new software makes it to these machines, and to think about these machines when new different software has special dependencies. For example, a new CGI hgGeneGraph was released in 2017, it would be good for the person with this testing responsibility to check the new CGI operated on genome-euro and genome-asia (it turned out the new CGI required some special admin-level installations). Otherwise, the testing is general ensuring things continue to operate.

First, go to /usr/local/apache/cgi-bin to make sure the new CGIs have made it to genome-euro (ls -l to check dates). Or check this in Safari on hgTracks. Minimally, the following should be performed:

• Monitor the error log on genome-euro for ~20 minutes (or for the duration of the testing, whichever is longer) to make sure no unexpected errors appear
  • The error log on genome-euro is located at: /var/log/httpd
  • Use the command: tail -f /usr/local/apache/logs/error_log | grep -v CGI_TIME
  • To stop monitoring, hit CTRL-C
  • If you want to also open a second log to watch all use you can use less and follow the log the with SHIFT-F
  • less -S /usr/local/apache/logs/error_log ...then type SHIFT-F
• View the default tracks in hgTracks for hg19, mm9 and mm10
• Click into a UCSC Genes details page
• Click into a RefSeq Genes details page
• Load an hg19 custom track
• Perform an hg19 blat search
• Perform in-silico PCR on hg19
• Perform a liftOver from hg19 to mm9
• Perform a conversion from hg19 to mm9

GBiB

• See Official GBiB Tester for specifics.
• All other QAers also test their tickets & CGIs on GBiB, see All Other QAers for details.

GBiC

• See GBiC_Testing for how to use vagrant to bring up different machine instances to test the GBiC install script..

Supporting Files / New Machine

There are some interesting supporting executables and files we need for some tracks and tools to work. These sometimes are forgotten when we upgrade to a new machine:

• Add psxy and ps2raster (GMT generic mapping tools) executables for HGDP world map (#11589 & mirror page)
• Add GhostWriter/ghostscript for PDF/PS to work properly
• Add ImageMagick for Public Sessions image summary
• Add loader/dot_static for Gene Interactions graph display
• Add epel-release first then R-core for GTEx hgc details page (#19830)
• Add rsync, curl, MySQL-Python, libpng12, httpd (mariadb-server)
• CRAM file support specifics -port 80 hosting .bai & another script that pulls in reference.
• Search a term like "ribosome" or "trna" that will have more than 500 hits. This can bring up a "too many open files" error where cluster-admin need to increase the number of open files. Request 10,000. But note a command cluster-admin might use to check the number is "ulimit -Hn" (*H*ard limit of *n*umber of files) and can show 10,000, but the CGIs may still be limited to 1024 files, as every program has its own limit.
• Test RNA plot output of hgGene, check picture, text, PS image.
• Ensure error logs united for https and http access & https certificate
• A coming new CGI, possibly hgCanary might help do these tests for new machines (likely under hg.conf control)

NOTE: whenever we update a new machine sometimes cronjobs need to have their known_hosts updated to remove the old machine IPs (edit in the .ssh directory) and also, if we update the RR or one of our mirrors, we might need to contact Public Hubs who restrict data to certain IP ranges (including genome-euro and genome-asia).

Javascript Changes

• If you see "hgTracks object" error think Javascript. In Chrome on Apples use option + command + J (⌥ + ⌘ + J) to opens the Developer Tools with the Console tab select.
• When big Javascript changes happen, here are some CGIs to consider looking at.
  • hgTracks
    • Track Search/RightClick/WiggleDetails
  • hgTrackUi
    • Matrix/subtracks/BigHubs
  • hgCollection
    • Double click to change colr selection/rename tracks
  • hgIntegrator
    • Drag drop order, select fields, select tracks.
  • hgGtexTrackSettings
    • Select tissues
  • hgTables
  • hgSessions
  • hgVai

New Track Types

When we add new track types, say like "bigTime" here are some areas to think about:

• hgTracks draws well (test multi-region)
• hgTracks ->Test PDF output (this has broken on some new display types)
• hgTracks -> Works with right-click configurations
• hgTracks ->Track Search page, shows up in types
• hgTrackUi -> schema links, settings can be updated
• hgc -> details display well
• hgTables -> can extract data, schema view
• hgIntegrator -> can extract data
• hubApi -> can extract data from API (or message sharing not available)
• hgSessions -> can download the data with new backup feature (if text upload)
• hgHubConnect -> can this type work in hubs/assembly hubs, if so, when connected, can you extract from hgTables/hgIntegrator/hubApi?
• Other: Consider if it should work in hgVai, potential impact on Mirrors?

---

• Most important: Documentation
  • Data Format page of types (is a new type specific page needed too?)
  • hgCustom.c list of types that can be loaded
  • Custom Track User Guide page, has a list of types (linking to data formats)
  • Track Hubs User Guide page, has a list of types (linking to data formats)
  • The Track Hubs Database Definitions Page
    • If a new type supported in hubs, important to share trackDb control hints for users.
    • BrianL should announce new type to track-hubs@soe.ucsc.edu address

Utilities

Testing an updated utility may require a make compile. Do the following:

cd ~/kent/src/utils/(utility directory)

e.g. cd ~/kent/src/utils/bedToBigBed

Then do the following steps:

git pull
make clean
make compile

You can test the utility from the kent/src tree:

~/kent/src/utils/(utility directory)/(utility)

e.g. ~/kent/src/utils/bedToBigBed/bedToBigBed

top