Microarray track: Difference between revisions
| (9 intermediate revisions by 6 users not shown) | |||
| Line 51: | Line 51: | ||
==microarrayGroups.ra== | ==microarrayGroups.ra== | ||
The microarrayGroups.ra files are located in kent/src/hg/makeDb/hgCgiData in a directory/file hierarchy similar to those of trackDb and hgNearData. During compilation and installation, these files are copied to the apache cgi-bin where various CGIs read them directly. Like other configuration .ra files, the microarrayGroups.ra files are meant to be relatively small | The microarrayGroups.ra files are located in kent/src/hg/makeDb/hgCgiData in a directory/file hierarchy similar to those of trackDb and hgNearData. During compilation and installation, these files are copied to the apache cgi-bin where various CGIs read them directly. Like other configuration .ra files, the microarrayGroups.ra files are meant to be relatively small compared to the size of the databases, and are flexibly structured. | ||
Here is an example of a microarrayGroups.ra file: | Here is an example of a microarrayGroups.ra file: | ||
| Line 82: | Line 82: | ||
groupSizes 3,3,3,3,3,3,3,3,3,3,3, | groupSizes 3,3,3,3,3,3,3,3,3,3,3, | ||
names breast,cerebellum,heart,kidney,liver,muscle,pancreas,prostate,spleen,testes,thyroid, | names breast,cerebellum,heart,kidney,liver,muscle,pancreas,prostate,spleen,testes,thyroid, | ||
name affyHumanExonSubsetReplicates | |||
type subset | |||
description Subset Arrays By Replicate | |||
expIds 0,3,6,9,12,15,18,21,24,27,30,1,4,7,10,13,16,19,22,25,28,31,2,5,8,11,14,17,20,23,26,29,32,0,3,6,9,12,15,18,21,24,27,30,2,5,8,11,14,17,20,23,26,29,32, | |||
groupSizes 11,11,11,22, | |||
names A,B,C,not B, | |||
name affyHumanExonDigestiveOrgans | |||
type subset | |||
description Subset of Digestive Organs | |||
expIds 9,10,11,12,13,14,18,19,20,9,10,11,12,13,14,18,19,20, | |||
groupSizes 3,3,3,9, | |||
names kidney,liver,pancreas,all digestive organs, | |||
Each paragraph of the microarrayGroups.ra file has the same basic fields '''name''' and '''type'''. In addition, with the exception of paragraphs with "type groupings", the paragraphs also have the '''description''', '''expIds''', '''groupSizes''', and '''names''' fields. | Each paragraph of the microarrayGroups.ra file has the same basic fields '''name''' and '''type'''. In addition, with the exception of paragraphs with "type groupings", the paragraphs also have the '''description''', '''expIds''', '''groupSizes''', and '''names''' fields. | ||
The "type groupings" paragraph is linked to the trackDb setting "groupings" through the | The "type groupings" paragraph is linked to the trackDb setting "groupings" through the '''name''' setting. This paragraph also defines which other other paragraphs (perhaps subparagraphs) are connected with it. The '''all''' setting (required) points to the paragraph defining the entire set of arrays ungrouped. The '''combine''' setting lists the paragraphs defining the other grouping methods for the set of arrays, and the '''combine.default''' mentions which of those is the default grouping method. | ||
The "type all" and "type combine" paragraphs have similar formats. The '''description''' setting (required) defines the label of the track in the browser. The '''expIds''' correspond to the same expIds as the BED, but unlike the BED, these may be in any order. The '''groupSizes''' and '''names''' settings must have the same number of elements in the comma-separated lists and the order is important. For example, in the affyHumanExonGroupByTissueMedian paragraph, the first of the names is "breast". The first of the groupSizes is "3", so the "breast" expIds are 0,1,2. Similarly, for cerebellum, the second groupSizes is 3, so those expIds are 3,4,5 | |||
If the paragraph type is "combine", then it requires an additional setting indicating how the arrays are combined. The two valid values are "median" and "mean". | |||
The "subset" type of paragraph is special, and supersedes combining. Unlike the combine paragraphs, the number of expIds listed need not be the same as the number of arrays available, but rather it must equal the sum of the values in the groupSizes line. The simplest explanation of the subset feature is that it allows the original set of expIds to be altered. Because of this, further combing by mean or median is affected. For example, if the user chooses to combine arrays by mean, and then in addition chooses to subset the arrays by replicate, and chooses the "not B" choice of subset, then the result will be the 11 tissues averaged each by their A and C replicate. Arrays mentioned in the expIds line can be mentioned multiple times, but not within the same subset. | |||
==Microarray Custom Tracks== | ==Microarray Custom Tracks== | ||
The microarray custom track format is similar to the normal BED custom track format with the addition of three required track line parameters in the header (expNames, expScale, and expStep) that mimic similar settings in the trackDb and microarrayGroups.ra files. | The microarray custom track format is similar to the normal BED custom track format with the addition of three required track line parameters in the header ('''expNames''', '''expScale''', and '''expStep''') that mimic similar settings in the trackDb and microarrayGroups.ra files. | ||
Here is an example of a custom track file that uses the BED 15 data above, but displays only 5 of the 33 arrays: | Here is an example of a custom track file that uses the BED 15 data above, but displays only 5 of the 33 arrays: | ||
| Line 103: | Line 119: | ||
<ul> | <ul> | ||
<li> | <li> | ||
'''expNames''' corresponds to the | '''expNames''' corresponds to the '''names''' setting in microarrayGroups.ra | ||
<li> | <li> | ||
'''expScale''' corresponds to the | '''expScale''' corresponds to the '''expScale''' setting in trackDb | ||
<li> | <li> | ||
'''expStep''' corresponds to the | '''expStep''' corresponds to the '''expStep''' setting in trackDb | ||
</ul> | </ul> | ||
These three parameters must be set if the '''type''' parameter is set to "array". | These three parameters must be set if the '''type''' parameter is set to "array". | ||
[[Category:Technical FAQ]] | A missing data point can be represented with an "expScores" value of -10000. It will be displayed in gray in the Genome Browser. | ||
The default coloring for microarray custom tracks is red/green. If you would like to use a different color scheme, set the extColor parameter like so: | |||
expColor="redGreen" (default color) | |||
Other options include: "redBlue", "yellowBlue", "redBlueOnWhite", or "redBlueOnYellow" | |||
[[Category:Technical FAQ]] [[Category:Custom tracks]] [[Category:Browser Linked]] | |||
Latest revision as of 00:22, 9 June 2011
This document explains the format of the data storage and display configuration files for microarray annotation tracks displayed on the UCSC Genome Browser. Note that the Gene Sorter has a somewhat different strategy for displaying microarray data that is not discussed here.
Database Representation
Microarray track data are stored in "BED 15" format. A typical line in a gapped BED (browser extensible display) file has 12 fields. Microarray, or "expRatio", BEDs have three additional fields: expCount, expIds, and expScores.
Here is an example of BED 15 format:
#chrom chromStart chromEnd name score strand thickStart thickEnd reserved blockCount blockSizes chromStarts expCount expIds expScores chr1 159639972 159640031 2440848 500 - 159639972 159640031 0 1 59, 0, 33 0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32, 0.593000,1.196000,-0.190000,-1.088000,0.093000,-0.731000,0.130000,-0.008000,-1.087000,0.609000,-1.061000,-1.092000,0.807000,0.499000,-0.322000,-0.985000,0.309000,0.000000,0.812000,-0.457000,-0.560000,0.096000,0.186000,-1.092000,0.045000,0.573000,1.170000,1.336000,1.251000,1.919000,-0.056000,-0.189000,0.028000, chr1 159640161 159640190 2440849 500 - 159640161 159640190 0 1 29, 0, 33 0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32, -0.906000,-1.247000,0.111000,-0.515000,-0.057000,-0.892000,0.167000,1.278000,0.051000,-0.596000,-0.251000,-0.826000,0.487000,0.714000,0.674000,1.046000,0.694000,0.236000,-0.718000,-1.196000,-1.274000,-1.278000,-1.055000,0.838000,-0.494000,1.137000,0.000000,0.690000,0.166000,-0.232000,0.174000,-1.253000,1.363000, chr1 159640215 159640242 2440850 500 - 159640215 159640242 0 1 27, 0, 33 0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32, -0.465000,0.127000,1.215000,-0.073000,-0.465000,-0.141000,0.507000,-0.462000,-0.464000,0.570000,1.356000,0.559000,-0.459000,-0.464000,-0.458000,0.000000,0.322000,-0.454000,0.887000,-0.464000,1.196000,-0.463000,0.376000,-0.461000,0.547000,0.032000,-0.464000,0.066000,0.762000,-0.465000,-0.456000,0.919000,-0.464000, chr1 159640256 159640309 2440851 500 - 159640256 159640309 0 1 53, 0, 33 0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32, 1.779000,1.039000,-0.068000,-0.066000,-0.050000,0.665000,0.861000,-0.067000,0.000000,0.090000,-0.067000,1.240000,-0.018000,-0.068000,0.122000,0.478000,-0.068000,-0.068000,0.630000,-0.068000,0.092000,0.620000,-0.066000,-0.068000,1.601000,0.537000,1.103000,0.720000,1.959000,1.703000,-0.061000,-0.067000,1.097000, chr1 159641139 159641250 2440852 300 - 159641139 159641250 0 1 111, 0, 33 0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32, 0.000000,1.569000,-0.090000,0.330000,-0.091000,-0.089000,0.634000,1.319000,-0.090000,-0.092000,-0.048000,-0.091000,-0.091000,0.815000,1.455000,1.127000,1.549000,0.337000,0.432000,-0.092000,-0.092000,-0.090000,0.397000,1.082000,-0.092000,-0.091000,1.780000,2.025000,1.988000,2.575000,0.224000,-0.092000,-0.092000, chr1 159642074 159642102 2440853 500 - 159642074 159642102 0 1 28, 0, 33 0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32, 2.204000,-1.053000,-0.756000,1.135000,0.393000,0.366000,-0.730000,-0.113000,-1.179000,1.079000,-1.194000,-1.195000,-0.675000,1.124000,0.382000,-1.196000,0.313000,-0.380000,0.244000,-0.258000,2.070000,1.312000,-1.195000,1.901000,-1.191000,0.438000,-1.195000,1.096000,1.635000,0.232000,-0.440000,0.528000,0.000000, chr1 159642586 159642611 2363661 600 + 159642586 159642611 0 1 25, 0, 33 0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32, 0.486000,0.065000,0.897000,-0.566000,-1.169000,-1.547000,-0.266000,0.370000,-0.864000,-0.802000,1.214000,-0.240000,0.269000,-0.067000,0.375000,0.258000,0.366000,-0.275000,0.965000,0.785000,0.571000,0.511000,0.759000,0.658000,-0.196000,-0.055000,-0.017000,-0.411000,-0.450000,-1.028000,0.104000,-0.228000,-0.000000, chr1 159642702 159642762 2363662 600 + 159642702 159642762 0 1 60, 0, 33 0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32, -0.772000,-0.532000,0.061000,-1.913000,-2.283000,-1.696000,-0.062000,0.200000,-0.609000,-2.054000,-1.184000,-2.465000,0.746000,0.673000,0.316000,0.041000,1.147000,0.590000,0.963000,0.919000,0.914000,0.000000,0.224000,-0.077000,0.395000,1.124000,0.690000,-0.369000,-1.427000,-0.970000,0.297000,0.335000,1.471000, chr1 159642857 159642883 2363663 200 + 159642857 159642883 0 1 26, 0, 33 0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32, 0.434000,-1.679000,-1.102000,0.529000,0.435000,0.614000,0.000000,0.175000,-1.517000,-1.148000,-0.180000,-1.379000,-0.798000,-0.163000,-0.219000,-1.936000,0.183000,0.627000,-0.933000,0.446000,1.150000,-2.162000,-0.018000,0.191000,1.351000,1.378000,-0.941000,0.743000,-1.632000,-0.415000,0.800000,-0.165000,1.257000, chr1 159642959 159642988 2363664 600 + 159642959 159642988 0 1 29, 0, 33 0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32, 0.000000,0.018000,-0.061000,-1.540000,-1.477000,-1.977000,-0.058000,-0.430000,-0.293000,-1.256000,-1.060000,-1.701000,0.715000,0.978000,0.980000,1.190000,1.248000,0.974000,0.907000,1.145000,0.782000,-0.152000,0.218000,-0.109000,1.550000,1.126000,1.537000,0.023000,-0.140000,-0.728000,0.520000,0.605000,0.597000, chr1 159643028 159643054 2363665 600 + 159643028 159643054 0 1 26, 0, 33 0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32, 0.291000,-0.093000,0.000000,-1.483000,-1.041000,-1.644000,0.219000,-0.189000,-0.466000,-0.942000,-0.832000,-1.026000,0.565000,0.868000,0.813000,0.786000,0.793000,0.574000,0.773000,0.764000,0.568000,-0.291000,-0.055000,-0.103000,0.916000,0.970000,0.993000,-0.106000,-0.230000,-0.402000,0.310000,0.433000,0.563000,
- expCount -- indicates how many microarrays the track is using
- expIds -- links a microarray label to its corresponding microarray measurement listed in the expScores field.
The microarray labeling information is not found in the BED 15 file at all; instead, it is part of the microarray configuration. Note that the expCount and expIds fields are constant across all rows within a file.
trackDb Settings
To display correctly in the Genome Browser, microarray tracks require the setting of several attributes in the trackDb file associated with the track's genome assembly. Here is a sample trackDb entry for a microarray track:
track affyHumanExon shortLabel Affy All Exon longLabel Affymetrix All Exon Chips group regulation priority 79.1 visibility hide type expRatio expScale 3.0 expStep 0.5 groupings affyHumanExonGroups
Of particular importance to microarray tracks are the type, expScale, expStep, and groupings parameters:
- type -- setting this parameter to "expRatio" indicates to the browser that the track's data set is a microarray.
- expScale -- an absolute value that reflects the dynamic range of the data and influences the coloring of the track. For example, an expScale setting of 3.0 indicates that most of the data lie between -3.0 and 3.0; the brightest green will be used for values less than or equal to -3.0 and the brightest red for values greater than or equal to 3.0.
- expStep -- controls the color key step values on the details page.
- groupings -- indicates the specific set of configurations to load from the microarrayGroups.ra file(s).
microarrayGroups.ra
The microarrayGroups.ra files are located in kent/src/hg/makeDb/hgCgiData in a directory/file hierarchy similar to those of trackDb and hgNearData. During compilation and installation, these files are copied to the apache cgi-bin where various CGIs read them directly. Like other configuration .ra files, the microarrayGroups.ra files are meant to be relatively small compared to the size of the databases, and are flexibly structured.
Here is an example of a microarrayGroups.ra file:
name affyHumanExonGroups type groupings all affyHumanExonAll combine affyHumanExonGroupByTissueMean,affyHumanExonGroupByTissueMedian, subset affyHumanExonSubsetByTissue,affyHumanExonSubsetByReplicate, combine.default affyHumanExonGroupByTissueMedian name affyHumanExonAll type all description All Arrays expIds 0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32, groupSizes 1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1, names breast_A,breast_B,breast_C,cerebellum_A,cerebellum_B,cerebellum_C,heart_A,heart_B,heart_C,kidney_A,kidney_B,kidney_C,liver_A,liver_B,liver_C,muscle_A,muscle_B,muscle_C,pancreas_A,pancreas_B,pancreas_C,prostate_A,prostate_B,prostate_C,spleen_A,spleen_B,spleen_C,testes_A,testes_B,testes_C,thyroid_A,thyroid_B,thyroid_C, name affyHumanExonGroupByTissueMean type combine mean description Arrays Grouped By Tissue Mean expIds 0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32, groupSizes 3,3,3,3,3,3,3,3,3,3,3, names breast,cerebellum,heart,kidney,liver,muscle,pancreas,prostate,spleen,testes,thyroid, name affyHumanExonGroupByTissueMedian type combine median description Arrays Grouped By Tissue Median expIds 0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32, groupSizes 3,3,3,3,3,3,3,3,3,3,3, names breast,cerebellum,heart,kidney,liver,muscle,pancreas,prostate,spleen,testes,thyroid, name affyHumanExonSubsetReplicates type subset description Subset Arrays By Replicate expIds 0,3,6,9,12,15,18,21,24,27,30,1,4,7,10,13,16,19,22,25,28,31,2,5,8,11,14,17,20,23,26,29,32,0,3,6,9,12,15,18,21,24,27,30,2,5,8,11,14,17,20,23,26,29,32, groupSizes 11,11,11,22, names A,B,C,not B, name affyHumanExonDigestiveOrgans type subset description Subset of Digestive Organs expIds 9,10,11,12,13,14,18,19,20,9,10,11,12,13,14,18,19,20, groupSizes 3,3,3,9, names kidney,liver,pancreas,all digestive organs,
Each paragraph of the microarrayGroups.ra file has the same basic fields name and type. In addition, with the exception of paragraphs with "type groupings", the paragraphs also have the description, expIds, groupSizes, and names fields.
The "type groupings" paragraph is linked to the trackDb setting "groupings" through the name setting. This paragraph also defines which other other paragraphs (perhaps subparagraphs) are connected with it. The all setting (required) points to the paragraph defining the entire set of arrays ungrouped. The combine setting lists the paragraphs defining the other grouping methods for the set of arrays, and the combine.default mentions which of those is the default grouping method.
The "type all" and "type combine" paragraphs have similar formats. The description setting (required) defines the label of the track in the browser. The expIds correspond to the same expIds as the BED, but unlike the BED, these may be in any order. The groupSizes and names settings must have the same number of elements in the comma-separated lists and the order is important. For example, in the affyHumanExonGroupByTissueMedian paragraph, the first of the names is "breast". The first of the groupSizes is "3", so the "breast" expIds are 0,1,2. Similarly, for cerebellum, the second groupSizes is 3, so those expIds are 3,4,5
If the paragraph type is "combine", then it requires an additional setting indicating how the arrays are combined. The two valid values are "median" and "mean".
The "subset" type of paragraph is special, and supersedes combining. Unlike the combine paragraphs, the number of expIds listed need not be the same as the number of arrays available, but rather it must equal the sum of the values in the groupSizes line. The simplest explanation of the subset feature is that it allows the original set of expIds to be altered. Because of this, further combing by mean or median is affected. For example, if the user chooses to combine arrays by mean, and then in addition chooses to subset the arrays by replicate, and chooses the "not B" choice of subset, then the result will be the 11 tissues averaged each by their A and C replicate. Arrays mentioned in the expIds line can be mentioned multiple times, but not within the same subset.
Microarray Custom Tracks
The microarray custom track format is similar to the normal BED custom track format with the addition of three required track line parameters in the header (expNames, expScale, and expStep) that mimic similar settings in the trackDb and microarrayGroups.ra files.
Here is an example of a custom track file that uses the BED 15 data above, but displays only 5 of the 33 arrays:
track type="array" expScale=3.0 expStep=0.5 expNames="breast_A,breast_B,breast_C,cerebellum_A,cerebellum_B," name="Microarray" description="Microarray custom track" chr1 159639972 159640031 2440848 500 - 159639972 159640031 0 1 59, 0, 5 0,1,2,3,4, 0.593000,1.196000,-0.190000,-1.088000,0.093000, chr1 159640161 159640190 2440849 500 - 159640161 159640190 0 1 29, 0, 5 0,1,2,3,4, -0.906000,-1.247000,0.111000,-0.515000,-0.057000,
In this example:
- expNames corresponds to the names setting in microarrayGroups.ra
- expScale corresponds to the expScale setting in trackDb
- expStep corresponds to the expStep setting in trackDb
These three parameters must be set if the type parameter is set to "array".
A missing data point can be represented with an "expScores" value of -10000. It will be displayed in gray in the Genome Browser.
The default coloring for microarray custom tracks is red/green. If you would like to use a different color scheme, set the extColor parameter like so:
expColor="redGreen" (default color)
Other options include: "redBlue", "yellowBlue", "redBlueOnWhite", or "redBlueOnYellow"
