Scroll to navigation

STILTS-TCUBE(1) Stilts commands STILTS-TCUBE(1)

NAME

stilts-tcube - Calculates N-dimensional histograms

SYNOPSIS

stilts tcube [cols=<expr> ... ...] [ifmt=<in-format>] [istream=true|false] [in=<table>] [icmd=<cmds>] [bounds=[<lo>]:[<hi>] ...] [binsizes=<size> ...] [nbins=<num> ...] [combine=sum|sum-per-unit|count|count-per-unit|mean|median|Q1|Q3|min|max|stdev|stdev_pop|hit] [out=<out-file>] [otype=byte|short|int|long|float|double] [scale=<expr>]

DESCRIPTION

tcube constructs an N-dimensional histogram, or density map, from N columns of an input table, and writes it out as an N-dimensional data cube. The parameters you supply define which N numeric columns of the input table you want to use and the dimensions (bounds and pixel sizes) of the output grid, as well as any weighting to be applied to each point and how the weighted quantities in a single bin are to be aggregated together. Each table row then defines a point in N-dimensional space. The program goes through each row, and if the point that row defines falls within the bounds of the output grid you have defined, associates the weight value with the relevant pixel in the output grid. When all the input values have been processed, the weights in each pixel are aggregated according to the requested combination method.

The resulting N-dimensional array, whose pixel values represent an aggregation of the rows associated with that region of the N-dimensional space, is then written out as a FITS file. In one dimension, this gives you a normal histogram of a given variable. In two dimensions it might typically be used to plot the density or weighted density on the sky of objects from a catalogue.

As with some of the other generic table commands, you can perform extensive pre-processing on the input table by use of the icmd parameter before the actual cube counts are calculated.

See also tgridmap, which does a similar job to this command but writes the output in table format.

OPTIONS

Columns to use for this task. One or more <expr> elements, separated by spaces, should be given. Each one represents a numeric value from the table, provided as a column name or algebraic expression.

The number of columns listed in the value of this parameter defines the dimensionality of the output data cube.

Specifies the format of the input table as specified by parameter in. The known formats are listed in SUN/256. This flag can be used if you know what format your table is in. If it has the special value (auto) (the default), then an attempt will be made to detect the format of the table automatically. This cannot always be done correctly however, in which case the program will exit with an error explaining which formats were attempted. This parameter is ignored for scheme-specified tables.

If set true, the input table specified by the in parameter will be read as a stream. It is necessary to give the ifmt parameter in this case. Depending on the required operations and processing mode, this may cause the read to fail (sometimes it is necessary to read the table more than once). It is not normally necessary to set this flag; in most cases the data will be streamed automatically if that is the best thing to do. However it can sometimes result in less resource usage when processing large files in certain formats (such as VOTable). This parameter is ignored for scheme-specified tables.

The location of the input table. This may take one of the following forms:

  • A filename.
  • A URL.
  • The special value "-", meaning standard input. In this case the input format must be given explicitly using the ifmt parameter. Note that not all formats can be streamed in this way.
  • A scheme specification of the form :<scheme-name>:<scheme-args>.
  • A system command line with either a "<" character at the start, or a "|" character at the end ("<syscmd" or "syscmd|"). This executes the given pipeline and reads from its standard output. This will probably only work on unix-like systems.

In any case, compressed data in one of the supported compression formats (gzip, Unix compress or bzip2) will be decompressed transparently.

Specifies processing to be performed on the input table as specified by parameter in, before any other processing has taken place. The value of this parameter is one or more of the filter commands described in SUN/256. If more than one is given, they must be separated by semicolon characters (";"). This parameter can be repeated multiple times on the same command line to build up a list of processing steps. The sequence of commands given in this way defines the processing pipeline which is performed on the table.

Commands may alternatively be supplied in an external file, by using the indirection character '@'. Thus a value of "@filename" causes the file filename to be read for a list of filter commands to execute. The commands in the file may be separated by newline characters and/or semicolons, and lines which are blank or which start with a '#' character are ignored. A backslash character '\fR' at the end of a line joins it with the following line.

Gives the bounds for each dimension of the cube in data coordinates. The form of the value is a space-separated list of words, each giving an optional lower bound, then a colon, then an optional upper bound, for instance "1:100 0:20" to represent a range for two-dimensional output between 1 and 100 of the first coordinate (table column) and between 0 and 20 for the second. Either or both numbers may be omitted to indicate that the bounds should be determined automatically by assessing the range of the data in the table. A null value for the parameter indicates that all bounds should be determined automatically for all the dimensions.

If any of the bounds need to be determined automatically in this way, two passes through the data will be required, the first to determine bounds and the second to populate the cube.

Gives the extent of of the data bins (cube pixels) in each dimension in data coordinates. The form of the value is a space-separated list of values, giving a list of extents for the first, second, ... dimension. Either this parameter or the nbins parameter must be supplied.

Gives the number of bins (cube pixels) in each dimension. The form of the value is a space-separated list of integers, giving the number of pixels for the output cube in the first, second, ... dimension. Either this parameter or the binsizes parameter must be supplied.

Defines how values contributing to the same density map bin are combined together to produce the value assigned to that bin. Possible values are:

  • sum: the sum of all the combined values per bin
  • sum-per-unit: the sum of all the combined values per unit of bin size
  • count: the number of non-blank values per bin (weight is ignored)
  • count-per-unit: the number of non-blank values per unit of bin size (weight is ignored)
  • mean: the mean of the combined values
  • median: the median
  • Q1: first quartile
  • Q3: third quartile
  • min: the minimum of all the combined values
  • max: the maximum of all the combined values
  • stdev: the sample standard deviation of the combined values
  • stdev_pop: the population standard deviation of the combined values
  • hit: 1 if any values present, NaN otherwise (weight is ignored)
  • Q.nnn: quantile nnn (e.g. Q.05 is the fifth percentile)

The location of the output file. This is usually a filename to write to. If it is equal to the special value "-" the output will be written to standard output.

The output cube is currently written as a single-HDU FITS file.

The type of numeric value which will fill the output array. If no selection is made, the output type will be determined automatically as the shortest type required to hold all the values in the array. Currently, integers are always signed (no BSCALE/BZERO), so for instance the largest value that can be recorded in 8 bits is 127.

Optionally gives a weight for each entry contributing to histogram bins. The value of this expression is accumulated, in accordance with the combine parameter, into the bin defined by its coordinates. If no expression is given, the value 1 is assumed.

SEE ALSO

stilts(1)

If the package stilts-doc is installed, the full documentation SUN/256 is available in HTML format:
file:///usr/share/doc/stilts/sun256/index.html

VERSION

STILTS version 3.5.1-debian

This is the Debian version of Stilts, which lack the support of some file formats and network protocols. For differences see
file:///usr/share/doc/stilts/README.Debian

AUTHOR

Mark Taylor (Bristol University)

Mar 2017