.\" Man page generated from reStructuredText. . .TH VTC 7 "" "" "" .SH NAME VTC \- Varnish Test Case Syntax . .nr rst2man-indent-level 0 . .de1 rstReportMargin \\$1 \\n[an-margin] level \\n[rst2man-indent-level] level margin: \\n[rst2man-indent\\n[rst2man-indent-level]] - \\n[rst2man-indent0] \\n[rst2man-indent1] \\n[rst2man-indent2] .. .de1 INDENT .\" .rstReportMargin pre: . RS \\$1 . nr rst2man-indent\\n[rst2man-indent-level] \\n[an-margin] . nr rst2man-indent-level +1 .\" .rstReportMargin post: .. .de UNINDENT . RE .\" indent \\n[an-margin] .\" old: \\n[rst2man-indent\\n[rst2man-indent-level]] .nr rst2man-indent-level -1 .\" new: \\n[rst2man-indent\\n[rst2man-indent-level]] .in \\n[rst2man-indent\\n[rst2man-indent-level]]u .. .\" Copyright (c) 2016-2017 Varnish Software AS .\" SPDX-License-Identifier: BSD-2-Clause .\" See LICENSE file for full text of license . .SH OVERVIEW .sp This document describes the syntax used by Varnish Test Cases files (.vtc). A vtc file describe a scenario with different scripted HTTP\-talking entities, and generally one or more Varnish instances to test. .SH PARSING .sp A vtc file will be read word after word, with very little tokenization, meaning a syntax error won\(aqt be detected until the test actually reach the relevant action in the test. .sp A parsing error will most of the time result in an assert being triggered. If this happens, please refer yourself to the related source file and line number. However, this guide should help you avoid the most common mistakes. .SS Words and strings .sp The parser splits words by detecting whitespace characters and a string is a word, or a series of words on the same line enclosed by double\-quotes ("..."), or, for multi\-line strings, enclosed in curly brackets ({...}). .SS Comments .sp The leading whitespaces of lines are ignored. Empty lines (or ones consisting only of whitespaces) are ignored too, as are the lines starting with "#" that are comments. .SS Lines and commands .sp Test files take at most one command per line, with the first word of the line being the command and the following ones being its arguments. To continue over to a new line without breaking the argument string, you can escape the newline character (\en) with a backslash (\e). .SH MACROS .sp When a string is processed, macro expansion is performed. Macros are in the form \fB${[,...]}\fP, they have a name followed by an optional comma\- or space\-separated list of arguments. Leading and trailing spaces are ignored. .sp The macros \fB${foo,bar,baz}\fP and \fB${ foo bar baz }\fP are equivalent. If an argument contains a space or a comma, arguments can be quoted. For example the macro \fB${foo,"bar,baz"}\fP gives one argument \fBbar,baz\fP to the macro called \fBfoo\fP\&. .sp Unless documented otherwise, all macros are simple macros that don\(aqt take arguments. .SS Built\-in macros .INDENT 0.0 .TP .B \fB${bad_backend}\fP A socket address that will reliably never accept connections. .TP .B \fB${bad_ip}\fP An unlikely IPv4 address. .TP .B \fB${date}\fP The current date and time formatted for HTTP. .TP .B \fB${listen_addr}\fP The default listen address various components use, by default a random port on localhost. .TP .B \fB${localhost}\fP The first IP address that resolves to "localhost". .TP .B \fB${pwd}\fP The working directory from which \fBvarnishtest\fP was executed. .TP .B \fB${string,[,...]}\fP The \fBstring\fP macro is the entry point for text generation, it takes a specialized action with each its own set of arguments. .TP .B \fB${string,repeat,,}\fP Repeat \fBuint\fP times the string \fBstr\fP\&. .TP .B \fB${testdir}\fP The directory containing the VTC script of the ongoing test case execution. .TP .B \fB${tmpdir}\fP The dedicated working directory for the ongoing test case execution, which happens to also be the current working directory. Useful when an absolute path to the working directory is needed. .TP .B \fB${topbuild}\fP Only present when the \fB\-i\fP option is used, to work on Varnish itself instead of a regular installation. .UNINDENT .SH SYNTAX .SS barrier .sp NOTE: This command is available everywhere commands are given. .sp Barriers allows you to synchronize different threads to make sure events occur in the right order. It\(aqs even possible to use them in VCL. .sp First, it\(aqs necessary to declare the barrier: .INDENT 0.0 .INDENT 3.5 .sp .nf .ft C barrier bNAME TYPE NUMBER [\-cyclic] .ft P .fi .UNINDENT .UNINDENT .sp With the arguments being: .INDENT 0.0 .TP .B bNAME this is the name of the barrier, used to identify it when you\(aqll create sync points. It must start with \(aqb\(aq. .TP .B TYPE it can be "cond" (mutex) or "sock" (socket) and sets internal behavior. If you don\(aqt need VCL synchronization, use cond. .TP .B NUMBER number of sync point needed to go through the barrier. .TP .B \-cyclic if present, the barrier will reset itself and be ready for another round once gotten through. .UNINDENT .sp Then, to add a sync point: .INDENT 0.0 .INDENT 3.5 .sp .nf .ft C barrier bNAME sync .ft P .fi .UNINDENT .UNINDENT .sp This will block the parent thread until the number of sync points for bNAME reaches the NUMBER given in the barrier declaration. .sp If you wish to synchronize the VCL, you need to declare a "sock" barrier. This will emit a macro definition named "bNAME_sock" that you can use in VCL (after importing the vtc vmod): .INDENT 0.0 .INDENT 3.5 .sp .nf .ft C vtc.barrier_sync("${bNAME_sock}"); .ft P .fi .UNINDENT .UNINDENT .sp This function returns 0 if everything went well and is the equivalent of \fBbarrier bNAME sync\fP at the VTC top\-level. .SS client/server .sp Client and server threads are fake HTTP entities used to test your Varnish and VCL. They take any number of arguments, and the one that are not recognized, assuming they don\(aqt start with \(aq\-\(aq, are treated as specifications, laying out the actions to undertake: .INDENT 0.0 .INDENT 3.5 .sp .nf .ft C client cNAME [...] server sNAME [...] .ft P .fi .UNINDENT .UNINDENT .sp Clients and server are identified by a string that\(aqs the first argument, clients\(aq names start with \(aqc\(aq and servers\(aq names start with \(aqs\(aq. .sp As the client and server commands share a good deal of arguments and specification actions, they are grouped in this single section, specific items will be explicitly marked as such. .sp SECTION: client\-server.macros Macros and automatic behaviour .sp To make things easier in the general case, clients will connect by default to a Varnish server called v1. To connect to a different Varnish server, use \(aq\-connect ${vNAME_sock}\(aq. .sp The \-vcl+backend switch of the \fBvarnish\fP command will add all the declared servers as backends. Be careful though, servers will by default listen to the 127.0.0.1 IP and will pick a random port, and publish 3 macros: sNAME_addr, sNAME_port and sNAME_sock, but only once they are started. For \(aqvarnish \-vcl+backend\(aq to create the vcl with the correct values, the server must be started first. .sp SECTION: client\-server.args Arguments .INDENT 0.0 .TP .B \-start Start the thread in background, processing the last given specification. .TP .B \-wait Block until the thread finishes. .TP .B \-run (client only) Equivalent to "\-start \-wait". .TP .B \-repeat NUMBER Instead of processing the specification only once, do it NUMBER times. .TP .B \-keepalive For repeat, do not open new connections but rather run all iterations in the same connection .TP .B \-break (server only) Stop the server. .TP .B \-listen STRING (server only) Dictate the listening socket for the server. STRING is of the form "IP PORT", or "/PATH/TO/SOCKET" for a Unix domain socket. In the latter case, the path must begin with \(aq/\(aq, and the server must be able to create it. .TP .B \-connect STRING (client only) Indicate the server to connect to. STRING is also of the form "IP PORT", or "/PATH/TO/SOCKET". As with "server \-listen", a Unix domain socket is recognized when STRING begins with a \(aq/\(aq. .TP .B \-dispatch (server only, s0 only) Normally, to keep things simple, server threads only handle one connection at a time, but the \-dispatch switch allows to accept any number of connection and handle them following the given spec. .sp However, \-dispatch is only allowed for the server name "s0". .TP .B \-proxy1 STRING (client only) Use the PROXY protocol version 1 for this connection. STRING is of the form "CLIENTIP:PORT SERVERIP:PORT". .TP .B \-proxy2 STRING (client only) Use the PROXY protocol version 2 for this connection. STRING is of the form "CLIENTIP:PORT SERVERIP:PORT". .UNINDENT .sp SECTION: client\-server.spec Specification .sp It\(aqs a string, either double\-quoted "like this", but most of the time enclosed in curly brackets, allowing multilining. Write a command per line in it, empty line are ignored, and long line can be wrapped by using a backslash. For example: .INDENT 0.0 .INDENT 3.5 .sp .nf .ft C client c1 { txreq \-url /foo \e \-hdr "bar: baz" rxresp } \-run .ft P .fi .UNINDENT .UNINDENT .INDENT 0.0 .TP .B accept (server only) Close the current connection, if any, and accept a new one. Note that this new connection is HTTP/1.x. .UNINDENT .INDENT 0.0 .TP .B chunked STRING Send STRING as chunked encoding. .UNINDENT .INDENT 0.0 .TP .B chunkedlen NUMBER Do as \fBchunked\fP except that the string will be generated for you, with a length of NUMBER characters. .UNINDENT .INDENT 0.0 .TP .B close (server only) Close the connection. Note that if operating in HTTP/2 mode no extra (GOAWAY) frame is sent, it\(aqs simply a TCP close. .UNINDENT .INDENT 0.0 .TP .B expect STRING1 OP STRING2 Test if "STRING1 OP STRING2" is true, and if not, fails the test. OP can be ==, <, <=, >, >= when STRING1 and STRING2 represent numbers in which case it\(aqs an order operator. If STRING1 and STRING2 are meant as strings OP is a matching operator, either == (exact match) or ~ (regex match). .sp varnishtest will first try to resolve STRING1 and STRING2 by looking if they have special meanings, in which case, the resolved value is use for the test. Note that this value can be a string representing a number, allowing for tests such as: .INDENT 7.0 .INDENT 3.5 .sp .nf .ft C expect req.http.x\-num > 2 .ft P .fi .UNINDENT .UNINDENT .sp Here\(aqs the list of recognized strings, most should be obvious as they either match VCL logic, or the txreq/txresp options: .INDENT 7.0 .IP \(bu 2 remote.ip .IP \(bu 2 remote.port .IP \(bu 2 remote.path .IP \(bu 2 req.method .IP \(bu 2 req.url .IP \(bu 2 req.proto .IP \(bu 2 resp.proto .IP \(bu 2 resp.status .IP \(bu 2 resp.reason .IP \(bu 2 resp.chunklen .IP \(bu 2 req.bodylen .IP \(bu 2 req.body .IP \(bu 2 resp.bodylen .IP \(bu 2 resp.body .IP \(bu 2 req.http.NAME .IP \(bu 2 resp.http.NAME .UNINDENT .UNINDENT .INDENT 0.0 .TP .B expect_close Reads from the connection, expecting nothing to read but an EOF. .UNINDENT .INDENT 0.0 .TP .B fatal|non_fatal Control whether a failure of this entity should stop the test. .UNINDENT .INDENT 0.0 .TP .B gunzip Gunzip the body in place. .UNINDENT .INDENT 0.0 .TP .B recv NUMBER Read NUMBER bytes from the connection. .UNINDENT .INDENT 0.0 .TP .B rxchunk Receive an HTTP chunk. .UNINDENT .INDENT 0.0 .TP .B rxpri (server only) Receive a preface. If valid set the server to HTTP/2, abort otherwise. .UNINDENT .INDENT 0.0 .TP .B rxreq (server only) Receive and parse a request\(aqs headers and body. .UNINDENT .INDENT 0.0 .TP .B rxreqbody (server only) Receive a request\(aqs body. .UNINDENT .INDENT 0.0 .TP .B rxreqhdrs (server only) Receive and parse a request\(aqs headers (but not the body). .UNINDENT .INDENT 0.0 .TP .B rxresp [\-no_obj] (client only) Receive and parse a response\(aqs headers and body. If \-no_obj is present, only get the headers. .UNINDENT .INDENT 0.0 .TP .B rxrespbody (client only) Receive (part of) a response\(aqs body. .UNINDENT .sp \-max : max length of this receive, 0 for all .INDENT 0.0 .TP .B rxresphdrs (client only) Receive and parse a response\(aqs headers. .UNINDENT .INDENT 0.0 .TP .B send STRING Push STRING on the connection. .UNINDENT .INDENT 0.0 .TP .B send_n NUMBER STRING Write STRING on the socket NUMBER times. .UNINDENT .INDENT 0.0 .TP .B send_urgent STRING Send string as TCP OOB urgent data. You will never need this. .UNINDENT .INDENT 0.0 .TP .B sendhex STRING Send bytes as described by STRING. STRING should consist of hex pairs possibly separated by whitespace or newlines. For example: "0F EE a5 3df2". .UNINDENT .INDENT 0.0 .TP .B settings \-dectbl INT Force internal HTTP/2 settings to certain values. Currently only support setting the decoding table size. .UNINDENT .INDENT 0.0 .TP .B shell Same as for the top\-level shell. .UNINDENT .INDENT 0.0 .TP .B stream HTTP/2 introduces the concept of streams, and these come with their own specification, and as it\(aqs quite big, have been moved to their own chapter. .UNINDENT .sp SECTION: stream stream .sp (note: this section is at the top\-level for easier navigation, but it\(aqs part of the client/server specification) .sp Streams map roughly to a request in HTTP/2, a request is sent on stream N, the response too, then the stream is discarded. The main exception is the first stream, 0, that serves as coordinator. .sp Stream syntax follow the client/server one: .INDENT 0.0 .INDENT 3.5 .sp .nf .ft C stream ID [SPEC] [ACTION] .ft P .fi .UNINDENT .UNINDENT .sp ID is the HTTP/2 stream number, while SPEC describes what will be done in that stream. .sp Note that, when parsing a stream action, if the entity isn\(aqt operating in HTTP/2 mode, these spec is ran before: .INDENT 0.0 .INDENT 3.5 .sp .nf .ft C txpri/rxpri # client/server stream 0 { txsettings rxsettings txsettings \-ack rxsettings expect settings.ack == true } \-run .ft P .fi .UNINDENT .UNINDENT .sp And HTTP/2 mode is then activated before parsing the specification. .sp SECTION: stream.actions Actions .INDENT 0.0 .TP .B \-start Run the specification in a thread, giving back control immediately. .TP .B \-wait Wait for the started thread to finish running the spec. .TP .B \-run equivalent to calling \fB\-start\fP then \fB\-wait\fP\&. .UNINDENT .INDENT 0.0 .TP .B timeout NUMBER Set the TCP timeout for this entity. .UNINDENT .INDENT 0.0 .TP .B txpri (client only) Send an HTTP/2 preface ("PRI * HTTP/2.0\er\en\er\enSM\er\en\er\en") and set client to HTTP/2. .UNINDENT .INDENT 0.0 .TP .B txreq|txresp [...] Send a minimal request or response, but overload it if necessary. .sp txreq is client\-specific and txresp is server\-specific. .sp The only thing different between a request and a response, apart from who can send them is that the first line (request line vs status line), so all the options are prety much the same. .INDENT 7.0 .TP .B \-method STRING (txreq only) What method to use (default: "GET"). .TP .B \-req STRING (txreq only) Alias for \-method. .TP .B \-url STRING (txreq only) What location to use (default "/"). .TP .B \-proto STRING What protocol use in the status line. (default: "HTTP/1.1"). .TP .B \-status NUMBER (txresp only) What status code to return (default 200). .TP .B \-reason STRING (txresp only) What message to put in the status line (default: "OK"). .UNINDENT .sp These three switches can appear in any order but must come before the following ones. .INDENT 7.0 .TP .B \-nohost Don\(aqt include a Host header in the request. .TP .B \-nolen Don\(aqt include a Content\-Length header. .TP .B \-hdr STRING Add STRING as a header, it must follow this format: "name: value". It can be called multiple times. .TP .B \-hdrlen STRING NUMBER Add STRING as a header with NUMBER bytes of content. .UNINDENT .sp You can then use the arguments related to the body: .INDENT 7.0 .TP .B \-body STRING Input STRING as body. .TP .B \-bodyfrom FILE Same as \-body but content is read from FILE. .TP .B \-bodylen NUMBER Generate and input a body that is NUMBER bytes\-long. .TP .B \-gziplevel NUMBER Set the gzip level (call it before any of the other gzip switches). .TP .B \-gzipresidual NUMBER Add extra gzip bits. You should never need it. .TP .B \-gzipbody STRING Gzip STRING and send it as body. .TP .B \-gziplen NUMBER Combine \-bodylen and \-gzipbody: generate a string of length NUMBER, gzip it and send as body. .UNINDENT .UNINDENT .INDENT 0.0 .TP .B write_body STRING Write the body of a request or a response to a file. By using the shell command, higher\-level checks on the body can be performed (eg. XML, JSON, ...) provided that such checks can be delegated to an external program. .UNINDENT .SS delay .sp NOTE: This command is available everywhere commands are given. .sp Sleep for the number of seconds specified in the argument. The number can include a fractional part, e.g. 1.5. .SS feature .sp Test that the required feature(s) for a test are available, and skip the test otherwise; or change the interpretation of the test, as documented below. feature takes any number of arguments from this list: .INDENT 0.0 .TP .B 64bit The environment is 64 bits .TP .B ipv4 127.0.0.1 works .TP .B ipv6 [::1] works .TP .B dns DNS lookups are working .TP .B topbuild The test has been started with \(aq\-i\(aq .TP .B root The test has been invoked by the root user .TP .B user_varnish The varnish user is present .TP .B user_vcache The vcache user is present .TP .B group_varnish The varnish group is present .TP .B cmd A command line that should execute with a zero exit status .TP .B ignore_unknown_macro Do not fail the test if a string of the form ${...} is not recognized as a macro. .TP .B persistent_storage Varnish was built with the deprecated persistent storage. .TP .B coverage Varnish was built with code coverage enabled. .TP .B asan Varnish was built with the address sanitizer. .TP .B msan Varnish was built with the memory sanitizer. .TP .B tsan Varnish was built with the thread sanitizer. .TP .B ubsan Varnish was built with the undefined behavior sanitizer. .TP .B sanitizer Varnish was built with a sanitizer. .TP .B workspace_emulator Varnish was built with its workspace emulator. .UNINDENT .sp A feature name can be prefixed with an exclamation mark (!) to skip a test if the feature is present. .sp Be careful with ignore_unknown_macro, because it may cause a test with a misspelled macro to fail silently. You should only need it if you must run a test with strings of the form "${...}". .SS haproxy .sp Define and interact with haproxy instances. .sp To define a haproxy server, you\(aqll use this syntax: .INDENT 0.0 .INDENT 3.5 .sp .nf .ft C haproxy hNAME \-conf\-OK CONFIG haproxy hNAME \-conf\-BAD ERROR CONFIG haproxy hNAME [\-D] [\-W] [\-arg STRING] [\-conf[+vcl] STRING] .ft P .fi .UNINDENT .UNINDENT .sp The first \fBhaproxy hNAME\fP invocation will start the haproxy master process in the background, waiting for the \fB\-start\fP switch to actually start the child. .sp Arguments: .INDENT 0.0 .TP .B hNAME Identify the HAProxy server with a string, it must starts with \(aqh\(aq. .TP .B \-conf\-OK CONFIG .INDENT 7.0 .TP .B Run haproxy in \(aq\-c\(aq mode to check config is OK stdout/stderr should contain \(aqConfiguration file is valid\(aq The exit code should be 0. .UNINDENT .TP .B \-conf\-BAD ERROR CONFIG .INDENT 7.0 .TP .B Run haproxy in \(aq\-c\(aq mode to check config is BAD. "ERROR" should be part of the diagnostics on stdout/stderr. The exit code should be 1. .UNINDENT .TP .B \-D Run HAproxy in daemon mode. If not given \(aq\-d\(aq mode used. .TP .B \-W Enable HAproxy in Worker mode. .TP .B \-S Enable HAproxy Master CLI in Worker mode .TP .B \-arg STRING Pass an argument to haproxy, for example "\-h simple_list". .TP .B \-cli STRING Specify the spec to be run by the command line interface (CLI). .TP .B \-mcli STRING Specify the spec to be run by the command line interface (CLI) of the Master process. .TP .B \-conf STRING Specify the configuration to be loaded by this HAProxy instance. .TP .B \-conf+backend STRING .INDENT 7.0 .TP .B Specify the configuration to be loaded by this HAProxy instance, all server instances will be automatically appended .UNINDENT .TP .B \-start Start this HAProxy instance. .TP .B \-wait Stop this HAProxy instance. .TP .B \-expectexit NUMBER Expect haproxy to exit(3) with this value .UNINDENT .SS logexpect .sp Reads the VSL and looks for records matching a given specification. It will process records trying to match the first pattern, and when done, will continue processing, trying to match the following pattern. If a pattern isn\(aqt matched, the test will fail. .sp logexpect threads are declared this way: .INDENT 0.0 .INDENT 3.5 .sp .nf .ft C logexpect lNAME \-v [\-g ] [\-d 0|1] [\-q query] \e [vsl arguments] { expect expect fail add fail clear abort ... } [\-start|\-wait] .ft P .fi .UNINDENT .UNINDENT .sp And once declared, you can start them, or wait on them: .INDENT 0.0 .INDENT 3.5 .sp .nf .ft C logexpect lNAME <\-start|\-wait> .ft P .fi .UNINDENT .UNINDENT .sp With: .INDENT 0.0 .TP .B lNAME Name the logexpect thread, it must start with \(aql\(aq. .TP .B \-v id Specify the varnish instance to use (most of the time, id=v1). .TP .B \-g Start processing log records at the head of the log instead of the tail. .TP .B \-q query Filter records using a query expression, see \fBman vsl\-query\fP for more information. Multiple \-q options are not supported. .TP .B \-m Also emit log records for misses (only for debugging) .TP .B \-err Invert the meaning of success. Usually called once to expect the logexpect to fail .TP .B \-start Start the logexpect thread in the background. .TP .B \-wait Wait for the logexpect thread to finish .UNINDENT .sp VSL arguments (similar to the varnishlog options): .INDENT 0.0 .TP .B \-C Use caseless regex .TP .B \-i Include tags .TP .B \-I <[taglist:]regex> Include by regex .TP .B \-T Transaction end timeout .UNINDENT .sp expect specification: .INDENT 0.0 .TP .B skip: [uint|*|?] Max number of record to skip .TP .B vxid: [uint|*|=] vxid to match .TP .B tag: [tagname|*|=] Tag to match against .TP .B regex: regular expression to match against (optional) .UNINDENT .sp For skip, vxid and tag, \(aq*\(aq matches anything, \(aq=\(aq expects the value of the previous matched record. The \(aq?\(aq marker is equivalent to zero, expecting a match on the next record. The difference is that \(aq?\(aq can be used when the order of individual consecutive logs is not deterministic. In other words, lines from a block of alternatives marked by \(aq?\(aq can be matched in any order, but all need to match eventually. .sp fail specification: .sp add: Add to the fail list .INDENT 0.0 .INDENT 3.5 Arguments are equivalent to expect, except for skip missing .UNINDENT .UNINDENT .sp clear: Clear the fail list .sp Any number of fail specifications can be active during execution of a logexpect. All active fail specifications are matched against every log line and, if any match, the logexpect fails immediately. .sp For a logexpect to end successfully, there must be no specs on the fail list, so logexpects should always end with .INDENT 0.0 .INDENT 3.5 expect fail clear .UNINDENT .UNINDENT .\" XXX can we come up with a better solution which is still safe? . .sp abort specification: .sp abort(3) varnishtest, intended to help debugging of the VSL client library itself. .SS process .sp Run a process with stdin+stdout on a pseudo\-terminal and stderr on a pipe. .sp Output from the pseudo\-terminal is copied verbatim to ${pNAME_out}, and the \-log/\-dump/\-hexdump flags will also put it in the vtc\-log. .sp The pseudo\-terminal is not in ECHO mode, but if the programs run set it to ECHO mode ("stty sane") any input sent to the process will also appear in this stream because of the ECHO. .sp Output from the stderr\-pipe is copied verbatim to ${pNAME_err}, and is always included in the vtc_log. .INDENT 0.0 .INDENT 3.5 .INDENT 0.0 .TP .B process pNAME SPEC [\-log] [\-dump] [\-hexdump] [\-expect\-exit N] [\-start] [\-run] [\-write STRING] [\-writeln STRING] [\-kill STRING] [\-stop] [\-wait] [\-close] .UNINDENT .UNINDENT .UNINDENT .INDENT 0.0 .TP .B pNAME Name of the process. It must start with \(aqp\(aq. .TP .B SPEC The command(s) to run in this process. .TP .B \-hexdump Log output with vtc_hexdump(). Must be before \-start/\-run. .TP .B \-dump Log output with vtc_dump(). Must be before \-start/\-run. .TP .B \-log Log output with VLU/vtc_log(). Must be before \-start/\-run. .TP .B \-start Start the process. .TP .B \-expect\-exit N Expect exit status N .TP .B \-wait Wait for the process to finish. .TP .B \-run Shorthand for \-start \-wait. .sp In most cases, if you just want to start a process and wait for it to finish, you can use the \fBshell\fP command instead. The following commands are equivalent: .INDENT 7.0 .INDENT 3.5 .sp .nf .ft C shell "do \-\-something" process p1 "do \-\-something" \-run .ft P .fi .UNINDENT .UNINDENT .sp However, you may use the the \fBprocess\fP variant to conveniently collect the standard input and output without dealing with shell redirections yourself. The \fBshell\fP command can also expect an expression from either output, consider using it if you only need to match one. .TP .B \-kill STRING Send a signal to the process. The argument can be either the string "TERM", "INT", or "KILL" for SIGTERM, SIGINT or SIGKILL signals, respectively, or a hyphen (\-) followed by the signal number. .sp If you need to use other signal names, you can use the \fBkill\fP(1) command directly: .INDENT 7.0 .INDENT 3.5 .sp .nf .ft C shell "kill \-USR1 ${pNAME_pid}" .ft P .fi .UNINDENT .UNINDENT .sp Note that SIGHUP usage is discouraged in test cases. .TP .B \-stop Shorthand for \-kill TERM. .TP .B \-winsz LIN COL Change the terminal window size to LIN lines and COL columns. .TP .B \-write STRING Write a string to the process\(aq stdin. .TP .B \-writeln STRING Same as \-write followed by a newline (\en). .TP .B \-writehex HEXSTRING Same as \-write but interpreted as hexadecimal bytes. .TP .B \-need\-bytes [+]NUMBER Wait until at least NUMBER bytes have been received in total. If \(aq+\(aq is prefixed, NUMBER new bytes must be received. .TP .B \-expect\-text LIN COL PAT Wait for PAT to appear at LIN,COL on the virtual screen. Lines and columns are numbered 1...N LIN==0 means "on any line" COL==0 means "anywhere on the line" .TP .B \-close Alias for "\-kill HUP" .TP .B \-screen_dump Dump the virtual screen into vtc_log .UNINDENT .SS setenv .sp Set or change an environment variable: .INDENT 0.0 .INDENT 3.5 .sp .nf .ft C setenv FOO "bar baz" .ft P .fi .UNINDENT .UNINDENT .sp The above will set the environment variable $FOO to the value provided. There is also an \fB\-ifunset\fP argument which will only set the value if the the environment variable does not already exist: .INDENT 0.0 .INDENT 3.5 .sp .nf .ft C setenv \-ifunset FOO quux .ft P .fi .UNINDENT .UNINDENT .SS shell .sp NOTE: This command is available everywhere commands are given. .sp Pass the string given as argument to a shell. If you have multiple commands to run, you can use curly brackets to describe a multi\-lines script, eg: .INDENT 0.0 .INDENT 3.5 .sp .nf .ft C shell { echo begin cat /etc/fstab echo end } .ft P .fi .UNINDENT .UNINDENT .sp By default a zero exit code is expected, otherwise the vtc will fail. .sp Notice that the commandstring is prefixed with "exec 2>&1;" to combine stderr and stdout back to the test process. .sp Optional arguments: .INDENT 0.0 .TP .B \-err Expect non\-zero exit code. .TP .B \-exit N Expect exit code N instead of zero. .TP .B \-expect STRING Expect string to be found in stdout+err. .TP .B \-match REGEXP Expect regexp to match the stdout+err output. .UNINDENT .SS Specification .sp The specification of a stream follows the exact same rules as one for a client or a server. .SS txreq, txresp, txcont, txpush .sp These four commands are about sending headers. txreq and txresp will send HEADER frames; txcont will send CONTINUATION frames; txpush PUSH frames. The only difference between txreq and txresp are the default headers set by each of them. .INDENT 0.0 .TP .B \-noadd Do not add default headers. Useful to avoid duplicates when sending default headers using \fB\-hdr\fP, \fB\-idxHdr\fP and \fB\-litIdxHdr\fP\&. .TP .B \-status INT (txresp) Set the :status pseudo\-header. .TP .B \-url STRING (txreq, txpush) Set the :path pseudo\-header. .TP .B \-method STRING (txreq, txpush) Set the :method pseudo\-header. .TP .B \-req STRING (txreq, txpush) Alias for \-method. .TP .B \-scheme STRING (txreq, txpush) Set the :scheme pseudo\-header. .TP .B \-hdr STRING1 STRING2 Insert a header, STRING1 being the name, and STRING2 the value. .TP .B \-idxHdr INT Insert an indexed header, using INT as index. .TP .B \-litIdxHdr inc|not|never INT huf|plain STRING Insert an literal, indexed header. The first argument specify if the header should be added to the table, shouldn\(aqt, or mustn\(aqt be compressed if/when retransmitted. .sp INT is the idex of the header name to use. .sp The third argument informs about the Huffman encoding: yes (huf) or no (plain). .sp The last term is the literal value of the header. .TP .B \-litHdr inc|not|never huf|plain STRING1 huf|plain STRING2 Insert a literal header, with the same first argument as \fB\-litIdxHdr\fP\&. .sp The second and third terms tell what the name of the header is and if it should be Huffman\-encoded, while the last two do the same regarding the value. .TP .B \-body STRING (txreq, txresp) Specify a body, effectively putting STRING into a DATA frame after the HEADER frame is sent. .TP .B \-bodyfrom FILE (txreq, txresp) Same as \fB\-body\fP but content is read from FILE. .TP .B \-bodylen INT (txreq, txresp) Do the same thing as \fB\-body\fP but generate a string of INT length for you. .TP .B \-gzipbody STRING (txreq, txresp) Gzip STRING and send it as body. .TP .B \-gziplen NUMBER (txreq, txresp) Combine \-bodylen and \-gzipbody: generate a string of length NUMBER, gzip it and send as body. .TP .B \-nostrend (txreq, txresp) Don\(aqt set the END_STREAM flag automatically, making the peer expect a body after the headers. .TP .B \-nohdrend Don\(aqt set the END_HEADERS flag automatically, making the peer expect more HEADER frames. .TP .B \-dep INT (txreq, txresp) Tell the peer that this content depends on the stream with the INT id. .TP .B \-ex (txreq, txresp) Make the dependency exclusive (\fB\-dep\fP is still needed). .TP .B \-weight (txreq, txresp) Set the weight for the dependency. .TP .B \-promised INT (txpush) The id of the promised stream. .TP .B \-pad STRING / \-padlen INT (txreq, txresp, txpush) Add string as padding to the frame, either the one you provided with \-pad, or one that is generated for you, of length INT is \-padlen case. .UNINDENT .SS txdata .sp By default, data frames are empty. The receiving end will know the whole body has been delivered thanks to the END_STREAM flag set in the last DATA frame, and txdata automatically set it. .INDENT 0.0 .TP .B \-data STRING Data to be embedded into the frame. .TP .B \-datalen INT Generate and INT\-bytes long string to be sent in the frame. .TP .B \-pad STRING / \-padlen INT Add string as padding to the frame, either the one you provided with \-pad, or one that is generated for you, of length INT is \-padlen case. .TP .B \-nostrend Don\(aqt set the END_STREAM flag, allowing to send more data on this stream. .UNINDENT .SS rxreq, rxresp .sp These are two convenience functions to receive headers and body of an incoming request or response. The only difference is that rxreq can only be by a server, and rxresp by a client. .SS rxhdrs .sp \fBrxhdrs\fP will expect one HEADER frame, then, depending on the arguments, zero or more CONTINUATION frame. .INDENT 0.0 .TP .B \-all Keep waiting for CONTINUATION frames until END_HEADERS flag is seen. .TP .B \-some INT Retrieve INT \- 1 CONTINUATION frames after the HEADER frame. .UNINDENT .SS rxpush .sp This works like \fBrxhdrs\fP, expecting a PUSH frame and then zero or more CONTINUATION frames. .INDENT 0.0 .TP .B \-all Keep waiting for CONTINUATION frames until END_HEADERS flag is seen. .TP .B \-some INT Retrieve INT \- 1 CONTINUATION frames after the PUSH frame. .UNINDENT .SS rxdata .sp Receiving data is done using the \fBrxdata\fP keywords and will retrieve one DATA frame, if you wish to receive more, you can use these two convenience arguments: .INDENT 0.0 .TP .B \-all keep waiting for DATA frame until one sets the END_STREAM flag .TP .B \-some INT retrieve INT DATA frames. .UNINDENT .sp Receive a frame, any frame. .SS sendhex .sp Push bytes directly on the wire. sendhex takes exactly one argument: a string describing the bytes, in hex notation, with possible whitespaces between them. Here\(aqs an example: .INDENT 0.0 .INDENT 3.5 .sp .nf .ft C sendhex "00 00 08 00 0900 8d" .ft P .fi .UNINDENT .UNINDENT .SS rxgoaway .sp Receive a GOAWAY frame. .SS gunzip .sp Same as the \fBgunzip\fP command for HTTP/1. .SS rxping .sp Receive a PING frame. .SS txping .sp Send PING frame. .INDENT 0.0 .TP .B \-data STRING specify the payload of the frame, with STRING being an 8\-char string. .TP .B \-ack set the ACK flag. .UNINDENT .SS rxprio .sp Receive a PRIORITY frame. .SS txprio .sp Send a PRIORITY frame .INDENT 0.0 .TP .B \-stream INT indicate the id of the stream the sender stream depends on. .TP .B \-ex the dependency should be made exclusive (only this streams depends on the parent stream). .TP .B \-weight INT an 8\-bits integer is used to balance priority between streams depending on the same streams. .UNINDENT .SS rxrst .sp Receive a RST_STREAM frame. .SS txrst .sp Send a RST_STREAM frame. By default, txrst will send a 0 error code (NO_ERROR). .INDENT 0.0 .TP .B \-err STRING|INT Sets the error code to be sent. The argument can be an integer or a string describing the error, such as NO_ERROR, or CANCEL (see rfc7540#11.4 for more strings). .UNINDENT .SS rxsettings .sp Receive a SETTINGS frame. .SS txsettings .sp SETTINGS frames must be acknowledge, arguments are as follow (most of them are from rfc7540#6.5.2): .INDENT 0.0 .TP .B \-hdrtbl INT headers table size .TP .B \-push BOOL whether push frames are accepted or not .TP .B \-maxstreams INT maximum concurrent streams allowed .TP .B \-winsize INT sender\(aqs initial window size .TP .B \-framesize INT largest frame size authorized .TP .B \-hdrsize INT maximum size of the header list authorized .TP .B \-ack set the ack bit .UNINDENT .SS rxwinup .sp Receive a WINDOW_UPDATE frame. .SS txwinup .sp Transmit a WINDOW_UPDATE frame, increasing the amount of credit of the connection (from stream 0) or of the stream (any other stream). .INDENT 0.0 .TP .B \-size INT give INT credits to the peer. .UNINDENT .INDENT 0.0 .TP .B write_body STRING Same as the \fBwrite_body\fP command for HTTP/1. .UNINDENT .SS expect .sp expect in stream works as it does in client or server, except that the elements compared will be different. .sp Most of these elements will be frame specific, meaning that the last frame received on that stream must of the correct type. .sp Here the list of keywords you can look at. .SS GOAWAY specific .INDENT 0.0 .TP .B goaway.err The error code (as integer) of the GOAWAY frame. .TP .B goaway.laststream Last\-Stream\-ID .TP .B goaway.debug Debug data, if any. .UNINDENT .SS PING specific .INDENT 0.0 .TP .B ping.data The 8\-bytes string of the PING frame payload. .TP .B ping.ack (PING) "true" if the ACK flag was set, "false" otherwise. .UNINDENT .SS PRIORITY specific .INDENT 0.0 .TP .B prio.stream The stream ID announced. .TP .B prio.exclusive "true" if the priority is exclusive, else "false". .TP .B prio.weight The dependency weight. .UNINDENT .SS PUSH_PROMISE specific .INDENT 0.0 .TP .B push.id The id of the promised stream. .UNINDENT .SS RESET_STREAM specific .INDENT 0.0 .TP .B rst.err The error code (as integer) of the RESET_STREAM frame. .UNINDENT .SS SETTINGS specific .INDENT 0.0 .TP .B settings.ack "true" if the ACK flag was set, else ""false. .TP .B settings.push "true" if the push settings was set to yes, "false" if set to no, and if not present. .TP .B settings.hdrtbl Value of HEADER_TABLE_SIZE if set, otherwise. .TP .B settings.maxstreams Value of MAX_CONCURRENT_STREAMS if set, otherwise. .TP .B settings.winsize Value of INITIAL_WINDOW_SIZE if set, otherwise. .TP .B setting.framesize Value of MAX_FRAME_SIZE if set, otherwise. .TP .B settings.hdrsize Value of MAX_HEADER_LIST_SIZE if set, otherwise. .UNINDENT .SS WINDOW_UPDATE specific .INDENT 0.0 .TP .B winup.size The size of the upgrade given by the WINDOW_UPDATE frame. .UNINDENT .SS Generic frame .INDENT 0.0 .TP .B frame.data Payload of the last frame .TP .B frame.type Type of the frame, as integer. .TP .B frame.size Size of the frame. .TP .B frame.stream Stream of the frame (correspond to the one you are executing this from, obviously). .TP .B frame.padding (for DATA, HEADERS, PUSH_PROMISE frames) Number of padded bytes. .UNINDENT .SS Request and response .sp Note: it\(aqs possible to inspect a request or response while it is still being construct (in\-between two frames for example). .INDENT 0.0 .TP .B req.bodylen / resp.bodylen Length in bytes of the request/response so far. .TP .B req.body / resp.body Body of the request/response so far. .TP .B req.http.STRING / resp.http.STRING Value of the header STRING in the request/response. .TP .B req.status / resp.status :status pseudo\-header\(aqs value. .TP .B req.url / resp.url :path pseudo\-header\(aqs value. .TP .B req.method / resp.method :method pseudo\-header\(aqs value. .TP .B req.authority / resp.authority :method pseudo\-header\(aqs value. .TP .B req.scheme / resp.scheme :method pseudo\-header\(aqs value. .UNINDENT .SS Stream .INDENT 0.0 .TP .B stream.window The current window size of the stream, or, if on stream 0, of the connection. .TP .B stream.weight Weight of the stream .TP .B stream.dependency Id of the stream this one depends on. .UNINDENT .SS Index tables .INDENT 0.0 .TP .B tbl.dec.size / tbl.enc.size Size (bytes) of the decoding/encoding table. .TP .B tbl.dec.size / tbl.enc.maxsize Maximum size (bytes) of the decoding/encoding table. .TP .B tbl.dec.length / tbl.enc.length Number of headers in decoding/encoding table. .TP .B tbl.dec[INT].key / tbl.enc[INT].key Name of the header at index INT of the decoding/encoding table. .TP .B tbl.dec[INT].value / tbl.enc[INT].value Value of the header at index INT of the decoding/encoding table. .UNINDENT .SS syslog .sp Define and interact with syslog instances (for use with haproxy) .sp To define a syslog server, you\(aqll use this syntax: .INDENT 0.0 .INDENT 3.5 .sp .nf .ft C syslog SNAME .ft P .fi .UNINDENT .UNINDENT .sp Arguments: .INDENT 0.0 .TP .B SNAME Identify the syslog server with a string which must start with \(aqS\(aq. .TP .B \-level STRING Set the default syslog priority level used by any subsequent "recv" command. Any syslog dgram with a different level will be skipped by "recv" command. This default level value may be superseded by "recv" command if supplied as first argument: "recv ". .TP .B \-start Start the syslog server thread in the background. .TP .B \-repeat .INDENT 7.0 .TP .B Instead of processing the specification only once, do it NUMBER times. .UNINDENT .TP .B \-bind Bind the syslog socket to a local address. .TP .B \-wait Wait for that thread to terminate. .TP .B \-stop Stop the syslog server thread. .UNINDENT .SS tunnel .sp The goal of a tunnel is to help control the data transfer between two parties, for example to trigger socket timeouts in the middle of protocol frames, without the need to change how both parties are implemented. .sp A tunnel accepts a connection and then connects on behalf of the source to the desired destination. Once both connections are established the tunnel will transfer bytes unchanged between the source and destination. Transfer can be interrupted, usually with the help of synchronization methods like barriers. Once the transfer is paused, it is possible to let a specific amount of bytes move in either direction. .sp SECTION: tunnel.args Arguments .INDENT 0.0 .TP .B \-start Start the tunnel in background, processing the last given specification. .TP .B \-start+pause Start the tunnel, but already paused. .TP .B \-wait Block until the thread finishes. .TP .B \-listen STRING Dictate the listening socket for the server. STRING is of the form "IP PORT", or "HOST PORT". .sp Listens by defaults to a local random port. .TP .B \-connect STRING Indicate the server to connect to. STRING is also of the form "IP PORT", or "HOST PORT". .sp Connects by default to a varnish instance called \fBv1\fP\&. .UNINDENT .sp SECTION: tunnel.spec Specification .sp The specification contains a list of tunnel commands that can be combined with barriers and delays. For example: .INDENT 0.0 .INDENT 3.5 .sp .nf .ft C tunnel t1 { barrier b1 sync pause delay 1 send 42 barrier b2 sync resume } \-start .ft P .fi .UNINDENT .UNINDENT .sp If one end of the tunnel is closed before the end of the specification the test case will fail. A specification that ends in a paused state will implicitely resume the tunnel. .SS varnish .sp Define and interact with varnish instances. .sp To define a Varnish server, you\(aqll use this syntax: .INDENT 0.0 .INDENT 3.5 .sp .nf .ft C varnish vNAME [\-arg STRING] [\-vcl STRING] [\-vcl+backend STRING] [\-errvcl STRING STRING] [\-jail STRING] [\-proto PROXY] .ft P .fi .UNINDENT .UNINDENT .sp The first \fBvarnish vNAME\fP invocation will start the varnishd master process in the background, waiting for the \fB\-start\fP switch to actually start the child. .sp Types used in the description below: .INDENT 0.0 .TP .B PATTERN is a \(aqglob\(aq style pattern (ie: fnmatch(3)) as used in shell filename expansion. .UNINDENT .sp Arguments: .INDENT 0.0 .TP .B vNAME Identify the Varnish server with a string, it must starts with \(aqv\(aq. .TP .B \-arg STRING Pass an argument to varnishd, for example "\-h simple_list". .TP .B \-vcl STRING Specify the VCL to load on this Varnish instance. You\(aqll probably want to use multi\-lines strings for this ({...}). .TP .B \-vcl+backend STRING Do the exact same thing as \-vcl, but adds the definition block of known backends (ie. already defined). .TP .B \-errvcl STRING1 STRING2 Load STRING2 as VCL, expecting it to fail, and Varnish to send an error string matching STRING1 .TP .B \-jail STRING Look at \fBman varnishd\fP (\-j) for more information. .TP .B \-proto PROXY Have Varnish use the proxy protocol. Note that PROXY here is the actual string. .UNINDENT .sp You can decide to start the Varnish instance and/or wait for several events: .INDENT 0.0 .INDENT 3.5 .sp .nf .ft C varnish vNAME [\-start] [\-wait] [\-wait\-running] [\-wait\-stopped] .ft P .fi .UNINDENT .UNINDENT .INDENT 0.0 .TP .B \-start Start the child process. .sp Once successfully started, the following macros are available for the default listen address: \fB${vNAME_addr}\fP, \fB${vNAME_port}\fP and \fB${vNAME_sock}\fP\&. Additional macros are available, including the listen address name for each address vNAME listens to, like for example: \fB${vNAME_a0_addr}\fP\&. .TP .B \-stop Stop the child process. .TP .B \-syntax Set the VCL syntax level for this command (default: 4.1) .TP .B \-wait Wait for that instance to terminate. .TP .B \-wait\-running Wait for the Varnish child process to be started. .TP .B \-wait\-stopped Wait for the Varnish child process to stop. .TP .B \-cleanup Once Varnish is stopped, clean everything after it. This is only used in very few tests and you should never need it. .TP .B \-expectexit NUMBER Expect varnishd to exit(3) with this value .UNINDENT .sp Once Varnish is started, you can talk to it (as you would through \fBvarnishadm\fP) with these additional switches: .INDENT 0.0 .INDENT 3.5 .sp .nf .ft C varnish vNAME [\-cli STRING] [\-cliok STRING] [\-clierr STRING] [\-clijson STRING] .ft P .fi .UNINDENT .UNINDENT .INDENT 0.0 .TP .B \-cli STRING|\-cliok STRING|\-clierr STATUS STRING|\-cliexpect REGEXP STRING All four of these will send STRING to the CLI, the only difference is what they expect the result to be. \-cli doesn\(aqt expect anything, \-cliok expects 200, \-clierr expects STATUS, and \-cliexpect expects the REGEXP to match the returned response. .TP .B \-clijson STRING Send STRING to the CLI, expect success (CLIS_OK/200) and check that the response is parsable JSON. .UNINDENT .sp It is also possible to interact with its shared memory (as you would through tools like \fBvarnishstat\fP) with additional switches: .INDENT 0.0 .TP .B \-expect !PATTERN|PATTERN OP NUMBER|PATTERN OP PATTERN Look into the VSM and make sure the first VSC counter identified by PATTERN has a correct value. OP can be ==, >, >=, <, <=. For example: .INDENT 7.0 .INDENT 3.5 .sp .nf .ft C varnish v1 \-expect SM?.s1.g_space > 1000000 varnish v1 \-expect cache_hit >= cache_hit_grace .ft P .fi .UNINDENT .UNINDENT .sp In the ! form the test fails if a counter matches PATTERN. .sp The \fBMAIN.\fP namespace can be omitted from PATTERN. .sp The test takes up to 5 seconds before timing out. .TP .B \-vsc PATTERN Dump VSC counters matching PATTERN. .TP .B \-vsl_catchup Wait until the logging thread has idled to make sure that all the generated log is flushed .UNINDENT .SS varnishtest .sp Alternate name for \(aqvtest\(aq, see above. .SS vtest .sp This should be the first command in your vtc as it will identify the test case with a short yet descriptive sentence. It takes exactly one argument, a string, eg: .INDENT 0.0 .INDENT 3.5 .sp .nf .ft C vtest "Check that vtest is actually a valid command" .ft P .fi .UNINDENT .UNINDENT .sp It will also print that string in the log. .SH HISTORY .sp This document has been written by Guillaume Quintard. .SH SEE ALSO .INDENT 0.0 .IP \(bu 2 \fIvarnishtest(1)\fP .IP \(bu 2 \fIvmod_vtc(3)\fP .UNINDENT .SH COPYRIGHT .sp This document is licensed under the same licence as Varnish itself. See LICENCE for details. .INDENT 0.0 .IP \(bu 2 Copyright (c) 2006\-2016 Varnish Software AS .UNINDENT .\" Generated by docutils manpage writer. .