NAME¶

faltest - Falcon unit test interface.

SYNOPSIS¶

faltest [-d unit_test_dir] [options] [unit_list] module_file.fam

DESCRIPTION¶

The faltest command line tool is a powerful interface to the Falcon unit testing system. The basic working principle of faltest is that of taking all the .fal script files contained in a directory, compile and execute them, eventually keeping track of errors, elapsed times and execution failures. After running all the scripts, faltest may print a report on what happened if requested to do so. A list of one or more unit test may be indicated in the faltest command line after the options. Also, the executed tests can be limited to named subsets. The unit test directory is added to the module load path, so load directives will be resolved searching the required scripts in the test path.

UNIT TEST SCRIPTS¶

Scripts being part of unit test have to start with a common header indicating some information about them. The header is actually a formatted Falcon comment which is read by the faltest utility. This is a typical header:

/**********************************************
* Falcon test suite
*
* ID: 10a
* Category: cat-name
* Subcategory: subcat-name
* Short: The short name
* Description:
*   Long description of this test
*   Spanning on many lines...
* [/Description]
**********************************************/

The header has a free form; faltest recognizes the following fields, being inside a comment and eventually preceded by a "*". ID: Category: Subcategory: Short: Description:

THE TESTSUITE MODULE¶

Falcon system provides a loadable module called "testsuite". The module is actually embedded in faltest , and is provided to all the scripts it runs. The module provides the following functions: success() failure( reason ) testReflect( item ) alive( percent ) timings( total_time, performed_ops ) timeFactor()

OPTIONS¶

-c <cat>

Select this category and ignore the rest.

 

-C <subcat>

Select this subcategory and ignore the rest.

 

-d

Directory from where to load the unit tests. If not specified, it will be the current directory.

 

-f <n>

Set time factor to N. Some scripts may use this information to perform more loops or lengthy operations.

 

-h

Show version and a short help.

 

-l

List the selected tests and exit. Combine with -v to have the path of the tests, as -l only lists the script ID, its short name and the category/subcategory pair.

 

-m

Do not compile in memory. Use disk temporary files.

 

-M

Checks for memory leaks. This option makes faltest to report unclaimed memory after each script execution, and a final report at the end of the tests. The check is extended to all the engine operations, so errors in the engine are detected too.

 

-o <file>

Write final report to the given output file.

 

-s

Perform module serialization test. Other than compiling the file, the module is also saved and then restored before being executed. This allows one to check for errors in module serialization (that is, loading of .fam files). The operation is performed in memory, unless the option -m is also specified.

 

-S

Compile via assembly. This test the correct behavior of the assembler generator and compiler instead of the binary module generator.

 

-t

Records and display timings. The statistics of compilation, linking and execution overall times are recorded and written as part of the report.

 

-T

Records timings() calls from scripts. This allows the scripts to declare their own performance ratings, and collects the results in the final report.

 

-v

Be verbose. Normally, execution and failures are sparsely reported. This is because the normal execution mode is meant for automated runs. Tests can be executed by automated utilities and errors can be reported to system administrator by simple checks on the output data.

 

A developer willing to fix a broken test must run that test alone with the -v enabled. A more complete error report (including compilation or execution errors, if they were the cause for the failure) will be then visualized. The -v options also allows one to see the path of the original script, which is otherwise hidden (masked by the testsuite ID).

 

-V

Prints version number and exits.

 

SAMPLE¶

This is a simple and complete example from the Falcon benchmark suite.

/*******************************************
* Falcon direct benchmarks
*
* ID: 2a
* Category: benchmark
* Subcategory: calls
* Short: Benchmark on function calls
* Description:
*    Performing repeated function calls and returns.
*    This test calls a function without parameters.
*
* [/Description]
******************************************/

loops = 1000000 * timeFactor()
each = int(loops/10)

function toBeCalled()
end

// getting time
time = seconds()
for i = 1 to loops

   // perform the call
   toBeCalled()

   if i % each == 0
      alive( i/loops*100 )
   end
end
// taking end time
time = seconds() - time

// subtract alive time
timeAlive = seconds()
for i = 1 to loops
   if i % each == 0
      alive( i/loops*100 )
   end
end
timeAlive = seconds() - timeAlive
time -= timeAlive

timings( time, loops )

/* end of file */

FILES¶

/usr/lib/libfalcon_engine.so

AUTHOR¶

Giancarlo Niccolai <gc@falconpl.org>

SEE ALSO¶

falcon(1)

LICENSE¶

This document is released under the "GNU Free Documentation License, version 1.2". On Debian systems, the complete text of the Free Documentation License, version 1.2, can be found in /usr/share/common-licenses/.