NAME¶

logswan — fast Web log analyzer using probabilistic data structures

SYNOPSIS¶

DESCRIPTION¶

logswan is a fast Web log analyzer using probabilistic data structures. It is targeted at very large log files, typically APIs logs. It has constant memory usage regardless of the log file size, and takes approximatively 4MB of RAM.

Unique visitors counting is performed using two HyperLogLog counters (one for IPv4, and another one for IPv6), providing a relative accuracy of 0.10%.

If file is a single dash (`-'), logswan reads from the standard input.

The options are as follows:

-d db

Specify path to a GeoIP database.

-g

Enable GeoIP lookups.

-h

Display usage.

-v

Display version.

EXAMPLES¶

The following script can be used to process all log files in the current directory and save the output in a file:

#!/bin/sh
for file in $(ls *.log)
do
	logswan $file > $file.json
done
exit 0

AUTHORS¶

logswan was written by Frederic Cambus.