table of contents
NFDUMP(1) | General Commands Manual | NFDUMP(1) |
NAME¶
nfdump
— flow
display and analysis program
SYNOPSIS¶
nfdump |
-r flowpath
[-w outfile]
[-f filterfile]
[-C config]
[-R filelist]
[-M dirlist]
[-O order]
[-t timewin]
[-c num]
[-a ] [-A
aggregation] [-b ]
[-B ] [-I ]
[-D nameserver]
[-G geoDB]
[-s statistic]
[-n num]
[-o format]
[-6 ] [-q ]
[-N ] [-i
ident] [-v
flowfile] [-E
flowfile] [-x
flowfile] [-z ]
[-y ] [-j ]
[-J num]
[-X ] [-Z ]
[-T ] [-V ]
[-h ] [filter] |
DESCRIPTION¶
nfdump
reads the flow data from one or
more binary files, created by any nfdump collector nfcapd,
nfpcapd and sfcapd. It processes and lists the
flows in many different output formats and can create a wide range of
statistics.
nfdump
has a very powerful flow filter to
process flows. The filter syntax is very similar to tcpdump, but adapted and
extended for flow filtering. A flow filter may also contain arrays of many
thousand IP addresses etc. to search for specific records.
nfdump
can aggreagte flows according to a
user defined number of elements. This masks certain elements and allows to
sum up flow records matching the same values.
The combination of flow filtering and aggregation as input for any
flow statistics allows complex flow processing. Pre-filtered and aggregated
flow data may also be written back into a binary flow file, which again may
be processed with nfdump
nfdump
can enrich the listing of flows
with geo location information and AS information, unless AS information is
already available in the flow records. IP addresses can be tagged with a two
letter country code, or with a longer location label containing the
geographic region, country and city. The geo location and AS information is
retrieved from the optional geoDB database, created by
the geolookup program from the nfdump tools.
geolookup uses the Maxmind
database GeoDB or GeoLite2 to
create a binary lookup database for nfdump
Please
check the geolooup(1) man page for more details.
The options are as follows:
-r
flowpath- Reads flow records from this path. flowpath may be a single file, or a directory containing any number of flow files or sub directories. All files are processed in the order, as listed by the OS.
-w
outfile- Writes all processed records into outfile instead of
printing. The flowfile is a binary flow file and may
be processed again with
nfdump
This can be useful to limit flows according to a flow filter and/or specific flow aggregation. -f
filterfile- Reads the flow filter from filterfile. This can be useful for very long or structured filters, with comments and long lists. Note: Any filter specified directly on the command line takes precedence over the filterfile.
-C
config- Read more options from file config.
nfdump
tries to read by default %prefix/etc/nfdump.config. This may be overwritten by the environment valiable NFCONF which again may be overwritten by this option-C.
In order to prevent reading any config file, even if it would exist set-C
none. A config file is not required, but may be handy for often used output formats etc. -O
order- Sets an output order for records to be printed as text output. This order
applies after all records processing, such as filtering, and aggregation
and before printing.
flows
- Sort according to the number of flows
packets
- Sort according to (in)packets
ipkg
- Same as packets
opkg
- Sort according to output packets
bytes
- Sort according to (in)bytes
ibyte
- Same as bytes
obyte
- Sort according to output bytes
pps
- Sort according to (in)packets per second
ipps
- Same as ipps
opps
- Sort according to out packets per second
bps
- Sort according to (in)bytes per second
ibps
- Same as bps
obps
- Sort according to output bytes per second
bpp
- Sort according to (in)bytes per packet
ibpp
- Same as bpp
obpp
- Sort according to output packets
tstart
- Sort according to start time of flow - former -m
tend
- Sort according to end time of flows
duration
- Sort according to duration of flows
-t
timewin- Set time window to process flows. This option is considered legacy andmay be replaced with a filter primitiv in future rleases. The time window is specified as: YYYY/MM/dd.hh:mm:ss[-YYYY/MM/dd.hh:mm:ss]. Any parts of the time spec may be omitted e.g YYYY/MM/dd expands to YYYY/MM/dd.00:00:00-infinity and processes all flow from a given day onwards. The time window may also be specified as +/- n. In this case it is relativ to the beginning or end of all flows. +10 means the first 10 seconds of all flows, -10 means the last 10 seconds of all flows.
-c
num- Limit the number of records to be processed to the first num records, which passwd the filter.
-a
- Aggregate flow records. The default aggregation is done at connection
level by taking the 5-tuple protocol, srcip, dstip,
srcport and dstport. This way of aggregation
may be overwritten by option
-A
-A
aggregation- Sets the list of elements in a flow record to be aggregated.
aggregation is a ',' separated list of any number of
v9/ipfix elements. The following elements are accepted:
proto
- IP protocol
srcip
- Source IP address
dstip
- Destination IP address
srcip4/net
- IPv4 source IP address with applied netmask
srcip6/net
- IPv6 source IP address with applied netmask
dstip4/net
- IPv4 destination IP address with applied netmask
dstip6/net
- IPv6 destination IP address with applied netmask
srcnet
- Apply netmask srcmask in netflow record for source IP
dstnet
- Apply netmask dstmask in netflow record for dest IP
srcport
- Source port
dstport
- Destination port
srcmask
- Source mask
dstmask
- Destination mask
srcvlan
- Source vlan label
dstvlan
- Destination vlan label
srcas
- Source AS number
dstas
- Destination AS number
nextas
- BGP Next AS
prevas
- BGP Previous AS
inif
- SNMP input interface number
outif
- SNMP output interface number
next
- IP next hop
bgpnext
- BGP next hop
insrcmac
- In source MAC address
outdstmac
- out destination MAC address
indstmac
- In destintation MAC address
outsrcmac
- Out source MAC address
tos
- Source type of service
srctos
- Source type of Service
dsttos
- Destination type of Service
mpls1
- MPLS label 1
mpls2
- MPLS label 2
mpls3
- MPLS label 3
mpls4
- MPLS label 4
mpls5
- MPLS label 5
mpls6
- MPLS label 6
mpls7
- MPLS label 7
mpls8
- MPLS label 8
mpls9
- MPLS label 9
mpls10
- MPLS label 10
router
- IP address of exporting router
odid
- observation domain ID
opid
- observation point ID
xsrcip
- X-late source IP address, if compiled with NSEL support
xdstip
- X-late destination IP address, if compiled with NSEL support
xsrcport
- X-late source port, if compiled with NSEL support
xdstport
- X-late destination port, if compiled with NSEL support
nfdump
automatically compiles the appropriate output format for the selected aggregation elements unless an explicit output format-o
is given. The automatic output format is identical to-o 'fmt:%ts %td <fields> %pkt %byt %bps %bpp %fl'
where <fields> represents the selected aggregation tags.
-b
- Aggregate flow records as bidirectional flows. This automatically implies
-a. Aggregation is done on connection level by taking the 5-tuple
protocol, srcip, dstip, srcport and
dstport The reverse order applies for the
corresponding reverse flow. Input and output packets/bytes are counted and
reported separately. Both flows are merged into a single record with
corresponding input and output counters. An appropriate output format is
selected automatically, which may be overwritten by any
-o
format option. -B
- Similar to option
-b
but tries to guess the correct client to server direction. Automagically swaps flows if src port is < dst port for TCP and UDP flows and src port < 1024 and dst port > 1024. Some exporters do not really care sending the flows in proper order. It's considered to be a conveniency option. -I
- Print flow statistics of a single file or the summary of all the files
specified by
-r
flowpath. -g
- Print for each flow file given by
-r
flowpath a one line summary, which can be easily used by gnu plot. -D
nameserver- Sets the nameserver to translate hostnames into IP addresses in filter expressions. See filter below for more details.
-G
geoDB- Use geoDB as geo lookup DB for geo location and AS
lookups.
nfdump
tries to read the environment variable NFGEODB for the path of geoDB. The option-G
overwrites NFGEODB. In order to prevent reading any geoDB file, even if it would exist set-G
none. -s
statistic [:p [/orderby]]- Generate the Top N flow record or flow element statistic. By optionally
adding :p to
statistic, the statistic is additionally split up
into the transport layer protocols. By default the statistic is transport
protocol independent. Each statistic may be ordered
by the optional parameter orderby This can be
flows,
packets, bytes, pps, bps or
bpp. You
may specify more than one orderby option, which
results in the same statistic but ordered differently. If no orderby is
given, the statistic is ordered by flows. You can specify as many -s flow
element statistics as needed on the command line for the same run.
statistic can be:
record
- aggregated netflow records.
srcip
- source IP addresses
dstip
- destination IP addresses
ip
- any (src or dst) IP addresses
nhip
- next hop IP addresses
nhbip
- BGP next hop IP addresses
router
- exporting router IP address
srcport
- source ports
dstport
- destination ports
port
- any (source or destination) ports
tos
- type of service - default src
srctos
- src type of service
dsttos
- dst type of service
dir
- flow directions ingress/egress
srcas
- source AS numbers
dstas
- destination AS numbers
srcgeo
- 2 letter geo source country code
dstgeo
- 2 letter geo destination country code
as
- any (source or destination) AS numbers
inif
- input interface
outif
- output interface
if
- any interface
inam
- input interface name
onam
- output interface name
srcmask
- src mask
dstmask
- dst mask
srcvlan
- src vlan label
dstvlan
- dst vlan label
vlan
- any vlan label
insrcmac
- input src MAC address
outdstmac
- output dst MAC address
indstmac
- input dst MAC address
outsrcmac
- output src MAC address
srcmac
- any src MAC address
dstmac
- any dst MAC address
inmac
- any input MAC address
outmac
- any output MAC address
mask
- any mask
proto
- IP protocols
mpls1
- MPLS label 1
mpls2
- MPLS label 2
mpls3
- MPLS label 3
mpls4
- MPLS label 4
mpls5
- MPLS label 5
mpls6
- MPLS label 6
mpls7
- MPLS label 7
mpls8
- MPLS label 8
mpls9
- MPLS label 9
mpls10
- MPLS label 10
sysid
- Internal SysID of exporter
nbar
- nbar ID
ja3
- ja3 hashes
odid
- observation domain ID
opid
- observation point ID
vrf/ivrf
- ingress vrf
evrf
- egress vrf
ivrfnam
- ingress vrf name
evrfnam
- egress vrf name
-
NSEL/ASA statistics
event
- NSEL/ASA event
xevent
- NSEL/ASA extended event
xsrcip
- NSEL/ASA translated src IP address
xsrcport
- NSEL/ASA translated src port
xdstip
- NSEL/ASA translated dst IP address
xdstport
- NSEL/ASA translated dst port
iacl
- NSEL/ASA ingress ACL
iace
- NSEL/ASA ingress ACE
ixace
- NSEL/ASA ingress xACE
eacl
- NSEL/ASA egress ACL
eace
- NSEL/ASA egress ACE
exace
- NSEL/ASA egress xACE
-
NAT statistics
nevent
- NAT event
nsrcip
- NAT src IP address
nsrcport
- NAT src port
ndstip
- NAT dst IP address
ndstport
- NAT dst port
% nfdump -s srcip -s ip/flows/bytes -s record/bytes
-n
num- Set the number of records to be printed to num. This
option applies to
-s
statistics as well as to ordered output-O
-or
-aggreated
-records
-a
The default is set to 10 for statistics and unlimited for the other use cases. To disable the limit, set num to 0. -o
format- Sets the output format to print flow records.
has many different output formats already predefined.
format may be one of the options below:raw
- Print the full flow record on multiple lines. This prints all available information.
fmt:
user- Print the flow records according the format user. This is a very flexible and powerful way to format flow records. See the section OUTPUT below for more details on how to compile your own format.
json
- Print full record as a separate json object.
csv
- Legacy .csv format - will get removed in future releases. Please use json instead.
pipe
- Legacy '|' separated format - will get removed in future releases. Please use json instead.
Already predefined fmt formats:
line
- Print each flow on one line. Default format.
long
- Print each flow on one line with more details
biline
- Same as line, but for bi-directional flows
bilong
- Same as long, but for bi-directional flows
gline
- Same as line, but add country code to IPs. If a geoDB file is supplied this is the default output format
glong
- Same as long, but add country code to IPs
extended
- Print each flow on one line with even more details.
nsel
- Print format for NSEL event records. Default format if NSEL/NAT support has been compiled in.
nel
- Print format for NAT event records.
The
nfdump
config file may contain additional formats. If you want to add new formats or change existing ones, check the config file.IPv6 addresses are printed condensed in any fmt defined format to prevent cluttering the output with large blank blocks. A condensed IPV6 uses max 16 characters. If it is longer, then the middle part of the IP is cut out and replaced be "..". For previewing an output, this fits most needs. For a listing with the full IPV6 addresses add option
-6.
-6
- Print full length of IPv6 addresses in output instead of condensed.
-q
- Quiet mode. Suppress the header line and the statistics at the bottom of text outputs.
-N
- Print plain numbers in output without scaling. Easier for output parsing with 3rd party tools.
-i
ident- Change the ident label in the file, specified by
-r
to ident -v
flowfile- Verify the consistency of flowfile and print the file parameters and number of records.
-E
flowfile- Print the exporter and sampler list if found in flowfile. Additional statistics per exporter are printed with number of flows, packets and sequence errors.
-x
flowfile- This options works on nfdump version 1.6.x files only and may get removed in future. Scans and prints extension maps located in flowfile
-z
- Compress flow files with LZO1X-1 compression. Fastest compression.
-y
- Compress flow files with LZ4 compression. Fast and efficient.
-j
- Compress flow files with bz2 compression. Slow but most efficient. May be used for archiving files or if you are really short of spce.
-J
num- Change compression for any number of files given by option
-r
flowpath num: 0 uncompress, 1: LZO1X-1, 2: bz2, 3: LZ4 compression. This option may be used for archiving flow files and changing the compression to use less disk space. -X
- Compiles the filter syntax and dumps the filter engine table to stdout. This is for debugging purpose only.
-Z
- Check filter syntax and exit. Sets the return value accordingly.
-R
filelist- Select a range of files. This option is mainly used by old NfSen and
documented here as legacy opton.
- /any/dir Read recursively all files in directory dir.
- /dir/file Read all files beginning with file.
- /dir/file1:file2 Read all files from file1 to file2.
-M
dirlist- Read the same file hierarchy from multiple directories. This option is mainly used by old NfSen and documented here as legacy option. Example: /any/path/to/dir1:dir2:dir3 etc. and will be expanded to the directories: /any/path/to/dir1, /any/path/to/dir2 and /any/path/to/dir3. Any number of colon separated directories may be given. The files to read are specified by -r or -R and are expected to exist in all the given directories. The options -r and -R must not contain any directories when used in combination with -M.
-T
- Tag IP addresses with a prepending cntrl-A character, to allow output parsers to hook in. This option is mainly used by old NfSen and documented here as legacy option.
-V
- Print
nfdump
version and exit. -h
- Print help text on stdout with all options and exit.
filter selects, which records will be further processed. If no filter is given, all records will be processed. Otherwise, only those flows matching the filter will be processed. Any IP address in a filter may be specified as IPv4 or IPv6.
The filter syntax is similar to tcpdump but adapted and extended for flow records. The filter can be either specified on the command line after all options or in a separate file. It can span several lines. Anything after a '#' is treated as a comment and ignored to the end of the line. There is virtually no limit in the length of the filter expression. All keywords are case insensitive.
A single filter primitiv filters a single element of a flow record. A filter consists of one or more primitives, which are linked together:
expr
and expr
expr
or
expr
not
expr and
(expr)
Possible filter primitives:
@include
file- Expands the content of file into the current filter
count
comp number- True if the comparison with the record counter matches number Each record gets assigned a record number at the time it is read from file. Therefore this record number is not unique and may change, depending on the order files are read.
ident
string- True if the record ident field matches string. This filter can be used to filter out different sources.
inet
ipv4
- True if source and destination IP of a record are IPv4 IPs.
inet6
ipv6
- True if source and destination IP of a record are IPv6 IPs.
proto
protocol- True if the record protocol field matches protocol.
protocol can be a symbolic name such as
tcp
,udp
,icmp
,ah
,esp
,ipip
, and many more or a protocol number, such as 6, 17 for protocol tcp and udp. tun proto
protocol- True if the record tunnel protocol field matches protocol. protocol may be a symbolic name or protocol number.
ip
ipaddrsrc ip
ipaddrdst ip
ipaddr- True if the respective IP field of the record matches
ipaddr. ipaddr may be an IPv4
or IPv6 address or a symbolic hostname. In this case a DNS lookup resolves
the hostname to one or more IP addresses. If more than one IP results, all
IPs are chained together in an
or
chain. (IP or IP or IP). Ifip
is not specified withsrc
ordst
the source or destination IP may match. host
ipaddrhost
is just a synonym forip
(See above)ip in
[iplist]src in ip
[iplist]dst ip
[iplist]- True if the respective IP field of the record is in
iplist. iplist is a space or ',' separated list of
IP addresses or networks in CIDR notation. This is the prefered way to
search in large list of IP addresses and networks and is much more
efficient than to chain all IP addresses together. (IP1 or IP2 or IP3).
The iplist may contain several hundreds to thousand
IPs and/or networks. For just a few IPs use an
or
chain, otherwise use an iplist Ifip
is not specified withsrc
ordst
the source or destination IP may match. net
network netmasksrc net
network netmaskdst net
network netmasknet
network/netbitssrc net
network/netbitsdst net
network/netbits- True if the respective IP field of the record matches the
network if the corresponding
netmask or netbits are applied
to the IP address. If
net
is not specified withsrc
ordst
the source or destination IP may match. geo
geolocsrc geo
geocodedst geo
geocode- True, if the 2-letter country code resolved by geolookup of the source or
destination IP address matches geocode. This filter
works only, if a valid geoDB is specified. See geo location option above.
The 2-letter country code corresponds to the maxmin DB definitions. if
geo
is not specified withsrc
ordst
the source or destination geo location code may match. tunip
ipaddrsrc tunip
ipaddrdst tunip
ipaddr- True if the respective tunnel IP field of the record matches
ipaddr. If
tunip
is not specified withsrc
ordst
the source or destination tunnel IP may match. port
comp numsrc port
comp numdst port
comp num- True if the comparison of the respective port field matches
num See comp for the
comparator details. If
port
is not specified withsrc
ordst
the source or destination port may match. port in
[portlist]src port in
[portlist]dst port in
[portlist]- True if the respective port field of the record is in
portlist. portlist is a space or ',' separated list
of port numbers. This is the prefered way to search in large list of port
numbers and is much more efficient than to chain all ports together.
(PORT1 or PORT2 or PORT3). portlist may contain
several hundreds to thousand of port numbers. If
port
is not specified withsrc
ordst
the source or destination port may match. icmp-type
numicmp-code
num- True if the respective icmp field of the record matches
num. This automatically implies
proto icmp.
engine-type
numengine-id
numsysid
num- True if the respective fields of the record matches
num engine type and ID are set by the exporting
device, sysid refers to the
nfdump
collector internal assigned number. See also option-E
above. if
numin if
numout if
num- True if the respective interface fields of the record matches
num. This ID may correspond to the SNMP ID of the
interface but depends on the exporter. If
if
is not specified within
orout
the input or output interface may match. as
comp numsrc as
comp numdst as
comp numprev as
comp numnext as
comp num- True if the comparison of the respective AS fields matches
nfdump
supports 32-bit AS numbers every where. Without or the source or destination AS may match. See comp for the comparator details. as in
[aslist]src as in
[aslist]dst as in
[aslist]prev as in
[aslist]next as in
[aslist]- True if the respective AS field of the record is in
aslist. aslist is a space or ',' separated list of
AS numbers. This is the prefered way to search in large list of AS numbers
and is much more efficient than to chain all ports together.
aslist may contain several hundreds to thousand of
AS numbers. If
as
is not specified withsrc, dst, prev
ornext
the source or destination AS may match. mask
bitssrc mask
bitsdst mask
bits- True if the respective mask bit field of the record matches
bits If
mask
is not specified withsrc
ordst
the source or destination mask bits may match. vlan
numsrc vlan
numdst vlan
num- True if the respective vlan field of the record matches
num If
vlan
is not specified withsrc
ordst
the source or destination vlan may match. flags
tcpflags- True if the respective tcp flags field of the record matches any of the
given tcpflags. tcpflags is a string combination of
all flags to be tested:
The order of the flags within tcpflags is not
relevant. Flags not mentioned are treated as don't care. In order to get
those flows with only the SYN flag set, use the syntax
flags S and not flags AFRPU
router ip
ipaddr- True if the ip address of the sending router matches ipaddr as valid IPv4/IPv6 address.
next ip
ipaddr- True if the field next-ip of the record matches ipaddr as valid IPv4/IPv6 address.
bgpnext ip
ipaddr- True if the field bgpnext-ip of the record matches ipaddr as valid IPv4/IPv6 address.
mac
macaddrin mac
macaddrin src mac
macaddrin dst mac
macaddrout mac
macaddrout src mac
macaddrout dst mac
macaddr- True if the respective mac address field of the record matches
macaddr By prepending
mac
with any combination of a direction specifier as defined by CISCO v9 the test is limited to thosemac
addresses only. Otherwise multiple matches are possible. Without any specifiers anymac
address is tested against macaddr mpls labelN
comp number- True if the comparison of the mpls label
N
withN
as mpls label number 1..10 matches number Filters according a specific number in the mpls label stack. mpls eos
comp number- True if the comparison of the end of stack mpls label matches number
mpls expN
comp number- True if the comparison of the experimental bits 0..7 of mpls label
N
withN
as mpls label number 1..10 matches number packets
comp numin packets
comp numout packets
comp num- True if the comparison of the packet counter in the flow record matches
num. num may contain any valid scaling factor such
as k, m, g Example: packets > 1k. For a single
flow
packets
andin packets
is equivalent and describes the number of packets from source to destination. In case of a bi-directional flow (sent by an exporter or combined by option--B
) the packet counter for the reverse flow can be tested without packet
bytes
comp numin bytes
comp numout bytes
comp num- True if the comparison of the byte counter in the flow record matches
num. num may contain any valid scaling factor such
as k, m, g Example: bytes > 1k
bytes
andin bytes
is equivalent and describes the number of bytes from source to destination. In case of a bi-directional flow (sent by an exporter or combined by option--B
) the byte counter for the reverse flow can be tested without bytes
flow
comp num- True if the comparison of the flow counter in the flow record matches num. num may contain any valid scaling factor such as k, m, g For each received flow, the flow counter is set to 1, unless the exporter sends this information. If multiple flows are aggregated, this counter is increased respectively.
tos
num- True if the type of service field of the flow record matches num
flowdir
direction- True, if the flow direction field in the flow record matches direction. direction may be ingress, egress, 0 for ingress, or 1 for egress
duration
comp time- True if the calculated duration of a flow (tend - tstart) compares to time. The duration is specified in msec (milliseconds)
pps
comp num- True if the calculated value of in-packets/duration (packets per second) compares with the number num. num may contain any valid scaling factor such as k, m, g
bps
comp num- True if the calculated value of 8*in-bytes/duration (bits per second) compares with the number num. num may contain any valid scaling factor such as k, m, g
bpp
comp num- True if the calculated value of in-bytes/in-packets (bytes per packet) compares with the number num. num may contain any valid scaling factor such as k, m, g
observation domain id
comp numberobservation point id
comp number- True if the comparison of the observation domain ID or point ID field respectively matches number
- payload filters
- Some exporters, such as
yaf
or the nfdump collectornfpcap can send payload data along the netflow information. If such
payloads are sent it can be filtered according the filter primitives below: payload content
'string'- True if the string string is found in the payload data. string must be quoted with single or double quotes: 'string', “string”
payload regex
'regex'payload regex
'regex' flags- True if regex matches the payload data.
regex
searches over the full payload length. A ' ' byte does not stop the match process. regex must be quoted with single or double quotes: 'regex' or “regex” The regex engine understands the following reduced syntax:- (...) subexpressions/capture ranges
- | the "or" operator
- ^and $ anchors
- [...] and [^...] character classes
- ?, *, +, simple quantifiers
- *?, +?, ?? lazy quantifiers
- {<num>}, {<num1>,<num2>} complex quantifiers
flags are otional can be:
payload ja3
md5string- True, if the payload contains the start of an SSL/TLS handshake and the calculated jas value of the handshake matches md5string
payload ja3 defined
- True, if the payload contains the start of an SSL/TLS handshake and a
valid ja3 value can be calculated. Useful to mask out all flow records
with no SSL/TLS traffic in order to generate a
-s
ja3 statistic. - nprobe implemented elements
client latency
comp timeserver latency
comp time- True, if the respective latency field in the flow record compares to time. time is specified in msec.
- CISCO ASA, network security event logging (NSEL) and NAT event logging (NEL) specific filters:
- NSEL specific filters:
asa event
event- True if the NSEL event type of an event record matches event which may be: ignore, create, term, delete, deny
asa event
comp number- True if the comparison of the NSEL event type of an event records matches number as a number.
asa event denied
reason- True if the event denied type of an event records matches reason which may be ingress, egress, interface, nosyn
asa xevent
comp num- True, if the comparison of the extended event field of the event record matches num
xip
ipaddrsrc xip
ipaddrdst xip
ipaddr- True, if the field of the translated source or destination IP address
matches ipaddr if
xip
is specified withoutsrc
ordst
both IP addresses may match. xport
ipaddrsrc xport
ipaddrdst xport
ipaddr- True, if the field of the translated source or destination IP address
matches ipaddr if
xport
is specified withoutsrc
ordst
both ports may match. xnet
network/masksrc xnet
network/maskdst xnet
network/mask- True if the translated source or destination IP address matches
network if mask mask is
applied. if
xnet
is specified withoutsrc
ordst
both IP addresses may match. ingress ACL
comp numberingress ACE
comp numberingress XACE
comp number- True if the comparison of the respective ingress field matches number
egress ACL
comp number- True if the comparison of the egress field matches number
- NEL specific filters:
nat event
event
- True if the NEL event type of an event record matches event. event may be add, delete
nat event
comp number- True if the comparison of the NEL event type of an event records matches number as a number.
nip
ipaddrsrc nip
ipaddrdst nip
ipaddr- True, if the field of the nat source or destination IP address matches
ipaddr if
nip
is specified withoutsrc
ordst
both IP addresses may match.It Cm nport Ar number
src nport
numberdst nport
number- True, if the field of the nat source or destination port matches
number if
nip
is specified withoutsrc
ordst
both ports may match. ingress vrf
number- True, if the field of the ingess vrf field of the event record matches number
pblock start
comp numberpblock step
comp numberpblock end
comp number- True if the comparison of the start, step or end of the NAT port block in the event record matches number
port in pblock
src port in pblock
dst port in pblock
- True, if the source or destination port field matches the NAT port block range
- comp
- Many filter elements support the comparison with a number. The following
comparators are supported for each of those filters:
=, ==, >, <, >=, <=
To prevent collisions with bash interpretion, alternative comparators are available:EQ, LT, GT, LE, GE
If comp is omitted, '==' is assumed.
OUTPUT FORMAT¶
This section describes how output formats are compiled.
nfdump
has a lot of already pre-defined output
formats such as raw, json, csv etc. One line formats
as described for option -o
can be compiled from
various elements of a flow record. As a flow record can contains man
different elements it is often useful to compile an output format for
spcific needs.
Format description¶
The output format is specified by -o
“fmt: string”
string contains the field tags
to be printed as well as other characters if needed. A
tag starts with a %
sign
followed by the field name. tags are separated by
spaces from other tags. Characters or other strings, not starting with a
%
sign are copied literally to the output.
Example:
-o
“fmt:%ts %td %pr %sap -> %dap %pkt %byt %fl”
This is the definition of the predined format
line.
It adds the elements tstart
duration protocol source IP address/port followed by the literal
characters -> and destination ip address/port packets,
bytes, flows counter. Depending on the task, different output formats
are required to see the required fields of a flow record. You can either
extend a predefined format or specifiy a new one at the command line.
Example: Extend the predefined format long
with the the IP address of the sending router
-o
“fmt:%long %ra”
Predefined formats can be extended by simply add their name with a
%
sign somewhere in the format string. As described
under the output option -o
Format definition¶
nfdump
has already many formats
predefined. Most of the time, these format are good enough. Sometimes you
may need different formats, which can be compiled as described above. In
order to prevent adding the same often used output format each time you run
nfdump
a new output format may be define in the
config file nfdump.conf The file nfdump.conf.dist
contains the definition of the already hard coded formats. These may be
uncommented and changed according to the specific needs. New formats may be
added using the following sytax:
fmt.newname = “fmt:%ts %td %pr
%sap -> %dap %pkt %byt %fl”
with newname any new or existing definition of output formats. Existing formats are overwriten with the new definition.
Tag definition¶
The following list contains all tags, which are available to compile the output format:
%<format>
- Inserts the predefined format at this position. e.g. %line
%cnt
- Record counter. record numbers are assigned dynamically assigned while reading readed from file.
%nfv
- Netflow version.
%ts
- Start Time - first seen
%tfs
- First seen - identical to %ts
%tsr
- Start Time, but in fractional seconds since the epoch (1970-01-01) UNIX format.
%te
- End Time - last seen
%ter
- End Time, in fractional seconds
%tr
- Time the flow was received by the collector
%trr
- Time the flow was received, in fractional seconds
%td
- Duration of flow. Displayed in ddHHMMSS.msec
%pr
- Transort protocol
%exp
- Exporter ID
%eng
- Engine Type/ID
%lbl
- Flowlabel
%sa
- Source Address
%da
- Destination Address
%sap
- Source Address:Port
%dap
- Destination Address:Port
%gsap
- Source Address(country code):Port
%gdap
- Destination Address(country code):Port
%sp
- Source Port
%dp
- Destination Port
%it
- ICMP-type
%ic
- ICMP-code
%sn
- Source Network, mask applied
%dn
- Destination Network, mask applied
%nh
- Next-hop IP Address
%nhb
- BGP Next-hop IP Address
%ra
- Router IP Address
%sas
- Source AS
%das
- Destination AS
%nas
- Next AS
%pas
- Previous AS
%in
- Input Interface num
%out
- Output Interface num
%pkt
- Packets - default input
%ipkt
- Input Packets
%opkt
- Output Packets
%byt
- Bytes - default input
%ibyt
- Input Bytes
%obyt
- Output Bytes
%fl
- Flows
%flg
- TCP Flags
%tos
- Tos - default src
%stos
- Src Tos
%dtos
- Dst Tos
%dir
- Direction: ingress, egress
%smk
- Src mask
%dmk
- Dst mask
%fwd
- Forwarding Status
%svln
- Src vlan label
%dvln
- Dst vlan label
%ismc
- Input Src Mac Addr
%odmc
- Output Dst Mac Addr
%idmc
- Input Dst Mac Addr
%osmc
- Output Src Mac Addr
%mpls1
- MPLS label 1
%mpls2
- MPLS label 2
%mpls3
- MPLS label 3
%mpls4
- MPLS label 4
%mpls5
- MPLS label 5
%mpls6
- MPLS label 6
%mpls7
- MPLS label 7
%mpls8
- MPLS label 8
%mpls9
- MPLS label 9
%mpls10
- MPLS label 10
%mpls
- MPLS labels 1-10
%bps
- bps - bits per second
%pps
- pps - packets per second
%bpp
- bps - Bytes per package
%sc
- src IP 2 letter country code
%dc
- dst IP 2 letter country code
%sloc
- src IP geo location info
%dloc
- dst IP geo location info
%n
- new line char \n
%ipl
- input payload
%opl
- ouput payload
%nbid
- nbar ID
%ja3
- ja3 hash
%sni
- sni name in tls handshake
%nbnam
- nbar name
%odid
- observation domainID
%opid
- observation pointID
- NSEL specific formats
%nfc
- NSEL connection ID
%evt
- NSEL event
%xevt
- NSEL extended event
%sgt
- NSEL Source security group tag
%msec
- NSEL event time in msec
%iacl
- NSEL ingress ACL
%eacl
- NSEL egress ACL
%xsa
- NSEL XLATE src IP address
%xda
- NSEL XLATE dst IP address
%xsp
- NSEL XLATE src port
%xdp
- NSEL SLATE dst port
%xsap
- Xlate Source Address:Port
%xdap
- Xlate Destination Address:Port
%uname
- NSEL user name
- NEL/NAT specific formats
%nevt
- NAT event - same as %evt
%ivrf
- NAT ingress VRF ID
%evrf
- NAT egress VRF ID
%nsa
- NAT src IP address
%nda
- NAT dst IP address
%nsp
- NAT src port
%ndp
- NAT dst port
%pbstart
- NAT pool block start
%pbend
- NAT pool block end
%pbstep
- NAT pool block step
%pbsize
- NAT pool block size
- Nprobe formats
%cl
- Client latency
%sl
- Server latency
%al
- Application latency
EXAMPLES¶
nfdump
processes files created by any
previous version of nfdump 1.6.x with some limitations for versions <
1.6.17. In order to convert flow files to the new 1.7.x binary format use
the following command to read//write files:
% nfdump -r oldfile -w
newfile
Print a statistic about the top 20 IP adresses, once sorted by flows and once by bytes
% nfdump -r flowfile -s
ip/flows/bytes -n 20
Print two statistics, one about the source IP and one about the destination IP address limited to flow with either source or destination port 443
% nfdump -r flowfile -s srcip/bytes
-s dstip/bytes -n 20 'port 443'
Print a statistic about the IP pairs, which exchanged most traffic.
% nfdump -r flowfile -s record/bytes
-A srcip,dstip
Print all flows in raw format with a HTTP header in the payload even if flow is not on port 80.
% nfdump -r flowfile -o raw
“payload regex 'GET|POST'”
Print a statistic about all ja3 md5 sums for those flows, which a valid ja3 can be calculated
% nfdump -r flowfile -s ja5 -n 0
'payload ja3 defined'
Aggregate all flows and write the result back to a binary file, sorted by the start time
% nfdump -r flowfile -a -Otstart -w
newfile
RETURN VALUES¶
nfdump
returns 0 on success and 255 if
processing failed.
SEE ALSO¶
https://www.iana.org/assignments/ipfix/ipfix.xhtml
https://www.cisco.com/en/US/technologies/tk648/tk362/technologies_white_paper09186a00800a3db9.html
BUGS¶
No software without bugs! Please report any bugs back to me.
October 21, 2024 | Debian |