NAME¶

softirqslower - Trace softirq handlers with latency exceeding a threshold

SYNOPSIS¶

softirqslower.py [min_us]
softirqslower.py [-c CPU] [min_us]

DESCRIPTION¶

The softirqslower tool traces softirq events that exceed a specified latency threshold. This tool helps diagnose interrupt handling bottlenecks by measuring two critical latency dimensions:

• irq(hard) to softirq: Time delay between hardware interrupt completion and softirq execution start
• softirq runtime: Actual execution duration of softirq handlers

Developed as part of the BCC (BPF Compiler Collection) tools, it's particularly useful for network stack optimization, scheduler tuning, and real-time systems analysis.

OPTIONS¶

min_us

Minimum latency threshold to trace (in microseconds). Default: 10,000 μs (10 ms)

-c, --cpu CPU

Trace only events on the specified CPU core

OUTPUT FORMAT¶

The tool outputs events in the following table format:

TIME STAGE SOFTIRQ LAT(us) CPU COMM

With fields defined as:

TIME

Event timestamp (HH:MM:SS format)

STAGE

Latency measurement type:

SOFTIRQ

SoftIRQ category (case-sensitive). Common values:

LAT(us)

Measured latency in microseconds

CPU

CPU core where softirq was handled (0-based numbering)

COMM

Process context handling the softirq:

EXAMPLES¶

Trace softirqs exceeding 10μs latency:

softirqslower 10

Monitor only CPU core 1:

softirqslower -c 1

Trace network-related softirq delays (>50μs):

softirqslower 50 | grep -E 'net_rx|net_tx'

Capture RCU delays longer than 100μs to file:

softirqslower 100 | grep rcu > rcu_latency.log

SIGNALS¶

Ctrl+C Stop tracing and exit gracefully

AUTHOR¶

Chenyue Zhou

SEE ALSO¶

runqslower(8), hardirqs(8), softirqs(8), trace(8), funclatency(8),

REPORTING BUGS¶

BCC Tools Issue Tracker: https://github.com/iovisor/bcc/issues