Scroll to navigation

Mail::SpamAssassin::FuzzyHash::Util(3pm) User Contributed Perl Documentation Mail::SpamAssassin::FuzzyHash::Util(3pm)

NAME

Mail::SpamAssassin::FuzzyHash::Util - Shared utilities for FuzzyHash algorithms

SYNOPSIS

  use Mail::SpamAssassin::FuzzyHash::Util qw(normalize_tokens hamming_distance);
  my @tokens = normalize_tokens($text);
  my $dist   = hamming_distance($hex_a, $hex_b);

FUNCTIONS

normalize_tokens($text)

Normalises $text and returns a list of unique tokens suitable for fuzzy-hash computation.

Steps: ensure UTF-8 flag is set, apply NFKC normalisation (folds full-width ASCII spam chars and ligatures), lowercase, insert space boundaries around CJK characters, strip anything that is not a Unicode letter, digit, or space, collapse whitespace, split and deduplicate.

Returns "('__empty__')" for texts that produce no tokens after normalisation.

hamming_distance($hex_a, $hex_b)

Returns the Hamming distance (number of differing bits) between two equal-length lowercase hex strings. The strings must have even length.

2026-06-29 perl v5.40.1