Scroll to navigation

Mail::SpamAssassin::Plugin::Redirectors(3pm) User Contributed Perl Documentation Mail::SpamAssassin::Plugin::Redirectors(3pm)

NAME

Redirectors - Check for redirected and shortened URLs

SYNOPSIS

  loadplugin    Mail::SpamAssassin::Plugin::Redirectors
  url_redirector bing.com
  url_shortener tinyurl.com
  body HAS_REDIR_URL          eval:redir_url()
  describe HAS_REDIR_URL      Message has one or more redirected URLs
  body REDIR_URL_VALID        eval:redir_url_valid()
  describe REDIR_URL_VALID    Message has a redirector that returned a valid redirection
  body REDIR_URL_CHAINED      eval:redir_url_chained()
  describe REDIR_URL_CHAINED  Message has redirected URL chained to other redirectors
  body REDIR_URL_CHAINED_DOM      eval:redir_url_chained_domain()
  describe REDIR_URL_CHAINED_DOM  Message has redirected URL chained to another redirector on the same domain
  body REDIR_URL_MAXCHAIN     eval:redir_url_maxchain()
  describe REDIR_URL_MAXCHAIN Message has redirected URL that causes too many redirections
  body REDIR_URL_LOOP         eval:redir_url_loop()
  describe REDIR_URL_LOOP     Message has redirected URL that loops back to itself
  body REDIR_URL_404          eval:redir_url_code('404') # Can check any non-redirect HTTP code
  describe REDIR_URL_404      Message has redirected URL returning HTTP 404
  body HAS_SHORT_URL          eval:short_url()
  describe HAS_SHORT_URL      Message has one or more shortened URLs
  body SHORT_URL_REDIR        eval:short_url_redir()
  describe SHORT_URL_REDIR    Message has shortened URL that resulted in a valid redirection
  body SHORT_URL_CHAINED      eval:short_url_chained()
  describe SHORT_URL_CHAINED  Message has shortened URL chained to other shorteners
  body SHORT_URL_MAXCHAIN     eval:short_url_maxchain()
  describe SHORT_URL_MAXCHAIN Message has shortened URL that causes too many redirections
  body SHORT_URL_LOOP         eval:short_url_loop()
  describe SHORT_URL_LOOP     Message has short URL that loops back to itself
  body SHORT_URL_200          eval:short_url_code('200') # Can check any non-redirect HTTP code
  describe SHORT_URL_200      Message has shortened URL returning HTTP 200
  body SHORT_URL_404          eval:short_url_code('404') # Can check any non-redirect HTTP code
  describe SHORT_URL_404      Message has shortened URL returning HTTP 404
  uri URI_TINYURL_BLOCKED      m,https://tinyurl\.com/app/nospam,
  describe URI_TINYURL_BLOCKED Message contains a tinyurl that has been disabled due to abuse
  uri URI_BITLY_BLOCKED       m,^https://bitly\.com/a/blocked,
  describe URI_BITLY_BLOCKED  Message contains a bit.ly URL that has been disabled due to abuse

DESCRIPTION

This plugin looks for URLs redirected or shortened by a list of URL redirector/shortener services. Upon finding a matching URL, plugin will send a HTTP request to the service and retrieve the Location-header which points to the actual destination URL. It then adds this URL to the list of URIs extracted by SpamAssassin which can then be accessed by uri rules and plugins such as URIDNSBL.

For hosts listed with "url_redirector" or "url_redirector_get", the request is made with Perl's LWP library (libwww-perl), without a browser. This documentation calls these two methods "LWP", in contrast to the Selenium browser used for hosts listed with "url_redirector_selenium".

This plugin will follow chained redirections, where a redirected URL leads to another redirector, in any combination and order -- for example a redirector that unwraps into what used to be called a "shortener", or vice versa. There is no functional difference between a "redirector" and a "shortener": both are just a domain whose HTTP response redirects somewhere else, and both are followed by the same code path. "url_shortener"/"url_shortener_get"/"url_shortener_custom_user_agent" and their "max_short_url*"/"url_shortener_cache_*"/etc. settings are kept as deprecated aliases of "url_redirector"/"url_redirector_get"/etc. for backwards compatibility, and are planned for removal in a future version. Likewise, short_url() and its sibling eval rules are aliases of redir_url() and friends. Redirection depth is limited by "max_redir_url_redirections", and "max_redir_urls" redirector URLs are checked in a message (10 by default); setting it to 0 disables HTTP requests, allowing only redir_url() to work and report found redirectors.

All supported rule types for checking redirected URLs and redirection status are documented in SYNOPSIS section.

NOTES

This plugin runs before priority 0 so that it may modify the parsed URI list prior to normal uri rules or the URIDNSBL plugin.

SECURITY CONSIDERATIONS

When "url_redirector_selenium_host" is set, the Selenium server runs a real browser that opens URLs chosen by the senders of incoming mail. Read Redirectors-Selenium-Security.pod before setting it. The file is in the docs directory of the SpamAssassin distribution, and "make install" installs it in the documentation directory, by default PREFIX/share/doc/spamassassin. To read it, run "perldoc" with the file's path.

ACKNOWLEDGEMENTS

The url_shortener functionality was originally provided by a separate DecodeShortURLs plugin; that functionality has been merged into this one, and "Mail::SpamAssassin::Plugin::DecodeShortURLs" is now a deprecated compatibility shim that loads this plugin.

ADMINISTRATOR SETTINGS

Domains that should be considered as a URL redirector.

Domain matching:

  • A bare domain (e.g. "bing.com") matches that exact host and also "www.bing.com".
  • A leading dot (e.g. ".sendgrid.com") matches any subdomain of the domain, to any depth. It does NOT match the bare domain itself; to match both, also list the bare domain ("sendgrid.com").

An optional "/path" may follow the domain to restrict matching to URLs whose path begins with that string and ends at a path-segment boundary. "/tr/op" matches "/tr/op", "/tr/op/foo", and "/tr/op?x=1", but NOT "/tr/open" -- the configured prefix is treated as one or more whole path segments. Append a trailing slash ("/tr/op/") to require at least one more segment after it (matches "/tr/op/foo" but not bare "/tr/op"). Multiple entries for the same domain are additive (allowlist); a URL is followed if it matches any entry.

A bare-domain entry without a path is equivalent to "domain/", which matches any path.

Example:

 url_redirector bing.com
 url_redirector .sendgrid.com
 url_redirector .sendibt2.com/tr/cl/

The last line follows "https://x.y.sendibt2.com/tr/cl/abc" but not "https://x.y.sendibt2.com/tr/op/abc".

"url_shortener" (and "url_shortener_get", "clear_url_shortener") are deprecated aliases of "url_redirector" (and "url_redirector_get", "clear_url_redirector") kept for backwards compatibility with configs written for the old DecodeShortURLs plugin, there is no functional difference between the two names, and the "url_shortener*" spelling is planned for removal in a future version.

Set Selenium host to use. Hosts listed in "url_redirector_selenium" are looked up only when this is set.

If the Selenium server does not answer when a session is created, each message that needs a Selenium lookup can be delayed by up to about 20 seconds; if it answers but has no free session, by about 10. Allow the server at least one session for each SpamAssassin process that makes Selenium lookups.

Set Selenium port to use.
If set, the browser session used for a message is kept for the next message instead of being closed, so cookies, stored data and any page still running carry over between messages. This turns off the per-message isolation described under "SECURITY CONSIDERATIONS". Read that before enabling it.

A kept session is closed if it does not answer when it is next needed, or if the message that needs it uses different browser settings; otherwise the Selenium server removes it when its session timeout expires.

The proxy the Selenium browser uses for the listed hosts. The scheme is "http", "https" or "socks5". An IPv6 host literal must be bracketed ("[::1]"). The port is required. Only one proxy is accepted. A proxy list, the per-scheme form ("http=host:port") or "direct://" is rejected, because it could let the browser bypass the proxy.

The host is the proxy address as seen from the Selenium host or container, not from SpamAssassin's host.

The browser's own host list applies with or without a proxy. A proxy configured as described in "SECURITY CONSIDERATIONS" looks up each listed host name itself, refuses it if it resolves to an internal address, and logs requests by host name. Without a proxy, the firewall and resolver of the Selenium host must refuse internal addresses.

Clear domains configured with url_redirector, url_redirector_get or url_redirector_selenium (or their url_shortener aliases), for example to override default settings from an update channel. If no arguments are given, all entries are cleared. If domains are specified, only those are removed.

When an entry includes a "/path", only that path is removed from the domain's allowlist; the domain entry itself is dropped only when its path list becomes empty. Use "domain/" to remove the "match any path" entry added by a bare-domain configuration.

Domains that should be considered as an URL redirector, accessed using the HTTP GET method instead of HEAD. Syntax and matching rules are the same as "url_redirector": a bare domain matches that host and "www.", a leading dot matches any subdomain but not the domain itself, and an optional "/path" prefix restricts the match.
Domains that should be fetched using Selenium (headless Chrome) instead of LWP. Use this only for hosts whose redirects are NOT discoverable via LWP, such as JavaScript-driven redirects. Syntax and matching rules are the same as "url_redirector".

Selenium executes JavaScript on the destination and always sends GET, which has weaker safety properties than the LWP path; reserve this list for hosts that genuinely require browser-driven redirect unwrapping.

The browser connects only to the hosts listed here and in "url_redirector_selenium_resource". Hosts listed here are matched as above except that paths play no part: a redirect or script that leads to another path on a listed host is followed, and one that leads to a host in neither list is not. Its URL is then recorded as the redirect target without the browser fetching it (see "SECURITY CONSIDERATIONS"). A page that needs resources from hosts not listed, such as scripts from a content delivery network, loads without them, and a redirect that depends on them is not seen; list such hosts with "url_redirector_selenium_resource". Each time it creates a browser session, the plugin checks that the browser refuses an address outside this list. If it does not, the plugin closes the session and skips Selenium lookups, with one warning per process.

If the same host appears in both "url_redirector_selenium" and "url_redirector"/"url_redirector_get", the directive parsed last sets the method for that host, including any paths listed for it under the other directive, and a warning is emitted.

Hosts in this list are looked up only when "url_redirector_selenium_host" is set. Until then they are skipped (there is no fallback to LWP, since these hosts wouldn't yield useful results via LWP), and redir_url() still fires for them so detection rules continue to work.

Due to how Selenium works, redir_url_code() and "redir_url_404" subs will not fire for hosts handled by Selenium; Selenium reports only "redirected" (301) or "not a redirect" (200), not real HTTP status codes.

Hosts the Selenium browser may also fetch from, for pages on "url_redirector_selenium" hosts that load scripts or other resources from another host. URLs on these hosts are not looked up.

Each entry matches only that exact host, not its "www." form or other subdomains. IP addresses, paths, a leading dot and names under ".localhost" are not accepted.

List only hosts where senders cannot publish content, not a storage service or a package CDN. See "SECURITY CONSIDERATIONS".

Example:

 url_redirector_selenium_resource code.jquery.com
Clear configured url_redirector_selenium_resource hosts, for example to override default settings from an update channel. If no arguments are given, all hosts are cleared. If hosts are specified, only those are removed.
Domains that, when found as a redirect destination, should cause the redirect chain to stop immediately. Useful to break loops caused by consent or gateway pages that redirect back into the chain.

Example:

 url_skip_redirect_to consent.google.com
Clear configured url_skip_redirect_to domains. If domains are specified, only those are removed from the list.
Regexp used to parse uri parameters in order to detect redirectors and to get redirected domains. The regexp must match only the redirected domain.
Custom HTTP user-agent to be used for specific domains, instead of the default specified in "url_redirector_user_agent". Required for some services like t.co to return blocked URL correctly.

Example:

 url_redirector_custom_user_agent t.co curl/8.6.0
The cache type that is being utilized. Currently only supported value is "dbi" that implies "url_redirector_cache_dsn" is a DBI connect string. DBI module is required.

Example: url_redirector_cache_type dbi

The DBI dsn of the database to use.

For SQLite, the database will be created automatically if it does not already exist, the supplied path and file must be read/writable by the user running spamassassin or spamd.

For MySQL/MariaDB or PostgreSQL, see sql-directory for database table creation clauses.

You will need to have the proper DBI module for your database. For example DBD::SQLite, DBD::mysql, DBD::MariaDB or DBD::Pg.

Minimum required SQLite version is 3.24.0 (available from DBD::SQLite 1.59_01).

Examples:

 url_redirector_cache_dsn dbi:SQLite:dbname=/var/lib/spamassassin/Redirectors.db

A config that still loads the deprecated "DecodeShortURLs" plugin (rather than loading "Redirectors" directly) keeps using that plugin's original "short_url_cache" table/columns, so an existing cache built before the plugins were merged keeps working unchanged. Switching "loadplugin" to "Redirectors" starts a fresh "redir_url_cache" table, as a one-time cost of migrating.

The username that should be used to connect to the database. Not used for SQLite.
The password that should be used to connect to the database. Not used for SQLite.
The length of time a cache entry will be valid for in seconds. Default is 86400 (1 day).

See "url_redirector_cache_autoclean" for database cleaning.

Deprecated alias of "url_redirector_cache_type" -- there is only one cache now, shared by everything this plugin fetches (redirectors and shorteners alike).
Deprecated alias of "url_redirector_cache_dsn".
Deprecated alias of "url_redirector_cache_username".
Deprecated alias of "url_redirector_cache_password".
Deprecated alias of "url_redirector_cache_ttl".

See "url_redirector_cache_autoclean" for database cleaning.

Automatically purge old entries from database. Value describes a random run chance of 1/x. The default value of 1000 means that cleaning is run approximately once for every 1000 messages processed. Value of 1 would mean database is cleaned every time a message is processed.

Set 0 to disable automatic cleaning and to do it manually.

If this option is enabled (set to 1), then redirected URLs and the decoded URLs will be logged with info priority.
Maximum time a redirection URL HTTP request can take, in seconds.
Maximum amount of redirector URLs that will be looked up per message. Chained redirections are not counted, only initial redirection URLs found.

Setting it to 0 disables HTTP requests, allowing only redir_url() test to work and report any found redirector URLs.

Maximum depth of chained redirections that a redirector can generate.

Redirection performed by the Selenium browser uses the browser's limits which look like a failed lookup if exceeded.

Set User-Agent header for HTTP requests. Some services require it to look like a common browser.
Deprecated alias of "url_redirector_cache_autoclean".
Deprecated alias of "url_redirector_loginfo".
Deprecated alias of "url_redirector_timeout".
Deprecated alias of "max_redir_urls" -- there is only one budget now, shared by everything this plugin fetches (redirectors and shorteners alike).
Deprecated alias of "max_redir_url_redirections".
Deprecated alias of "url_redirector_user_agent". Per-domain overrides use "url_redirector_custom_user_agent" ("url_shortener_custom_user_agent" is itself a deprecated alias of that).
2026-10-24 perl v5.42.3