NAME¶

Sympa::WWW::Crawlers - Regular expression for User-Agent of web crawlers

DESCRIPTION¶

This module keeps definition of regular expressions used by Sympa software.

The regular expression is generated from the data provided by the project below.

SEE ALSO¶

•

Syntactic patterns of HTTP user-agents used by bots / robots / crawlers / scrapers / spiders

<https://github.com/monperrus/crawler-user-agents>

HISTORY¶

Crawler detection feature of WWSympa was introduced on Sympa 5.4a.4 which derives information provided by <http://www.useragentstring.com>.

On Sympa 6.2.74, it was replaced with regular expression matching using information provided by crawler-user-agents project above.