NAME¶

ExtractText - extracts text from documenmts.

SYNOPSIS¶

loadplugin Mail::SpamAssassin::Plugin::ExtractText

ifplugin Mail::SpamAssassin::Plugin::ExtractText

  extracttext_external  pdftotext  /usr/bin/pdftotext -nopgbrk -layout -enc UTF-8 {} -
  extracttext_use       pdftotext  .pdf application/pdf
  # http://docx2txt.sourceforge.net
  extracttext_external  docx2txt   /usr/bin/docx2txt {} -
  extracttext_use       docx2txt   .docx application/docx
  extracttext_external  antiword   /usr/bin/antiword -t -w 0 -m UTF-8.txt {}
  extracttext_use       antiword   .doc application/(?:vnd\.?)?ms-?word.*
  extracttext_external  unrtf      /usr/bin/unrtf --nopict {}
  extracttext_use       unrtf      .doc .rtf application/rtf text/rtf
  extracttext_external  odt2txt    /usr/bin/odt2txt --encoding=UTF-8 {}
  extracttext_use       odt2txt    .odt .ott application/.*?opendocument.*text
  extracttext_use       odt2txt    .sdw .stw application/(?:x-)?soffice application/(?:x-)?starwriter
  extracttext_external  tesseract  {OMP_THREAD_LIMIT=1} /usr/bin/tesseract -c page_separator= {} -
  extracttext_use       tesseract  .jpg .png .bmp .tif .tiff image/(?:jpeg|png|x-ms-bmp|tiff)
  # QR-code decoder
  extracttext_external  zbar       /usr/bin/zbarimg -q -D {}
  extracttext_use       zbar       .jpg .png .pdf image/(?:jpeg|png) application/pdf
  add_header   all          ExtractText-Flags _EXTRACTTEXTFLAGS_
  header       PDF_NO_TEXT  X-ExtractText-Flags =~ /\bpdftotext_NoText\b/
  describe     PDF_NO_TEXT  PDF without text
  score        PDF_NO_TEXT  0.001
  header       DOC_NO_TEXT  X-ExtractText-Flags =~ /\b(?:antiword|openxml|unrtf|odt2txt)_NoText\b/
  describe     DOC_NO_TEXT  Document without text
  score        DOC_NO_TEXT  0.001
  header       EXTRACTTEXT  exists:X-ExtractText-Flags
  describe     EXTRACTTEXT  Email processed by extracttext plugin
  score        EXTRACTTEXT  0.001

endif

DESCRIPTION¶

This module uses external tools to extract text from message parts, and then sets the text as the rendered part. External tool must output plain text, not HTML or other non-textual result.

How to extract text is completely configurable, and based on MIME part type and file name.

CONFIGURATION¶

All configuration lines in user_prefs files will be ignored.

extracttext_maxparts (default: 10)

Configure the maximum mime parts number to analyze, a value of 0 means all mime parts will be analyzed

extracttext_timeout (default: 5 10)

Configure the timeout in seconds of external tool checks, per attachment.

Second argument speficies maximum total time for all checks.

        extracttext_use  antiword  .doc application/(?:vnd\.?)?ms-?word.*
        extracttext_use  openxml   .docx .dotx .dotm application/(?:vnd\.?)openxml.*?word.*
        extracttext_use  openxml   .doc .dot application/(?:vnd\.?)?ms-?word.*
        extracttext_use  unrtf     .doc .rtf application/rtf text/rtf

        extracttext_external  antiword  /usr/bin/antiword -t -w 0 -m UTF-8.txt {} -
        extracttext_external  unrtf     /usr/bin/unrtf --nopict {}
        extracttext_external  odt2txt   /usr/bin/odt2txt --encoding=UTF-8 {}

        header    PDF_NO_TEXT  X-ExtractText-Flags =~ /\bpdftotext_Notext\b/
        describe  PDF_NO_TEXT  PDF without text

ADMINISTRATOR SETTINGS¶

extracttext_cache_type (default: none)

The cache type that is being utilized. Currently only supported value is "dbi" that implies "extracttext_cache_dsn" is a DBI connect string. DBI module is required.

Example: extracttext_cache_type dbi

extracttext_cache_dsn (default: none)

The DBI dsn of the database to use.

For SQLite, the database will be created automatically if it does not already exist, the supplied path and file must be read/writable by the user running spamassassin or spamd.

For MySQL/MariaDB or PostgreSQL, see sql-directory for database table creation clauses.

You will need to have the proper DBI module for your database. For example DBD::SQLite, DBD::mysql, DBD::MariaDB or DBD::Pg.

Minimum required SQLite version is 3.24.0 (available from DBD::SQLite 1.59_01).

Examples:

 extracttext_cache_dsn dbi:SQLite:dbname=/var/lib/spamassassin/ExtractText.db
    

extracttext_cache_username (default: none)

The username that should be used to connect to the database. Not used for SQLite.

extracttext_cache_password (default: none)

The password that should be used to connect to the database. Not used for SQLite.

extracttext_cache_ttl (default: 86400)

The length of time a cache entry will be valid for in seconds. Default is 86400 (1 day).

See "extracttext_cache_autoclean" for database cleaning.

extracttext_cache_autoclean (default: 1000)

Automatically purge old entries from database. Value describes a random run chance of 1/x. The default value of 1000 means that cleaning is run approximately once for every 1000 messages processed. Value of 1 would mean database is cleaned every time a message is processed.

Set 0 to disable automatic cleaning and to do it manually.