History¶

The original txt2html script was written by Seth Golub (see http://www.aigeek.com/txt2html/), and converted to a perl module by Kathryn Andersen (see http://www.katspace.com/tools/text_to_html/) and made into a sourceforge project by Sun Tong (see http://sourceforge.net/projects/txt2html/). Earlier versions of the HTML::TextToHTML module called the included script texthyper so as not to clash with the original txt2html script, but now the projects have all been merged. UPDATING: currently, the project is available on GitHub at https://github.com/resurrecting-open-source-projects/txt2html

new¶

    $conv = new HTML::TextToHTML()
    $conv = new HTML::TextToHTML(titlefirst=>1,
        ...
    );

Create a new object with new. If arguments are given, these arguments will be used in invocations of other methods.

See "OPTIONS" for the possible values of the arguments.

args¶

    $conv->args(short_line_length=>60,
        titlefirst=>1,
        ....
    );

Updates the current arguments/options of the HTML::TextToHTML object. Takes hash of arguments, which will be used in invocations of other methods. See "OPTIONS" for the possible values of the arguments.

process_chunk¶

$newstring = $conv->process_chunk($mystring);

Convert a string to a HTML fragment. This assumes that this string is at the least, a single paragraph, but it can contain more than that. This returns the processed string. If you want to pass arguments to alter the behaviour of this conversion, you need to do that earlier, either when you create the object, or with the "args" method.

    $newstring = $conv->process_chunk($mystring,
                            close_tags=>0);

If there are open tags (such as lists) in the input string, process_chunk will automatically close them, unless you specify not to, with the close_tags option.

    $newstring = $conv->process_chunk($mystring,
                            is_fragment=>1);

If you want this string to be treated as a fragment, and not assumed to be a paragraph, set is_fragment to true. If there is more than one paragraph in the string (ie it contains blank lines) then this option will be ignored.

process_para¶

$newstring = $conv->process_para($mystring);

Convert a string to a HTML fragment. This assumes that this string is at the most a single paragraph, with no blank lines in it. If you don't know whether your string will contain blank lines or not, use the "process_chunk" method instead.

This returns the processed string. If you want to pass arguments to alter the behaviour of this conversion, you need to do that earlier, either when you create the object, or with the "args" method.

    $newstring = $conv->process_para($mystring,
                            close_tags=>0);

If there are open tags (such as lists) in the input string, process_para will automatically close them, unless you specify not to, with the close_tags option.

    $newstring = $conv->process_para($mystring,
                            is_fragment=>1);

If you want this string to be treated as a fragment, and not assumed to be a paragraph, set is_fragment to true.

txt2html¶

    $conv->txt2html(%args);

Convert a text file to HTML. Takes a hash of arguments. See "OPTIONS" for the possible values of the arguments. Arguments which have already been set with new or args will remain as they are, unless they are overridden.

init_our_data¶

$self->init_our_data();

Initializes the internal object data.

deal_with_options¶

$self->deal_with_options();

do extra processing related to particular options

escape¶

$newtext = escape($text);

Escape & < and >

demoronize_char¶

$newtext = demoronize_char($text);

Convert Microsoft character entities into characters.

Added by Alan Jackson, alan at ajackson dot org, and based on the demoronize script by John Walker, http://www.fourmilab.ch/

demoronize_code¶

$newtext = demoronize_code($text);

convert Microsoft character entities into HTML code

get_tag¶

$tag = $self->get_tag($in_tag);

$tag = $self->get_tag($in_tag, tag_type=>TAG_START, inside_tag=>'');

output the tag wanted (add the <> and the / if necessary) - output in lower or upper case - do tag-related processing options:
tag_type=>TAG_START | tag_type=>TAG_END | tag_type=>TAG_EMPTY
(default start)
inside_tag=>string (default empty)

close_tag¶

$tag = $self->close_tag($in_tag);

close the open tag

hrule¶

   $self->hrule(para_lines_ref=>$para_lines,
             para_action_ref=>$para_action,
             ind=>0);

Deal with horizontal rules.

shortline¶

    $self->shortline(line_ref=>$line_ref,
                     line_action_ref=>$line_action_ref,
                     prev_ref=>$prev_ref,
                     prev_action_ref=>$prev_action_ref,
                     prev_line_len=>$prev_line_len);

Deal with short lines.

is_mailheader¶

    if ($self->is_mailheader(rows_ref=>$rows_ref))
    {
        ...
    }

Is this a mailheader line?

mailheader¶

    $self->mailheader(rows_ref=>$rows_ref);

Deal with a mailheader.

mailquote¶

    $self->mailquote(line_ref=>$line_ref,
                     line_action_ref=>$line_action_ref,
                     prev_ref=>$prev_ref,
                     prev_action_ref=>$prev_action_ref,
                     next_ref=>$next_ref);

Deal with quoted mail.

subtract_modes¶

    $newvector = subtract_modes($vector, $mask);

Subtracts modes listed in $mask from $vector.

paragraph¶

    $self->paragraph(line_ref=>$line_ref,
                     line_action_ref=>$line_action_ref,
                     prev_ref=>$prev_ref,
                     prev_action_ref=>$prev_action_ref,
                     line_indent=>$line_indent,
                     prev_indent=>$prev_indent,
                     is_fragment=>$is_fragment,
                     ind=>$ind);

Detect paragraph indentation.

listprefix¶

    ($prefix, $number, $rawprefix, $term) = $self->listprefix($line);

Detect and parse a list item.

startlist¶

    $self->startlist(prefix=>$prefix,
                     number=>0,
                     rawprefix=>$rawprefix,
                     term=>$term,
                     para_lines_ref=>$para_lines_ref,
                     para_action_ref=>$para_action_ref,
                     ind=>0,
                     prev_ref=>$prev_ref,
                     total_prefix=>$total_prefix);

Start a list.

endlist¶

    $self->endlist(num_lists=>0,
        prev_ref=>$prev_ref,
        line_action_ref=>$line_action_ref);

End N lists

continuelist¶

    $self->continuelist(para_lines_ref=>$para_lines_ref,
                        para_action_ref=>$para_action_ref,
                        ind=>0,
                        term=>$term);

Continue a list.

liststuff¶

    $self->liststuff(para_lines_ref=>$para_lines_ref,
                     para_action_ref=>$para_action_ref,
                     para_line_indent_ref=>$para_line_indent_ref,
                     ind=>0,
                     prev_ref=>$prev_ref);

Process a list (higher-level method).

get_table_type¶

    $table_type = $self->get_table_type(rows_ref=>$rows_ref,
                                        para_len=>0);

Figure out the table type of this table, if any

is_aligned_table¶

    if ($self->is_aligned_table(rows_ref=>$rows_ref, para_len=>0))
    {
        ...
    }

Check if the given paragraph-array is an aligned table

is_pgsql_table¶

    if ($self->is_pgsql_table(rows_ref=>$rows_ref, para_len=>0))
    {
        ...
    }

Check if the given paragraph-array is a Postgresql table (the ascii format produced by Postgresql)

A PGSQL table can start with an optional table-caption,

    then it has a row of column headings separated by |
    then it has a row of ------+-----
    then it has one or more rows of column values separated by |
    then it has a row-count (N rows)

is_border_table¶

    if ($self->is_border_table(rows_ref=>$rows_ref, para_len=>0))
    {
        ...
    }

Check if the given paragraph-array is a Border table.

A BORDER table can start with an optional table-caption,

    then it has a row of +------+-----+
    then it has a row of column headings separated by |
    then it has a row of +------+-----+
    then it has one or more rows of column values separated by |
    then it has a row of +------+-----+

is_delim_table¶

    if ($self->is_delim_table(rows_ref=>$rows_ref, para_len=>0))
    {
        ...
    }

Check if the given paragraph-array is a Delimited table.

A DELIM table can start with an optional table-caption, then it has at least two rows which start and end and are punctuated by a non-alphanumeric delimiter.

    | val1 | val2 |
    | val3 | val4 |

tablestuff¶

    $self->tablestuff(table_type=>0,
                      rows_ref=>$rows_ref,
                      para_len=>0);

Process a table.

make_aligned_table¶

    $self->make_aligned_table(rows_ref=>$rows_ref,
                              para_len=>0);

Make an Aligned table.

make_pgsql_table¶

    $self->make_pgsql_table(rows_ref=>$rows_ref,
                              para_len=>0);

Make a PGSQL table.

make_border_table¶

    $self->make_border_table(rows_ref=>$rows_ref,
                             para_len=>0);

Make a BORDER table.

make_delim_table¶

    $self->make_delim_table(rows_ref=>$rows_ref,
                            para_len=>0);

Make a Delimited table.

is_preformatted¶

    if ($self->is_preformatted($line))
    {
        ...
    }

Returns true if the passed string is considered to be preformatted.

split_end_explicit_preformat¶

    $front = $self->split_end_explicit_preformat(para_ref=>$para_ref);

Modifies the given string, and returns the front preformatted part.

endpreformat¶

    $self->endpreformat(para_lines_ref=>$para_lines_ref,
                        para_action_ref=>$para_action_ref,
                        ind=>0,
                        prev_ref=>$prev_ref);

End a preformatted section.

preformat¶

    $self->preformat(mode_ref=>$mode_ref,
                     line_ref=>$line_ref,
                     line_action_ref=>$line_action_ref,
                     prev_ref=>$prev_ref,
                     next_ref=>$next_ref,
                     prev_action_ref);

Detect and process a preformatted section.

make_new_anchor¶

    $anchor = $self->make_new_anchor($heading_level);

Make a new anchor.

anchor_mail¶

    $self->anchor_mail($line_ref);

Make an anchor for a mail section.

anchor_heading¶

    $self->anchor_heading($heading_level, $line_ref);

Make an anchor for a heading.

heading_level¶

    $self->heading_level($style);

Add a new heading style if this is a new heading style.

is_ul_list_line¶

    if ($self->is_ul_list_line($line))
    {
        ...
    }

Tests if this line starts a UL list item.

is_heading¶

    if ($self->is_heading(line_ref=>$line_ref, next_ref=>$next_ref))
    {
        ...
    }

Tests if this line is a heading. Needs to take account of the next line, because a standard heading is defined by "underlining" the text of the heading.

heading¶

    $self->heading(line_ref=>$line_ref,
        next_ref=>$next_ref);

Make a heading. Assumes is_heading is true.

is_custom_heading¶

    if ($self->is_custom_heading($line))
    {
        ...
    }

Check if the given line matches a custom heading.

custom_heading¶

    $self->custom_heading(line_ref=>$line_ref);

Make a custom heading. Assumes is_custom_heading is true.

unhyphenate_para¶

    $self->unhyphenate_para($para_ref);

Join up hyphenated words that are split across lines.

tagline¶

    $self->tagline($tag, $line_ref);

Put the given tag around the given line.

iscaps¶

    if ($self->iscaps($line))
    {
        ...
    }

Check if a line is all capitals.

caps¶

    $self->caps(line_ref=>$line_ref,
                line_action_ref=>$line_action_ref);

Detect and deal with an all-caps line.

do_delim¶

    $self->do_delim(line_ref=>$line_ref,
                    line_action_ref=>$line_action_ref,
                    delim=>'*',
                    tag=>'STRONG');

Deal with a line which has words delimited by the given delimiter; this is used to deal with italics, bold and underline formatting.

glob2regexp¶

    $regexp = glob2regexp($glob);

Convert very simple globs to regexps

add_regexp_to_links_table¶

    $self->add_regexp_to_links_table(label=>$label,
                                     pattern=>$pattern,
                                     url=>$url,
                                     switches=>$switches);

Add the given regexp "link definition" to the links table.

add_literal_to_links_table¶

    $self->add_literal_to_links_table(label=>$label,
                                      pattern=>$pattern,
                                      url=>$url,
                                      switches=>$switches);

Add the given literal "link definition" to the links table.

add_glob_to_links_table¶

    $self->add_glob_to_links_table(label=>$label,
                                   pattern=>$pattern,
                                   url=>$url,
                                   switches=>$switches);

Add the given glob "link definition" to the links table.

parse_dict¶

    $self->parse_dict($dictfile, $dict);

Parse the dictionary file. (see also load_dictionary_links, for things that were stripped)

setup_dict_checking¶

    $self->setup_dict_checking();

Set up the dictionary checking.

in_link_context¶

    if ($self->in_link_context($match, $before))
    {
        ...
    }

Check if we are inside a link (<a ...>); certain kinds of substitution are not allowed here.

apply_links¶

    $self->apply_links(para_ref=>$para_ref,
                       para_action_ref=>$para_action_ref);

Apply links and formatting to this paragraph.

check_dictionary_links¶

    $self->check_dictionary_links(line_ref=>$line_ref,
                                  line_action_ref=>$line_action_ref);

Check (and alter if need be) the bits in this line matching the patterns in the link dictionary.

load_dictionary_links¶

    $self->load_dictionary_links();

Load the dictionary links.

do_file_start¶

    $self->do_file_start($outhandle, $para);

Extra stuff needed for the beginning: HTML headers, and prepending a file if desired.

do_init_call¶

    $self->do_init_call();

Certain things, like reading link dictionaries, need to be done only once.

Link Dictionary¶

A link dictionary file contains patterns to match, and what to convert them to. It is called a "link" dictionary because it was intended to be something which defined what a href link was, but it can be used for more than that. However, if you wish to define your own links, it is strongly advised to read up on regular expressions (regexes) because this relies heavily on them.

The file consists of comments (which are lines starting with #) and blank lines, and link entries. Each entry consists of a regular expression, a -> separator (with optional flags), and a link "result".

In the simplest case, with no flags, the regular expression defines the pattern to look for, and the result says what part of the regular expression is the actual link, and the link which is generated has the href as the link, and the whole matched pattern as the visible part of the link. The first character of the regular expression is taken to be the separator for the regex, so one could either use the traditional / separator, or something else such as | (which can be helpful with URLs which are full of / characters).

So, for example, an ftp URL might be defined as:

    |ftp:[\w/\.:+\-]+|      -> $&

This takes the whole pattern as the href, and the resultant link has the same thing in the href as in the contents of the anchor.

But sometimes the href isn't the whole pattern.

    /<URL:\s*(\S+?)\s*>/ --> $1

With the above regex, a () grouping marks the first subexpression, which is represented as $1 (rather than $& the whole expression). This entry matches a URL which was marked explicitly as a URL with the pattern <URL:foo> (note the &lt; is shown as the entity, not the actual character. This is because by the time the links dictionary is checked, all such things have already been converted to their HTML entity forms, unless, of course, the escape_HTML_chars option was turned off) This would give us a link in the form <A HREF="foo">&lt;URL:foo&gt;</A>

The h flag

However, if we want more control over the way the link is constructed, we can construct it ourself. If one gives the h flag, then the "result" part of the entry is taken not to contain the href part of the link, but the whole link.

For example, the entry:

    /&lt;URL:\s*(\S+?)\s*&gt;/ -h-> <A HREF="$1">$1</A>

will take <URL:foo> and give us <A HREF="foo">foo</A>

However, this is a very powerful mechanism, because it can be used to construct custom tags which aren't links at all. For example, to flag *italicised words* the following entry will surround the words with EM tags.

    /\B\*([a-z][a-z -]*[a-z])\*\B/ -hi-> <EM>$1</EM>

The i flag

This turns on ignore case in the pattern matching.

The e flag

This turns on execute in the pattern substitution. This really only makes sense if h is turned on too. In that case, the "result" part of the entry is taken as perl code to be executed, and the result of that code is what replaces the pattern.

The o flag

This marks the entry as a once-only link. This will convert the first instance of a matching pattern, and ignore any others further on.

For example, the following pattern will take the first mention of HTML::TextToHTML and convert it to a link to the module's home page.

    "HTML::TextToHTML"  -io-> http://www.katspace.com/tools/text_to_html/

Input File Format¶

For the most part, this module tries to use intuitive conventions for determining the structure of the text input. Unordered lists are marked by bullets; ordered lists are marked by numbers or letters; in either case, an increase in indentation marks a sub-list contained in the outer list.

Headers (apart from custom headers) are distinguished by "underlines" underneath them; headers in all-capitals are distinguished from those in mixed case. All headers, both normal and custom headers, are expected to start at the first line in a "paragraph".

In other words, the following is a header:

    I am Head Man
    -------------

But the following does not have a header:

    I am not a head Man, man
    I am Head Man
    -------------

Tables require a more rigid convention. A table must be marked as a separate paragraph, that is, it must be surrounded by blank lines. Tables come in different types. For a table to be parsed, its --table_type option must be on, and the --make_tables option must be true.

ALIGN Table Type

Columns must be separated by two or more spaces (this prevents accidental incorrect recognition of a paragraph where interword spaces happen to line up). If there are two or more rows in a paragraph and all rows share the same set of (two or more) columns, the paragraph is assumed to be a table. For example

    -e  File exists.
    -z  File has zero size.
    -s  File has nonzero size (returns size).

becomes

    <table>
    <tr><td>-e</td><td>File exists.</td></tr>
    <tr><td>-z</td><td>File has zero size.</td></tr>
    <tr><td>-s</td><td>File has nonzero size (returns size).</td></tr>
    </table>

This guesses for each column whether it is intended to be left, centre or right aligned.

BORDER Table Type

This table type has nice borders around it, and will be rendered with a border, like so:

    +---------+---------+
    | Column1 | Column2 |
    +---------+---------+
    | val1    | val2    |
    | val3    | val3    |
    +---------+---------+

The above becomes

    <table border="1">
    <thead><tr><th>Column1</th><th>Column2</th></tr></thead>
    <tbody>
    <tr><td>val1</td><td>val2</td></tr>
    <tr><td>val3</td><td>val3</td></tr>
    </tbody>
    </table>

It can also have an optional caption at the start.

         My Caption
    +---------+---------+
    | Column1 | Column2 |
    +---------+---------+
    | val1    | val2    |
    | val3    | val3    |
    +---------+---------+

PGSQL Table Type

This format of table is what one gets from the output of a Postgresql query.

     Column1 | Column2
    ---------+---------
     val1    | val2
     val3    | val3
    (2 rows)

This can also have an optional caption at the start. This table is also rendered with a border and table-headers like the BORDER type.

DELIM Table Type

This table type is delimited by non-alphanumeric characters, and has to have at least two rows and two columns before it's recognised as a table.

This one is delimited by the '| character:

    | val1  | val2  |
    | val3  | val3  |

But one can use almost any suitable character such as : # $ % + and so on. This is clever enough to figure out what you are using as the delimiter if you have your data set up like a table. Note that the line has to both begin and end with the delimiter, as well as using it to separate values.

This can also have an optional caption at the start.

Create a new object¶

    my $conv = new HTML::TextToHTML();
    my $conv = new HTML::TextToHTML(title=>"Wonderful Things",
                            default_link_dict=>$my_link_file,
      );

Add further arguments¶

    $conv->args(short_line_length=>60,
               preformat_trigger_lines=>4,
               caps_tag=>"strong",
      );

Convert a file¶

    $conv->txt2html(infile=>[$text_file],
                     outfile=>$html_file,
                     title=>"Wonderful Things",
                     mail=>1
      );

Make a pipleline¶

    open(IN, "ls |") or die "could not open!";
    $conv->txt2html(inhandle=>[\*IN],
                     outfile=>'-',
      );