NAME¶

actsync, actsyncd - Synchronize newsgroups

SYNOPSIS¶

actsync [-AkmT] [-b hostid] [-d hostid] [-g max] [-i ignore-file] [-I hostid] [-l hostid] [-n name] [-o format] [-p min-unchanged] [-q hostid] [-s size] [-t hostid] [-v verbosity] [-w seconds] [-z seconds] [host] host

actsyncd config [debug-level [debug-format]]

IN A NUTSHELL¶

These programs permit keeping the list of newsgroups carried by your news server synchronized with an external source.

For instance, you can decide to carry the same newsgroups as another news server or as listed in a file from an external FTP or web site, and then synchronize with the chosen source on a daily basis by running actsyncd in a cron job.

If you only want a subset of newsgroups from that source, it can be parameterized in the actsync.ign configuration file in the pathetc directory.

INN comes with a default configuration for fetching the list of newsgroups from "downloads.isc.org" (also known as "ftp.isc.org"). You can read about the policies used for maintaining that active file at <https://downloads.isc.org/pub/usenet/CONFIG/README>. Just make sure actsync.cfg (the configuration file) and actsync.ign (the synchronization rules) suit your needs, and run:

    actsyncd <pathetc>/actsync.cfg

You'll find more detailed examples of use below in this manual page.

DESCRIPTION¶

actsync permits one to synchronize, compare, or merge two active files. With this utility one may add, change, or remove newsgroups on the local news server to make it similar to the list of the newsgroups found on another system or file. The synchronization need not be exact. Local differences in newsgroup lists may be maintained and preserved. Certain newsgroup errors may be detected and optionally corrected.

There are several reasons to run actsync (or actsyncd) on a periodic basis. Among the reasons are:

• A control message to add, change or remove a newsgroup may fail to reach your site.
• Your control.ctl file may be out of date or incomplete.
• News articles for a new newsgroup may arrive ahead (sometimes days ahead) of the control message.
• Control messages may be forged, thus bypassing the restrictions found in control.ctl unless you set up PGP authentication (and even then, not all hierarchies use PGP authentication).
• Your active file may have been trashed.

If either host argument begins with "." or "/", it is assumed to be the name of a file containing information in the active(5) format. Newsgroup information from a file may be treated as if it was obtained from a host. In this manual page, the host arguments on the command line are called hosts, even though they may be file names.

If a host argument does not begin with "." or "/", it is assumed to be a hostname or Internet address. In this case, actsync will attempt to use the NNTP protocol to obtain a copy of the specified system's active file. If the host argument contains ":", the right side will be considered the port to connect to on the remote system. If no port number is specified, actsync will connect to port "119".

Regardless how the active file information is obtained, the actions of actsync remain the same.

The first host specified is taken to be the local host, the one where any changes would be made. The second host specified is the remote host that is being synchronized with. If only one host is specified, it is assumed to be the remote host to synchronize with, and the local host is assumed to be the default local NNTP server as specified by the NNTPSERVER environment variable or by the server value found in inn.conf.

If either host is specified as "-", the default server will be used for that host, determined as described above.

The newsgroup synchronization, by default, involves all newsgroups found on both hosts. One may also synchronize a subset of newsgroups by directing actsync to ignore certain newsgroups from both systems. Only newsgroups with valid names will be synchronized. To be valid, a newsgroup name must consist only of alphanumeric characters, ".", "+", "-", and "_". One may not have two "." characters in a row. The first character must be alphanumeric, as must any character following ".". The name may not end in a "." character.

The actsyncd daemon provides a convenient interface to configure and run actsync. If a host is not initially reachable, the daemon will retry up to 9 additional times, waiting 6 minutes before each retry. This daemon runs in the foreground, sending output to standard output and standard error. actsyncd runs the getlist(1) utility to obtain a copy of a remote system's active file via its NNTP server, or runs an FTP or web client program (wget or ncftpget if installed and found at configure time, or the simpleftp(1) program shipped with INN) to fetch a list of newsgroups from an external archive (such as <https://downloads.isc.org/pub/usenet/CONFIG/active>; see more about this below). It then uses mod-active(8) to update the active file if there are commands for ctlinnd in its output.

The configuration filename for the daemon is given as a command line argument, usually actsync.cfg in pathetc. This file can contain the following options:

    host=<host>
    path=<path-to-active>
    protocol=<protocol>
    flags=<actsync-options>
    ignore_file=<ignore-file>

The "host=", "flags=", and "ignore_file=" lines are mandatory. Each keyword must start at the beginning of the line, and there may be no whitespace before the "=" character. Blank lines are ignored, as are comment lines that start with "#". Any other lines may produce undefined results.

The <host> setting refers to the second (remote) host parameter to actsync. If <path-to-active> is not provided, <protocol> is considered to be "nntp", and <host> is accessed as a remote NNTP server. If <path-to-active> is provided, <host> is accessed as an FTP or web server depending on the value of <protocol> ("ftp", "http" or "https", defaulting to "ftp" when unset), retrieving the active file at the <path-to-active> absolute path. If the filename ends in ".bz2", ".gz" or ".Z", it will be automatically uncompressed after retrieval. <actsync-options> contains any other flags that you wish to pass to actsync. <ignore-file> names the ignore file used by actsync (the -i option).

Note that one should not include -i or -o options in the "flags=" line; they are automatically taken care of by actsyncd.

One may produce a trial actsyncd run without changing anything on the server by supplying the debug-level argument:

    actsyncd <pathetc>/actsync.cfg 2

The debug-level causes actsyncd to run actsync with a -v debug-level flag (overriding any -v flag on the "flags=" line), not make any changes to the active file, write a new active file to standard output, and write debug messages to standard error. Note that using debug-level is only supported when synchronizing with another news server, not with FTP or HTTP(S).

If the debug-format argument is also given to actsyncd, the data written to standard output will be in "-o debug-format" instead of in "-o a1" format.

INN comes with default values of "downloads.isc.org" for <host> and "/pub/usenet/CONFIG/active.gz" for <path-to-active>:

    host=downloads.isc.org
    path=/pub/usenet/CONFIG/active.gz
    protocol=https
    flags=-v 0 -p 80
    ignore_file=actsync.ign

You can read about the policies used for maintaining that active file at <https://downloads.isc.org/pub/usenet/CONFIG/README>. Consider synchronizing from this file on a daily basis by using a cron job.

OPTIONS¶

actsync takes the following options.

In all of the following options, the hostid parameter takes one of the following values:

    0    neither server
    1    local default server
    2    remote server
    12   both servers
    21   both servers

In other words, "1" refers to the local host (the first host argument on the actsync command line) and "2" refers to the remote host (the second host argument, or the only one if only one is given).

-A

actsync tries to authenticate using the username and password information in passwd.nntp(5) before issuing the LIST command.

-b hostid

This flag causes actsync to ignore for synchronization purposes newsgroups with "bork.bork.bork"-style names (newsgroups whose last 3 components are identical). For example, the following newsgroups have bork-style names:

    alt.helms.dork.dork.dork
    alt.auto.accident.sue.sue.sue
    alt.election.vote.vote.vote
    

The default is "-b 0"; no newsgroups are ignored because of bork-style names.

-d hostid

This flag causes actsync to ignore newsgroups that have all numeric path components. For example, the following newsgroups have numeric path components:

    alt.prime.chongo.23209
    391581.times.2.to_the.216193.power.-1
    99.bottles.of.treacle.on.the.wall
    linfield.class.envio_bio.101.d
    

The newsgroups directory of a newsgroup with a all numeric component could conflict with an article from another group if stored using the tradspool storage method; see storage.conf(5). For example, the directory for the first newsgroup listed above is the same path as article number 23209 from the newsgroup:

    alt.prime.chongo
    

The default is "-d 0"; all numeric newsgroups from both hosts will be processed.

-g max

Ignore any newsgroup with more than max levels. For example, "-g 6" would ignore:

    alt.feinstien.votes.to.trash.freedom.of.speech
    alt.senator.exon.enemy.of.the.internet
    alt.crypto.export.laws.dumb.dumb.dumb
    

but would not ignore:

    alt.feinstien.acts.like.a.republican
    alt.exon.amendment
    alt.crypto.export.laws
    

If max is "0", then the max level feature is disabled.

By default, the max level feature is disabled.

-i ignore-file

The ignore-file, usually actsync.ign in pathetc, allows one to have a fine degree of control over which newsgroups are ignored. It contains a set of rules that specifies which newsgroups will be checked and which will be ignored.

By default, these rules apply to both hosts. This can be modified by using the -I flag.

Blank lines and text after a "#" are considered comments and are ignored.

Rule lines consist of tokens separated by whitespace. Rule lines may be one of two forms:

    c <newsgroup> [<type> ...]
    i <newsgroup> [<type> ...]
    

If the rule begins with a "c", the rule requests certain newsgroups to be checked. If the rule begins with an "i", the rule requests certain newsgroups to be ignored. The <newsgroup> field may be a specific newsgroup, or a uwildmat pattern.

If one or more <type>s are specified, then the rule applies to the newsgroup only if it is of the specified type. Types refer to the 4th field of the active file; that is, a type may be one of:

    y
    n
    m
    j
    x
    =group.name
    

Unlike active files, the "group.name" in an alias type may be a newsgroup name or a uwildmat pattern. Also, "=" is equivalent to "=*".

On each rule line, no pattern type may be repeated. For example, one may not have more than one type that begins with "=", per line. However, one may achieve an effect equivalent to using multiple "=" types by using multiple rule lines affecting the same newsgroup.

By default, all newsgroups are candidates to be checked. If no ignore-file is specified, or if the ignore file contains no rule lines, all newsgroups will be checked. If an ignore file is used, each newsgroup in turn is checked against the ignore file. If multiple lines match a given newsgroup, the last line in the ignore file is used.

For example, consider the following ignore file lines:

    i *.general
    c *.general m
    i nsa.general
    

The newsgroups ba.general and mod.general would be synchronized if moderated and ignored if not moderated. The newsgroup nsa.general would be ignored regardless of moderation status. All newsgroups not matching *.general would be synchronized by default.

-I hostid

This flag restricts which hosts are affected by the ignore file. This flag may be useful in conjunction with the -m flag. For example:

    actsync -i actsync.ign -I 2 -m host1 host2
    

will keep all newsgroups currently on host1. It will also only compare host1 groups with non-ignored newsgroups from host2.

The default is "-I 12"; newsgroups from both hosts are ignored per the file specified with -i.

-k

By default, any newsgroup on the local host that has an invalid name will be considered for removal. This causes actsync simply ignore such newsgroups. This flag, used in combination with -m, will prevent any newsgroup from being scheduled for removal.

-l hostid

This flag causes problem newsgroups of type "=" to be considered as errors. Newsgroups of type "=" are newsgroups active entries that have a fourth field that begins with "="; i.e., newsgroups that are aliased to other newsgroups. A problem newsgroup is one for which one of the following is true:

-m

Merge newsgroups instead of sync. By default, if a newsgroup exists on the local host but not the remote, it will be scheduled to be removed. This flag disables this process, permitting newsgroups unique to the local host to be kept.

-n name

Depending on -o, the ctlinnd(8) command may be used to create newsgroups as necessary. When this is done, the default creator name used is "actsync". This flag changes the creator name to name.

-o format

Determine the output or action format of this utility. format may be one of:

-p min-unchanged

By default, the actsync utility has safeguards against performing massive changes. If fewer than min-unchanged percent of the non-ignored lines from the local host remain unchanged, no actions (output, execution, etc.) are performed and actsync exits with a non-zero exit status. The min-unchanged value may be a floating point value such as "66.667".

A change is a local newsgroup line that was removed, added, changed, or found to be in error. Changing the 2nd or 3rd active fields via "-o ak" or "-o aK" are not considered changes by -p.

To force actsync to accept any amount of change, use the "-p 0" option. To force actsync to reject any changes, use the "-p 100" option.

Care should be taken when producing active(5) formatted output. One should check to be sure that actsync exited with a zero status prior to using such output. Also one should realize that such output will not contain lines ignored due to -i even if "-p 100" is used.

By default, 96% of the lines not ignored in the first host argument on the actsync command line must be unchanged. That is, by default, "-p 96" is assumed.

-q hostid

By default, all newsgroup errors are reported on standard error. This flag quiets errors from the specified hostid.

-s size

If size is greater than 0, then ignore newsgroups with names longer than size and ignore newsgroups aliased (by following "=" chains) to names longer than size. Length checking is performed on both the local and remote hosts.

By default, size is "0" and thus no length checking is performed.

-t hostid

Ignore improper newsgroups consisting of only a top component from the specified hostid. The following newsgroups are considered proper newsgroups despite top only names and therefore are exempt from this flag:

    control
    general
    junk
    test
    to
    

For example, the following newsgroup names are improper because they only contain a top level component:

    dole_for_pres
    dos
    microsoft
    windows95
    

The default is "-t 2"; that is, all improper top-level-only newsgroups from the remote host are ignored.

-T

This flag causes newsgroups on the remote host in new hierarchies to be ignored. Normally a newsgroup which only exists on the remote host, chongo.was.here for example, is created on the local host. However, if this flag is given and the local host does not have any other newsgroups in the same hierarchy (chongo.* in this case), the newsgroup in question will be ignored and will not be created on the local host.

-v verbosity

By default, actsync is not verbose. This flag controls the verbosity level as follows:

-w seconds

If "-o x" or "-o xi" is selected, ctlinnd will wait seconds seconds before timing out. The default value is "-w 30".

-z seconds

If "-o x" is selected, actsync will pause for seconds seconds before each command is executed. This helps prevent innd from being busied-out if a large number of ctlinnd commands are needed. One can entirely disable this sleeping by using "-z 0".

By default, actsync will pause for "4" seconds before each command is executed if "-o x" is selected.

EXAMPLES¶

Determine the difference (but don't change anything) between your newsgroup set and the one of another news server:

    actsync news.server.com

Same as above, with full debug and progress reports:

    actsync -v 4 news.server.com

Force a site to have the same newsgroups as some other site:

    actsync -o x master

This may be useful to sync a slave site to its master, or to sync internal site to a gateway.

Compare your site with news.server.com, disregarding local groups and certain local differences with it. Produce a report if any differences were encountered:

    actsync -v 2 -i actsync.ign news.server.com

where actsync.ign contains:

    # Don't compare to.* groups as they will differ.
    #
    i       to.*
    # These are our local groups that nobody else
    # (should) carry.  So ignore them for the sake
    # of the compare.
    #
    i       nsa.*
    # These groups are local favorites, so keep them
    # even if news.server.com does not carry them.
    #
    i       ca.dump.bob.dorman
    i       ca.keep.bob.dorman
    i       alt.tv.dinosaurs.barney.die.die.die
    i       alt.tv.dinosaurs.barney.love.love.love
    i       alt.sounds.*    =alt.binaries.sounds.*

To interactively sync against news.server.com, using the same ignore file:

    actsync -o xi -v 2 -i actsync.ign news.server.com

Based on newsgroups that you decided to keep, one could make changes to the actsync.ign file:

    # Don't compare to.* groups as they will differ.
    #
    i       to.*
    # These are our local groups that nobody else
    # (should) carry.  So ignore them for the sake
    # of the compare.
    #
    i       nsa.*
    # These groups are local favorites, so keep them
    # even if news.server.com does not carry them.
    #
    i       ca.dump.bob.dorman
    i       alt.tv.dinosaurs.barney.die.die.die
    i       alt.sounds.*    =alt.binaries.sounds.*
    # Don't sync test groups, except for ones that are
    # moderated or that are under the gnu hierarchy.
    #
    i       *.test
    c       *.test m        # check moderated test groups
    c       gnu.*.test
    c       gnu.test        # just in case it ever exists

Automatic processing may be set up by using the following actsync.cfg file:

    # Host to sync off of (host2).
    host=news.server.com
    # Location of the ignore file.
    ignore_file=<pathetc in inn.conf>/actsync.ign
    # actsync(8) flags
    #
    # Automatic execs, report if something was done,
    # otherwise don't say anything, don't report
    # news.server.com active file problems, just ignore
    # the affected entries.
    flags=-o x -v 2 -q 2

and then by running actsyncd with the path to the config file:

    actsyncd <pathetc>/actsync.cfg

The command

    actsyncd <pathetc>/actsync.cfg 4 >cmd.log 2>dbg.log

will operate in debug mode, not change the active file, write ctlinnd style commands to cmd.log, and write debug statements to dbg.log. (Note that using debug-level is only supported when synchronizing with another news server, not with FTP or HTTP(S).)

To check only the major hierarchies against news.server.com, use the following actsync.ign file:

    # By default, ignore everything.
    #
    i       *
    # Check the major groups.
    #
    c       alt.*
    c       comp.*
    c       gnu.*
    c       humanities.*
    c       misc.*
    c       news.*
    c       rec.*
    c       sci.*
    c       soc.*
    c       talk.*

and the command:

    actsync -i actsync.ign news.server.com

To determine the differences between your old active and your current default server:

    actsync <pathetc>/active.old -

To report but not fix any newsgroup problems with the current active file:

    actsync - -

To detect any newsgroup errors on your local server, and to remove any *.bork.bork.bork-style silly newsgroup names:

    actsync -b 2 - -

The active file produced by:

    actsync <flags> -o x erehwon.honey.edu

is effectively the same as the active file produced by:

    cd <pathdb>
    ctlinnd pause 'running actsync'
    rm -f active.new
    actsync <flags> -o a1 erehwon.honey.edu > active.new
    rm -f active.old
    ln active active.old
    mv active.new active
    ctlinnd reload active 'running actsync'
    ctlinnd go 'running actsync'

It should be noted that the final method above, pausing the server and simply replacing the active file, may be faster if you are making lots of changes.

FILES¶

pathbin/actsync

The C program itself used to synchronize, compare, or merge two active files.

pathbin/actsyncd

The Shell daemon which provides a convenient interface to configure and run actsync.

pathetc/actsync.cfg

The configuration file which specifies the settings to use.

pathetc/actsync.ign

The ignore file which contains a set of synchronization rules that specifies which newsgroups will be checked and which will be ignored.

CAUTION¶

Careless use of this tool may result in the unintended addition, change, or removal of newsgroups. You should avoid using the "x" output format until you are sure it will do what you want.

BUGS¶

If a newsgroup appears multiple times, actsync will treat all copies as errors. However, if the group is marked for removal, only one rmgroup will be issued.

HISTORY¶

Written by Landon Curt Noll <chongo@toad.com> for InterNetNews. Updated to support FTP fetching by David Lawrence <tale@isc.org>, and to HTTP(S) fetching by Julien Élie. Converted to POD by Russ Allbery <eagle@eyrie.org>.

By Landon Curt Noll <chongo@toad.com> (chongo was here /\../\).

Copyright (c) Landon Curt Noll, 1993. All rights reserved.

Permission to use and modify is hereby granted so long as this notice remains. Use at your own risk. No warranty is implied.

SEE ALSO¶

active(5), ctlinnd(8), getlist(1), inn.conf(5), libinn_uwildmat(3), mod-active(8), passwd.nntp(5), simpleftp(1).