update from 2.5.1 -> 2.5.2

Vendor update message (excerpt):

Many thanks to Julien �LIE for preparing this release.

Special notes:

The way checkpoints are handled by innreport for innd and innfeed has totally changed to provide more accurate daily statistics. The first Usenet report after an upgrade to INN 2.5.2 will probably contain incorrect statistics for incoming and outgoing articles because the beginning of the log files that will be used was generated by a previous version of INN.

A new version of innreport.conf is shipped with INN 2.5.2 but, in order to preserve any local changes, will not be automatically installed with make update. The changes are minor and not mandatory for the upgrade.

Changes from 2.5.1 to 2.5.2

    * Julien Elie has implemented in innd the new version of the NNTP protocol described in RFC 3977, RFC 4643 and RFC 4644, and innd now recognizes the CAPABILITIES command. Despite these standards, three commands (IHAVE, CHECK and TAKETHIS) will continue, for interoperability reasons, to return a reject code (respectively 435, 438, and 439) when the command contains a syntax error instead of 501. The mandatory username argument for authenticated peers is not enforced in INN 2.5.2 but will be be enforced by INN 2.6.0 when it is released.

Major improvements are:

    * innd now has a decent parser for NNTP commands. The parser is more correct (commands like "IHAVEZ<>", without a space between the command and its argument, are no longer valid) and allows leading and trailing whitespaces in commands. innd also now checks the length of the NNTP command sent by the client. If the command contains more than 512 bytes (or 497 bytes for an argument), an error is returned and the command is discarded. After ten unrecognized commands, innd closes the connection with the appropriate code (400 instead of 500).
    * The output of the HELP command specifies the arguments expected by NNTP commands, similar to nnrpd's HELP command.
    * LIST ACTIVE, LIST ACTIVE.TIMES and LIST NEWSGROUPS now allow an optional wildmat argument to restrict the results of those commands to specific newsgroups.
    * When using HEAD or STAT with an article number or a range, 412 (no group selected) is now returned instead of 501 (syntax error).
    * Jeffrey M. Vinocur has implemented support in both innd and nnrpd for whitespace in usernames/passwords provided with AUTHINFO USER/PASS. They were previously treated as invalid arguments or incorrectly parsed. innd and nnrpd now treat everything after the first whitespace character following AUTHINFO USER/PASS, up to, but not including, the final CRLF, as the username/password, in conformity with RFC 4643.
    * The syntax of message-IDs is now based on RFC 5536 (USEFOR) instead of RFC 1036. The major change is that quoted-pairs have been removed from the syntax.
    * The Perl and Python filters for innd now check the message-ID of articles arriving through TAKETHIS. Only CHECK and IHAVE commands previously used them.
    * Case-insensitive matches are now used for distributions, path identities, IMAP commands, header names, and control commands. (Newsgroups are still matched case-sensitively.) Message-IDs are case-sensitively matched, except for history hashes.
    * The new Archive:, Archive-At:, Comments:, and Summary: header fields defined in RFC 5064 and RFC 5536 can be used in innd filters. nnrpd now checks at injection time that an article does not contain an Injection-Info: header, that an Injection-Date: header (if provided) is valid, and that the Path: header does not contain ".POSTED". Note that INN does not yet generate these two injection fields or include the new Path: header field ".POSTED" keyword. These new features will be in the next major release of INN.
    * LIST SUBSCRIPTIONS now accepts an optional wildmat argument to restrict the results of this command to specific newsgroups.
    * nnrpd now supports a new LIST variant named COUNTS. LIST COUNTS is a combination of LIST ACTIVE and GROUP. It returns the same result as LIST ACTIVE except that the number of articles in a newsgroup is inserted before its status.
    * A new flag has been added to newsfeeds entries: "Aj", when present, says to feed articles accepted and filed in "junk" (due to *wanttrash*) to peers based on their newsfeeds feed patterns applied to the Newsgroups: header as though the article were accepted and all those groups were locally carried. This is useful if you want to run INN with a minimal active file and propagate all posts. Thanks to Andrew Gierth for the patch.
    * A new parameter has been added to inn.conf: *logtrash* defines whether a line for articles posted to groups not locally carried by the news server should be added in the news log file to report unwanted newsgroups. The default is true but it can be useful to set it to false (especially when *wanttrash* is also used).
    * The procbatchdir keyword has been added to news.daily to specify the backlog directory of innfeed. This is useful when several instances of innfeed are running or when its configuration file is not the default one.
    * sm now supports a new flag, -c, which shows a decoded form of the storage API token. This was previously done by the contrib showtoken script developed by Olaf Titz and Marco d'Itri.
    * The O flag in newsfeeds now relies on the contents of the Injection-Info: header field if it is present to determine the origin of an article. It falls back on X-Trace: if there is no Injection-Info: header field.
    * A new "unsigned long" type bas been added to the configuration parser. It will properly warn the news administrator when a variable supposed to be positive contains a negative integer. It will prevent INN from crashing due to misconfiguration at several places where it did not expect negative values.
    * innxbatch and innxmit now recognize the new 403 code introduced by RFC 3977 for a problem preventing the requested action from being taken.
    * HDR and OVER commands now return the correct 423 code (instead of 420) when the current article number is used but the article no longer exists.
    * actsync, inews, innxbatch, innxmit, nntpget and rnews can now authenticate to news servers which only expect a username, without password, conforming to RFC 4643.
    * The keyword generation code now generates a Keywords: header only if the original article does not already have one. The generated Keywords: header no longer begins with a comma. If keyword generation is set to true in inn.conf but the Keywords: header is not stored in the overview, the news administrator is warned and keyword generation deactivated, since it exists only to populate the overview data.
    * Two segfaults in keyword generation were fixed. The first occurred when an article already had a Keywords: header longer than the *keylimit* parameter. The second was caused by a possible invalid pointer beyond the newly allocated Keywords: header.
    * Fixed innd handling of empty lines. innd was not properly discarding an empty command and was closing the connection when it received only whitespace in a command.
    * Fixed a bug in how innd responded to reader commands when readers were not allowed. A superfluous blank line was sent in its response.
    * Fixed a bug in innd's response to TAKETHIS when authentication is required. Previously, 480 code was returned immediately without accepting the multi-line data block first, which broke synchronization in the NNTP protocol.
    * Fixed a bug in recognizing the article terminator when empty articles were fed to innd via IHAVE or TAKETHIS, leading to treating subsequent NNTP commands as part of the article.
    * When innd could not provide information for LIST ACTIVE.TIMES and LIST NEWSGROUPS, it was returning an invalid error message without a response code. The proper 503 answer code is now returned.
    * When an unauthenticated user tried to post an article, nnrpd replied 440 (posting not allowed) instead of the correct 480 (authentication required) response if the user might be able to post after authentication. Thanks to Daniel Weber for the bug report.
    * Fixed a bug in both innd and nnrpd answers to LIST commands where the output was not checked for valid dot stuffing.
    * Fixed a bug leading to junked non-control articles being sent to control-only feeds, and also fixed handling of poisoned control groups. Thanks to Andrew Gierth for the patch.
    * Fixed a bug in innreport leading to incorrect summing of innd stats when *hostname* was set to an IPv6 address instead of a fully-qualified domain name. Thanks to Petr Novopashenniy for the bug report.
    * Changed how innreport uses innd and innfeed checkpoint messages. Previously, connections held open for multiple days led to skewed and incorrect statistics on how many articles had been received or sent. The count is now more accurate and, for each connection of a feed, only depends on *incominglogfrequency* in inn.conf and *stats-period* in innfeed.conf.
    * Fixed a bug in nnrpd Perl filter: a header field whose name begins with the name of a standardized header field was not properly handled.
    * Fixed a bug in how innd was parsing Message-ID: and Supersedes: headers which contained trailing whitespace. The article was corrupted by an unexpected "\r" in the middle of the header. nnrpd now checks the syntax of the Message-ID: header field, if present.
    * Fixed various bugs in how leading whitespace was treated in headers. The HDR, XHDR and XPAT commands were not properly showing leading whitespace in header values. Lone "\n" and "\r" characters are now changed into spaces and "\r\n" is just removed. archive, makehistory, and tdx-util now keep leading whitespace in headers when generating overview data, and archive now changes "\n" (when not preceded by "\r") into a space when generating overview data.
    * Fixed a bug in the generation of overview data which may corrupt previously generated overview data when a pseudo Xref: header field is injected in an extra overview field.
    * Fixed a bug in the parsing of the *ovgrouppat* wildmat in inn.conf that prevented overview data from being generated when poisoned groups were specified but a latter sub-pattern matched the group. A uwildmat expression is now correctly handled, and a potential segfault has been fixed. Thanks to Dieter Stussy for the bug report.
    * Fixed a bug when HDR, XHDR and XPAT were used when *virtualhost* was set to true in readers.conf. The Xref: header of articles posted to only one newsgroup appeared empty.
    * Fixed a bug in tdx-util in parsing empty overview fields when called with -A or -F.
    * Fixed a bug in cvtbatch, which was returning only the size of the headers of an article when the "b" parameter was used with the -w flag. It now correctly returns the size of the whole article, which is what "b" was documented to do. cvtbatch also has a new "t" parameter, which can be used with the -w flag to retrieve the arrival time of an article.
    * Fixed a bug in how mailpost handles cross-posting feature. It was not properly detaching from sendmail. Thanks to Harald Dunkel for the patch.
    * Fixed a bug in the newsfeeds C flag: the count of followup groups was one less than the real number. When the value of the Followup-To: header field is "poster", it is no longer considered to be a followup. Thanks to Dieter Stussy for the patch.
    * When using tradindexed, the overview data for a cancelled article is now immediately removed from the overview. Thanks to Lars Magne Ingebrigtsen for the patch.
    * batcher has not supported the retrieval of an article with its file name for a long time. The -S flag has therefore been removed.
    * inews no longer rejects articles that contain more than 50 header fields. Thanks to Torsten Jerzembeck for the bug report.
    * news.daily no longer sends superfluous mails when the nomail keyword is given. Mail is only sent when there is real output. Previously, there would always be headings and empty lines left over from the structuring of the full report, which are now ommitted. Also, the output of programs executed with postexec is now included in the regular mail. Thanks to Florian Schlichting for the patch.
    * innconfval no longer maps NULL string or list values to an empty string or list and instead maps them to undefined values. This fixes an issue reported by Kamil Jonca: nnrpd was inserting an empty Organization: header when the *organization* parameter in inn.conf was unset.
    * Other minor bug fixes and documentation improvements.


(spz)