regress/infra-unittests: clean up test code for subst.mk

There had been unnecessary variations in the code.

(rillig)

regress/infra-unittests: replace remaining double with single quotes

(rillig)

regress/infra-unittests: replace double quotes with single quotes

Strings in single quotes are simpler to understand and also need fewer
pixels on the screen.

(rillig)

mk/subst.mk: fix edge case in detection of identity substitutions

In a basic regular expression, a dollar-sign only means end-of-string if
it appears at the end of the pattern, or (at the choice of the
implementation) at the end of a \(...\) subexpression.

This affects the package converters/help2man that uses a regular
expression containing a dollar in a non-final position.  This regular
expression had not been detected as an identity substitution even though
it is one.

(rillig)

regress/infra-unittests: add tests for identity substitutions

(rillig)

regress/infra-unittests: run tests in a temporary directory

This isolates the tests from the test infrastructure and allows the test
infrastructure to create arbitrary files for its own purpose without
affecting any of the tests.

(rillig)

www/thunderbird60: skip SUBST block for Python interpreter

The mentioned file only contains <(PYTHON), but not <(python).

(rillig)

hs-*: add PLIST files

These PLIST files have been autogenerated by mk/haskell.mk using
HS_UPDATE_PLIST=yes during a bulk build.  They will help to track changes
to the packages.  The Haskell packages didn't have PLIST files because
their paths contained package hashes.  These hashes are now determined by
mk/haskell.mk, which makes it easy to generate easy to read PLIST files.

(rillig)

regress/tools-platform: demonstrate bug in nbsed-20120308

(rillig)

devel/hs-base-orphans: add PLIST

(rillig)

devel/swig2: skip check for unknown configure options

(rillig)

lang/mono6: fix build in strict environment

(rillig)

mk/haskell.mk: allow to generate PLIST files

Since GHC 7.10 or 7.8, the Haskell packages are installed in directories
whose name contains the package hash.  This made it harder to predict the
exact pathname.  Havin the exact pathnames in the PLIST file is the
ideal, it also helps to record the general structure of the installed
files to see whether some file unexpectedly appear or disappear.

To enable this for Haskell packages, the various base directories are
replaced with placeholders during print-PLIST.  These placeholders are
translated back to their respective paths when the +PLIST is generated
from the PLIST in the package directory.

Except for 2 packages, all Haskell packages in main pkgsrc had their
package PLIST file removed.  To help in adding them back, the pkgsrc
developer can set HS_UPDATE_PLIST=yes in mk.conf, which will generate the
PLIST directly into ${PKGDIR}/PLIST upon installation.

Most packages in pkgsrc-wip still have their old PLIST, and these are
migrated automatically as well.

(rillig)

security/libssh2: remove unknown configure options

(rillig)

devel/sfslite: suppress USE_TOOLS+=perl warning

(rillig)

devel/subversion-base: suppress USE_TOOLS+=perl warning

(rillig)

devel/swig3: skip check for unknown configure options

(rillig)

graphics/goocanvas2: suppress USE_TOOLS+=perl warning

(rillig)

graphics/libpano13: suppress USE_TOOLS+=perl warning

(rillig)

mk/tools: don't create empty symlink for wrong tool definition

This case can only happen in the following special case:

TOOLS_CREATE+= asdf
TOOLS_PATH.asdf= # empty

If there is a lonely TOOLS_CREATE without a corresponding TOOLS_PATH, it
defaults to ${FALSE} and thus doesn't trigger this code.

(rillig)

multimedia/gst-plugins0.10-base: allow SUBST block to be a no-op

(rillig)

multimedia/gst-plugins1-base: sort "used by" lines

(rillig)

net/bind914: remove unknown configure option

(rillig)

net/isc-dhcp4: skip check for unknown configure options

(rillig)

net/ns: suppress USE_TOOLS+=perl warning

(rillig)

net/proftpd: skip check for unknown configure options

(rillig)

net/wide-dhcpv6: clean up SUBST_FILES

(rillig)

news/suck: suppress USE_TOOLS+=perl warning

(rillig)

parallel/ganglia-monitor-core: clean up SUBST_FILES

The package has switched to autoconf's .in files to get the paths into
the source code.

(rillig)

security/p5-Authen-PAM: use perl as a tool

This is not only a Perl module, it also runs the Perl interpreter
directly.

(rillig)

sysutils/ups-nut-*: suppress USE_TOOLS+=perl warning

(rillig)

textproc/ruby-nqxml: allow to skip rinstall SUBST block

(rillig)

wm/sawfish: fix location of main.jl.in

(rillig)

www/nostromo: remove outdated SUBST_MESSAGE

(rillig)

www/ruby-div: fix strict build with SUBST_NOOP_OK=no

(rillig)

x11/xpmroot: suppress USE_TOOLS+=perl warning

(rillig)

x11/qt4-libs: suppress USE_TOOLS+=perl warning

(rillig)

databases/mariadb55-client: clean up REPLACE_PERL

Some of the mentioned files didn't exist anymore.

(rillig)

mark perl as not needed

Before, it had the same effect but generate a USE_TOOLS+=perl warning.

(rillig)

cross/cc65: remove unnecessary dependency on Perl

(rillig)

graphics/graphwiz: remove unknown configure option

(rillig)

lang/quickjs: install documentation and examples

(rillig)

doc: Updated lang/quickjs to 20200412

(rillig)

lang/quickjs: update to 2020-04-12

Changes since 2020-01-19:

2020-04-12:

- added cross realm support
- added AggregateError and Promise.any
- added env, uid and gid options in os.exec()
- misc bug fixes

2020-03-16:

- reworked error handling in std and os libraries: suppressed I/O
  exceptions in std FILE functions and return a positive errno value
  when it is explicit
- output exception messages to stderr
- added std.loadFile(), std.strerror(), std.FILE.prototype.tello()
- added JS_GetRuntimeOpaque(), JS_SetRuntimeOpaque(), JS_NewUint32()
- updated to Unicode 13.0.0
- misc bug fixes

(rillig)

mk/scripts: fix a few of the HTML injections in README generation

These are needed for the djb-errno-hack option, which includes the <>
characters.

(rillig)

mail/mailfront: fix PKG_SUGGESTED_OPTIONS

The package-specific options.mk is included by djbware.mk and must
therefore not be included by the package Makefile itself.  This fixes the
PKG_SUPPORTED_OPTIONS displayed by show-options.

Found by making the package-settable variables in mk/bsd.options.mk
read-only.

(rillig)

mk: move locking.mk from internal to misc

Locking is not really internal since it has user-settable variables.

(rillig)

mk/tools/perl.mk: allow to silence the USE_TOOLS+=perl warning

Packages that don't declare USE_TOOLS+=perl and whose configure script
invokes perl produce a warning.

Usually warnings are ignored, but they can also be configured as errors,
for example during a strict bulk build.  In this situation it is
necessary to override the default behavior of the perl tool to fail
silently.  Up to now, defining both TOOLS_BROKEN+=perl and
TOOLS_FAIL+=perl produced a duplicate make target.

To handle this situation, let TOOLS_BROKEN+=perl take precedence over
TOOLS_FAIL+=perl.  This is much easier than finding out in each case how
to disable the perl check in the configure script, which is most often
done by adding any of the following to CONFIGURE_ENV: PERL=#none,
ac_cv_prog_PERL=#none, ac_cv_path_PERL=#none.

(rillig)

devel/libatomic_ops: fix layout of MESSAGE file

(rillig)

regress/buildlink-transform: clean up code

(rillig)

regress/buildlink-transform: repair most of the regression test

This test is for the old compiler wrapper.
Using it with the cwrappers from 2014 makes several test cases fail.

(rillig)

regress/replace-interpreter: fix package installation

(rillig)

regress/make-shell: add missing variable for the shell test

(rillig)

regress/ignore-tools: remove

This regression test has been broken for a long time now.  It tested the
_TOOLS_OPSYS_INCOMPAT variable, which does not exist anymore.

(rillig)

mk/tools/create.mk: make tool wrappers smaller

There is no need to include the comments from the shquote function.

(rillig)

mk/tools/bsd.tools.mk: include _TOOLS_VARNAME in show-all-tools

This information is useful for getting the variable name that corresponds
to a tool.  In most cases this is just the uppercase name of the tool,
but there are exceptions like ${SETENV} for env, ${HOSTNAME_CMD} for
hostname.

(rillig)

mk/tools/create.mk: add new variable TOOLS_ALWAYS_WRAP for debugging

(rillig)

security/py-ecdsa: remove unnecessary REPLACE_PYTHON

These files are not installed as executables, therefore they don't need a
script interpreter. The paths to the files were wrong anyway.

(rillig)

sysutils/burp: fix path to configuration files

(rillig)

sysutils/py-google-compute-engine: allow no-op SUBST block

It looks wrong that the paths in constants.py depend on the operating
system, but that's not caused by the latest SUBST changes, so I'm leaving
that as-is.  Someone familiar with this package should have a look.

(rillig)

www/viewvc: remove no-op file from SUBST_FILES

(rillig)

www/w3c-httpd: remove duplicate file from SUBST_FILES

(rillig)

mk/check/check-portability.mk: improve documentation

(rillig)

doc: Updated pkgtools/pkglint to 20.1.5

(rillig)

pkgtools/pkglint: update to 20.1.5

Changes since 20.1.4:

No more wrong warnings about the Solaris /bin/sh. These warnings had been
there for 14 years, preventing pkgsrc developers from using the $$(...)
command substitution and negation in shell conditions.

https://mail-index.netbsd.org/pkgsrc-changes/2020/05/01/msg212194.html

(rillig)

devel/go-protobuf: fix shell portability

(rillig)

net/go-dns: fix shell portability

(rillig)

sysutils/beats: skip portability check on unused parts

(rillig)

emulators/aranym: skip portability check

On macOS, /bin/sh is a bash.

(rillig)

security/aide: skip portability check

(rillig)

regress/replace-interpreter: ensure order of targets

(rillig)

lang/gcc9: fix build in strict environment

(rillig)

lang/gcc49: fix build in strict environment

(rillig)

lang/gcc44: fix build in strict environment

(rillig)

cross/powerpc-none-elf-gcc: fix build in strict environment

(rillig)

cross/pdp11-aout-gcc: fix build in strict environment

(rillig)

cross/h8300-elf-gcc: fix typo from previous commit

(rillig)

cross/or1k-none-elf-gcc: fix build in strict environment

(rillig)

cross/h8300-elf-gcc: fix build in strict environment

(rillig)

cross/arm-none-eabi-gdb: suppress shell portability

(rillig)

regress/infra-unittests: parse command line options [-kv] [-f filter]

(rillig)

regress/infra-unittests: indent here-documents

(rillig)

lang/mono6: fix shell portability

(rillig)

mk/pkginstall: fix output of show-all-pkginstall

(rillig)

lang/mono6: clean up CONF_FILES

(rillig)

lang/mono6: fix path to Python interpreter

(rillig)

lang/sbcl: remove no-op SUBST block

There is no file that contains the word "tar" anymore.

(rillig)

lang/rust: remove no-op file from SUBST_FILES

(rillig)

print/py-weasyprint: remove no-op SUBST block

The word PYVERSSUFFIX does not appear in setup.cfg.

(rillig)

www/davical: remove no-op SUBST block

The word EGDIRCONFIG does not appear anywhere else.

(rillig)

devel/ocaml-findlib: remove no-op SUBST block

There is no file called *reg* anymore.

(rillig)

lang/mono: fix hard-coded X11BASE

(rillig)

security/isakmpd: remove no-op files from SUBST_FILES

(rillig)

graphics/irrlicht: remove no-op SUBST block

(rillig)

www/horde: remove no-op file from SUBST_FILES

(rillig)

audio/fasttracker2: remove no-op SUBST block

(rillig)

misc/ec2-ami-tools: remove no-op file from SUBST_FILES

(rillig)

games/ccgo: remove no-op file from SUBST_FILES

(rillig)

misc/brs: fix hard-coded paths in patch

(rillig)

lang/ruby27-base: fix path to power_assert/console

(rillig)

doc: Updated pkgtools/pkglint to 20.1.4

(rillig)

pkgtools/pkglint: update to 20.1.4

Changes since 20.1.3:

For patches that patch a single file, the filename of the patch should
correspond to the patched file. There are a few different naming schemes
in action, therefore the check is relatively loose. Patches that are
called patch-[a-z][a-z] continue to be allowed for historic reasons.
Patches that are called patch-CVE-* are also allowed.

The entries in doc/CHANGES-* are checked for consistency. For example,
it doesn't make sense to add a package twice or "update" a package from
version 1.0 to version 1.0. All version numbers in these entries must
be valid pkgsrc versions, i.e. start with a digit and only use
characters from -.0-9A-Z_a-z.

(rillig)

doc/CHANGES-2012: remove duplicate entries

(rillig)

doc/CHANGES-2019: clean up redundant entries

An update from 1.23 to 1.23 is not really an update.

(rillig)

net/knot: remove nonexistent file from REPLACE_PYTHON

(rillig)

devel/cscope: remove unknown configure arguments

(rillig)

lang/llvm: remove nonexistent file from REPLACE_PYTHON

(rillig)

net/yaz: remove unknown configure option

The configure script does not contain "openssl" anywhere.

(rillig)

lang/clang: fix detection of libstdc++

In the previous commit, I had missed a negation in the empty(...)
condition.

PKGREVISION++ to notify clang users of the fix.

https://mail-index.netbsd.org/pkgsrc-users/2020/05/07/msg031132.html

(rillig)

www/ap2-python: omit unknown configure option

This fixes the build with GNU_CONFIGURE_STRICT=yes.

(rillig)

mk/subst.mk: allow identity substitutions with escaped dots

This fixes the build of converters/help2man in SUBST_NOOP_OK=no mode.

(rillig)

devel/libthrift: fix bmake warnings about duplicate perl tool

Adding perl to TOOLS_BROKEN leads to these warnings:

make: "/usr/pkgsrc/mk/tools/create.mk" line 142:
warning: duplicate script for target ".../.tools/bin/perl" ignored

Since perl is not needed to build this package, disable it in the
configure script.

(rillig)

graphics/OpenRM: skip shell portability check

(rillig)

print/LPRng: skip shell portability check for RedHat files

They use the [[ keyword but are not needed for the pkgsrc build.

(rillig)

emulators/BasiliskII: ignore bashism on macOS

(rillig)

regress/infra-unittests: add tests for portability checks

Files like Makefile.am and configure.ac are usually not used during a
build, therefore there's no point in checking these for shell portability
issues.

(rillig)

mk/check/check-portability: opt-in for newer portability checks

There are a few portability checks that have been existing for years.
Later additions need an opt-in phase to avoid breaking existing usages.

https://mail-index.netbsd.org/tech-pkg/2020/05/04/msg023084.html

(rillig)

mk/check/check-portability: fix error message when no patches are found

(rillig)

mk/check/check-portability: skip X.in if X is patched

A commonly occuring scenario is that a package patches the configure
script, but that the corresponding configure.in contains shell code that
is not portable.  In cases like these, configure.in is typically not used
during the build, therefore there is no need to check it for portability.

This also applies to all other combinations where a file is patched and
the corresponding file.in contains unportable shell code.

(rillig)

regress/check-portability: fix regression test for added line numbers

The line numbers were added to the output in check-portability.awk r1.12.

(rillig)

converters/help2man: remove redundant condition around SUBST block

Since subst.mk r1.85 from 2020-04-29 20:00, a few hours after this
condition had been added, this is not necessary anymore.

(rillig)

mk/configure/pkg-config-override.mk: ignore no-op SUBST blocks

There are many .pc files that don't have a "Libs:" line.  That is totally
ok, therefore just skip these files.

(rillig)

lang/ruby: make patching config_file.rb optional

ruby26-base and beyond don't need this patch anymore.  They get the
configuration directory from Gem::ConfigFile::SYSTEM_CONFIG_PATH, which
is set to RbConfig::CONFIG["sysconfdir"], which in turn is set to
PKGSYSCONFDIR.

(rillig)

devel/py-coverage: skip shell portability check for unneeded file

(rillig)

devel/ruby-redmine: remove no-op files from SUBST_FILES

(rillig)

emulators/qemu: remove no-op file from SUBST_FILES

(rillig)

devel/protobuf: skip shell portability check

(rillig)

lang/openjdk8: document safe use of array[char]

(rillig)

mk/defaults/options.description: describe a few options

(rillig)

www/aws: clean up package definition

(rillig)

net/libktorrent: move documentation from share/docs to share/doc

(rillig)

mk/bsd.options.mk: make deprecated options more visible

Up to now, the generated warnings were not prefixed with anything.

(rillig)

cross/freemint-mintlib: skip portability check

(rillig)

cross/aarch64-none-elf-gcc: skip check for unknown configure options

(rillig)

lang/gcc-aux: fix shell portability

(rillig)

lang/gcc6-aux: fix shell portability

(rillig)

devel/gdb7: fix shell portability

(rillig)

lang/gcc48: ignore unknown configure options, skip portability check

(rillig)

lang/gcc49: ignore unknown configure options, skip portability check

(rillig)

devel/gdb: fix shell portability in test file

(rillig)

net/delegate: fix potential endless loop in CGI part

Not reported upstream since the homepage is not reachable.

(rillig)

ham/cwtext: document safe usage of array[char]

(rillig)

lang/cu-prolog: suppress harmless array[char] compiler warnings

(rillig)

lang/cu-prolog: re-wrap DESCR to look more natural

(rillig)

news/bystand: document harmless array[char]

(rillig)

games/bridge-hands: document harmless array[char]

(rillig)

devel/bcc: add license, document that array[char] are harmless

(rillig)

lang/awka: fix homepage

(rillig)

lang/awka: fix build with -Werror=char-subscripts

(rillig)

misc/astrolog: document harmless instances of array[char]

(rillig)

biology/arka: document 2 instances of char as array index

(rillig)

lang/STk: fix array index using char

(rillig)

net/IglooFTP: fix potential endless loop for long passwords

(rillig)

net/DarwinStreamingServer: allow build even with -Werror=char-subscripts

(rillig)

games/7kaa: build even when CFLAGS has -Werror=char-subscripts

The upstream code looks very suspicious.  It forces compilation with
-fsigned-char and then uses signed characters to index a 256-entry table,
for example in IMGbltAreaRemap.  But only in the C++ code.
The corresponding assembler code uses unsigned bytes.
I'll let them fix this bug themselves.

(rillig)

devel/binutils: suppress USE_TOOLS+=perl warning

(rillig)

cross/avr-gcc: skip portability check, allow unknown configure options

(rillig)

cross/mingw-w64-i686-gcc-bootstrap: configure options, portability

(rillig)

mk/help: clean up code

(rillig)

mk/help: remove negation from variable names

(rillig)

mk/bsd.pkg.mk: improve documentation

(rillig)

mk/bsd.options.mk: fix comment for bmake help

Before, "bmake help" only printed the second section of the help topic.

(rillig)

mk/bdb.buildlink3.mk: fix outdated comments

(rillig)

mk/help: do not treat continuation lines as textual comments

(rillig)

mk/help: make variable more precise

The word "ignore" was too unspecific.  There are lines that should be
printed, and there are lines that should be evaluated.

(rillig)

mk/bsd.pkg.mk: fix typos in documentation

(rillig)

mk/help: clean up code for selecting relevant help topics

(rillig)

mk/help: only list help topics that have associated keywords

(rillig)

regress/infra-unittests: demonstrate wrong help topics without keywords

(rillig)

doc/guide: remove documentation of WRAPPER_TRANSFORM_CMDS

It is not used anymore.

(rillig)

devel/idutils: suppress USE_TOOLS+=perl warning

(rillig)

textproc/grep: suppress USE_TOOLS+=perl warning

(rillig)

math/R-nloptr: remove no-op SUBST block

(rillig)

regress/tools-platform: document which shell pkgsrc uses

(rillig)

graphics/GraphicsMagick: remove unknown configure options

(rillig)

mk/wrapper/transform-gcc: pass -Wno-* without warning

Seen in lang/mono6.

(rillig)

mail/up-imappproxy: remove obsolete patch, fix directories in README

(rillig)

www/py-werkzeug: remove nonexistent file from SUBST_FILES

There is no contrib directory anymore.

(rillig)

graphics/ImageMagick6: remove unknown configure options

(rillig)

time/libical: remove broken REPLACE_INTERPRETER block

The block was called "perl" but replaced "tclsh". Its empty REPLACE_FILES
triggered a warning, which I converted to an error in my local bulk
build.

(rillig)

multimedia/gst-plugins1-base: remove unknown configure option

(rillig)

textproc/enchant2: remove unknown configure option

(rillig)

sysutils/dbus-python-common: fix unknown configure option

(rillig)

cross/arm-none-eabi-gcc: skip check for unknown configure options

(rillig)

net/wistumbler2: remove no-op SUBST block

That file only contains the macro ETC_CFGFILE, not an absolute path.

(rillig)

wm/windowmaker: remove WindowMaker.in from SUBST_FILES

That file does not contain the paths that are fixed.

(rillig)

www/siege: remove no-op SUBST block

The files don't contain EGDIR.

(rillig)

x11/qt4-libs: remove no-op SUBST group

The mm file does not contain the word prefix.

(rillig)

net/publicfile: remove redundant file for errno hack

That substitution is already done by patch-leapsecs__read.c.

(rillig)

www/drupal8: remove SUBST block for nonexistent file

(rillig)

math/pari: remove no-op SUBST block

When building on NetBSD-8.0-x86_64, the file pari.cfg does not contain
any references to TOOLS_DIR.

(rillig)

net/netatalk3: remove no-op SUBST block

There is no mention of dvips in the whole package.

(rillig)

sysutils/qlogtools: skip nonexistent error.h when fixing errno

Some packages using mk/djbware.mk do not have error.h but need to fix the
errno declaration in other files.  Up to now, there was no good way of
achieving exactly this.

Building packages like these in a SUBST_NOOP_OK=no build makes these
packages fail.  To fix these packages, the configuration of the files
needed to be a bit more flexible.

(rillig)

mk/djbware.mk: indent nested .if and .include directives

(rillig)

mail/qmail: doesn't need the errno hack anymore

notqmail-1.07 correctly includes <errno.h> instead of declaring errno as
a process-global variable.

(rillig)

sysutils/py-diffoscope: remove no-op SUBST block

diffoscope has switched to libarchive instead of depending on an external
gtar.

(rillig)

graphics/py-cairocffi: remove no-op SUBST block

There is neither PREFIX nor SOEXT in cairocffi/__init__.py.

(rillig)

textproc/py-4Suite: remove no-op pattern from SUBST_FILES

The Borrowed tests don't need a #! interpreter, but one of the Core tests
does.

(rillig)

x11/x11vnc: allow no-op SUBST block for pthread

On NetBSD-8.0-x86_64, the generated configure script does not contain
"pthread" at all. Maybe it is different on other platforms, therefore
leave the possible substitution as-is, but don't error out.

(rillig)

www/py-werkzeug: remove examples/cookieauth.py from SUBST_FILES

It doesn't exist anymore.

(rillig)