pax: don't overwrite destination if -r -w copy fails

Add more error handling to pax -r -w so that any failure
during the copy to the temporary file (including a failed flush)
prevents any existing destination file from being replaced
with the partial (including possibly empty) temporary file.
The partial temporary file is removed.  pax still exists non-zero.

Thanks to Michael van Elst (mlelstv@) for the analysis
of the problem in the PR.

Should fix PR misc/33753.

(lukem)

pax: exit 0 if stdin filelist is used and empty

If copying a list of files from stdin, exit zero instead of non-zero
if there are no files supplied.

AFAICT, POSIX doesn't require a non-zero an error in this situation,
since there are no files to not match.

Fix from PR bin/41736 by Lloyd Parkes.

(lukem)

Second arg to fdt_memory_remove_range is a size so pass dtbsize and not
dtb + dtbsize

(skrll)

deprecate remnants of MKCRYPTO

MKCRYPTO was removed by riastradh@ on 2017-05-22,
so remove references to it in comments.

(lukem)

bsd.own.mk: deprecate NOCRYPTO

MKCRYPTO was removed by riastradh@ on 2017-05-22,
so the NOCRYPTO override is now unnecessary.

(lukem)

Tickets #175 - #184

(martin)

Pull up following revision(s) (requested by nat in ticket #184):

usr.sbin/bta2dpd/bta2dpd/sbc_encode.c: revision 1.11

With the latest changes, pad(4) will return partial reads to allow a
more fine grained pacing of audio data. But this broke bta2dpd which
relied on full buffers returned (like reading from a file).

Replace the single read() in bta2dpd with a loop that fetches a full
buffer. This restores the old behaviour but loops in userland instead
of the kernel at the cost of a few extra system calls.

(martin)

share/mk: finish USE_COMBINE, NOCOMBINE deprecation

Remove the leftover remnants of deprecated USE_COMBINE and NOCOMBINE,
as gcc -combine has been EOL for many years.

Completes the work done by maya@ on 2019-05-10 in bsd.kmodule.mk rev 1.61.

(lukem)

Ticket #1634

(martin)

Pull up following revision(s) (requested by andvar in ticket #1634):

external/apache2/mDNSResponder/dist/mDNSPosix/PosixDaemon.c: revision 1.15

reapply changes for the built-in drop-privs support by tsarna.

this commit doesn't reapply "dumping of the unicast server list
to the DumpStateLog debugging output" enhancement.
It doesn't build anymore, no idea how to rewrite.

Should fix PR 57442. Needs pull-ups for netbsd-9, netbsd-10.

(martin)

Pull up following revision(s) (requested by andvar in ticket #183):

external/apache2/mDNSResponder/dist/mDNSPosix/PosixDaemon.c: revision 1.15

reapply changes for the built-in drop-privs support by tsarna.

this commit doesn't reapply "dumping of the unicast server list
to the DumpStateLog debugging output" enhancement.
It doesn't build anymore, no idea how to rewrite.

Should fix PR 57442. Needs pull-ups for netbsd-9, netbsd-10.

(martin)

Pull up following revision(s) (requested by bouyer in ticket #182):

sys/arch/arm/nxp/imx6_platform.c: revision 1.8 (via patch)
sys/arch/arm/nxp/imx6_platform.c: revision 1.9 (via patch)
sys/arch/arm/nxp/imx6_ccmvar.h: revision 1.2
sys/arch/arm/nxp/imx6_ccmvar.h: revision 1.3
distrib/sets/lists/dtb/ad.earmv7hf: revision 1.10
sys/arch/arm/nxp/imx6sx_clk.c: revision 1.1
sys/arch/arm/nxp/imx6_ccm.c: revision 1.3
sys/arch/arm/nxp/imx6sx_clk.c: revision 1.2
sys/arch/arm/nxp/imx6_ccm.c: revision 1.4
sys/arch/arm/nxp/imx6sx_clk.c: revision 1.3
sys/arch/arm/nxp/imx6_spi.c: revision 1.8
sys/arch/arm/nxp/imx_sdhc.c: revision 1.8
sys/arch/arm/imx/imxusb.c: revision 1.19
sys/arch/arm/nxp/imx6_usb.c: revision 1.7
sys/arch/arm/nxp/imx6_usb.c: revision 1.8
sys/arch/arm/nxp/files.imx: revision 1.3
sys/arch/arm/nxp/imx6_clk.c: revision 1.5
sys/arch/arm/nxp/imx6_clk.c: revision 1.6
sys/arch/arm/nxp/imx6_reg.h: revision 1.2
sys/arch/arm/nxp/imx6_gpc.c: revision 1.4
sys/arch/arm/imx/imx23_usb.c: revision 1.6
sys/dtb/arm/Makefile: revision 1.6
sys/arch/arm/imx/imxusbvar.h: revision 1.7
sys/arch/evbarm/conf/GENERIC: revision 1.119
sys/arch/arm/nxp/imx6_ccmreg.h: revision 1.2
sys/arch/arm/nxp/imx6_iomux.c: revision 1.3
sys/arch/arm/nxp/imx6_usbphy.c: revision 1.3
sys/arch/arm/nxp/imx6_pcie.c: revision 1.7

In preparation of imx6sx support, move imx6q-specific definitions from
imx6_ccm.c to imx6_clk.c, and prefix with IMX6Q/imx6q:
- Clock IDs and related struct imx_clock_id
- Clock Parents and Divider Tables
- struct imx6_clk
and related static functions

Add a pointer to struct imx6_clk, and it's size to imx6ccm_softc

Pass a pointer to imx6ccm_softc to all functions from imx6_ccm.c
NFCI

i.mx6sx CPU support in the CCM module: the clock tree si different from
the i.mx6q
- move i.mx6q-specific functions and data to imx6_clk.c
- add i.mx6sx specific imx6sx_clk.c
- add a imx6sxccm device
i.mx6sx platform support:
- the i.mx6sx has a third AIPS, so KERNEL_IO_IOREG map has to be larger
- the uart clock is at 24Mhz instead of 80.

Add i.mx6sx compatible entries to drivers that should work as is.

Fix typo, preventing i2c4 from attaching
remove commented out entries, leftover from the imx6q ccm code.
introduce imx6sx_platform_bootstrap(), which calls imx_platform_bootstrap()
and then checks for an "arm,cortex-a9-twd-timer" compatible entry in the
fdt. If not present, create one so that a9ptmr will attach.
We need this entry as this is the only timer we support for this platform,
but the upstream imx6sx.dtsi is missing the entry for it (and all A9 CPUs
have it anyway).

Thanks to Jared McNeill for advices and review.

Also build dtb files for CONFIG_SOC_IMX6SX

experimental IMX6SX support:
- add options SOC_IMX6SX
- add imx6sxccm device
tested on a UDOO Neo Full board.

known to work:
- uart (console)
- sdmmc0
- ethernet (enet0)
known to not work:
- USB (device not detected).

needs a modified device tree at this time (add arm,cortex-a9-twd-timer entry
copied from the imx6qdl dtsi); for unkown reason the imx6sx.dtsi file lacks
an entry for the a9ptmr although it is present in the soc (and, from what I
understood, in all cortex A9 SoCs).

Add imx6sx dtb files

The i.mx6sx has 2 OTG and one host-only USB controller, while the 6q has
only one OTG.

Add a "uintptr_t data" argument to all sc_*_md_hook callbacks, which
gets the sc_md_hook_data value when called.

In imx6_usb.c use this to pass the number of OTG controllers to the callbacks.
imx6_usb_init() can then properly call init_otg() or init_h1() for unit 1.

In imx6_usb_attach(), test if there is a vbus-supply property in the fdt,
and enable the regulator if present.

Now the USB port of the UDOO Neo works.

(martin)

Pull up following revision(s) (requested by lukem in ticket #181):

usr.bin/m4/gnum4.c: revision 1.13

use strchr

(martin)

Pull up following revision(s) (requested by lukem in ticket #180):

usr.bin/m4/gnum4.c: revision 1.12

m4 -g: { and } aren't supported in patsubst() or regexp()

When running in -g (GNU m4) emulation, patsubst() and regexp()
use the GNU m4 emacs-like regexes as implemented by gnulib,
which don't support {..} intervals.

When converting a GNU m4 regex to a POSIX ERE, escape raw { and }.
Autoconf relies on the GNU m4 regex behaviour.

See:
- https://www.gnu.org/software/autoconf/manual/autoconf-2.69/html_node/Redefined-M4-Macros.html
- https://www.gnu.org/software/gnulib/manual/html_node/emacs-regular-expression-syntax.html

This fixes the tools/compat/configure regen.

PR toolchain/57431

(martin)

Pull up following revision(s) (requested by hannken in ticket #179):

external/bsd/unbound/include/config.h: revision 1.11

Set PID path back to "/var/run/unbound.pid" so rc scripts work again.
Ok: Christos Zoulas

PR bin/57242 unbound rc.d script does not work with chrooted unbound

(martin)

Pull up following revision(s) (requested by lukem in ticket #178):

etc/etc.mac68k/Makefile.inc: revision 1.24
distrib/cdrom/Makefile: revision 1.53

Creating Type&Creator mappings for .bin (macbinary encoded) files,
mkisofs changes files, then turns around and complains they have been
changed.

This is a self-inflicted wound, so demote the error to a warning.

Reported upstream as <https://codeberg.org/schilytools/schilytools/issues/58>.

Arguably fixes PR toolchain/42166

Request pull-ups to netbsd-{8,9,10}.
XXX What about macppc?

(martin)

Pull up following revision(s) (requested by lukem in ticket #177):

tools/Makefile.gnuhost: revision 1.56
external/gpl3/gcc/usr.bin/host-libcpp/Makefile: revision 1.6

Fix passing -j NNN to gmake

Use a more restrictive pattern to extract -j NNN from MAKEFLAGS
into GMAKE_J_ARGS, to avoid false positives when the source directory
has "-j" in the path (e.g "amd64-job-000012" or "src-j9999").

Previously this could pass either -"-j" or "-j BIGNUM" to gmake
and result in "vfork: Resource temporarily unavailable" failures.

PR misc/54456

(martin)

Pull up following revision(s) (requested by lukem in ticket #176):

external/gpl3/gcc/usr.bin/libdecnumber/Makefile: revision 1.9
external/gpl3/gcc/usr.bin/common/Makefile: revision 1.12
external/gpl3/gcc/usr.bin/backend/Makefile: revision 1.67
external/gpl3/gcc/usr.bin/common-target/Makefile: revision 1.12
external/gpl3/gcc/usr.bin/frontend/Makefile: revision 1.15
external/gpl3/gcc/usr.bin/libcpp/Makefile: revision 1.10

gcc: fix build with clang++ HOST_CXX

Define HOSTPROG_CXX before .include anything that brings in bsd.own.mk.

This ensures that HOST_DBG (etc) gets assigned before HOST_CFLAGS
and HOST_CXXFLAGS is created.

backend: .include <bsd.init.mk> much earlier, as per the other directories.

Fixes backend build when using clang++ as the host compiler (e.g., macOS),
because backend host tools are now built with -O.

Inspired by https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=255760

Note that gcc.old/Makefile.prog may be mis-used because it defines HOSTPROG_CXX
and this Makefile fragment is included after bsd.*.mk in other Makefiles,
but they seem to build ok so leaving it alone for now.

Fixes PR toolchain/57014

(martin)

Pull up following revision(s) (requested by gutteridge in ticket #175):

sys/sys/proc.h: revision 1.371
sys/kern/vfs_lookup.c: revision 1.234

Default PROC_MACHINE_ARCH to machine_arch and use this for magic
symlinks to resolve "@machine_arch".

This keeps behaviour of magic symlinks and 'uname -p' output the same.
Fixes PR 57320.

(martin)

s/Resture/Restore/ and s/restared/restarted/ in comments.

(andvar)

Add RUMPHIJACK option "blanket=/DEV" so mount_ffs may canonicalise
and mount the device path.  Cannot use "/rump/DEV" here as the device
path is embedded in "struct ufs_args" where it doesnt get hijacked.

(hannken)

s/explcit/explicit/ in comment.

(andvar)

s/sessoin/session/ in warning message.

(andvar)

With the latest changes, pad(4) will return partial reads to allow a
more fine grained pacing of audio data. But this broke bta2dpd which
relied on full buffers returned (like reading from a file).

Replace the single read() in bta2dpd with a loop that fetches a full
buffer. This restores the old behaviour but loops in userland instead
of the kernel at the cost of a few extra system calls.

(mlelstv)

In sys/param.h document that MAXPATHLEN includes space for a null byte.

Also note that it's the same as PATH_MAX. Reflow the comment.

Suggested by Thierry Laronde.

(dholland)

add a fortune

(dholland)

fix word endings toin -> tion in comments.

(andvar)

Explicitly run openssl's "gen" script in a shell

The script has permissions 755 in the repo but ends up with permissions
644 in the source sets, therefore we cannot invoke it simply as
path/to/gen .

(hgutch)

reapply changes for the built-in drop-privs support by tsarna.
this commit doesn't reapply "dumping of the unicast server list
to the DumpStateLog debugging output" enhancement.
It doesn't build anymore, no idea how to rewrite.
Should fix PR 57442. Needs pull-ups for netbsd-9, netbsd-10.

(andvar)

Revert to previous.

This change broke the intention of recent changes to pad(4).

The solution to address PR/57436 must be done in bta2dpd(8).

(nat)

openssl: Add missing sparc compat openssl engines to set list.

(riastradh)

openssl: Delete capi makefiles -- not likely to ever see use here.

(riastradh)

openssl: Nix capi.so engine -- it's for the Windows crypto API.

Not likely to work in NetBSD base!

(riastradh)

openssl: Fix typo in mips set list.

(riastradh)

openssl: padlock.so engine for VIA CPUs makes sense only on x86.

(riastradh)

openssl: Link engines against libcrypto so they can be loaded.

(riastradh)

openssl: Wire up dynamic engine build goo.

This restores the devcrypto engine, which was converted from static
to dynamic since the previous OpenSSL update.

XXX Pretty sure I got some of the set list package names wrong, but
it's not clear what the right ones are or what consequences any of
this has.

XXX Needs testing.

(riastradh)

openssl: Add /usr/lib/${ARCHDIR}/openssl/engines to aarch64 build.

(riastradh)

Revert "openssl: Nix build goo for dynamically loaded engines."

Turns out there is a regression lurking here: the devcrypto engine
was made dynamic-only (https://github.com/openssl/openssl/pull/7859),
so to maintain that functionality, we need to build dynamic engines.
Will fix the aarch64 build issue and wire things up separately.

(riastradh)

Fix potential null dereference by simply adding last_gd != NULL check
around the code block with last_gd usage after the inner loop.
It may mainly happen on bus_dmamap_load error.
Reported by Mootja _14, 2017.

(andvar)

include arm/arm32/pte.h to gemini_ipm.c to fix GEMINI_MASTER/SLAVE config builds.
They fail with 'L1_S_SIZE' undeclared error otherwise from gemini.h declarations.
gemini_machdep.c includes it transitively, thus doesn't faili on GEMINI config.
Likely nobody is using them anyway though...

(andvar)

openssl: Nix build goo for dynamically loaded engines.

- Engines are deprecated in openssl 3.

- We never actually had the .so engines wired up to be built in
  openssl 1.1, and judging by the lack of obsolete entries in the set
  lists, I don't think we ever had them wired up to be built at all.

(riastradh)

Fix previous.

The previous commit was misleading due to the flag names used.

The rationale for this change is that it is ok to return EAGAIN on first
read as playback has not started.  Then it shoud wait on every subsequent read.

This is what makes the high precsision throttling of pad(4), for which these
degree of fine-grained sleeps are not possible in userland.  This is a
requirement of users of pad(4) such as bta2dpd(8).

(nat)

/etc/rc.d/sshd: New check cmd and reload precmd.

- check cmd: run `sshd -t' to check sshd_config file

- reload precmd: run check cmd before reloading so we don't nuke sshd
  if there's an error in the sshd_config file

(It is still possible to effectively nuke sshd by changing the
configuration tosomething that won't work on your network, but at
least we avoid making sshd just exit on reload when you make a typo
in a config option.)

XXX pullup-9
XXX pullup-10

(riastradh)

Fix logic for timing.

Addresses PR/57436.

XXX pullup-10.

(nat)

set error to EIO on SKEY_ABORTED_COMMAND.
It is usually the case for other SCSI drivers, though they do implement retries too.
potentially same should be committed to prep(?).
discussed with riastradh.

(andvar)

initialize error on definition to fix 'error' may not be initialized issue.
the commit unifies code between bebox and prep machines.
reported by Mootja _08.

(andvar)

+CRDT  conflict-free replicated data type
+UDS    Unix Domain Socket

(jschauma)

ddb: PR kern/57435 - fix duplicate "show mount"

Bolivar cannot carry double.  One of these commands should have been
"show mounts", cf. page/pages and a few others.  While here fix the
function name to be db_show_all_mounts to conform to the pattern.

(uwe)

openssl: CWARNFLAGS, not COPTS, for -Wno-error=...

COPTS gets put too early, so these are cancelled out by -W flags
added later by bsd.sys.mk.  (How did this ever work???)

(riastradh)

openssl: Note that -Wno-error=char-subscripts likely masks real bugs.

(riastradh)

openssl: Consolidate new CWARNFLAGS.clang stuff in Makefile.openssl.

(riastradh)

pathadj() is required to succeed.  If it cannot, simply issuing a
warning and continuing is not good enough.  Change the warning to
an error (and hence immediate exit) instead, that's all we can do.

Problem pointed out by tlaronde@polynum.com

XXX - pullup -10 (others?)

(kre)

If it isn't possible to determine the realpath() of ".", then simply
give up on attempting to resolve any relative path - there's no hope.

(kre)

add a little const to help clang

(christos)

openssl: Suppress clang warnings for using string literals as char *.

Not sure this is actually forbidden by the C standard.  There may be
bugs lurking here, but OpenSSL uses this extensively, so let's just
get the build going again.

(riastradh)

openssl: Nix trailing whitespace in build goo.

No functional change intended.

(riastradh)

libpthread: New pthread__smt_wait to put CPU in low power for spin.

This is now distinct from pthread__smt_pause, which is for spin lock
backoff with no paired wakeup.

On Arm, there is a single-bit event register per CPU, and there are two
instructions to manage it:

- wfe, wait for event -- if event register is clear, enter low power
  mode and wait until event register is set; then exit low power mode
  and clear event register

- sev, signal event -- sets event register on all CPUs (other
  circumstances like interrupts also set the event register and cause
  wfe to wake)

These can be used to reduce the power consumption of spinning for a
lock, but only if they are actually paired -- if there's no sev, wfe
might hang indefinitely.  Currently only pthread_spin(3) actually
pairs them; the other lock primitives (internal lock, mutex, rwlock)
do not -- they have spin lock backoff loops, but no corresponding
wakeup to cancel a wfe.

It may be worthwhile to teach the other lock primitives to pair
wfe/sev, but that requires some performance measurement to verify
it's actually worthwhile.  So for now, we just make sure not to use
wfe when there's no sev, and keep everything else the same -- this
should fix severe performance degredation in libpthread on Arm
without hurting anything else.

No change in the generated code on amd64 and i386.  No change in the
generated code for pthread_spin.c on arm and aarch64 -- changes only
the generated code for pthread_lock.c, pthread_mutex.c, and
pthread_rwlock.c, as intended.

PR port-arm/57437

XXX pullup-10

(riastradh)

libpthread: Use __nothing, not /* nothing */, for empty macros.

No functional change intended -- just safer to do it this way in case
the macros are used in if branches or comma expressions.

PR port-arm/57437 (pthread__smt_pause/wake issue)

XXX pullup-10

(riastradh)

osnet: on macOS, use <mach/boolean.h> for boolean_t

macOS/x86_64 defines boolean_t as 'unsigned int' not 'int',
which causes a build issue with tools/ctfmerge on that host
after my recent fixes for macOS semaphores.
So use the <mach/boolean.h> instead of a local typedef ifdef __APPLE__.

May fix a macOS/x86_64 build issue reported by cjep@.
Builds fine on NetBSD/amd64 or macOS/arm.

Note: this compat stuff is clunky, and based on the commit log,
annoyingly error prone. A newer sync of osnet from upstream /may/
improve a lot of these compat typedef workarounds for solaris types...

(lukem)

revert my previous commit as there is no agreement on it being either desired
or appropriate

(jschauma)

ctfmerge: fix macOS semaphore implementation, part 2

dispatch_semaphore_signal() doesn't return an error, just an
indicator of whether a thread was woken or not, so there's
no need to fail on non-zero return.

(lukem)

amdgpu: Fix mostly harmless merge botch.

Avoids confusing error message that should have been confined to an
error branch.

(riastradh)

Fix CLKF_INTR so that not all time is shown as being spent in interrupts.

(skrll)

chflags(1): fix wrong markup in previous

(uwe)

Briefly describe the 'arch' and 'nodump' flags.

(jschauma)

use strchr

(christos)

m4 -g: { and } aren't supported in patsubst() or regexp()

When running in -g (GNU m4) emulation, patsubst() and regexp()
use the GNU m4 emacs-like regexes as implemented by gnulib,
which don't support {..} intervals.
When converting a GNU m4 regex to a POSIX ERE, escape raw { and }.

Autoconf relies on the GNU m4 regex behaviour.

See:
- https://www.gnu.org/software/autoconf/manual/autoconf-2.69/html_node/Redefined-M4-Macros.html
- https://www.gnu.org/software/gnulib/manual/html_node/emacs-regular-expression-syntax.html

This fixes the tools/compat/configure regen.

PR toolchain/57431

(lukem)

revert previous: autoconf doesn't use EREs

autoconf relies upon GNU m4 emacs-style regexs, which
don't support intervals so don't escape these.
A fix to usr.bin/m4 -g (GNU m4 emulation) is incoming.

(lukem)

entropy(9): Avoid race between rnd_add_data and ioctl(RNDCTL).

XXX pullup-10

(riastradh)

entropy(9): On flags change, cancel any scheduled consolidation.

We've been instructed to lose confidence in existing entropy sources,
so let's make sure to re-gather enough entropy before the next
consolidation can happen, in case some of what would be counted in
consolidation is from those entropy sources.

XXX pullup-10

(riastradh)

match the new compiler warning

(christos)

no more md5 for you.

(christos)

introduce imx6sx_platform_bootstrap(), which calls imx_platform_bootstrap()
and then checks for an "arm,cortex-a9-twd-timer" compatible entry in the
fdt. If not present, create one so that a9ptmr will attach.
We need this entry as this is the only timer we support for this platform,
but the upstream imx6sx.dtsi is missing the entry for it (and all A9 CPUs
have it anyway).

Thanks to Jared McNeill for advices and review.

(bouyer)

quote { to make regcomp happy

(christos)

CHANGES: rm SHAREDSTRINGS, mkstr, xstr

(lukem)

obsolete mkstr(1) and xstr(1)

These were only used on PDP-11 for two programs we don't ship,
and have been obsolete since the VAX days.

xstr never worked in the build.sh cross-build environment (22 years), or
parallel make environment (nearly 28 years), didn't work in the orignal 386bsd
import, and has never been needed in NetBSD as we don't have the older BSD
programs (pascal, pre-nvi ex) that needed mkstr/xstr on PDP-11.

PR toolchain/35964

(lukem)

deprecate SHAREDSTRINGS build option

Hasn't worked for at least 20 years (and never in the cross-build environment,
nor did it work with parallel make), and has never been needed in NetBSD as we
don't have the older BSD programs (pascal, pre-nvi ex) that needed mkstr/xstr
on PDP-11.

PR toolchain/35964

(lukem)

efi(4): Fix logic to handle buffer sizing.

Can't KASSERT(datasize <= databufsize) because the caller is allowed
to pass in a too-small size and get ERR_BUFFER_TOO_SMALL back, with
the actual size returned so it can resize its buffer.  So just clamp
the size to the smaller of what the caller provided and what the
firwmare provided, instead of asserting anything.

PR kern/57076

XXX pullup-10

(riastradh)

ctfmerge tool fixes might need a clean tools build

(lukem)

ctfmerge: pthread error handling. macos host fixes

(lukem)

tools/compat: regen for dispatch_semaphore_create

(lukem)

tools/compat: MAKEVERBOSE support in regen

(lukem)

autoconf: AC_FUNC_ALLOCA(): add local fix for BSD support

Add local fix to AC_FUNC_ALLOCA() to include <stdlib.h> on BSD:
# elif defined(__NetBSD__) || defined(__FreeBSD__) || defined(__DragonFly__) || defined(__OpenBSD__)
#  include <stdlib.h>

This change avoids regen losing this change in tools/compat/configure
that was added in rev 1.75 on 2013-03-05 of tools/compat/configure,
and subsequent commits that keep re-adding these lines back
manually (e.g., rev 1.97 on 2019-06-22 of tools/compat/configure)

XXX: Arguably this could be fixed in a separate way to allow us to
use an unmodified modern autoconf in the future ?

(lukem)

Set PID path back to "/var/run/unbound.pid" so rc scripts work again.

Ok: Christos Zoulas

PR bin/57242 unbound rc.d script does not work with chrooted unbound

(hannken)

ctfmerge: fix macOS semaphore implementation

Use dispatch_semaphore_create() if present instead of sem_init().

macOS doesn't actually implement sem_init() (et al)
(even though it provides the prototypes as deprecated).
This was detected by the previous commit to ctfmerge
that added error handling.

Implement ctfmerge's barrier operations in terms of
dispatch(3) APIs such as dispatch_semaphore_create() (et al).

Update tools/compat/configure.ac to find dispatch_semaphore_create().

Fixes ctfmerge on macOS hosts.

Inspired by https://stackoverflow.com/a/27847103.

(lukem)

ctfmerge: error check sem_*() and pthread_*() APIs

terminate() if sem_*() returns -1 or pthread_*() returns != 0.
(Set errno from pthread_*() so terminate() prints the strerror message).

Note: Failing on errors instead of ignoring them helps identify
reasons for intermittent failures, such as those on macOS host builds:
  ERROR: nbctfmerge: barrier_init: sem_init(bar_sem): Function not implemented

(lukem)

indent: separate code for handling enums from the lexer

The lexer's responsibility is to generate tokens, it's not supposed to
update the parser state.  Centralize the state transitions that control
indentation of enum constants to keep the lexer code clean.

Skip comments, newlines and preprocessing lines when updating the parser
state for enum constants and for '*' in declarations.

(rillig)

indent: fix indentation of struct declarations

(rillig)

indent: split debug output into paragraphs

The paragraphs separate the different processing steps: getting a token
from the lexer, processing the token, updating the parser state, sending
a finished line to the output.

(rillig)

centralize the warnings, try to fix clang

(christos)

centralie warnings

(christos)

indent: extract processing of a single token to separate function

No functional change.

(rillig)

s/configuratin/configuration/ in comment, from Brad Harder (bch) by email.

(andvar)

use macros compatible with xf86-video-suncg14 to issue SX instructions
much more readable, alignment weirdness is handled automatically and code is
interchangable

(macallan)

autoconf(9): Omit config_detach kernel lock assertion too for now.

like in config_attach_pseudo, this assertion almost certainly
indicates real bugs, but let's try to get the tests back and running
again before addressing those.

(riastradh)

indent: fix spacing around '*' in declarations

(rillig)

indent: fix spacing in declarations in for loops

(rillig)

tests/indent: add code snippets found in make and lint1

(rillig)

autoconf(9): Omit config_attach_pseudo kernel lock assertion for now.

Breaks too many things that I didn't test in the branch (cgd, fss,
&c.); let's address all forty-odd cases before turning it on.

(riastradh)

indent: fix spacing between block braces

(rillig)

tests/indent: refine a few tests

(rillig)

Use the offset in mmap calculations.

It fixes wsfb(4) when used with intelfb(4).

Ok riastradh@

XXX pullup-10

(nat)

indent: adjust indentation in lexer

No binary change.

(rillig)

tests/lint1: allow whitespace in files containing the expected output

This reduces the amount of wizardry needed to interpret the files.  For
a more gentle introduction of the file format, see ../lint2/msg_000.ln.

The whitespace that is removed is very similar to the one in the lint2
tests, the difference is that '%<space>' and '%#' need to be preserved
in abbreviated printf and scanf format strings.

(rillig)

lint.1: add a few examples

In default mode, lint runs in a mode that aids in migrating from
traditional C to C90, which is not widely useful nowadays.  Instead,
suggest some examples of using lint in more practical ways.

(rillig)

lint: use consistent variable name for redeclared symbol

No binary change.

(rillig)

lint: fix emitted type for arrays of unknown size

(rillig)

lint: clean up the name of a local variable

No binary change.

(rillig)

i386/GENERIC: Enable EFI runtime support.

PR kern/57076

XXX pullup-10

(riastradh)

i386/ALL: Enable EFI runtime support.

PR kern/57076

XXX pullup-10

(riastradh)

efi(4): Translate between size_t and unsigned long.

Fixes i386 build.

PR kern/57076

XXX pullup-10

(riastradh)

efi(4): Implement EFIIOC_GET_TABLE on x86.

PR kern/57076

XXX pullup-10

(riastradh)

efi(4): Implement MI parts of EFIIOC_GET_TABLE.

Intended to be compatible with FreeBSD.

Not yet supported on any architectures.

PR kern/57076

XXX pullup-10

(riastradh)

efi(4): Move error macros to efi.h.

PR kern/57076

XXX pullup-10

(riastradh)

efi(4): Parenthesize EFIERR argument out of paranoia.

PR kern/57076

XXX pullup-10

(riastradh)

dtrace_fbt: Read and write instructions appropriately-endian on arm.

arm is a little more complicated because it has three cases:
- big-endian data, big-endian instructions
- big-endian data, little-endian instructions
- little-endian data, little-endian instructions

(riastradh)

dk(4): Add locking notes.

(riastradh)

dk(4): Explain why no need for device reference in dksize, dkdump.

(riastradh)

dk(4): Strengthen preconditions of various devsw operations.

These can only happen between dkopen and dkclose, so there's no need
to test -- we can assert instead that the wedge exists and is fully
initialized.

(riastradh)

dk(4): Strengthen dkclose preconditions.

Like dkopen, except it is possible for this to be called after the
wedge has transitioned to dying.

XXX sc_state read here races with sc_state write in dkwedge_detach.
Could change this to atomic_load/store.

(riastradh)

dk(4): Strengthen dkopen preconditions.

This cannot be called before dkwedge_attach for the same unit
returns, so sc->sc_dev is guaranteed to be set to a nonnull device_t
and the state is guaranteed not to be larval.

And this cannot be called concurrently with dkwedge_detach, or after
dkwedge_detach does vdevgone until another wedge with the same number
is attached (which can't happen until dkwedge_detach completes), so
the state is guaranteed not to be dying or dead.

Hence sc->sc_dev != NULL and sc->sc_state == DKW_STATE_RUNNING.

(riastradh)

dk(4): Prevent race between dkwedge_get_parent_name and wedge detach.

Still races with parent detach but maybe this is better.

XXX Maybe we should ditch dkwedge_get_parent_name -- it's used only
by rf_containsboot, which kinda suggests it shouldn't exist.

(riastradh)

dk(4): Split unsafe lookups into safe subroutines and unsafe wrappers.

No functional change intended.

Eventually we should adjust the callers to use the safe subroutines
instead and device_release when done.

(riastradh)

dk(4): Don't hold lock around uiomove in dkwedge_list.

Instead, hold a device reference.  dkwedge_detach will not run until
the device reference is released.

(riastradh)

dk(4): Skip larval wedges in various lookup routines.

These have not yet finished a concurent dkwedge_attach, so there's
nothing we can safely do with them.  Just pretend they don't exist --
as if we had arrived at the lookup a moment earlier.

(riastradh)

dk(4): Simplify dkwedge_delall by detaching directly.

No need for O(n^2) algorithm and potentially racy lookups -- not that
n is large enough for n^2 to matter, but the mechanism is simpler
this way.

(riastradh)

dk(4): Use device_lookup_private for dkwedge_lookup.

No longer necessary to go through the dkwedges array.

Currently device_lookup_private still involves touching other global
locks, but that will change eventually to a lockless pserialized fast
path.

(riastradh)

dk(4): dkunit is no longer needed; nix it.

dkwedges array indexing now coincides with autoconf device numbering.

(riastradh)

dk(4): Use config_attach_pseudo_acquire to create wedges.

This way, indexing of the dkwedges array coincides with numbering of
autoconf dk(4) instances.

As a side effect, this plugs a race in dkwedge_add with concurrent
drvctl -r.  There are a lot of such races in dk(4) left -- to be
addressed with more device references.

(riastradh)

autoconf(9): New functions for referenced attach/detach.

New functions:

- config_found_acquire(dev, aux, print, cfargs)
- config_attach_acquire(parent, cf, aux, print, cfargs)
- config_attach_pseudo_acquire(cf, aux)
- config_detach_release(dev, flags)
- device_acquire(dev)

The config_*_acquire functions are like the non-acquire versions, but
they return a referenced device_t, which is guaranteed to be safe to
use until released.  The device's detach function may run while it is
referenced, but the device_t will not be freed and the parent's
.ca_childdetached routine will not be called.

=> config_attach_pseudo_acquire additionally lets you pass an aux
  argument to the device's .ca_attach routine, unlike
  config_attach_pseudo which always passes NULL.

=> Eventually, config_found, config_attach, and config_attach_pseudo
  should be made to return void, because use of the device_t they
  return is unsafe without the kernel lock and difficult to use
  safely even with the kernel lock or in a UP system.  For now,
  they require the caller to hold the kernel lock, while
  config_*_acquire do not.

config_detach_release is like device_release and then config_detach,
but avoids the race inherent with that sequence.

=> Eventually, config_detach should be eliminated, because getting at
  the device_t it needs is unsafe without the kernel lock and
  difficult to use safely even with the kernel lock or in a UP
  system.  For now, it requires the caller to hold the kernel lock,
  while config_detach_release does not.

device_acquire acquires a reference to a device.  It never fails and
can be used in thread context (but not interrupt context, hard or
soft).  Caller is responsible for ensuring that the device_t cannot
be freed; in other words, the device_t must be made unavailable to
any device_acquire callers before the .ca_detach function returns.
Typically device_acquire will be used in a read section (mutex,
rwlock, pserialize, &c.) in a data structure lookup, with
corresponding logic in the .ca_detach function to remove the device
from the data structure and wait for all read sections to complete.

Proposed on tech-kern:
https://mail-index.netbsd.org/tech-kern/2023/05/10/msg028889.html

(riastradh)

tty(9): Make ttwrite update uio with only how much it has consumed.

As is, it leaves uio in an inconsistent state.  Good enough for the
write(2) return value to be correct for a userland caller to restart
write(2) where it left off, but not good enough for a loop in the
kernel to reuse the same uio.

Reported-by: syzbot+e0f56178d0add0d8be20@syzkaller.appspotmail.com
https://syzkaller.appspot.com/bug?id=6290eb02b8fe73361dc15c7bc44e1208601e6af8

Reported-by: syzbot+7caa189e8fccd926357e@syzkaller.appspotmail.com
https://syzkaller.appspot.com/bug?id=c0a3b77b4831dfa81fc855857bde81755d246bd3

Reported-by: syzbot+4a1eff91eb4e7c1970b6@syzkaller.appspotmail.com
https://syzkaller.appspot.com/bug?id=10523a633a4ad9749f57dc7cf03f9447d518c5b8

Reported-by: syzbot+1d3c280f59099dc82e17@syzkaller.appspotmail.com
https://syzkaller.appspot.com/bug?id=8e02ebb0da76a8e286461f33502117a1d30275c6

Reported-by: syzbot+080d51214d0634472b12@syzkaller.appspotmail.com
https://syzkaller.appspot.com/bug?id=1f617747db8087e5554d3df1b79a545dee26a650

Reported-by: syzbot+dd50b448e49e5020131a@syzkaller.appspotmail.com
https://syzkaller.appspot.com/bug?id=f71c8cef4110b7eeac6eca67b6a4d1f4a8b3e96f

Reported-by: syzbot+26b675ecf0cc9dfd8586@syzkaller.appspotmail.com
https://syzkaller.appspot.com/bug?id=57b1901f5b3e090a964d08dd0d729f9909f203be

Reported-by: syzbot+87f0df2c9056313a5c4b@syzkaller.appspotmail.com
https://syzkaller.appspot.com/bug?id=67994a3da32d075144e25d1ac314be1d9694ae6e

Reported-by: syzbot+e5bc98e18aa42f0cb25d@syzkaller.appspotmail.com
https://syzkaller.appspot.com/bug?id=6374bd286532423c63f2b331748280729134224c

Reported-by: syzbot+7e587f4c5aaaf80e84b3@syzkaller.appspotmail.com
https://syzkaller.appspot.com/bug?id=976210ed438d48ac275d77d7ebf4a086e43b5fcb

(riastradh)

uiomove(9): Add uiopeek/uioskip operations.

This allows a caller to grab some data, consume part of it, and
atomically update the uio with only the amount it consumed.  This
way, the caller can use a buffer of a size that doesn't depend on how
much it will actually consume, which it may not know in advance --
e.g., because it depends on how much an underlying hardware tty
device will accept before it decides it has had too much.

Proposed on tech-kern:
https://mail-index.netbsd.org/tech-kern/2023/05/09/msg028883.html

(Opinions were divided between `uioadvance' and `uioskip'.  I stuck
with `uioskip' because that was less work for me.)

(riastradh)

man0: support objdirs, tools, parallel builds

man0 creates files, so write to the the OBJDIR instead of the
source directory.

Use TOOL_ appropriately.
Add MAKEVERBOSE support.

Add separate rules to generate:
- all.files: the list of all man files to process (based on mkman)
- manNpages.files: per-section files from all.files. (based on mkman)
- manNpages.ps: per-section .ps from manNpages.files, with
  per-file MAKEVERBOSE messages to stderr (as before, just prettier).
- manNpages.pdf: per-section .pdf from manNpages.ps.

Remove the now-obsolete mkman script.

Add TOOL_GROPS and _MKSHMSG_FORMAT to <bsd.own.mk>

(lukem)

lint: rename constant NOTSPEC to NO_TSPEC

It was too easy to misread the old name as NOT_SPEC instead of the
intended NO_TSPEC.

(rillig)

man0: add bind/dist/bin/tests to noman

(lukem)

tests/lint: demonstrate wrong emitted array length in initialization

Seen in usr.bin/indent/debug.c, in the various 'name' arrays.

(rillig)

indent: implement suppressing optional blank lines

(rillig)

Fix a comment

(skrll)

tests/indent: skip creating an intermediate file in tests

(rillig)