i386/ALL: Enable EFI runtime support.

PR kern/57076

XXX pullup-10

(riastradh)

efi(4): Translate between size_t and unsigned long.

Fixes i386 build.

PR kern/57076

XXX pullup-10

(riastradh)

efi(4): Implement EFIIOC_GET_TABLE on x86.

PR kern/57076

XXX pullup-10

(riastradh)

efi(4): Implement MI parts of EFIIOC_GET_TABLE.

Intended to be compatible with FreeBSD.

Not yet supported on any architectures.

PR kern/57076

XXX pullup-10

(riastradh)

efi(4): Move error macros to efi.h.

PR kern/57076

XXX pullup-10

(riastradh)

efi(4): Parenthesize EFIERR argument out of paranoia.

PR kern/57076

XXX pullup-10

(riastradh)

dtrace_fbt: Read and write instructions appropriately-endian on arm.

arm is a little more complicated because it has three cases:
- big-endian data, big-endian instructions
- big-endian data, little-endian instructions
- little-endian data, little-endian instructions

(riastradh)

dk(4): Add locking notes.

(riastradh)

dk(4): Explain why no need for device reference in dksize, dkdump.

(riastradh)

dk(4): Strengthen preconditions of various devsw operations.

These can only happen between dkopen and dkclose, so there's no need
to test -- we can assert instead that the wedge exists and is fully
initialized.

(riastradh)

dk(4): Strengthen dkclose preconditions.

Like dkopen, except it is possible for this to be called after the
wedge has transitioned to dying.

XXX sc_state read here races with sc_state write in dkwedge_detach.
Could change this to atomic_load/store.

(riastradh)

dk(4): Strengthen dkopen preconditions.

This cannot be called before dkwedge_attach for the same unit
returns, so sc->sc_dev is guaranteed to be set to a nonnull device_t
and the state is guaranteed not to be larval.

And this cannot be called concurrently with dkwedge_detach, or after
dkwedge_detach does vdevgone until another wedge with the same number
is attached (which can't happen until dkwedge_detach completes), so
the state is guaranteed not to be dying or dead.

Hence sc->sc_dev != NULL and sc->sc_state == DKW_STATE_RUNNING.

(riastradh)

dk(4): Prevent race between dkwedge_get_parent_name and wedge detach.

Still races with parent detach but maybe this is better.

XXX Maybe we should ditch dkwedge_get_parent_name -- it's used only
by rf_containsboot, which kinda suggests it shouldn't exist.

(riastradh)

dk(4): Split unsafe lookups into safe subroutines and unsafe wrappers.

No functional change intended.

Eventually we should adjust the callers to use the safe subroutines
instead and device_release when done.

(riastradh)

dk(4): Don't hold lock around uiomove in dkwedge_list.

Instead, hold a device reference.  dkwedge_detach will not run until
the device reference is released.

(riastradh)

dk(4): Skip larval wedges in various lookup routines.

These have not yet finished a concurent dkwedge_attach, so there's
nothing we can safely do with them.  Just pretend they don't exist --
as if we had arrived at the lookup a moment earlier.

(riastradh)

dk(4): Simplify dkwedge_delall by detaching directly.

No need for O(n^2) algorithm and potentially racy lookups -- not that
n is large enough for n^2 to matter, but the mechanism is simpler
this way.

(riastradh)

dk(4): Use device_lookup_private for dkwedge_lookup.

No longer necessary to go through the dkwedges array.

Currently device_lookup_private still involves touching other global
locks, but that will change eventually to a lockless pserialized fast
path.

(riastradh)

dk(4): dkunit is no longer needed; nix it.

dkwedges array indexing now coincides with autoconf device numbering.

(riastradh)

dk(4): Use config_attach_pseudo_acquire to create wedges.

This way, indexing of the dkwedges array coincides with numbering of
autoconf dk(4) instances.

As a side effect, this plugs a race in dkwedge_add with concurrent
drvctl -r.  There are a lot of such races in dk(4) left -- to be
addressed with more device references.

(riastradh)

autoconf(9): New functions for referenced attach/detach.

New functions:

- config_found_acquire(dev, aux, print, cfargs)
- config_attach_acquire(parent, cf, aux, print, cfargs)
- config_attach_pseudo_acquire(cf, aux)
- config_detach_release(dev, flags)
- device_acquire(dev)

The config_*_acquire functions are like the non-acquire versions, but
they return a referenced device_t, which is guaranteed to be safe to
use until released.  The device's detach function may run while it is
referenced, but the device_t will not be freed and the parent's
.ca_childdetached routine will not be called.

=> config_attach_pseudo_acquire additionally lets you pass an aux
  argument to the device's .ca_attach routine, unlike
  config_attach_pseudo which always passes NULL.

=> Eventually, config_found, config_attach, and config_attach_pseudo
  should be made to return void, because use of the device_t they
  return is unsafe without the kernel lock and difficult to use
  safely even with the kernel lock or in a UP system.  For now,
  they require the caller to hold the kernel lock, while
  config_*_acquire do not.

config_detach_release is like device_release and then config_detach,
but avoids the race inherent with that sequence.

=> Eventually, config_detach should be eliminated, because getting at
  the device_t it needs is unsafe without the kernel lock and
  difficult to use safely even with the kernel lock or in a UP
  system.  For now, it requires the caller to hold the kernel lock,
  while config_detach_release does not.

device_acquire acquires a reference to a device.  It never fails and
can be used in thread context (but not interrupt context, hard or
soft).  Caller is responsible for ensuring that the device_t cannot
be freed; in other words, the device_t must be made unavailable to
any device_acquire callers before the .ca_detach function returns.
Typically device_acquire will be used in a read section (mutex,
rwlock, pserialize, &c.) in a data structure lookup, with
corresponding logic in the .ca_detach function to remove the device
from the data structure and wait for all read sections to complete.

Proposed on tech-kern:
https://mail-index.netbsd.org/tech-kern/2023/05/10/msg028889.html

(riastradh)

tty(9): Make ttwrite update uio with only how much it has consumed.

As is, it leaves uio in an inconsistent state.  Good enough for the
write(2) return value to be correct for a userland caller to restart
write(2) where it left off, but not good enough for a loop in the
kernel to reuse the same uio.

Reported-by: syzbot+e0f56178d0add0d8be20@syzkaller.appspotmail.com
https://syzkaller.appspot.com/bug?id=6290eb02b8fe73361dc15c7bc44e1208601e6af8

Reported-by: syzbot+7caa189e8fccd926357e@syzkaller.appspotmail.com
https://syzkaller.appspot.com/bug?id=c0a3b77b4831dfa81fc855857bde81755d246bd3

Reported-by: syzbot+4a1eff91eb4e7c1970b6@syzkaller.appspotmail.com
https://syzkaller.appspot.com/bug?id=10523a633a4ad9749f57dc7cf03f9447d518c5b8

Reported-by: syzbot+1d3c280f59099dc82e17@syzkaller.appspotmail.com
https://syzkaller.appspot.com/bug?id=8e02ebb0da76a8e286461f33502117a1d30275c6

Reported-by: syzbot+080d51214d0634472b12@syzkaller.appspotmail.com
https://syzkaller.appspot.com/bug?id=1f617747db8087e5554d3df1b79a545dee26a650

Reported-by: syzbot+dd50b448e49e5020131a@syzkaller.appspotmail.com
https://syzkaller.appspot.com/bug?id=f71c8cef4110b7eeac6eca67b6a4d1f4a8b3e96f

Reported-by: syzbot+26b675ecf0cc9dfd8586@syzkaller.appspotmail.com
https://syzkaller.appspot.com/bug?id=57b1901f5b3e090a964d08dd0d729f9909f203be

Reported-by: syzbot+87f0df2c9056313a5c4b@syzkaller.appspotmail.com
https://syzkaller.appspot.com/bug?id=67994a3da32d075144e25d1ac314be1d9694ae6e

Reported-by: syzbot+e5bc98e18aa42f0cb25d@syzkaller.appspotmail.com
https://syzkaller.appspot.com/bug?id=6374bd286532423c63f2b331748280729134224c

Reported-by: syzbot+7e587f4c5aaaf80e84b3@syzkaller.appspotmail.com
https://syzkaller.appspot.com/bug?id=976210ed438d48ac275d77d7ebf4a086e43b5fcb

(riastradh)

uiomove(9): Add uiopeek/uioskip operations.

This allows a caller to grab some data, consume part of it, and
atomically update the uio with only the amount it consumed.  This
way, the caller can use a buffer of a size that doesn't depend on how
much it will actually consume, which it may not know in advance --
e.g., because it depends on how much an underlying hardware tty
device will accept before it decides it has had too much.

Proposed on tech-kern:
https://mail-index.netbsd.org/tech-kern/2023/05/09/msg028883.html

(Opinions were divided between `uioadvance' and `uioskip'.  I stuck
with `uioskip' because that was less work for me.)

(riastradh)

openssl: Limit armv8 stuff to aarch64.

Strictly speaking there is armv8 aarch32, e.g. AES instructions are
exposed to 32-bit mode, but OpenSSL's `*_armv8.S' files are staunchly
aarch64 and don't work when built for aarch64 32-bit compat libraries
(which could take advantage of armv8 instructions, but only if the
code is actually aarch32, which it isn't).

The armv8 stuff is still included in aarch64 /usr/lib/libcrypto, just
not in the 32-bit compat libraries.

With any luck, this will fix the aarch64 clang build (again).

(riastradh)

openssl compat: Try this again!

compatdir appears to be for things like /usr/lib that have
/usr/lib/eabi as a subdirectory, so for /usr/lib/openssl it would
mean we have /usr/lib/openssl/eabi as a subdirectory.  But that's not
what we have; we have /usr/lib/eabi/openssl.

Let's try using NetBSD.dist.compat.in instead, which appears to be
intended for this purpose, from what I can gather.

(riastradh)

kern/sys_descrip.c: Nix trailing whitespace.

(riastradh)

x86/efiboot: Nix trailing whitespace.

(riastradh)

x86/locore.S: Nix trailing whitespace.

(riastradh)

x86/boot(8): Nix trailing whitespace.

(riastradh)

options(4): Nix trailing whitespace.

(riastradh)

openssl: More set list fixes.

(riastradh)

openssl: More tests need -Wno-missing-field-initializers.

(riastradh)

openssl: libcrypto bf test needs -Wno-missing-field-initializers

(riastradh)

aarch64: New directories for compat openssl on aarch64.

Hope I got the setlist tags right, not really sure how this works.

(riastradh)

libfido2: Needs -Wno-error=pointer-sign on arm.

(riastradh)

libcryptotest: Suppress some warnings.

Detected by clang aarch64 32-bit arm compat library build.

(riastradh)

compat: Disable relro in 32-bit arm compat.

Not supported by binutils:

ld: warning: -z relro ignored

PR toolchain/57405

(riastradh)

compat: Nix trailing whitespace.

No functional change intended.

(riastradh)

dk(4): Need pdk->dk_openlock to read pdk->dk_wedges.

(riastradh)

distrib/sets/lists: Nix trailing whitespace.

No functional change intended.

(riastradh)

i915: Avoid dereferencing null fence if resv has changed.

PR kern/57402

XXX pullup-10

(riastradh)

dk(4): Make it clearer that dkopen EROFS branch doesn't leak.

It looked like we may need to sometimes call dklastclose in error
branch for the case of (flags & ~sc->sc_mode & FWRITE) != 0, but it
is not actually possible to reach that case: if the caller requested
read/write, and the parent is read-only, and it is the first time
we've opened the parent, then dkfirstopen will fail with EROFS so we
never get there.

But this is confusing and it looked like the error branch is wrong,
so let's rearrange the conditional to make it clearer that we cannot
goto out after dkfirstopen has succeeded.  And then assert that the
case cannot happen when we do call dkfirstopen.

(riastradh)

opl(4): Use config_detach_children.

(riastradh)

acpi(4): Fix membars in ACPI_ACQUIRE/RELEASE_GLOBAL_LOCK.

XXX pullup-8 (by patch with membar_enter/exit)
XXX pullup-9 (by patch with membar_enter/exit)
XXX pullup-10

(riastradh)

uts(4): Use config_detach_children.

(riastradh)

udsbr(4): Use config_detach_children.

(riastradh)

irmce(4): Use config_detach_children.

(riastradh)

xirc(4): Use config_detach_children.

(riastradh)

pcmcom(4): Use config_detach_children.

(riastradh)

mhzc(4): Use config_detach_children.

(riastradh)

ismt(4): Use config_detach_children.

(riastradh)

esa(4): Use config_detach_children.

(riastradh)

emuxki(4): Use config_detach_children.

(riastradh)

cxdtv(4): Use config_detach_children.

(riastradh)

auich(4): Use config_detach_children.

(riastradh)

nandemulator(4): Use config_detach_children.

(riastradh)

wb(4): Use config_detach_children.

(riastradh)

spc(4): Use config_detach_children.

(riastradh)

aic(4): Use config_detach_children.

(riastradh)

ims(4): Use config_detach_children.

(riastradh)

hpf1275a(4): Use config_detach_children.

(riastradh)

hilms(4): Use config_detach_children.

(riastradh)

gpiosim(4): Use config_detach_children.

(riastradh)

gpioow(4): Use config_detach_children.

(riastradh)

gpioiic(4): Use config_detach_children.

(riastradh)

acpiwmi(4): Use config_detach_children.

(riastradh)

sandpoint: Use config_detach_children to reduce error branch bugs.

(riastradh)

powerpc: Use config_detach_children to reduce error branch bugs.

(riastradh)

mips: Use config_detach_children to reduce error branch bugs.

(riastradh)

x86/imc(4): Use config_detach_children.

(riastradh)

ichlpcib(4): Use config_detach_children.

Delete a lot of unnecessary code with broken error branches involving
config_detach which have probably seldom if ever been exercised.

No substantive functional change intended.  Low risk because
ichlpcib(4) is not a removable device, so you have to go out of your
way to exercise detach.

(riastradh)

ichlpcib(4): KNF.  No functional change intended.

(riastradh)

bsd.lib.mk: Nix trailing whitespace.

(riastradh)

dtrace_fbt: Read and write instruction words in little-endian.

Step toward making this work on aarch64eb.

(riastradh)

dk(4): Fix typo: sc_state, not sc_satte.

Had tested a patch series, but not every patch in it, and I
inadvertently fixed the typo in a later patch in the series, not in
the one I committed.

(riastradh)

dk(4): Omit needless sc_iopend, sc_dkdrn mechanism.

vdevgone guarantees that all instances are closed by the time it
returns, which in turn guarantees all I/O operations (read, write,
ioctl, &c.) have completed, and, if the block device is open,
vinvalbuf(V_SAVE) -> vflushbuf has completed, which forces all
buffered transfers to be issued and waits for them to complete.

So by the time vdevgone returns, no further transfers can be
submitted and the bufq must be empty.

(riastradh)

ioctl(DIOCRMWEDGES): Delete only idle wedges.

Don't forcibly delete busy wedges.

Reported-by: syzbot+e46f31fe56e04f567d88@syzkaller.appspotmail.com
https://syzkaller.appspot.com/bug?id=8a00fd7f2e7459748d7a274098180a4708ff0f61

Fixes accidental destruction of the busy wedge that the root file
system is mounted on, triggered by syzbot's ioctl(DIOCRMWEDGES).

(riastradh)

dk(4): dkclose must handle a dying wedge too to close the parent.

Otherwise the parent open leaks on detach (or revoke) when the wedge
was open and had to be forcibly closed.

Reported-by: syzbot+e46f31fe56e04f567d88@syzkaller.appspotmail.com
https://syzkaller.appspot.com/bug?id=8a00fd7f2e7459748d7a274098180a4708ff0f61

Fixes assertion sc->sc_dk.dk_openmask == 0.

(riastradh)

mutex(9): Write comments in terms of ordering semantics.

Phrasing things in terms of implementation details like `acquiring
and locking cache lines' both suggests a particular cache coherency
protocol, paints an incomplete picture for more involved protocols,
and doesn't really help to prove theorems the way ordering relations
do.

No functional change intended.

(riastradh)

mutex(9): Omit needless membar_consumer.

In practical terms, this is not necessary because MUTEX_SET_WAITERS
already issues MUTEX_MEMBAR_ENTER, which on all architectures is a
sequential consistency barrier, i.e., read/write-before-read/write,
subsuming membar_consumer.

In theoretical terms, MUTEX_MEMBAR_ENTER might imply only
write-before-read/write, so one might imagine that the
read-before-read ordering of membar_consumer _could_ be necessary.
However, the memory operations that are significant here are:

1. load owner := mtx->mtx_owner
2. store mtx->mtx_owner := owner | MUTEX_BIT_WAITERS
3. load owner->l_cpu->ci_curlwp to test if equal to owner

(1) is program-before (2) and at the same memory location,
mtx->mtx_owner, so (1) happens-before (2).

And (2) is separated in program order by MUTEX_MEMBAR_ENTER from (3),
so (2) happens-before (3).

So even if the membar_consumer were intended to guarantee that (1)
happens-before (3), it's not necessary, because we can already prove
it from MUTEX_MEMBAR_ENTER.

But actually, we don't really need (1) happens-before (3), exactly;
what we really need is (2) happens-before (3), since this is a little
manifestation of Dekker's algorithm between cpu_switchto and
mutex_exit, where each CPU sets one flag and must ensure it is
visible to the other CPUs before testing the other flag -- one flag
here is the MUTEX_BIT_WAITERS bit, and the other `flag' here is the
condition owner->l_cpu->ci_curlwp == owner; the corresponding logic,
in cpu_switchto, is:

1'. store owner->l_cpu->ci_curlwp := owner
2'. load mtx->mtx_owner to test if MUTEX_BIT_WAITERS set

(riastradh)

drm: KASSERT(A && B) -> KASSERT(A); KASSERT(B)

(riastradh)

xen_clock(4): Sprinkle dtrace probes.

(riastradh)

sys/sdt.h: Omit spurious trailing semicolon in DTRACE_PROBE*.

(riastradh)

sys/sdt.h: Nix needless SDT_VAR_DECL.

Previously introduced as a provisional build fix, its need has been
obviated by making SDT_PROBE* do __USE on the arguments in builds
without KDTRACE_HOOKS.

(riastradh)

kern/vfs_subr.c: Revert previous build fixes, no longer needed.

SDT_PROBE* will now DTRT here.

(riastradh)

sys/sdt.h: Make sure SDT_PROBE*, DTRACE_PROBE* compile the arguments.

This way there's no need to fuss around with conditional declarations
and assignments in the callers to avoid `variable set but not used'
errors in builds without KDTRACE_HOOKS.

(riastradh)

sys/cdefs.h: New __MACROUSE for macros to type-check arguments.

This way, if there's a macro F(x) with a conditional definition --
e.g., conditional on DIAGNOSTIC or KDTRACE_HOOKS -- it can do
__MACROUSE(x) in the definition that's supposed to compile away,
without triggering `variable set but not used' errors, while still
catching type errors in the expression x.  But there's no cost or
side effects incurred in the generated code because the value is not
computed.

This should be the same as __USE, but I haven't figured out how to
make it work for both bit fields (which are likely to appear in macro
arguments) and aggregates (structs/unions, which are likely to appear
as variable declarations), so for now we use two separate macros.

(riastradh)

sys/sdt.h: Whitespace cleanup (and internal macro semicolon tidying).

No functional change intended.

(riastradh)

sys/sdt.h: Properly parenthesize macro arguments.

(riastradh)

sys/sdt.h: Nix trailing whitspace.

No functional change intended.

(riastradh)

dk(4): Rename label for consistency.  No functional change intended.

(riastradh)

vfs: Sprinkle dtrace probes into syncer.

(riastradh)

vfs(9): Move SDT_PROVIDER_DEFINE(vfs) from vfs_cache.c to vfs_init.c.

Not a namecache-specific thing.

(riastradh)

kern/vfs_init.c: Sort includes.  No functional change intended.

(riastradh)

sys/vnode_impl.h: Declare vfs sdt provider.

This way it can be used by more than just vfs_cache.c.

(riastradh)

kern/vfs_subr.c: Sort includes.  No functional change intended.

(riastradh)

tmpfs: Assert no arithmetic overflow in directory node tn_size.

Need >2^57 directory entries before this is a problem.  If we created
a million per second, this would take over 4000 years.

(riastradh)

tmpfs: Refuse sizes that overflow round_page.

Reported-by: syzbot+8dbeee84de15f86df65b@syzkaller.appspotmail.com
https://syzkaller.appspot.com/bug?id=4a27b9fe074f8d4b0afbe22969339b8dfdb157e8

(riastradh)

kern/vfs_syscalls.c: Nix trailing whitesapce.

No functional change intended.

(riastradh)

tmpfs: Nix trailing whitespace.  No functional change intended.

(riastradh)

sys/vnode_impl.h: Nix trailing whitespace.

No functional change intended.

(riastradh)

dk(4): Fix lock assertion in size increase: parent's, not wedge's.

Reported-by: syzbot+d4dc610473cacc5183dd@syzkaller.appspotmail.com
https://syzkaller.appspot.com/bug?id=e18ddae8283d6fab44cfb1ac7e3f8e791f8c0700

(riastradh)

dlopen(3): Read _rtld_objtail under the lock.

XXX pullup-8
XXX pullup-9
XXX pullup-10

(riastradh)

rumpdev: Catch up to buf_syncwait -> vfs_syncwait rename.

Do similarly with a do_sys_sync weak stub so we can drop the rumpvfs
dependency.

(riastradh)

rumpvfs: Link in secmodel_extensions_vfs.c as intended, like kernel.

Forgot to commit this part when splitting up secmodel_extensions.c.

(riastradh)

specfs: KNF.  No functional change intended.

(riastradh)

fcntl(2), flock(2): Assert FHASLOCK is clear if no fo_advlock.

(riastradh)

fcntl(2), flock(2): Unify error branches.

Let's make this a bit less error-prone by having everything converge
in the same place instead of multiple returns in different contexts.

(riastradh)

fcntl(2), flock(2): Fix missing fd_putfile in error branch.

Oops!

(riastradh)

Welcome to 10.99.4, courtesy of struct fileops additions!

(riastradh)

rump: Move vnode_if.c from rumpkern to rumpvfs.

This has become increasingly less of a `fully dynamic interface', and
the need for it in the rest of sys/kern/ has been obviated, so let's
put it where it belongs in rumpvfs.

(riastradh)

secmodel_extensions: Split vfs part into separate .c file.

This way we can provide weak rumpkern stubs that don't require
rumpvfs for things that are relevant to vfs, but if you do link
rumpvfs then you get the same logic in secmodel extensions.

(riastradh)

rump: Move ubc_uiomove, ubc_zerorange from rumpvfs to rumpkern.

Needed by kern_ksyms.c.  We could split kern_ksyms.c up into kernel-
internal interfaces and the user/kernel /dev/ksyms interface, but
there's nothing vfs-specific about ubc_uiomove and ubc_zerorange
anyway, so let's just define them in rumpkern.

(riastradh)

rumpkern: Provide stub pnbuf_cache weak alias.

Needed only by sys_module.c handle_modctl_load, which won't work if
there's no rumpvfs linked in anyway.

(riastradh)

rumpkern: Provide stub uvm_vnodeops weak alias.

Needed for UVM_OBJ_IS_VNODE, but not for anything else in rumpkern
without rumpvfs.

(riastradh)

rumpkern: Provide fs_filtops weak alias.

We should introduce a link set for sys_kfilters, but this works
around the problem for now -- librump's dependency on librumpvfs
symbols.

(riastradh)

file(9): New fo_posix_fadvise operation.

XXX kernel revbump -- changes struct fileops API and ABI

(riastradh)

file(9): New fo_fpathconf operation.

XXX kernel revbump -- struct fileops API and ABI change

(riastradh)

file(9): New fo_advlock operation.

This moves the vnode-specific logic from sys_descrip.c into
vfs_vnode.c, like we did for fo_seek.

XXX kernel revbump -- struct fileops API and ABI change

(riastradh)

sys/file.h: Sort forward declarations.

No functional change intended.

(riastradh)

dk(4): Convert tests to assertions in various devsw operations.

.d_cancel, .d_strategy, .d_read, .d_write, .d_ioctl, and .d_discard
are only ever used between successful .d_open return and entry to
.d_close.  .d_open doesn't return until sc is nonnull and sc_state is
RUNNING, and dkwedge_detach waits for the last .d_close before
setting sc_state to DEAD.  So there is no possibility for sc to be
null or for sc_state to be anything other than RUNNING or DYING.

There is a small functional change here but only in the event of a
race: in the short window between when dkwedge_detach is entered, and
when .d_close runs, any I/O operations (read, write, ioctl, &c.) may
be issued that would have failed with ENXIO before.

This shouldn't matter for anything: disk I/O operations are supposed
to complete reasonably promptly, and these operations _could_ have
begun milliseconds prior, before dkwedge_detach was entered, so it's
not a significant distinction.

Notes:

- .d_open must still contend with trying to open a nonexistent wedge,
  of course.

- .d_close must also contend with closing a nonexistent wedge, in
  case there were two calls to open in quick succession and the first
  failed while the second hadn't yet determined it would fail.

- .d_size and .d_dump are used from ddb without any open/close.

(riastradh)

dk(4): Fix racy access to sc->sc_dk.dk_openmask in dkwedge_delall1.

Need sc->sc_parent->dk_rawlock for this, as used in dkopen/dkclose.

(riastradh)

disk(9): Fix missing unlock in error branch in previous change.

Reported-by: syzbot+870665adaf8911c0d94d@syzkaller.appspotmail.com
https://syzkaller.appspot.com/bug?id=a4ae17cf66b5bb999182ae77fd3c7ad9ad18c891

(riastradh)

readdir(2), lseek(2): Fix races in access to struct file::f_offset.

For non-directory vnodes:
- reading f_offset requires a shared or exclusive vnode lock
- writing f_offset requires an exclusive vnode lock

For directory vnodes, access (read or write) requires either:
- a shared vnode lock AND f_lock, or
- an exclusive vnode lock.

This way, two files for the same underlying directory vnode can still
do VOP_READDIR in parallel, but if two readdir(2) or lseek(2) calls
run in parallel on the same file, the load and store of f_offset is
atomic (otherwise, e.g., on 32-bit systems it might be torn and lead
to corrupt offsets).

There is still a potential problem: the _whole transaction_ of
readdir(2) may not be atomic.  For example, if thread A and thread B
read n bytes of directory content, thread A might get bytes [0,n) and
thread B might get bytes [n,2n) but f_offset might end up at n
instead of 2n once both operations complete.  (However, f_offset
wouldn't be some corrupt garbled number like n & 0xffffffff00000000.)
Fixing this would require either:
(a) using an exclusive vnode lock in vn_readdir,
(b) introducing a new lock that serializes vn_readdir on the same
    file (but ont necessarily the same vnode), or
(c) proving it is safe to hold f_lock across VOP_READDIR, VOP_SEEK,
    and VOP_GETATTR.

(riastradh)

dk(4): Narrow the scope of the device numbering lookup on detach.

Just need it for vdevgone, order relative to other things in detach
doesn't matter.

No functional change intended.

(riastradh)

dk(4): dkdump: Simplify.  No functional change intended.

(riastradh)

dk(4): Omit needless locking in dksize, dkdump.

All the members these use are stable after initialization, except for
the wedge size, which dkwedge_size safely reads a snapshot of without
locking in the caller.

(riastradh)

dk(4): Take a read-lock on dkwedges_lock if we're only reading.

- dkwedge_find_by_name
- dkwedge_find_by_parent
- dkwedge_print_wnames

(riastradh)

dk(4): Set .d_cfdriver and .d_devtounit to plug open/detach race.

This way, opening dkN or rdkN will wait if attach or detach is still
in progress, and vdevgone will wake up such pending opens and make
them fail.  So it is no longer possible for a wedge to be detached
after dkopen has already started using it.

For now, we use a custom .d_devtounit function that looks up the
autoconf unit number via the dkwedges array, which conceivably may
use an independent unit numbering system -- nothing guarantees they
match up.  (In practice they will mostly match up, but concurrent
wedge creation could lead to different numbering.)  Eventually this
should be changed so the two numbering systems match, which would let
us delete the new dkunit function and just use dev_minor_unit like
many other drivers can.

(riastradh)

dk(4): Use disk_begindetach and rely on vdevgone to close instances.

The first step is to decide whether we can detach (if forced, yes; if
not forced, only if not already open), and prevent new opens if so.
There's no need to start closing open instances at this point --
we're just making a decision to detach, and preventing new opens by
transitioning state that dkopen will respect[*].

The second step is to force all open instances to close.  This is
done by vdevgone.  By the time vdevgone returns, there can be no open
instances, so if there _were_ any, closing them via vdevgone will
have passed through dklastclose.

After that point, there can be no opens and no I/O operations, so
dk_openmask must already be zero and the bufq must be empty.

Thus, there's no need to have an explicit call to dklastclose (via
dkwedge_cleanup_parent) before or after making the decision to
detach.

[*] Currently access to this state is racy: nothing serializes
    dkwedge_detach's state transition with dkopen's test.  TBD in a
    separate commit shortly.

(riastradh)

dk(4): Fix callout detach race.

1. Set a flag sc_iostop under the lock sc_iolock so dkwedge_detach
  and dkstart don't race over it.

2. Decline to schedule the callout if sc_iostop is set.  The callout
  is already only ever scheduled while the lock is held.

3. Use callout_halt to wait for any concurrent callout to complete.
  At this point, it can't reschedule itself.

Without this change, the callout could be concurrently rescheduling
itself as we issue callout_stop, leading to use-after-free later.

(riastradh)

dk(4): Add null d_cancel routine to devsw.

This way, dkclose is guaranteed that dkopen, dkread, dkwrite,
dkioctl, &c., have all returned before it runs.  For block opens,
setting d_cancel also guarantees that any buffered writes are flushed
with vinvalbuf before dkclose is called.

(riastradh)

disk(9): Fix use-after-free race with concurrent disk_set_info.

This can happen with dk(4), which allows wedges to have their size
increased without destroying and recreating the device instance.

Drivers which allow concurrent disk_set_info and disk_ioctl must
serialize disk_set_info with dk_openlock.

(riastradh)

dk(4): Require dk_openlock in dk_set_geometry.

Not strictly necessary but this makes reasoning easier and documents
with an assertion how disk_set_info is serialized.

(riastradh)

dk(4): Assert dkwedges[unit] is the sc we're about to free.

(riastradh)

dk(4): Assert parent vp is nonnull before we stash it away.

Let's enable early attribution if this goes wrong.

If it's not the parent's first open, also assert the parent vp is
already nonnull.

(riastradh)

dk(4): Don't touch dkwedges or ndkwedges outside dkwedges_lock.

(riastradh)

dk(4): Move CFDRIVER_DECL and CFATTACH_DECL3_NEW earlier in file.

Follows the pattern of most drivers, and will be necessary for
referencing dk_cd in dk_bdevsw and dk_cdevsw soon, to prevent
open/detach races.

No functional change intended.

(riastradh)

dk(4): Prevent races in access to struct dkwedge_softc::sc_size.

Rules:

1. Only ever increases, never decreases.

  (Decreases require removing and readding the wedge.)

2. Increases are serialized by dk_openlock.

3. Reads can happen unlocked in any context where the softc is valid.

Access is gathered into dkwedge_size* subroutines -- don't touch
sc_size outside these.  For now, we use rwlock(9) to keep the
reasoning simple.  This should be done with atomics on 64-bit
platforms and a seqlock on 32-bit platforms to avoid contention.
However, we can do that in a later change.

(riastradh)

dk(4): <sys/rwlock.h> for rwlock(9).

(riastradh)

dk(4): KNF: Sort includes.

No functional change intended.

(riastradh)

dk(4): ENXIO, not ENODEV, means no such device.

ENXIO is `device not configured', meaning there is no such device.

ENODEV is `operation not supported by device', meaning the device is
there but refuses the operation, like writing to a read-only medium.

Exception: For undefined ioctl commands, it's not ENODEV _or_ ENXIO,
but rather ENOTTY, because why make any of this obvious when you
could make it obscure Unix lore?

(riastradh)

dk(4): Fix typo in comment: dkstrategy, not dkstragegy.

No functional change intended.

(riastradh)

dk(4): Omit needless void * cast.

No functional change intended.

(riastradh)

dk(4): KNF: Whitespace.

No functional change intended.

(riastradh)

dk(4): KNF: return (v) -> return v.

No functional change intended.

(riastradh)

dk(4): Avoid holding dkwedges_lock while allocating array.

This is not great -- we shouldn't be choosing the unit number here
anyway; we should just let autoconf do it for us -- but it's better
than potentially blocking any dk_openlock or dk_rawlock (which are
sometimes held when waiting for dkwedges_lock) for memory allocation.

(riastradh)

dk(4): Restore assertions in dklastclose.

We only enter dklastclose if the wedge is open (sc->sc_dk.dk_openmask
!= 0), which can happen only if dkfirstopen has succeeded, in which
case we hold a dk_rawopens reference to the parent that prevents
anyone else from closing it.  Hence sc->sc_parent->dk_rawopens > 0.

On open, sc->sc_parent->dk_rawvp is set to nonnull, and it is only
reset to null on close.  Hence if the parent is still open, as it
must be here, sc->sc_parent->dk_rawvp must be nonnull.

(riastradh)

autoconf(9): Add a comment where we risk arithmetic overflow.

(riastradh)

sb(4): Fix missing mutex initialization at isapnp and ofisa.

PR kern/57363

XXX pullup-8
XXX pullup-9
XXX pullup-10

(riastradh)

viocon(4): Show error in assertion failure.

(riastradh)

virtio@pci: Fix assertion on detach.

If the child never attached in the first place, it's OK for it to not
have detached.

XXX This should not be a set of flags; this should be a state
enumeration, because some flags make no sense, like FINISHED|FAILED.

XXX This should not be asserted separately in each bus; there should
be a single place in virtio.c to assert this, uniformly in all buses.

PR kern/57357

XXX pullup-10

(riastradh)

autoconf(9): Assert alldevs_lock held in config_unit_nextfree.

The one caller, config_unit_alloc, guarantees it through
config_alldevs_enter/exit.

(riastradh)

autoconf(9): Avoid potential ABA bug in config_makeroom.

When we unlock alldevs_lock to allocate a new cd_devs array nsp,
other threads may have:

1. freed the old one (osp),
2. done some other memory allocation,
3. allocated a new _larger_ array whose address happens to concide
  with osp (e.g., in (2) the page was recycled for a different pool
  cache), and
4. updated cd_devs back to osp but increased cd_ndevs.

In that case, the memory may be corrupted: we try to copy the wrong
number of device_t pointers into nsp and we free osp with the wrong
(stale) length.

Avoid this by checking whether cd_ndevs has changed too -- if not,
osp might have been recycled but at least the lengths we're about to
copy and free are still correct so there's no harm in an ABA
situation.

XXX pullup-8
XXX pullup-9
XXX pullup-10

(riastradh)

pserialize(9): Fix buggy assertion inside assertion.

If there has been a context switch, then we can't be in a pserialize
read section -- _and_ whatever we read out of curcpu() is stale, so
we can't assert anything based on it.  So just delete the assertion.

Reported-by: syzbot+d7065ab16d7cd9cc09af@syzkaller.appspotmail.com
https://syzkaller.appspot.com/bug?id=0c63db890575d29c6a51fb24b003d3bab1f7f04b

(riastradh)

pserialize(9): Fix bug in recent micro-optimization.

curcpu() is always a valid struct cpu_info pointer, but curlwp->l_cpu
is not necessarily -- early at boot on some architectures (like x86,
apparently) it may be null.

Not clear that l->l_cpu is always a cost win over curcpu(): on some
architectures curcpu() is basically a macro for curlwp->l_cpu, while
on others it is a fixed VA.  So let's use the one that always works
instead of the one that's sometimes cheaper and sometimes crashier.

(riastradh)

dk(4): Explain why dk_rawopens can't overflow and assert it.

(riastradh)

sys/modules/Makefile: Use MACHINE_CPU, not MACHINE_ARCH, for aarch64.

This way it covers both little-endian and big-endian.

(riastradh)

powerpc/ddb: Fix one more load to use db_read_bytes.

Fix some typos in crash(8) comments too.

XXX pullup-8
XXX pullup-9
XXX pullup-10

(riastradh)

powerpc/ddb: Use db_read_bytes, not direct pointer access.

Mark some powerpc-variant ifdefs with XXX crash(8), not sure yet what
to do about them.

XXX pullup-8
XXX pullup-9
XXX pullup-10

(riastradh)

crash(8): Add powerpc support.

XXX pullup-8
XXX pullup-9
XXX pullup-10

(riastradh)

Welcome to 10.99.3!

Courtesy of:
- deleted ttycheckoutq parameter
- deleted mutex_owner symbol
- tco(4) at ichsmb(4)
- new m_get_n, m_gethdr_n symbols

(riastradh)

mbuf(9): New m_get_n, m_gethdr_n.

m_get_n(how, type, alignbytes, nbytes) returns an mbuf with no packet
header having space for nbytes, with an internal buffer pointer
aligned by alignbytes (typically ETHER_ALIGN or similar, if not
zero).

m_gethdr_n(how, type, alignbytes, nbytes) does the same but for an
mbuf with a packet header.

These return NULL on failure, which can happen either:
(a) because how is M_DONTWAIT and allocating memory would sleep, or
(b) because alignbytes + nbytes > MCLBYTES.

On exit, m_len is set to nbytes, as is m_pkthdr.len for m_gethdr_n.

These should be used to systematically replace all calls to m_get,
m_gethdr, MGET, MGETHDR, and m_getcl.  Most calls to m_clget and
MCLGET will probably evaporate as a consequence.

Proposed on tech-net last year:
https://mail-index.netbsd.org/tech-net/2022/07/16/msg008285.html

(riastradh)

ichsmb(4), tco(4): Add support for TCO on newer Intel chipsets.

TCO (`Total Cost of Ownership', Intel's bizarre name for a watchdog
timer) used to hang off the Intel I/O platform controller hub's (ICH)
low-pin-count interface bridge (LPC IB), or ichlpcib(4).  On newer
devices, it hangs off the ICH SMBus instead.

Tested on INTEL 100SERIES_SMB (works) and INTEL 100SERIES_LP_SMB
(doesn't work, still not sure why).

XXX kernel revbump: This breaks the module ABI -- tco(4) modules
older than the change to make ta_has_rcba into ta_version will
incorrectly attach at buses they do not understand.  (However, the
tco(4) driver is statically built into GENERIC, so maybe it's safe
for pullup since the module wouldn't have worked anyway.)

(riastradh)

kern: Nix mutex_owner.

There is no valid reason to use this except in assertions of the form

KASSERT(mutex_owner(lock) == curlwp),

which is more obviously spelled as

KASSERT(mutex_owned(lock)).

Exception: There's one horrible kludge in zfs that abuses this, which
should be eliminated.

XXX kernel revbump -- deleting symbol

PR kern/47114

(riastradh)

ttycheckoutq(9): wait=0 always, parameter no longer useful, nix it.

XXX kernel revbump

(riastradh)

lfs: Assert page identity doesn't change.

Forgot what I was debugging when I inserted a relookup in my local
tree months or years ago, but whatever it was, if that solved a
problem, this KDASSERT will make the problem more obvious.

(riastradh)

style(5): Omit redundant proscription against .c externs in previous.

(riastradh)

rndctl(8): Take omitted name/type as empty device name.

This can now be used to change flags for all sources at once.

XXX pullup-10

(riastradh)

x86: Omit needless membar_sync in intr_disestablish_xcall.

Details in comments.

(riastradh)

thmap(9): kmem_zalloc(KM_SLEEP) never fails.  Prune dead branch.

(riastradh)

com(4): Note ttylock in comment, not tty_lock.

(riastradh)

evbarm/plcom(4): Note ttylock in comment, not tty_lock.

(riastradh)

evbarm/plcom(4): Sync some recent changes from com(4).

- Nix quirky `integrate' macro.
- Omit needless spltty in plcomstart.
- Comment on lock order.
- Update confusing comment about hangup delay.
- Add include guards.

(riastradh)

com(4): Note timecounter_lock in lock order comments.

(riastradh)

aarch64: Build drm modules.

(riastradh)

ttwrite(9): Assert we mangle uio_resid only if we also return error.

(riastradh)

uiomove(9): Stronger assertions about iov array.

(riastradh)

kmsan: Format exact instruction addresses relative to symbols.

(riastradh)

usb(9): Assert ud_ifaces is null before we clobber it.

(riastradh)

arm/rockchip: Omit needless functions.

(riastradh)

xhci(4): Avoid crash in suspend/resume/resume if first resume fails.

Rather than try to recover from this, just make new commands fail so
at least we don't deadlock.

XXX pullup-9
XXX pullup-10

(riastradh)

uvm: Simplify assertion in uvn_get.

No functional change intended.

(riastradh)

nfs: Simplify assertion.  No functional change intended.

(riastradh)

dwc2: KASSERT(A && B) -> KASSERT(A); KASSERT(B)

(riastradh)

compat_linux32: KASSERT(A && B) -> KASSERT(A); KASSERT(B)

(riastradh)

genfs: KASSERT(A && B) -> KASSERT(A); KASSERT(B)

(riastradh)

pool(9): Tweak branch prediction in pool_cache_get_paddr assertion.

No functional change intended.

(riastradh)

kpause(9): Simplify assertion.  No functional change intended.

(riastradh)

pool(9): Simplify assertion in pool_update_curpage.

Add message while here.

(riastradh)

kern: KASSERT(A && B) -> KASSERT(A); KASSERT(B)

(riastradh)

uvm(9): KASSERT(A && B) -> KASSERT(A); KASSERT(B)

(riastradh)

kmem(9): Tweak branch predictions in fast paths.

(riastradh)

pserialize(9): Micro-optimize pserialize_read_exit.

Most of the time we're not cold, so let's convince gcc to make that a
statically predicted-not-taken branch.

(riastradh)

i386: Make curlwp and curcpu() flushable.

The only effect of the `volatile' qualifier on an asm block with
outputs is to force the instructions to appear in the generated code,
even if the outputs end up being unused.  Since these instructions
have no (architectural) side effects -- provided %fs is set
correctly, which must be the case here -- there's no need for the
volatile qualifier, so nix it.

(riastradh)

amd64: Make curlwp and curcpu() flushable.

The only effect of the `volatile' qualifier on an asm block with
outputs is to force the instructions to appear in the generated code,
even if the outputs end up being unused.  Since these instructions
have no (architectural) side effects -- provided %gs is set
correctly, which must be the case here -- there's no need for the
volatile qualifier, so nix it.

(riastradh)

pserialize(9): Micro-optimize pserialize_not_in_read_section_p.

Load l_ncsw to test whether we have been preempted, rather than
loading and storing l_nopreempt (via function call) to prevent it.

(riastradh)

ASSERT_SLEEPABLE(9): Micro-optimize this a little bit.

This convinces gcc to do less -- make a smaller stack frame, compute
fewer conditional moves in favour of predicted-not-taken branches --
in the fast path where we are sleepable as the caller expects.

Wasn't able to convince it to do the ncsw loop with a
predicted-not-taken branch, but let's leave the __predict_false in
there anyway because it's still a good prediction.

(riastradh)

xhci(4): Defer root intr xfers while polling.

Root intr xfers require taking adaptive locks, which is forbidden
while polling.

This is not great -- any USB transfer completion callbacks might try
to take adaptive locks, not just uhub_intr, and that will always
causes trouble.  We get lucky with ukbd_intr because it's not
MP-safe, so it relies only on the kernel lock (a spin lock) anyway.
But this change brings xhci in line with ehci.

PR kern/57326

XXX pullup-8
XXX pullup-9
XXX pullup-10

(riastradh)

libc atomic: Make previous work a little less accidentally.

(riastradh)