Reverse sense of NO_STATIC_MODULES -> OPENPAM_STATIC_MODULES.

This avoids leaking NO_STATIC_MODULES into the public header, which
has led to considerable confusion and workarounds in pkgrsc.

PR security/39313
PR security/55216

ok christos

(riastradh)

Make FUTEX_WAIT_BITSET(bitset=0) fail with EINVAL to match Linux.

(riastradh)

Fix waiting on a zero bitset.

The logic in futex_wait assumes there are two paths out:

1. Error (signal or timeout), in which case we take ourselves off the
  queue.

2. Wakeup, in which case the waker takes us off the queue.

But if the user does FUTEX_WAIT_BITSET(bitset=0), as in the
futex_wait_pointless_bitset test, then we will never even go to
sleep, so there will be nobody to wake us as in (2), but it's not an
error as in (1) either.  As a result, we're left on the queue.

Instead, don't bother with any of the wait machinery in that case.
This does not actually match Linux semantics -- Linux returns EINVAL
if bitset is zero.  But let's make sure this passes the releng test
rig as the tests are written now, and then fix both the logic and the
tests -- this is a candidate fix for:

lib/libc/sys/t_futex_ops (277/847): 20 test cases
    futex_basic_wait_wake_private: [6.645189s] Passed.
    futex_basic_wait_wake_shared: [6.572692s] Passed.
    futex_cmp_requeue: [4.624082s] Passed.
    futex_requeue: [4.427191s] Passed.
    futex_wait_pointless_bitset: [0.202865s] Passed.
    futex_wait_timeout_deadline: [ 9074.4164779] panic: TAILQ_INSERT_TAIL 0xffff000056a1ad48 /tmp/bracket/build/2020.04.28.03.00.23-evbarm-aarch64/src/sys/kern/sys_futex.c:826
[ 9074.4340691] cpu0: Begin traceback...
[ 9074.4340691] trace fp ffffc0004ceffb40
[ 9074.4340691] fp ffffc0004ceffb60 vpanic() at ffffc000004aac58 netbsd:vpanic+0x160
[ 9074.4441432] fp ffffc0004ceffbd0 panic() at ffffc000004aad4c netbsd:panic+0x44
[ 9074.4441432] fp ffffc0004ceffc60 futex_wait_enqueue() at ffffc000004b7710 netbsd:futex_wait_enqueue+0x138
[ 9074.4555795] fp ffffc0004ceffc80 futex_func_wait.part.5() at ffffc000004b82f4 netbsd:futex_func_wait.part.5+0x17c
[ 9074.4660518] fp ffffc0004ceffd50 do_futex() at ffffc000004b8cd8 netbsd:do_futex+0x1d0
[ 9074.4660518] fp ffffc0004ceffdf0 sys___futex() at ffffc000004b9078 netbsd:sys___futex+0x50

(riastradh)

Rename futex_get -> futex_lookup_create.  Remove futex_put.

Just use futex_rele instead of futex_put.  There may once have been a
method to the madness this alias in an early draft but there is no
longer.

No functional change; all names are private to sys_futex.c.

(riastradh)

Fix races in aborted futex waits.

- Re-check the wake condition in futex_wait in the event of error.
  => Otherwise, if futex_wait times out in cv_timedwait_sig but
    futex_wake wakes it while cv_timedwait_sig is still trying to
    reacquire fw_lock, the wake would be incorrectly accounted.

- Fold futex_wait_abort into futex_wait so it happens atomically.
  => Otherwise, if futex_wait times out and release fw_lock, then,
    before futex_wait_abort reacquires the lock and removes it from
    the queue, the waiter could be woken by futex_wake.  But once we
    enter futex_wait_abort, the decision to abort is final, so the
    wake would incorrectly accounted.

- In futex_wait_abort, mark each waiter aborting while we do the lock
  dance, and skip over aborting waiters in futex_wake and
  futex_requeue.
  => Otherwise, futex_wake might move it to a new futex while
    futex_wait_abort has released all the locks -- but
    futex_wait_abort still has the old futex, so TAILQ_REMOVE will
    cross the streams and bad things will happen.

- In futex_wait_abort, release the futex we moved the waiter off.
  => Otherwise, we would leak the futex reference acquired by
    futex_func_wait, in the event of aborting.  (For normal wakeups,
    futex_wake releases the reference on our behalf.)

- Consistently use futex_wait_dequeue rather than TAILQ_REMOVE so that
  all changes to fw_futex and the waiter queue are isolated to
  futex_wait_enqueue/dequeue and happen together.

Patch developed with and tested by thorpej@.

(riastradh)

Speed up cksum word-by-word with a 15 KB table generated dynamically.

(Really we could generate the main table dynamically too.)

From anonymous.

(riastradh)

More -Wno-error=address-of-packed-member to placate clang.

(riastradh)

Match Linux's criterion for byte-swapping on big-endian CPUs.

(riastradh)

Need <sys/types.h> for uint32_t and uintptr_t.

(riastradh)

do {...} while (0) to make DTRACE_PROBE macros proper statements.

(riastradh)

Fix trailing whitespace.

(riastradh)

Reject overly large widths, from mlelstv.

We are returning an ACPI_INTEGER (= uint64_t), so it doesn't make
sense to handle more than 64 bits.

Apparently there are some ACPIs out there that ask for unreasonably
large widths here.  Just reject those requests, rather than writing
past the caller's stack buffer.

Previously we attempted to fix this by copying byte by byte as large
as the caller asked, in order to avoid the undefined behaviour of
shifting past the size of ACPI_INTEGER, but that just turned a shift
(which might have been harmless on real machines) into a stack buffer
overflow (!).

ok msaitoh

(riastradh)

KNF

(riastradh)

Revert acpi_ec.c 1.76.

We will do this another way, and separate KNF fixes from the critical
functional change.

ok msaitoh

(riastradh)

Revert acpi_ec.c 1.77.

We will do this another way.

ok msaitoh

(riastradh)

Mark res __diagused; #ifdef DIAGNOSTIC panic ---> KASSERT.

(riastradh)

Suppress spurious address-of-packed error in rump lfs too.

(riastradh)

Fix clang build after packed lfs64 accessor change.

(riastradh)

Revert another historic $Hdr$ collapse.

(riastradh)

Avoid misaligned access to lfs64 on-disk records in memory.

lfs64 directory entries are only 32-bit aligned in order to conserve
space in directory blocks, and we had a hack to stuff a 64-bit inode
in them.  This replaces the hack by __aligned(4) __packed, and goes
further:

1. It's not clear that all the other lfs64 data structures are 64-bit
  aligned on disk to begin with.  We can go through these later and
  upgrade them from

struct foo64 {
...
} __aligned(4) __packed;

union foo {
struct foo64 f64;
...
};

  to

struct foo64 {
...
};

union foo {
struct foo64 f64 __aligned(8);
...
} __aligned(4) __packed;

  if we really want to take advantage of 64-bit memory accesses.

  However, the __aligned(4) __packed must remain on the union
  because:

2. We access even the lfs32 data structures via a union that has
  lfs64 members, and it turns out that compilers will assume access
  through a union with 64-bit aligned members implies the whole
  union has 64-bit alignment, even if we're only accessing a 32-bit
  aligned member.

(riastradh)

CTASSERT lfs on-disk structure sizes.

(riastradh)

Restore historic $Hdr: ...$ text.

This was presumably eaten by git cvsexportcommit, which is curious
because I thought I had gotten out of the habit of passing -k to it.

(riastradh)

Avoid unnecessary zil_commit on rm.

1. Issue zil_commit only if we're actually updating something --
  there's no need to commit if we're unlinking the file or if
  there's no atime update being applied.

2. Issue zil_commit only if the zfs has sync=always set -- for
  sync=standard there's no need for us to commit anything here since
  no application asked for an explicit sync.

Speeds up untarring base.tgz on top of itself by a factor of about
2x, and speeds up rm by a factor of about 10x, on my system with an
SSD SLOG over SATA.  Histogram of unlink, rmdir, and rename timing
shows dramatic reduction in latency for most samples.

(To be fair, this was not an improvement over zfs; issuing the
unnecessary zil_commit was a self-inflicted performance wound.)

(riastradh)

Include "opt_uvm_page_trkown.h" for UVM_PAGE_TRKOWN.

(riastradh)

Fix userland build by surrounding stuff with #ifdef _KERNEL.

(...Why does this header file get exposed to userland at all?)

(riastradh)

Revert "Add fixup operation to remove private mutex types for x86."

The x86 (and arm) definitions of struct kmutex are no longer defined
differently depending on __MUTEX_PRIVATE.

(riastradh)

Remove __MUTEX_PRIVATE conditional in definition of struct kmutex.

This doesn't buy us anything but the need to hack around it in
ctfmerge to avoid massive duplication of kernel types -- which only
worked for the x86 definition.

This changes only x86 and arm for now, pending compile-testing the
remaining architectures.

(riastradh)

Revert "Include opt_diagnostic.h for DIAGNOSTIC."

This did not do what I thought it did.  opt_diagnostic.h is only for
the unused _DIAGNOSTIC, which seems like an abortive attempt to
incrementally convert DIAGNOSTIC to an opt_*.h option rather than a
command-line option.

(riastradh)

Hackily rename enum pipe for drm, without patching.

The patch we used to maintain to do this rename doubled the pain
of merging drm updates.  But ctf gets confused if we have a struct
and an enum both called `pipe', and it's technically against C99
to do so.  So #define it in all drm code for now to fix dtrace.

(riastradh)

Include opt_diagnostic.h for DIAGNOSTIC.

...at least, in header files, which may not have already included
libkern.h.

(riastradh)

Avoid struct inode.

This is an fs-independent structure in Linux.  We don't actually use
it as such; it's just a dummy struct tag.  But we do have an actual
struct inode in ufs and in lfs, and using the same struct tag here
confuses ctf leading to four copies of pretty much every drm data
structure.

(riastradh)

Avoid duplicate definition of internal_state struct.

(riastradh)

Need opt_inet.h for #ifdef INET, INET6.

(riastradh)

Use vpanic, not vprintf and then panic.

(riastradh)

Remove unfinished hack I accidentally committed in 2017.

This caused make to unconditionally take ages running useless
submakes in every subdirectory.  Accidentally committed during the
MKCRYPTO option removal when I was presumably experimenting with
automating library dependency generation in lib.

Should shave a few seconds at least off every build!

(riastradh)

New ioctl DIOCGSECTORALIGN returns sector alignment parameters.

struct disk_sectoralign {
/* First aligned sector number.  */
uint32_t dsa_firstaligned;

/* Number of sectors per aligned unit.  */
uint32_t dsa_alignment;
};

- Teach wd(4) to get it from ATA.
- Teach cgd(4) to pass it through from the underlying disk.
- Teach dk(4) to pass it through with adjustments.
- Teach zpool (zfs) to take advantage of it.
  => XXX zpool doesn't seem to understand when the vdev's starting
    sector is misaligned.

Missing:

- ccd(4) and raidframe(4) support -- these should support _using_
  DIOCGSECTORALIGN to decide where to start putting ccd or raid
  stripes on disk, and these should perhaps _implement_
  DIOCGSECTORALIGN by reporting the stripe/interleave factor.

- sd(4) support -- I don't know any obvious way to get it from SCSI,
  but if any SCSI wizards know better than I, please feel free to
  teach sd(4) about it!

- any ld(4) attachments -- might be worth teaching the ld drivers for
  nvme and various raid controllers to get the aligned sector size

There's some duplicate logic here for now.  I'm doing it this way,
rather than gathering the logic into a new disklabel_sectoralign
function or something, so that this change is limited to adding a new
ioctl, without any new kernel symbols, in order to make it easy to
pull up to netbsd-9 without worrying about the module ABI.

(riastradh)

Include kern_crashme.c in non-DEBUG kernels.

This is useful for simulating crashes in production to test failover.

(riastradh)

Create /dev/ipmi0 in `MAKEDEV all'.

Works on my machine with sysutils/ipmitool if the latter is built
with CONFIGURE_ARGS+= --enable-intf-open.

(riastradh)

Add a flag to dk_dump for virtual disk devices.

If a disk is backed by a physical medium other than itself, such as
cgd(4), then it passes DK_DUMP_RECURSIVE to disable the recursion
detection for dk_dump.

If, however, a device represents a physical medium on its own, such
as wd(4), then it passes 0 instead.

With this, I can now dump to dk on cgd on dk on wd.

(riastradh)

Allow dumping to cgd(4) on a dk(4).

(Technically this also allows dumping to a dk(4) on which there
happens to be a cgd(4) configured, but I'm not sure how to
distinguish that case here.  So don't do that!)

(riastradh)

Teach zfs bdevsw to do b_psize.

This is needed, among other things, to swap on zvols.

Attempting to swap on zvols currently deadlocks but that's a separate
issue that needs to be fixed too!

(riastradh)

Create some more cgd device nodes.

(riastradh)

Add cgd to evbarm GENERIC.common.

(riastradh)

Take a mutex around mode config logic as the comment advises.

XXX pullup

(riastradh)

kernfs no longer needs a hacky workaround to make rootdev appear.

(riastradh)

uugetty is no more

(riastradh)

Draft man pages for the standard users and groups.

These are currently listed in order of uid because I went through
src/etc/group and src/etc/master.passwd line by line, and sorting any
other way after the fact -- like lexicographically, how it should be
-- was kinda inconvenient.

Feel free to sort, add information, add historical references,
correct any mistakes, &c., so that these remain living documents
describing NetBSD's standard users and groups and practices around
them.

(riastradh)

Fix missing <sys/evcnt.h> by removing the evcnts instead.

Just wanted to confirm that a race might happen, and indeed it did.
These serve little diagnostic value otherwise.

(riastradh)

Fix userland references to LFS_ORPHAN_NEXTFREE.

Forgot to grep for these or do a full distribution build, oops!

(riastradh)

Simplify some branches and kassert some redundant assignments.

(riastradh)

Fix mistake in use of usbd_xfer_schedule_timeout in motg.

This code path is used both for xfers that are new, and xfers that
are being done piece by piece and are partway done.  For the latter
case, skip usbd_xfer_schedule_timeout so we schedule it only once per
xfer.

(riastradh)

If zfs=YES, unconditioally do zfs mount/unmount -a.

If you set zfs=YES, presumably you positively want the automatic rc.d
actions, so if there's no /sbin/zfs or if zfs can't find pools with
/etc/zfs/zpool.cache, presumably you would like feedback about that
in rc.log.

(riastradh)

Allow random seed on zfs.

(riastradh)

Dust off the orphan detection code and try to make it work.

(riastradh)

Teach LFS_ORPHAN_NEXTFREE about lfs64.

(riastradh)

lfs_writer_enter can't fail; keep it simple and don't pretend it can.

Assert that mtsleep can't fail either -- it doesn't catch signals and
there's no timeout.

(riastradh)

Call lfs_orphan in lfs_rename while we're still in the dirop.

(riastradh)

In lfs_update, hold lfs_writer around lfs_vflush.

Otherwise, we might do

lfs_vflush
-> lfs_seglock
-> lfs_segwait(SEGM_CKP)
  -> lfs_writer_enter

which is the reverse of the lfs_writer -> lfs_seglock ordering.

(riastradh)

Initialize/destroy lfs_allclean_wakeup in modcmd, not lfs_mountfs.

Fixes reloading lfs.kmod.

(riastradh)

Break deadlock in PR kern/52301.

The lock order is lfs_writer -> lfs_seglock.  The problem in 52301 is
that lfs_segwrite violates this lock order by sometimes doing
lfs_seglock -> lfs_writer, either (a) when doing a checkpoint or (b),
opportunistically, when there are no dirops pending.  Both cases can
deadlock, because dirops sometimes take the seglock (lfs_truncate,
lfs_valloc, lfs_vfree):

(a) There may be dirops pending, and they may be waiting for the
seglock, so we can't wait for them to complete while holding the
seglock.

(b) The test for fs->lfs_dirops == 0 happens unlocked, and the state
may change by the time lfs_writer_enter acquires lfs_lock.

To resolve this in each case:

(a) Do lfs_writer_enter before lfs_seglock, since we will need it
unconditionally anyway.  The worst performance impact of this should
be that some dirops get delayed a little bit.

(b) Create a new lfs_writer_tryenter to use at this point so that the
test for fs->lfs_dirops == 0 and the acquisition of lfs_writer happen
atomically under lfs_lock.

(riastradh)

Don't lfs_writer_enter while holding v_interlock.

There's no need to lfs_writer_enter at all here, as far as I can see.
lfs_flush_fs will do it for us.

(riastradh)

Take a reference and fix assertions in lfs_flush_dirops.

Fixes panic:

KASSERT((ip->i_state & IN_ADIROP) == 0) at lfs_vnops.c:1670
lfs_flush_dirops
lfs_check
lfs_setattr
VOP_SETATTR
change_mode
sys_fchmod
syscall

This assertion -- and the assertion that vp->v_uflag has VU_DIROP set
-- is valid only until we release lfs_lock, because we may race with
lfs_unmark_dirop which will remove the nodes and change the flags.

Further, vp itself is valid only as long as it is referenced, which it
is as long as it's on the dchain, but lfs_unmark_dirop drops the
dchain's reference.

(riastradh)

Change some cheap KDASSERT into KASSERT.

(riastradh)

Serialize access to the splay tree with lfs_lock.

(riastradh)

Assert lfs_writer where I think we can now prove it.

(riastradh)

Prevent new dirops while we issue lfs_flush_dirops.

lfs_flush_dirops assumes (by KASSERT((ip->i_state & IN_ADIROP) == 0))
that vnodes on the dchain will not become involved in active dirops
even while holding no other locks (lfs_lock, v_interlock), so we must
set lfs_writer here.  All other callers already set lfs_writer.

We set fs->lfs_writer++ without explicitly doing lfs_writer_enter
because

(a) we already waited for the dirops to drain, and
(b) we hold lfs_lock and cannot drop it before setting lfs_writer.

(riastradh)

Teach lfs to transition ro<->rw.

(riastradh)

Just use VOP_BWRITE for lfs_bwrite_log.

Hope this doesn't cause trouble with vfs_suspend.

(riastradh)

Use a marker node to iterate lfs_dchainhd / i_lfs_dchain.

I believe elements can be removed while the lock is dropped,
including the next node we're hanging on to.

(riastradh)

Teach device nodes on zfs to handle fsync by calling spec_fsync too.

(riastradh)

Teach zfs spec nodes to VOP_BWRITE too.

Fixes hang on writing to, e.g., ffs mounted on a device node that
lives on zfs.

XXX pullup

(riastradh)

OOPS -- fix mistake in previous commit.

bbusy really needs to return the error; otherwise things are very
bad!

(riastradh)

Fix kassert in lfs by initializing vp first.

(riastradh)

Buffer cache SDT probes.

(riastradh)

Use vn_bwrite, not genfs_nullop, for VOP_BWRITE.

VOP_BWRITE is responsible for calling biodone; can't just leave it
hanging.

XXX pullup

(riastradh)

Mark previous #ifdef __NetBSD__, per request from hannken.

(riastradh)

Report the OS name and release appropriately for NetBSD.

We are not OpenSolaris or uts!

(riastradh)

usbdi dtrace probes

(riastradh)

Sprinkle some dtrace probes into scsi(4).

(riastradh)

C99 initializers for scsipi_bustype.  No functional change intended.

(riastradh)

umass dtrace probes

(riastradh)

New dtrace usb provider, with a handful of probes in usb.c.

(riastradh)

Make sure the umass transfer callback is run in error branches.

(riastradh)

Consolidate logic to call the transfer callback.

No functional change intended.

(riastradh)

Fix mistakes in timeout/abort/completion changes in motg(4).

- Call usbd_xfer_schedule_timeout so we actually do time out.

- Don't call usbd_xfer_trycomplete until all the data have been
  transferred -- it commits to completion, not timeout.

- Use xfer->ux_status != USBD_IN_PROGRESS to test whether, after a
  partial write, an xfer has been interrupted or timed out and need
  not be continued.

- Remove wrong assertion.

(riastradh)

Abort default pipe too on detach before detaching children.

This ensures that pending xfers on the default pipe will be aborted
before we wait for children, which, in the case of scsibus -> sd,
means waiting for pending xfers to complete -- xfers that may never
complete if something is wedged.

(riastradh)

Define VOP_STRATEGY on zfs device nodes too.

Fixes eternal hangs in attempts to do I/O on device nodes on zfs.

XXX pullup

(riastradh)

Fix non-DIAGNOSTIC build with UVM_PAGE_TRKOWN.

(riastradh)

Fix wrong KASSERT in motg abort.

This has been wrong since last summer when we did the transition to
xfer->ux_status = USBD_CANCELLED earlier.

XXX pullup-9

(riastradh)

When aborting a pipe, set pipe->up_running = 0.

This prevents any concurrent usb_transfer_complete from restarting
the pipe with usbd_start_next.

(riastradh)

Move another file from drm2 to common for <linux/kernel.h>.

(riastradh)

Missed a drm2 -> common move.

(riastradh)

Open /dev/urandom with O_CLOEXEC.

Let's avoid bleeding file descriptors into our clients' children,
shall we?

XXX pullup

(riastradh)

Let's not write temporary files to fixed paths in /tmp, shall we?

XXX pullup

(riastradh)

Teach zfs to revoke vnodes the usual way.

XXX pullup

(riastradh)

#ifdef DIAGNOSTIC --> __diagused

(riastradh)

Fix non-DIAGNOSTIC builds.

(riastradh)

Fix mistakes in previous sloppy change with root intr xfers.

- Make sure ux_status is set to USBD_IN_PROGRESS when started.
  Otherwise, if it is still in flight when we abort the pipe,
  usbd_ar_pipe will skip calling upm_abort.

- Initialize ux_status under the lock; in principle a completion
  interrupt (or a delay) could race with the initialization.

- KASSERT that the xfer is in progress when we're about to complete
  it.

Candidate fix for PR kern/54963 for other HCI drivers than uhci.

ok nick
ok phone

(This is the change that nick evidently MEANT to ok when he ok'd the
previous one!)

(riastradh)

Set ux_isdone in uhci_poll_hub for DIAGNOSTIC.

(riastradh)

Initialize xfer->ux_status in uhci_root_intr_start.

Otherwise, it will be USBD_NOT_STARTED, so usbd_ar_pipe will skip
calling upm_abort.

Candidate fix for PR kern/54963, same problem as reported at:
https://mail-index.NetBSD.org/current-users/2020/02/13/msg037740.html

(riastradh)

Nix another conditional ALIGN.

(riastradh)

Reduce some more cosmetic diffs.

(riastradh)

Fix undoing of -Wpointer-arith workaround.

(riastradh)

Carry new CWARNFLAGS over to module reachover makefiles.

Forgot these modules existed!

(riastradh)

Placate -Wsign-compare.

(riastradh)

Add LIST_HEAD to <linux/nbsd-namespace.h> too.

(riastradh)

Implement (obsolete) simple_strtol stub; reduce diff.

(riastradh)

Implement drm userptr with uvm_vslock & bus_dmamap_load_uio.

(riastradh)

Revert unnecessary change.

The only way this logic can reach the reference to stat is if repcnt
goes to zero, which -- since it starts at 4 -- can only happen if we
go through several iterations of the loop, which in turn implies that
stat is initialized.

If GCC still complains about this, it's wrong and should be dealt
with some other way.

(riastradh)

DRM_HZ -> HZ, to reduce diffs.

(riastradh)

Revert cosmetic changes.

(riastradh)

Stub out console_lock/unlock and drm_fb_helper_set_suspend.

Reduce some more diffs.

(riastradh)

Reduce diffs by nixing DRM_SUSER macro, deleted upstream long ago.

Instead stub Linux capable(CAP_SYS_ADMIN).

(riastradh)

Provide necessary forward declarations or includes.

(riastradh)

Undo the `enum pipe' -> `enum i915_pipe' rename.

(riastradh)

Use a header file hack to cope with Linux/NetBSD namespace clashes.

Currently serves for:

ALIGN
mutex_init
mutex_destroy

(riastradh)

Eliminate -Wpointer-arith patches; use -Wno-pointer-arith instead.

(riastradh)

We use -Wno-shadow for radeon; remove unused diff.

(riastradh)

New stub to reduce diff.

(riastradh)

Fix steady state of root intr xfers.

Why?

- Avoid completing a root intr xfer multiple times in races.
- Avoid potential use-after-free in poll_hub callouts (uhci, ahci).

How?

- Use sc->sc_intr_xfer or equivalent to store only a pending xfer
  that has not yet completed -- whether successfully, by timeout, or
  by synchronous abort.  When any of those happens, set it to null
  under the lock, so the xfer is completed only once.

- For hci drivers that use a callout to poll the root hub (uhci, ahci):

  . Pass the softc pointer, not the xfer, to the callout, so the
    callout is not even tempted to use xfer after free -- if the
    callout fires, but the xfer is synchronously aborted before the
    callout can do anything, the xfer might be freed by the time the
    callout starts to examine it.

  . Teach the callout to do nothing if it is callout_pending after it
    has fired.  This way:

    1. completion or synchronous abort can just callout_stop
    2. start can just callout_schedule

    If the callout had already fired before (1), and doesn't acquire
    the bus lock until after (2), it may be tempted to abort the new
    root intr xfer just after submission, which would be wrong -- so
    instead we just have the callout do nothing if it notices it has
    been rescheduled, since it will fire again after the appropriate
    time has elapsed.

(riastradh)

Factor out HCI-independent xfer completion logic.

New API for HCI drivers to synchronize hardware completion
interrupts, synchronous aborts, and asynchronous timeouts:

- When submitting an xfer to hardware, call
  usbd_xfer_schedule_timeout(xfer).

- On HCI completion interrupt for xfer completion:

if (!usbd_xfer_trycomplete(xfer))
return; /* timed out or aborted, ignore it */

- In upm_abort methods, call usbd_xfer_abort(xfer).

For HCI drivers that use this API (not needed in drivers that don't,
or for xfers like root intr xfers that don't use it):

- New ubm_abortx method serves role of former *hci_abort_xfer, but
  without any logic for wrangling timeouts/callouts/tasks -- caller
  in usbd_xfer_abort has already handled them.

- New ubm_dying method, returns true if the device is in the process
  of detaching, used by the timeout logic.

Converted and tested:
- ehci
- ohci

Converted and compile-tested:
- ahci (XXX did this ever work?)
- dwc2
- motg (XXX missing usbd_xfer_schedule_timeout in motg_*_start?)
- uhci
- xhci

Not changed:

- slhci (sys/dev/ic/sl811hs.c) -- doesn't use a separate per-xfer
  callout for timeouts (XXX but maybe should?)

- ugenhc (sys/rump/dev/lib/libugenhc/ugenhc.c) -- doesn't manage its
  own transfer timeouts

- vhci -- times transfers out only on detach; could be adapted easily
  if we wanted to use the xfer->ux_callout

(riastradh)

Fix steady state of timeouts in ehci.

This is complicated because:

1. There are three ways that an xfer can be completed:
  (a) hardware interrupt completes xfer
  (b) software decision aborts xfer with USBD_CANCELLED
  (c) timeout aborts xfer with USBD_TIMEOUT

2. The timeout abort can't be done in callout because ehci_sync_hc,
  called unconditionally by ehci_abort_xfer to wait until the device
  has finished using any references to the xfer, may sleep.  So we
  have to schedule a callout that, when run, will schedule a usb_task.

3. The hardware completion interrupt can't sleep waiting for a callout
  or task to finish -- can't use callout_halt or usb_rem_task_wait.
  So the callout and usb_task must be able to run _after_ the hardware
  completion interrupt, and recognize that they're late to the party.

  (Note, though, that usbd_free_xfer does wait for the callout and
  task to complete, so there's no danger they may use themselves after
  free.)

4. The xfer may resubmitted -- and the timeout may be rescheduled --
  immediately after the hardware completion interrupt, _while_ the
  callout and/or usb_task may still be scheduled.  Specifically, we
  may have the following sequence of events:
  (a) hardware completion interrupt
  (b) callout or usb_task fires
  (c) driver resubmits xfer
  (d) callout or usb_task acquires lock and looks around dazed and
      bewildered at the firehose of events like reading the news in 2019

The mechanism for sorting this out is that we have two bits of state:

- xfer->ux_timeout_set informs the driver, when submitting an xfer and
  setting up its timeout, whether either the callout or usb_task is
  already scheduled or not.

- xfer->ux_timeout_reset informs the callout or usb_task whether it
  should reschedule the callout, because the xfer got resubmitted, or
  not.

(riastradh)

New xfer state variables ux_timeout_set and ux_timeout_reset.

These are needed because:

- The host controller interrupt cannot wait for the callout or task
  to finish running.

- Nothing in the USBD API as is waits for the callout or task to
  finish running.

- Callers expect to be able to resubmit USB xfers from xfer callbacks
  without waiting for anything to finish running.

The variable ux_timeout_set can be used by a host controller to
decide on submission whether to schedule the callout or to ask an
already-scheduled callout or already-queued task to reschedule the
callout, by setting the variable ux_timeout_reset to true.

When the callout or task runs and sees that ux_timeout_reset is true,
rather than queue the task or abort the xfer, it can instead just
schedule the callout anew.

(riastradh)

Nothing guarantees xfer's timeout has completed.

Wait for it when we free the xfer.

(riastradh)

New function usb_task_pending for diagnostic assertions.

Usable only for negative diagnostic assertions:

KASSERT(!usb_task_pending(dev, task))

If you can think of a better name for this than !usb_task_pending,
I'm all ears.

(riastradh)

Teach usb_rem_task to return whether removed from queue or not.

(riastradh)

cyclic.kmod needs -Wno-sign-compare for aarch64 CPU_INFO_FOREACH.

Provisional workaround; feel free to fix.

(riastradh)

Define the MULTIPROCESSOR cpu_number() for modules too.

Modules should work whether the main kernel is multiprocessor or not.
In particular, dtrace should not think cpu_number() is 0 while
cpu_index(curcpu()) and curcpu()->ci_index are nonzero, leading to
rather spectacularly bogus results...

(riastradh)

Switch from db_write_bytes to using direct-mapping.

This way there's no dependency on ddb.

(riastradh)

Change the address ranges that aarch64 considers toxic for dtrace.

`Toxic' means dtrace forbids D scripts from even attempting to read
or write at them.

Previously we considered [0, VM_MIN_KERNEL_ADDRESS) toxic, but
VM_MIN_KERNEL_ADDRESS is only the minimum address of the kernel map;
the direct-mapped region lies below it, and with PMAP_MAP_POOLPAGE we
allocate virtual pages for pool backing directly from physical pages
through the direct-mapped region.  Also, this did not consider I/O
mappings to be toxic, which they probably should be.

Instead, treat:

[0, AARCH64_KSEG_START)
and
[VM_KERNEL_IO_ADDRESS, 0xfff...ff)

as toxic.  (The upper bound for 0xfff...ff ought to be inclusive, not
exclusive, but I think we'll need another mechanism for expressing
that to dtrace!)

(riastradh)

Count the number of artificial frames in aarch64 fbt probe correctly.

(riastradh)

Teach dtrace about el1_trap_exit frames on aarch64.

Implement dtrace_getarg and dtrace_getreg while here.

(riastradh)

Use /dev/ksyms, not /netbsd, for the running kernel's symbols.

(riastradh)

Tidy up a bit: don't set things we won't use; assert nonzeroness.

(riastradh)

Tidy up a bit.  No functional change intended.

aarch64 fbt_invop doesn't actually use the argument, but it would
make more sense for it to be the return value and/or first argument
register.  Certainly it's not `eax'!

(riastradh)

Use db_write_bytes to overwrite kernel text.

(riastradh)

Create a buffer space of 512 bytes before the trapframe.

dtrace fbt needs enough space to emulate an

stp x29, x30, [sp,#-FRAMESIZE]!

instruction in a function prologue.  In the aarch64 instruction
encoding, FRAMESIZE can be as large as 512 bytes, so reserve this
much space when KDTRACE_HOOKS is enabled.

(riastradh)

Mark compilation with -pg as xfail on aarch64.

XXX file a PR to xref

(riastradh)

Test that fpu state is preserved by fork.

(riastradh)

Preserve pcu(9) state in fork.

There should perhaps be a pcu_fork operation to keep this factored
neatly but this will be simpler to pull up.

In practical terms, this may not affect most architecture that use
pcu(9) -- alpha, arm32, mips, powerpc, riscv -- but it does affect
aarch64, in which v8-v15 are callee-saves, and GCC actually takes
advantage of them, and for more than just floating-point data too.

XXX pullup

(riastradh)

Delete aarch64 cpu_in_cksum.S draft.

This isn't actually used in the kernel; it is only used to cause the
in_cksum tests to fail.

If you want to revive it and make it work, you can pull it out of the
attic.

(riastradh)

aarch64 doesn't trap integer division by zero either.

(riastradh)

Show errno on failure.

(riastradh)

Omit duplicate rnd_init in rump.

(riastradh)

Initialize struct cpu_info::ci_cpuname (= ci_data.cpu_name) in rump.

(riastradh)

Switch from ad-hoc logging to dtrace probes.

(riastradh)

Teach threadpool(9) to use percpu_create, mostly.

(riastradh)

Fix previous brainfart.

Don't use the uninitialized trng node as the root node -- derp.
Instead, use the root node as the root node, and initialize the trng
node here.

(riastradh)

Use specfs vnops for specnodes in kernfs.

While here, don't filter out rootdev and rrootdev merely because
they're not cached.

Fixes the elusive /kern/rootdev and /kern/rrootdev nodes, which only
appeared sometimes when they felt like it, and fixes operations on
/kern/rootdev and /kern/rrootdev always returning EOPNOTSUPP.

We didn't seem to have a single PR for these issues but the following
PRs are all relevant:

PR bin/13564
PR kern/38265
PR kern/38778
PR kern/45974

XXX pullup-9, pullup-8, pullup-7, pullup-6, pullup-5, pullup-4, pullup-3, pullup-2, pullup-1.4T...

(riastradh)

Fix abuse of TAILQ_CONCAT.

Other parts of this expect that the entries will be on wq->wq_queue
or wq->wq_dqueue, so we can't just move a batch of entries onto a
temporary queue.  Instead, use a marker node to delimit when the
batch ends.

XXX pullup

(riastradh)

opencrypto orders new_session -> process for us.

No need for membar_producer.

Discussed with hikaru@.

(riastradh)

softint_disestablish does xc_barrier(0) for us already.

(riastradh)

Redocument xc_barrier.

Be more clearer, and more correcter, about what it does.

(riastradh)

KNF

(riastradh)

Reduce some ifdefs.

(riastradh)

pic_pending_zero is unnecessary; percpu_alloc already zeroes.

(riastradh)

Switch arm pic allocation and initialization to percpu_create.

(riastradh)

Switch sys/net to percpu_create.

(riastradh)

Switch opencrypto to percpu_create.

Can't sleep for allocation in percpu_foreach.

(riastradh)

KNF

(riastradh)

Use __read_mostly and gather related __cacheline_aligned together.

Should save a few bytes of kernel.

(riastradh)

New function percpu_create.

Associates a constructor and destructor with the percpu.  Currently
the constructor runs immediately, but in principle we could use the
same API for future CPU hotplug support.

This lets you sleep for allocation or draining users before
deallocation when setting up or tearing down a percpu -- currently we
have many abuses of percpu_foreach in tree for that purpose.

Proposed on tech-kern:
https://mail-index.NetBSD.org/tech-kern/2020/01/30/msg026036.html

(riastradh)

Switch if_vlan to atomic_load/store_*.

Fix missing membar_datadep_consumer -- now atomic_load_consume -- in
vlan_lookup_tag_psref.

(riastradh)

Switch if_l2tp to atomic_load/store_*.

Fix missing membar_datadep_consumer -- now atomic_load_consume -- in
l2tp_lookup_session_ref.

(riastradh)

Fix order in rollback case; switch if_ipsec to atomic_load/store_*.

(riastradh)

Switch if_gif to atomic_load/store_*.

(riastradh)

Fix wrong memory order and switch pfil to atomic_load/store_*.

(riastradh)

Fix wrong memory order and switch bpf to atomic_load/store_*.

(riastradh)

Use atomic_load_acquire for FreeBSDish atomic_load_acq_uint shim.

(riastradh)

Use atomic_load/store_* in ena(4), not questionable membar_sync.

(riastradh)

Load struct fdfile::ff_file with atomic_load_consume.

Exceptions: when we're only testing whether it's there, not about to
dereference it.

Note: We do not use atomic_store_release to set it because the
preceding mutex_exit should be enough.

(That said, it's not clear the mutex_enter/exit is needed unless
refcnt > 0 already, in which case maybe it would be a win to switch
from the membar implied by mutex_enter to the membar implied by
atomic_store_release -- which I would generally expect to be much
cheaper.  And a little clearer without a long comment.)

(riastradh)

Load struct filedesc::fd_dt with atomic_load_consume.

Exceptions: when fd_refcnt <= 1, or when holding fd_lock.

While here:

- Restore KASSERT(mutex_owned(&fdp->fd_lock)) in fd_unused.
  => This is used only in fd_close and fd_abort, where it holds.
- Move bounds check assertion in fd_putfile to where it matters.
- Store fd_dt with atomic_store_release.
- Move load of fd_dt under lock in knote_fdclose.
- Omit membar_consumer in fdesc_readdir.
  => atomic_load_consume serves the same purpose now.
  => Was needed only on alpha anyway.

(riastradh)

Use __BIT.

(riastradh)

Unconditionally enable UATP_DEBUG.

(riastradh)

Downgrade noisy message from aprint_verbose to DPRINTF.

(riastradh)

Work around quirk of Fountain and Geyser 1 trackpads.

Sometimes they get desynchronized, but we know the last packet is a
17-byte packet, so if we get one early then stop here.

Tested by macallan on an iBook and a PowerBook.  This code path
shouldn't break anything on MacBooks because they have different
total numbers of sensors so this branch won't be reached.

(riastradh)

Fix (presently harmless) psato.

Don't overwrite cy_root_node; use cy_trng_node as intended.

(riastradh)

Replace kooky sctp random number generation by cprng_strong32().

(riastradh)

Remove filemon(4).

Discussed on tech-kern:
https://mail-index.NetBSD.org/tech-kern/2020/01/13/msg025938.html

This was never (intentionally) enabled by default, and the design has
some shortcomings.  You can get mostly the same results with ktrace,
as in usr.bin/make/filemon/filemon_ktrace.c which is now used instead
of filemon for make's meta mode.

If applications require higher performance than ktrace, or nesting
that ktrace doesn't support, we might consider adding something back
into the vfs system calls themselves, without hijacking the syscall
table.  (Might want a more reliable output format too, e.g. one that
can handle newlines in file names.)

(riastradh)

Missed a spot -- define _KERNTYPES to get register_t.

(riastradh)

Tweak makefile for USE_FILEMON=dev, from sjg.

(riastradh)

Per sjg's suggestion, split filemon API into separate back ends.

By default we use the ktrace back end, but the /dev/filemon back end
is available as a compile-time option, by setting USE_FILEMON=dev in
make.  sjg raised concerns about ktrace performance and would like to
continue using /dev/filemon on FreeBSD (which has seen more
maintenance kernel-side) without forking make.

(riastradh)

Reimplement make(1) meta mode without filemon(4).

(riastradh)

Enable sun8icrypto in GENERIC64.

But set its entropy estimate to zero until we have a better idea of
how it works.  Can't really hurt this way unless sun8icrypto is
maliciously dependent on all other inputs to the entropy pool, which
seems unlikely.

(riastradh)

Implement rndseed support in efiboot and fdt arm.

The EFI environment variable `rndseed' specifies the path to the
random seed.  It is loaded only for fdt platforms at the moment.

Since the rndseed (an rndsave_t object as defined in <sys/rndio.h>)
is 536 bytes long (for hysterical raisins), and to avoid having to
erase parts of the fdt tree, we load it into a physical page whose
address is passed in the fdt tree, rather than passing the content of
the file as an fdt node directly; the kernel then reserves the page
from uvm, and maps it into kva to call rnd_seed.

For now, the only kernel that does use efiboot with fdt is evbarm,
which knows to handle the rndseed.  Any new kernels that use efiboot
with fdt must do the same; otherwise uvm may hand out the page with
the secret key on it for a normal page allocation in the kernel --
which should be OK if there are no kernel memory disclosure bugs, but
would lead to worse consequences than simply loading the seed late in
userland with /etc/rc.d/random_seed otherwise.

ok jmcneill

(riastradh)

New function cpu_startup_hook on arm.

Called at end of cpu_startup.  Can be defined in, e.g., evbarm to do
additional stuff after cpu_startup.  Defined as a weak alias to a
function that does nothing, so optional.

ok jmcneill

(riastradh)

Reduce entropy estimate for sun8icrypto TRNG.

NIST's SP800-90B entropy estimation tools put it at no more than .08
bits of entropy per byte of data(!), so estimate 100 bits of data per
bit of entropy.  This is probably not conservative enough -- the NIST
tools were written without knowledge of how this alleged TRNG works!
Knowledge of the physics of how the TRNG is supposed to work could
probably enable a better job at predicting the outputs.

While here, bump the size of data we can sample directly with sysctl
to 4096 bytes.

(riastradh)

Add vhci to x86/ALL.

(riastradh)

Fix typo -- acknowledge interrupts _and_ errors.

(riastradh)

Nix trailing whitespace.

(riastradh)

Remove never-used Mersenne twister from libkern.

(riastradh)

Omit vestigial unused commented-out experiment.

(riastradh)

Just use KASSERTMSG and panic.  No need for custom wrappers.

(riastradh)

Remove charade dancing around now-dead RCU patent.

(riastradh)

Skip npf_config_sync if nothing to do.

Saves an unnecessary pserialize_perform every second.

(riastradh)

Oops -- forgot to kmem_free.

(riastradh)

Tidy up comments.

(riastradh)

Avoid a race between interruption and reacquisition of lock.

Otherwise, we would have leaked the memory in this case.

(riastradh)

Make sure ERESTART doesn't come flying out to userland.

I picked ERESTART to mean `all channels are occupied' because that's
what opencrypto(9) uses to decide whether to queue a request, but
it's not appropriate for sysctl(2) to return that.

(riastradh)

Factor out some of the self-test logic used for debugging.

Add missing bus_dmamap_sync(POSTWRITE) while here.

(riastradh)