heartbeat(9): Make heartbeat_suspend/resume nestable.

And make them bind to the CPU as a side effect, instead of requiring
the caller to have already done so.

This lets us eliminate the assertions so we can use them in ddb even
when things are going haywire and we just want to get diagnostics.

XXX kernel revbump -- struct cpu_info change

(riastradh)

drmkms: Fix module build.

(riastradh)

drm: Fix conditionals around drmkms_pci and agp.

Kernel should build now with all pci drm drivers stripped out but
DRM_LEGACY still enabled.  (Might not be very useful, but it'll
build.  Maybe we should also have DRM_LEGACY_PCI so those drivers can
be modloaded later.)

(riastradh)

certctl(8): Fix permissions on ca-certificates.crt bundle: 0644.

While here, write it atomically: write to .tmp first, then rename
when done; this way applications never see a partially-written bundle
at /etc/openssl/certs/ca-certificates.crt.

(riastradh)

certctl(8): Test permissions of ca-certificates.crt.

Inadvertently created 0600 instead of 0644 due to copying file
created by mktemp(1) with cp(1).

(riastradh)

lists: Remove more bogus shlib obsolete entries.

Too much trouble to have to remember about libuv.so, libgmp.so, &c.
Just apply the rule uniformly.

All that's left is test libraries and loadable modules.

Searched with:

git grep 'lib.*\.so\.[0-9][0-9]*.*obsolete' distrib/sets/lists \
| grep -v '/modules/' \
| grep -v '/locale/lib/' \
| grep -v '/lib/npf/' \
| grep -v '/lib/librump' \
| grep -v '/lib/netbsd/libclang_rt' \
| grep -v '/runemodule/'

(riastradh)

lists: Remove bogus libfoo.so.N and libfoo.so.N.M obsolete entries.

These must stay around so applications linked against them will still
work after upgrade, even if libfoo.so now points to libfoo.so.(N+1)
or libfoo.so.N.(M+1).

Exceptions:

- I'm willing to believe the rump modules have a different story so I
  left those obsolete entries alone.

- libuv.so was never supposed to be exposed publicly anyway and never
  went out in a release.  (Maybe this information should be recorded
  somewhere?)

- Same is probably true of lib{gmp,mpc,mpfr}.so, not sure of the
  history.  Maybe libg2c.so too, no idea what that is.

- libisns.so was moved from /usr/lib to /lib, so it's legitimate for
  the debug data to live there too now.  (XXX Maybe we should have a
  separate marker for this.)

- Libraries under /usr/tests are not used by normal applications, so
  they can safely be deleted when obsoleted.

Note: The libfoo.so symlink for a library that has been deleted
altogether, not just upgraded, can be obsoleted.  Loadable modules
that applications aren't linked against can be obsoleted, even if
some of them like npf ext_*.so or pam_*.so are formally versioned
(for reasons unclear to me).

Note: This means that incremental builds may complain about these
.so.N and .so.N.M files in destdir (PR misc/57581), but it's much
worse for an upgrade to break working applications.

(riastradh)

base, debug: Remove obsolete entries for older libfido2.

The shared libraries need to stay around for applications on upgrade
(and the debug data should be kept too if the shared libraries are
kept).

(riastradh)

libc_aligned, libc_fp: Add missing close-paren.

Did this ever work?

(riastradh)

libss, liblwres: Delete makefiles for long-dead external libraries.

No functional change intended -- these haven't been used in years.

(riastradh)

liblutok.so: Bump major for recent Lua update.

XXX pullup-10

(riastradh)

heimdal/libsl: Belatedly bump major.

This is to address the major bump of libterminfo.so.9 in:

Author: roy <roy@NetBSD.org>
Date:  Fri Mar 13 15:19:24 2020 +0000

    terminfo: promote numeric parameters from short to int

That commit caught all the other dependent libraries except libsl.

XXX pullup-10

(riastradh)

Recursively revbump all dependents of libcrypto.

Otherwise any existing software linked against the openssl11
libcrypto.so.14 and any of these libraries will suddenly start
pulling in libcrypto.so.15 at the same time, leading to mayhem in the
address space.

PR lib/57603

XXX pullup-10

(riastradh)

postinstall(8): Handle various certs.conf scenarios gracefully.

Tested the following scenarios:

1. fresh install
  empty /etc/openssl/certs
  default /etc/openssl/certs.conf
  - opensslcertsconf
    [x] check: pass
    [x] fix: pass -- nothing
  - opensslcertsrehash
    [x] check: fail -- needs rehash
    [x] fix: pass -- quietly rehash successfully (go to 4)

2. fresh upgrade
  empty /etc/openssl/certs
  no /etc/openssl/certs.conf
  - opensslcertsconf
    [x] check: fail -- complain missing /etc/openssl/certs.conf
    [x] fix: pass -- install default /etc/openssl/certs.conf (go to 1)
  - opensslcertsrehash
    [x] check: fail -- complain missing /etc/openssl/certs.conf
    - [x] fix: fail -- complain missing /etc/openssl/certs.conf

3. upgrade from certctl, changes to certs
  certctl-managed /etc/openssl/certs
  default /etc/openssl/certs.conf
  - opensslcertsconf
    [x] check: pass
    [x] fix: pass -- nothing
  - opensslcertsrehash
    [x] check: fail -- needs rehash
    [x] fix: pass -- quietly rehash successfully (go to 4)

4. upgrade from certctl, no changes to certs
  certctl-managed /etc/openssl/certs
  default /etc/openssl/certs.conf
  - opensslcertsconf
    [x] check: pass
    [x] fix: pass -- nothing
  - opensslcertsrehash
    [x] check: pass
    [x] fix: pass -- quietly rehash successfully (go to 4)

5. upgrade from mozilla-rootcerts
  populated /etc/openssl/certs
  no /etc/openssl/certs.conf
  - opensslcertsconf:
    [x] check: fail -- complain missing /etc/openssl/certs.conf
    [x] fix: pass -- install manual /etc/openssl/certs.conf (go to 7)
  - opensslcertsrehash:
    [x] check: fail -- complain missing /etc/openssl/certs.conf
    [x] fix: fail -- complain missing /etc/openssl/certs.conf

6. upgrade from mozilla-rootcerts with etcupdate naively
  populated /etc/openssl/certs
  default /etc/openssl/certs.conf
  - opensslcertsconf:
    [x] check: pass
    [x] fix: pass -- nothing
  - opensslcertsrehash:
    [x] check: fail -- complain mismatched certs/ and certs.conf
    [x] fix: fail -- complain mismatched certs/ and certs.conf

7. upgrade from mozilla-rootcerts with etcupdate manually
  populated /etc/openssl/certs
  manual /etc/openssl/certs.conf
  - opensslcertsconf:
    [x] check: pass
    [x] fix: pass -- nothing
  - opensslcertsrehash:
    [x] check: pass
    [x] fix: pass -- skip rehash because manual (go to 7)

XXX Someone should draft automatic tests for postinstall.  It has a
very good track record, but it sure would be nice to automate this
testing rather than redo it each time I make a tiny change.

(riastradh)

certctl(8): Install certs.conf in /usr/share/examples too.

This way postinstall(8) can refer to the default one when you've done
an upgrade without etcupdate or similar to pull in new config files
from etc.tgz.

Not great -- we should do this systematically for all config files in
/etc, but this one-off hack is less risky for 10.

(riastradh)

heartbeat(9): Move #ifdef HEARTBEAT to sys/heartbeat.h.

Less error-prone this way, and the callers are less cluttered.

(riastradh)

heartbeat(9): Move panicstr check into the IPI itself.

We can't return early from defibrillate because the IPI may have yet
to run -- we can't return until the other CPU is definitely done
using the ipi_msg_t we created on the stack.

We should avoid calling panic again on the patient CPU in case it was
already in the middle of a panic, so that we don't re-enter panic
while, e.g., trying to print a stack trace.

Sprinkle some comments.

(riastradh)

heartbeat(9): More detail about manual test success criteria.

Changes comments only, no functional change.

(riastradh)

heartbeat(9): Ignore stale tc if primary CPU heartbeat is suspended.

The timecounter ticks only on the primary CPU, so of course it will
go stale if it's suspended.

(It is, perhaps, a mistake that it only ticks on the primary CPU,
even if the primary CPU is offlined or in a polled-input console
loop, but that's a separate issue.)

(riastradh)

cons(9): Suspend heartbeat checks while in polled-input mode.

This goes into a tight loop at high IPL, so it is to be expected that
the heartbeats will stop happening.

Should fix heartbeat panics at root device prompt on boot.

(riastradh)

cons(9): Sort includes.

No functional change intended.

(riastradh)

heartbeat(9): New flag SPCF_HEARTBEATSUSPENDED.

This way we can suspend heartbeats on a single CPU while the console
is in polling mode, not just when the CPU is offlined.  This should
be rare, so it's not _convenient_, but it should enable us to fix
polling-mode console input when the hardclock timer is still running
on other CPUs.

(riastradh)

cpu_setstate: Fix call to heartbeat_suspend.

Do this on successful offlining, not on failed offlining.

No functional change right now because heartbeat_suspend is
implemented as a noop -- heartbeat(9) will just check the
SPCF_OFFLINE flag.  But if we change it to not be a noop, well, then
we need to call it in the right place.

(riastradh)

ukbd(4): Sort includes.

No functional change intended.

(riastradh)

ukbd(4): Fix ordering in ukbd_cnpollc exit.

This is probably an MP-safety issue waiting to happen, but let's at
least make the wind and unwind sequences mirror images.

(riastradh)

certctl(8): Fix quoting and whitespace style in evilpath test.

No functional change intended.

(riastradh)

certctl(8): Fix some bugs with evil pathnames.

(riastradh)

certctl(8): Test more evil pathnames.

(riastradh)

certctl(8): Minor man page clarifications.

- Specify exactly what /etc/openssl/certs gets populated with.
- Change HTTPS to TLS.
- Specify the permitted character class in certs.conf.
  (Maybe more conservative than strictly needed; but let's stay on
  the safe side.)

(riastradh)

hier(7): Document /etc/openssl.

(riastradh)

mozilla-certdata: Install relative symlinks.

Slightly more compact this way, and you can examine them in a destdir
without chrooting.  Not terribly important, but a minor convenience.

(riastradh)

etc/mtree/special: Fix spaces/tabs.

No functional change intended.

(riastradh)

certctl(8): Set certs.conf 644 and add it to etc/mtree/special.

(riastradh)

distrib/sets/lists: certs.conf belongs in etc, not in base.

Oops.

(riastradh)

postinstall(8): Fail if `certctl rehash' fails.

Not using `set -e' here, evidently (maybe we should), so the separate
return 0 suppressed the error.

(riastradh)

certctl(8): Avoid clobbering prepopulated /etc/openssl/certs.

Also avoid clobbering some other edge cases like symlinks or
non-directories there.

This way, we have the following transitions on system updates:

- If /etc/openssl/certs is empty (as in default NetBSD<10 installs):
  quietly populated on rehash.

- If /etc/openssl/certs is nonempty (you've added things to it,
  e.g. by hand or with mozilla-rootcerts) and has never been managed
  by certctl(8): left alone on rehash, with an error message to
  explain what you need to do.

- If /etc/openssl/certs has been managed by certctl(8): quietly
  updated on rehash.

Note: This means current installations made since certctl(8) was
added will be treated like /etc/openssl/certs is nonempty and has
never been managed by certctl(8).  To work around this, you can just
delete /etc/openssl/certs and rerun `certctl rehash'.

(riastradh)

certctl(8): Test prepopulated /etc/openssl/certs.

This is the scenario when you have previously populated
/etc/openssl/certs manually, or with a package like mozilla-rootcerts
or mozilla-rootcerts-openssl, and you update to a version of NetBSD
with certctl(8).  In this case, certctl(8) should avoid destroying
your work.

While here, also test some related but less likely edge cases:

- nonexistent
- symlink
- regular file

(riastradh)

certctl(8): Exit nonzero on missing certs.conf.

(riastradh)

certctl(8): Add xfail test for missing certs.conf.

Command should fail, i.e., exit with nonzero status, but it exits
with zero instead.

(riastradh)

postinstall(8): Add opensslcerts item to regen /etc/openssl/certs.

Works only with destdir /, since it relies on running openssl(1),
which is not available as a tool or required in the cross-build
environment.

(riastradh)

mozilla-certdata: Connect it up to the build.

(riastradh)

mozilla-certdata: regen

(actually, just `gen', this first time)

(riastradh)

mozilla-certdata: Makefile infrastructure.

(riastradh)

mozilla-certdata: Record in doc/3RDPARTY.

(riastradh)

certctl(8): New tool for managing OpenSSL CA certificates.

Same command-line syntax as FreeBSD, clearer semantics about which
parts are config and which parts are cache.

(riastradh)

ext2fs: Nix trailing whitespace.

(riastradh)

ext2fs.h: Restore e2fs_cgload/cgsave for libsa and userland use.

Stop-gap until they can be taught to handle the new version that was
moved to ext2fs_vfsops.c, presumably.

(riastradh)

xen: Provide definitions or ifdefs to make drm build in XEN3_DOM0.

No idea if it works, but it builds now.

PR port-xen/49330

(riastradh)

gdb/psim: Suppress typedef-redefinition errors in the clang build.

Seems clang disagrees with gcc about whether this is allowed under
-std=gnu99.

(riastradh)

distrib/sets/lists/tests: Correction: h_inotify_* are MI.

Debug lists are probably still wrong, though -- looks like there's no
rule to change h_inotify_*.out.debug to h_inotify_*.debug.  Out of
energy to look into this further, though; please fix me!

(riastradh)

distrib/sets/lists: /usr/tests/compat/linux/h_* is amd64-only for now.

(riastradh)

distrib/sets/lists: compat/linux/h_*.debug is amd64-only for now.

(riastradh)

distrib/sets/lists: Add .debug files for new compat/linux tests.

(riastradh)

etc/mtree/NetBSD.dist.tests: Update for new compat linux tests.

(riastradh)

options(4), sysctl(7): Document options HEARTBEAT.

(riastradh)

gdb/doc/GDBvn.texi: Omit spurious trailing blank line.

Reduces diff from upstream.

(riastradh)

vis(3): Per KNF, sys/param.h comes before sys/types.h.

Which is nice because that's also lexicographic.

(riastradh)

vis(3): Need <stdint.h> for SIZE_MAX, per C standard.

>From Kyle Evans <kevans@FreeBSD.org>.

Followup to PR lib/57573.

XXX pullup-10
XXX pullup-9
XXX pullup-8

(riastradh)

vis(3): Sort includes.  No functional change intended.

Prompted by PR lib/57573.

XXX pullup-10
XXX pullup-9
XXX pullup-8

(riastradh)

strncpy(3): More on how strlcpy is not a safe strncpy replacement.

(riastradh)

base/shl.mi: Tag libasan.so and libubsan.so with gcc like before.

Not in clang builds.  This tag got dropped in the gcc12 update.

(riastradh)

vis(3): Fix one more buffer overrun in an edge case.

PR lib/57573

XXX pullup-10
XXX pullup-9
XXX pullup-8

(riastradh)

vis(3): Fix main part of PR lib/57573.

>From Kyle Evans <kevans@FreeBSD.org>.

XXX pullup-10
XXX pullup-9
XXX pullup-8

(riastradh)

vis(3): Avoid potential arithmetic overflow in maxolen.

Can't easily prove that this overflow is impossible, so let's add a
check.

Prompted by PR lib/57573.

XXX pullup-10
XXX pullup-9
XXX pullup-8

(riastradh)

vis(3): Call wcslen(start) only once.

It had better not change between these two times!

Prompted by PR lib/57573.

XXX pullup-10
XXX pullup-9
XXX pullup-8

(riastradh)

vis(3): Avoid arithmetic overflow before calloc(3).

Prompted by PR lib/57573.

XXX pullup-10
XXX pullup-9
XXX pullup-8

(riastradh)

vis(3): Make mbslength unsigned.

Sprinkle assertions and comments justifying the proposition that it
would never go negative if signed.

Obviates need to worry about mblength > SSIZE_MAX.

Prompted by PR lib/57573.

XXX pullup-10
XXX pullup-9
XXX pullup-8

(riastradh)

vis(3): Make maxolen unsigned size_t, not ssize_t.

It is initialized once either to *dlen, which is unsigned size_t, or
to wcslen(start) * MB_MAX_LEN + 1, and wcslen returns unsigned size_t
too.  So there appears to have never been any reason for this to be
signed.

Part of PR lib/57573.

XXX pullup-10
XXX pullup-9
XXX pullup-8

(riastradh)

vis(3) tests: Test another overflow edge case.

Related to PR lib/57573.

XXX pullup-10
XXX pullup-9
XXX pullup-8

(riastradh)

vis(3) tests: Expand tests and diagnostic outputs on failure.

PR lib/57573

XXX pullup-10
XXX pullup-9
XXX pullup-8

(riastradh)

vis(3) tests: Add xfail test for encoding overflow.

>From Kyle Evans <kevans@FreeBSD.org>.

PR lib/57573

XXX pullup-10
XXX pullup-9
XXX pullup-8

(riastradh)

strncpy(3): Fix column sizing.

(riastradh)

strlcpy(3): Tweak markup.

(riastradh)

strlcpy(3), strncpy(3): Omit needless (void) casts in examples.

The return values are not critical.

(riastradh)

strcpy(3), strlcpy(3), strncpy(3): Just say `byte', not `character'.

(riastradh)

strncpy(3): Tiny wording tweak.

(riastradh)

strncpy(3): Reword to make sentence structure parallel.

(riastradh)

strncpy(3): Fix typo -- stpncpy, not stpcpy which is different.

(riastradh)

strncpy(3): Take another whack at clarifying this.

Emphasize the fixed-buffer nature of it, and that NUL-termination is
neither required on input nor guaranteed on output.

(riastradh)

strlcpy(3): Nix stray space between `NUL' and `-terminating'.

(riastradh)

strcpy(3): Note that strlcpy(3) is a safer replacement for strcpy(3).

Suggest snprintf("%s") as a more portable alternative too.

Note that both strlcpy and snprintf still require the input to be
NUL-terminated.

(riastradh)

strncpy(3): Rework the example in an attempt to improve exposition.

(riastradh)

strncpy(3): Note strcpy(3) man page revision this forked from.

(riastradh)

strcpy(3), strlcpy(3), strncpy(3): Use `.Tn NUL' for the zero byte.

Let's be consistent within these man pages.  (If someone else really
likes the unpronounceable `.Ql \e0' better, that's fine, you can go
through and systematically change all the man pages to use that after
we're done clarifying strcpy(3), strncpy(3), and strlcpy(3).)

(riastradh)

strncpy(3): Slightly more consistency about NUL vs '\0' in the text.

(riastradh)

strncpy(3): Qualify example of strlcpy(3) with a major caveat.

(riastradh)

strncpy(3), stpncpy(3): Split man page out of strcpy(3), stpcpy(3).

These are for substantively different purposes (fixed-width fields
with optional NUL padding vs NUL-terminated strings), so they don't
belong together.

Be more specific about the security issues.

(riastradh)

acpiwmi(4): Fix abuse of char buffer for struct guid_t content.

Nothing guarantees alignment, so this is all undefined behaviour,
even if we don't touch the uninitialized members.

XXX pullup-10
XXX pullup-9
XXX pullup-8

(riastradh)

strlcpy(3): Rework man page to clarify relation to strncpy(3).

Add caveats explaining when strlcpy(3) and strlcat(3) are dangerously
inadequate or inappropriate.

XXX pullup-10
XXX pullup-9
XXX pullup-8

(riastradh)

workqueue(9) tests: Fix mistake in rev. 1.9.

Somehow, despite manually verifying a build/install/test of every
revision in my patch series, I managed to commit the wrong version of
the file for what became rev. 1.9, so the test was just broken when
it went in and remained broken in the commit where I fixed the real
bug and removed the xfail marker on the test.

PR kern/5757

XXX pullup-10
XXX pullup-9
XXX pullup-8

(riastradh)

vmxnet(4): Fix various MP bugs.

- Defer reset to workqueue.
  => vmxnet3_stop_locked is forbidden in softint.
  => XXX Problem: We still take the core lock in softint, and we
    still take the core lock around vmxnet3_stop_locked.  TBD.
- Touch if_flags only under IFNET_LOCK.
  => Cache ifp->if_flags & IFF_PROMISC in vmxnet3_ifflags_cb.
  => Don't call vmxnet3_set_rxfilter unless up and running; cache
    this as vmx_mcastactive.  Use ENETRESET in vmxnet3_ifflags_cb
    instead of calling vmxnet3_set_rxfilter directly.
    . (The cache is currently serialized by the core lock, but it
      might reasonably be serialized by an independent lock like in
      usbnet(9).)
- Fix vmxnet3_stop_rendezvous so it actually does something.
  => New vxtxq_stopping, vxrxq_stopping variables synchronize with
    Rx/Tx interrupt handlers.
- Sprinkle IFNET_LOCK and core lock assertions.

(riastradh)

trousers: Make this build again.

- Downgrade address-of-packed-member errors to warnings.  Not sure if
  this is safe, but there's too many to audit.

- Silence deprecation warnings for openssl3.

- Address removal of const qualifier in iconv.

- Nix unused definitions in a .h file, which cause trouble now that
  -fno-common is the default.

(riastradh)

xvif(4): Omit needless membars in xennetback_connect.

xneti is a private data structure to which we have exclusive access
here; ordering the stores doesn't make sense.

(riastradh)

xvif(4): Omit needless membars in xennetback_rx_copy_process.

- No need for barrier around touching req_cons and rsp_prod_pvt,
  which are private.

- RING_PUSH_RESPONSES_AND_CHECK_NOTIFY updates the shared req_prod and
  then issues xen_mb, which is all that we need between the update of
  shared req_prod and hypervisor_notify_via_evtchn.

  (Between updating the shared req_prod and issuing
  hypervisor_notify_via_evtchn, only xen_wmb is needed.  But after
  writing to the shared req_prod, RING_PUSH_REQUESTS_AND_CHECK_NOTIFY
  must also read from the shared rsp_event, which requires the
  store-before-load ordering that only xen_mb provides.)

(riastradh)

xvif(4): Simplify while loop in xennetback_evthandler.

No functional change intended.

(riastradh)

xvif(4): Omit needless membars in xennetback_evthandler.

This should improve throughput without any impact on correctness.

(riastradh)

xvif(4): Move expensive xen_mb out of xennetback_evthandler loop.

Use the cheaper RING_HAS_UNCONFIRMED_REQUESTS for most of the loop.

This should improve throughput without any impact on correctness.

(riastradh)

xvif(4): Omit local variable aliasing xneti->xni_txring.req_cons.

No functional change intended.

(riastradh)

xvif(4): Add missing xen_rmb in xennetback_evthandler.

(riastradh)

xvif(4): Comment on memory barriers in xennetback_evthandler.

Note which ones appear unnecessary and which ones appear too strong,
but don't change them.

No functional change intended.

(riastradh)

workqueue(9): Factor out wq->wq_flags & WQ_FPU in workqueue_worker.

No functional change intended.  Makes it clearer that s is
initialized when used.

(riastradh)

workqueue(9): Sort includes.

No functional change intended.

(riastradh)

workqueue(9): Avoid unnecessary mutex_exit/enter cycle each loop.

(riastradh)

workqueue(9): Stop violating queue(3) internals.

(riastradh)

workqueue(9): Sprinkle dtrace probes for workqueue_wait edge cases.

Let's make it easy to find out whether these are hit.

(riastradh)

workqueue(9): Avoid touching running work items in workqueue_wait.

As soon as the workqueue function has called, it is forbidden to
touch the struct work passed to it -- the function might free or
reuse the data structure it is embedded in.

So workqueue_wait is forbidden to search the queue for the batch of
running work items.  Instead, use a generation number which is odd
while the thread is processing a batch of work and even when not.

There's still a small optimization available with the struct work
pointer to wait for: if we find the work item in one of the per-CPU
_pending_ queues, then after we wait for a batch of work to complete
on that CPU, we don't need to wait for work on any other CPUs.

PR kern/57574

XXX pullup-10
XXX pullup-9
XXX pullup-8

(riastradh)

workqueue(9) tests: Add test for PR kern/57574.

XXX pullup-10
XXX pullup-9
XXX pullup-8

(riastradh)

workqueue(9) tests: Destroy struct work immediately on entry.

(riastradh)

workqueue(9) tests: Nix trailing whitespace.

(riastradh)

tests/rump/rumpkern: Use PROGDPLIBS, not explicit -L/-l.

This way we relink the t_* test programs whenever changes under
tests/rump/kernspace change libkernspace.a.

(riastradh)

sqlite3/lib/Makefile: -Wno-error=restrict is a gccism.

Let's try to get at least one working clang autobuild this summer,
shall we?

(riastradh)

libc/resolv/res_debug.c: Minor whitespace fixes.

(riastradh)

traceroute/Makefile: Nix trailing whitespace.

(riastradh)

gcc/README.warnings: Nix trailing whitespace.

(riastradh)

t_cdb: No need for weird padding any more.

cdbw_output never needed it at runtime, and the declaration no longer
makes gcc angry about not having it.

(riastradh)

libnpf(3): No need for weird padding any more.

cdbw_output never needed it at runtime, and the declaration no longer
makes gcc angry about not having it.

(riastradh)

npftest(8): No need for weird padding any more.

cdbw_output never needed it at runtime, and the declaration no longer
makes gcc angry about not having it.

(riastradh)

dev_mkdb(8): No need for weird padding any more.

cdbw_output never needed it at runtime, and the declaration no longer
makes gcc angry about not having it.

(riastradh)

services_mkdb(8): No need for weird padding any more.

cdbw_output never needed it at runtime, and the declaration no longer
makes gcc angry about not having it.

(riastradh)

cdbw(3): Make cdbw_output descr parameter type less confusing.

This is a string of _up to_ 16 bytes, used with strncpy(..., 16).
Specifying `const char descr[16]', while formally equivalent to
`const char *descr' in standard C, now provokes the ire of gcc when
the caller does not provide a buffer of at least 16 bytes.

(riastradh)

style(5): Advise against new struct typedefs and explain why.

Proposed on tech-kern:
https://mail-index.netbsd.org/tech-kern/2023/07/11/msg028950.html

Positive feedback to general concept, negative feedback to specifics
and phrasing of the first iteration but no objections to latest
iteration after several weeks at:
https://mail-index.netbsd.org/tech-kern/2023/07/16/msg028994.html

(riastradh)

radeon: Suspend ioctls while device is suspended.

XXX pullup-10

(riastradh)

nouveau: Suspend ioctls while device is suspended.

XXX pullup-10

(riastradh)

amdgpu: Suspend ioctls while device is suspended.

XXX pullup-10

(riastradh)

distrib/sets/lists: Fix libtsan entries.

- No obsolete .so.N or .so.N.M entries.
- libtsan.so for gcc.
- libtsan.so.1* for gcc=10.
- libtsan.so.2* for gcc=12.
- No libtsan.so.1* on aarch64, libtsan on aarch64 is new in gcc12.

(riastradh)

netbt(4): Initialize bt_lock earlier.

Use a driver-class module modcmd init function, instead of a socket
domain init function; the socket-domain ones don't run until after
configure, but we need this to be initialized before configure so
that Bluetooth HCI drivers like ubt(4) can use it.

This is suboptimal but it's the least intrusive way I've thought of
to get this working, even if it's a little grody to make netbt a
`driver-class' (builtin) module.  Note that this doesn't mean netbt
becomes dynamically loadable or unloadable; we're just using a module
for initialization ordering.

PR kern/56988

XXX pullup-10

(riastradh)

sys/lwp.h: Slightly more paranoia re curlwp in PR port-evbarm/57564.

(riastradh)

sys/lwp.h: KASSERT -> KASSERTMSG

Might help diagnose PR port-evbarm/57564.

(riastradh)

libm: Add dummy remainderl and remquol.

These are pretty bad -- these aren't transcendental functions; not
rocket science to make them correctly rounded -- but let's just make
sure they're available in libm for netbsd-10.

XXX pullup-10

(riastradh)

rnd(4): Document `entropy: best effort' in random(4).

(riastradh)

xcall(9): Rename condvars to be less confusing.

The `cv' suffix is not helpful and `xclocv' looks like some kind of
clock at first glance.  Just say `xclow' and `xchigh'.

(riastradh)

xen/x86: Get the right intrframe pointer in ddb ipi.

This was broken with the transition from evtchn_set_handler to
intr_establish_xname in 2017, remained broken with the transition
from intr_establish_xname to xen_intr_establish_xname in 2018, and
still remained broken when xen_intr_establish_xname was changed back
to evtchn_set_handler in 2020.

The mechanism is grody -- instead of a secret second argument to the
interrupt handler, the intrframe pointer should be replaced by a
struct cpu_info member that is saved and restored by the interrupt
handler calling logic.  But we should make sure the replacement
actually works first -- which is not trivial in part because the
users are hidden behind sketchy function pointer casts.

With any luck, this will make `mach cpu N' work in ddb on Xen.

XXX pullup-10
XXX pullup-9 (by patch)

(riastradh)

xen/x86: Enable heartbeat checks.

(riastradh)

tests/crypto/opencrypto: test_asymfeat requires privileges.

(riastradh)

tests/net/inpcb: Tests require root.

(riastradh)

tests/net/net/t_bind: IP_BINDANY and IPV6_BINDANY require root.

(riastradh)

tests/lib/librumphijack: Avoid trying to run rpcbind as non-root.

Can probably make this work through rumphijack, but there's no sense
in even trying the test if we can't, so let's reduce the unprivileged
false alarms.

(riastradh)

tests/lib/libnvmm: Tests require privileges.

(riastradh)

t_nullmnt: Need privileges for mount.

(unless vfs.generic.usermount=1 but let's keep it simple)

(riastradh)

tests/include/t_paths: Test all paths, nonfatally.

This way a single failure doesn't suppress failure reports for all
the other paths to test.

Omit some needless blank lines while here.

(riastradh)

tests/include/t_paths: /dev/ksyms requires root.

(riastradh)

Revert "rump: Set mp_online = true and start threads _after_ cold = 0."

This breaks some tests, e.g. dev/scsipi/t_cd:noisyeject, which relies
on config_finalize to wait for driver threads.  Trouble is, the
actual setting of cold=0 happens near the call to config_finalize in
RUMP__FACTION_DEV.  Need to think harder about this.

(riastradh)

cprng_fast(9): Drop and retake percpu reference across cprng_strong.

cprng_strong may sleep on an adaptive lock (via entropy_extract),
which invalidates percpu(9) references.

Discovered by stumbling upon this panic in a test run:

panic: kernel diagnostic assertion "(cprng == percpu_getref(cprng_fast_percpu)) && (percpu_putref(cprng_fast_percpu), true)" failed: file "/home/riastradh/netbsd/current/src/sys/rump/librump/rumpkern/../../../crypto/cprng_fast/cprng_fast.c", line 117

XXX pullup-10

(riastradh)

cprng(9): Drop and retake percpu reference across entropy_extract.

entropy_extract may sleep on an adaptive lock, which invalidates
percpu(9) references.

Add a note in the comment over entropy_extract about this.

Discovered by stumbling upon this panic during a test run:

[  1.0200050] panic: kernel diagnostic assertion "(cprng == percpu_getref(cprng_fast_percpu)) && (percpu_putref(cprng_fast_percpu), true)" failed: file "/home/riastradh/netbsd/current/src/sys/rump/librump/rumpkern/../../../crypto/cprng_fast/cprng_fast.c", line 117

XXX pullup-10

(riastradh)

base/md.amd64, base/md.sparc64: Mark openssl compat directories.

Should fix MKCOMPAT=no build.

PR port-amd64/57567

(riastradh)

comp/mi: /usr/include/g++/bit is gcc-only, not for clang (or pcc)

(riastradh)

rump: Set mp_online = true and start threads _after_ cold = 0.

Otherwise we may have threads running while cold, which is a
contradiction in terms.

Deferring mp_online = true is necessary because things like xcall(9)
use rely on it to decide whether to wait for threads on other CPUs.

(riastradh)

memfd(2): Run all tests; don't stop after the first failure.

(riastradh)

t_ubsan, t_ubsanxx: Remove gcc=10, gcc=12 conditionals.

These are built unconditionally.  Should help fix the clang build.

(riastradh)

Revert "xennetback(4): Fix xennetback_evthandler loop."

PR kern/57560

(riastradh)

Revert "xennetback(4): Fix membars in xennetback_rx_copy_process."

PR kern/57560

(riastradh)

Revert "xennetback(4): Omit needless membars in xennetback_connect."

PR kern/57560

(riastradh)

entropy(9): Disable !cold assertion in rump for now.

Evidently rump starts threads before it sets cold = 0, and deferring
the call to rump_thread_allow(NULL) in rump.c rump_init_callback
until after setting cold = 0 doesn't work either -- rump kernels just
hang.  To be investigated -- for now, let's just stop breaking
thousands of tests.

(riastradh)

Revert "softint(9): Sprinkle KASSERT(!cold)."

Temporary workaround for PR kern/57563 -- to be fixed properly after
analysis.

(riastradh)

softint(9): Sprinkle KASSERT(!cold).

Softints are forbidden to run while cold.  So let's make sure nobody
even tries it -- if they do, they might be delayed indefinitely,
which is going to be much harder to diagnose.

(riastradh)

entropy(9): Simplify stages.  Split interrupt vs non-interrupt paths.

- Nix the entropy stage (cold, warm, hot).  Just use the usual kernel
  `cold' (cold: single-core, single-thread; interrupts may happen),
  and don't make any three-way distinction about whether interrupts
  or threads or other CPUs can be running.

  Instead, while cold, use splhigh/splx or forbid paths to come from
  interrupt context, and while warm, use mutex or the per-CPU hard
  and soft interrupt paths for low latency.  This comes at a small
  cost to some interrupt latency, since we may stir the pool in
  interrupt context -- but only for a very short window early at boot
  between configure and configure2, so it's hard to imagine it
  matters much.

- Allow rnd_add_uint32 to run in hard interrupt context or with spin
  locks held, but defer processing to softint and drop samples on the
  floor if buffer is full.  This is mainly used for cheaply tossing
  samples from drivers for non-HWRNG devices into the entropy pool,
  so it is often used from interrupt context and/or under spin locks.

- New rnd_add_data_intr provides the interrupt-like data entry path
  for arbitrary buffers and driver-specified entropy estimates: defer
  processing to softint and drop samples on the floor if buffer is
  full.

- Document that rnd_add_data is forbidden under spin locks outside
  interrupt context (will crash in LOCKDEBUG), and inadvisable in
  interrupt context (but technically permitted just in case there are
  compatibility issues for now); later we can forbid it altogether in
  interrupt context or under spin locks.

- Audit all uses of rnd_add_data to use rnd_add_data_intr where it
  might be used in interrupt context or under a spin lock.

This fixes a regression from last year when the global entropy lock
was changed from IPL_VM (spin) to IPL_SOFTSERIAL (adaptive).  Thought
I'd caught all the problems from that, but another one bit three
different people this week, presumably because of recent changes that
led to more non-HWRNG drivers entering the entropy consolidation
path from rnd_add_uint32.

In my attempt to preserve the rnd(9) API for the (now long-since
abandoned) prospect of pullup to netbsd-9 in my rewrite of the
entropy subsystem in 2020, I didn't introduce a separate entry point
for entering entropy from interrupt context or equivalent, i.e., spin
locks held, and instead made rnd_add_data rely on cpu_intr_p() to
decide whether to process the whole sample under a lock or only take
as much as there's buffer space for before scheduling a softint.  In
retrospect, that was a mistake (though perhaps not as much of a
mistake as other entropy API decisions...), a mistake which is
finally getting rectified now by rnd_add_data_intr.

XXX pullup-10

(riastradh)

fileassoc(9): Fast paths to skip global locks when not in use.

PR kern/57552

(riastradh)

percpu(9): percpu_create ctor may be called later, not synchronously.

(riastradh)

xen: Fix previous commit, forgot to amend it before committing.

(riastradh)

xen: Report when hardclock jump exceeds timecounter(9) limit.

(riastradh)

ihidev(4): Use iic_use_direct_match as intended.

This appears to have been a mistake; there's no obvious explanation
in the commit history for why this is different from all other
iic_use_direct_match users.

Patch from Vladimir 'phcoder' Serbinenko <phcoder@gmail.com>, thanks!

(If it really is intended to ues I2C_MATCH_DIRECT_COMPATIBLE here, we
need a clear explanation of why, written down in a nearby comment.)

(riastradh)

timecounter(9): Nix trailing whitespace.

No functional change intended.

(riastradh)

wsmouse(4): Make wsmouse_input safe to call from MP-safe interrupts.

XXX pullup-10

(riastradh)

ld.elf_so: Add some known-answer tests for hash functions.

Make sure the testing mechanism detects the traditional overflow bug.

(riastradh)

ld.elf_so: Split hash functions into a separate file.

This way we can test them in isolation.

No functional change intended.

(riastradh)

ld.elf_so: Sort SRCS.

No functional change intended.

(riastradh)

ld.elf_so: Split SRCS onto multiple lines.

Makes updates easier.

No functional change intended.

(riastradh)

drm/linux_ww_mutex: Fix ww acquire context ordering.

XXX pullup-8
XXX pullup-9
XXX pullup-10

(riastradh)

drm/linux_ww_mutex: Fix wait loops.

If cv_wait_sig returns because a signal is delivered, we may
nonetheless have been granted the lock.  It is harmless for us to
ignore this fact in three of the four paths, but in
ww_mutex_state_wait_sig, we may now have ownership of the lock and
MUST NOT return failure because the caller MUST release the lock
before destroying the ww_acquire_ctx.

While here, restructure the other three loops for clarity, so they
match the structure of the fourth and so they have a little less
impenetrable negation.

PR kern/57537

XXX pullup-8
XXX pullup-9
XXX pullup-10

(riastradh)

sys/memfd.h: Fix include guards after rename.

(riastradh)

sys/memfd.h: Tidy.

- Sort includes.
- #include <sys/types.h> for size_t.

(riastradh)

sys: Rename sys/miscfd.h -> sys/memfd.h.

Let's not create new dumping grounds for miscellaneous stuff; one
header file for one purpose.

(riastradh)

memfd(2): Convert KASSERT to CTASSERT.

Move it closer to where it's relevant too.

(riastradh)

xen_clock(4): New hardclock dtrace probes.

sdt:xen:hardclock:tick(last, now)
  fires on every hardclock tick

sdt:xen:hardclock:jump(last, now, nticks)
  fires on every hardclock tick when (now - last) >= 2*NS_PER_TICK,
  i.e., this call to the timer interrupt handler requires multiple
  hardclock ticks

(riastradh)

xen_clock(4): Compute NS_PER_TICK only once per call.

This involves a division by an unknown number, so let's cache it.

XXX Could do better by precomputing it in xen_resumeclocks, with some
effort to ensure concurrent calls don't stomp on each other.

(riastradh)

xen_clock(4): Fix whitespace and sprinkle comments.

No functional change intended.

(riastradh)

timecounter(9): Link to phk's timecounter paper for reference.

No functional change intended.

(riastradh)

kern: Restore non-atomic time_second symbol.

This is used by savecore(8), vmstat(8), and possibly other things.

XXX Should really teach dump and savecore(8) to use an intentionally
designed header, rather than relying on kvm(3) -- and make it work
for saving cores from other kernels so you don't have to boot the
same buggy kernel to get a core dump.

(riastradh)

x86/pmap: Print quantities in failed assertions in pmap_load.

(riastradh)

x86/errata.c: Only say the errata revision search for cpu0.

(riastradh)

x86/errata.c: Say what revision we're searching for.

(riastradh)

x86/errata.c: Link to original AMD errata guide.

This one is no longer updated; need to link to newer ones for
individual families too.  That's where all the cryptic nomenclature
comes from here.

(riastradh)

i386/XEN3PAE_DOMU: Pass -g to build debug data like GENERIC.

Needed for CTF data by dtrace when MKDEBUG=no MKDEBUGKERNEL=no.

XXX pullup-10

(riastradh)

i386/XEN3PAE_DOM0: Pass -g to build debug data like GENERIC.

Needed for CTF data by dtrace when MKDEBUG=no MKDEBUGKERNEL=no.

XXX pullup-10

(riastradh)

amd64/XEN3_DOMU: Pass -g to build debug data like GENERIC.

Needed for CTF data by dtrace when MKDEBUG=no MKDEBUGKERNEL=no.

XXX pullup-10

(riastradh)

bsd.own.mk: Use MACHINE_ARCH for default MKCTF/MKDTRACE=yes x86.

The substantive impact of this is that it enables MKCTF=yes for Xen
kernels.  This is a change because, when building a Xen kernel
(XEN3_DOM0, XEN3_DOMU), MACHINE is set to `xen', not to `i386' or
`amd64', so the conditional never took effect.

(The side effect of setting MKDTRACE=yes when building Xen kernels is
unlikely to matter; that affects module and userland builds.)

PR port-xen/57535

XXX pullup-10

(riastradh)

rbtree(3): No (rb_tree_t) for constant initializer.

This is a compound literal, which is not formally valid as a constant
initializer.

XXX pullup-10

(riastradh)

i386/XEN3PAE_DOMU: Enable KDTRACE_HOOKS.

XXX pullup-10

(riastradh)

i386/XEN3PAE_DOM0: Enable KDTRACE_HOOKS.

XXX pullup-10

(riastradh)

amd64/XEN3_DOMU: Enable KDTRACE_HOOKS.

XXX pullup-10

(riastradh)

amdgpu: Mark float-dependent variables volatile.

This way they are computed -- using FP instructions -- before
DC_FP_END, after which point FP instructions will trap or behave
unpredictably.

This is a workaround, not a proper solution -- really, the
DC_FP_START/END calls should happen in a separate compilation unit
built without -mhard-float or whatever, but that's a lot more work to
make happen for now.

PR kern/57059

XXX pullup-10

(riastradh)

xhci(4): Don't panic on suspend if previous suspend/resume failed.

Trying to resume again probably won't make the situation much worse,
but panicking can definitely make it worse.

XXX pullup-10

(riastradh)

hdafg(4): Do hotplug detection in kthread, not callout.

This can sometimes take a while (~1ms), and the logic to suspend the
callout on device suspend/resume was racy (PR kern/57322).

XXX pullup-8
XXX pullup-9
XXX pullup-10

(riastradh)

x86/fpu: In kernel mode fpu traps, print the instruction pointer.

(riastradh)

autoconf(9): Print `waiting for devices' normally once a minute.

(riastradh)

rbtree(3): Fix RBDEBUG build with RB_TREE_INITIALIZER.

XXX pullup-10

(riastradh)

libc: Use RB_TREE_INITIALIZER to nix initfini.c/_env.c coupling.

Doesn't actually help to remove _env.c from programs that don't use
it, though, because various internal things like __diagassert13 still
call getenv.

(riastradh)

rbtree(3): New RB_TREE_INITIALIZER macro.

Allows static initialization of an rbtree.

XXX pullup-10

(riastradh)

acpiec(4): One more debug message about read/write polling timeout.

(riastradh)

acpiec(4): Take a lock around acpiec_cold updates.

Otherwise we race with readers -- probably harmlessly, but let's
avoid the appearance of problems.

XXX Maybe acpiec_suspend and acpiec_shutdown should interrupt
transactions and force them to fail promptly?

XXX This looks bad because acpiec_cold is global and sc->sc_mtx
doesn't look like it's global, but we expect to have only one
acpiec(4) device anyway from what I understand.  Maybe we should move
acpiec_cold into the softc?

(riastradh)