acpiec(4): Factor out if (state == FREE) cv_signal(sc_cv).

In principle this could have a functional change, but at worst, it is
to signal more wakeups than needed, which should always be safe.

(riastradh)

acpiec(4): Pass softc, not device_t, to acpiec_space_handler.

Better to keep the device_t isolated to public interfaces.  Simpler
internally this way.

No functional change intended.

(riastradh)

acpiec(4): Pass softc, not device_t, to acpiec_gpe_query thread.

Simpler.

No functional change intended.

(riastradh)

acpiec(4): Pass softc, not device_t, to acpiec_read/write.

Simpler, type-safer.

No functional change intended.

(riastradh)

acpiec(4): Pass softc, not device_t, to acpiec_lock/unlock.

Simpler, type-safer.

No functional change intended.

(riastradh)

acpiec(4): Pass softc, not device_t, to acpiec_gpe_handler.

Simpler.

No functional change intended.

(riastradh)

acpiec(4): Pass softc, not device_t, to acpiec_callout.

Simpler.

No functional change intended.

(riastradh)

acpiec(4): Pass softc, not device_t, to acpiec_gpe_state_machine.

Simpler, type-safer.

No functional change intended.

(riastradh)

acpiec(4): Factor wait logic out.

No functional change intended.

(riastradh)

acpiec(4): Merge returns in acpiec_read/write.

No functional change intended.

(riastradh)

acpiec(4): Don't touch sc->sc_state outside sc->sc_mtx.

(riastradh)

acpiec(4): Fix cv_timedwait abuse in acpiec_read/write.

(riastradh)

acpiec(4): Fix interrupt wait loop in acpiec_gpe_query thread.

(riastradh)

acpiec(4): Fix cv_wait loop around sc->sc_got_sci.

That is, make it actually loop as required, so it gracefully handles
spurious wakeups instead of barging into invalid states.

(riastradh)

acpiec(4): Set sc_got_sci only when a transaction is over.

Before, when the acpiec thread noticed an SCI had been requested and
entered acpiec_gpe_state_machine to send the query command, it would
see the SCI is still requested -- because it had yet to acknowledge
it by setting the query command! -- and think the EC was asking for a
_second_ SCI.

So once the first SCI transaction was over, it would start a second
one, even though the EC hadn't asked for another -- and this would
wedge on some ECs.

Now, acpiec_gpe_state_machine waits to see what state we transition
to before taking the SCI bit to mean we need to notify the acpiec
thread to handle another query.

That way, when the acpiec thread enters acpiec_gpe_state_machine with
EC_STATE_QUERY, it can send the query command first, with the side
effect of clearing the SCI bit in subsequent reads of the status
register, and it won't think another SCI has been requested until it
returns to EC_STATE_FREE and sees the SCI bit set again in the status
register.

Possibly relevant PRs:

PR kern/53135
PR kern/52763
PR kern/57162

(riastradh)

acpiec(4): Assert state is free when we start a transaction.

No functional change intended.

(riastradh)

acpiec(4): Sprinkle comments.

Note where this code is abusing cv_wait and needs a loop to handle
spurious wakeups.

No functional change intended.

(riastradh)

acpiec(4): Clarify lock order and sprinkle lock assertions.

No functional change intended.

(riastradh)

acpiec(4): New ACPIEC_DEBUG option.

Value is bit mask of debug messages to enable.

Enable in x86/ALL kernels.

No functional change intended when the option is off.

(riastradh)

acpiec(4): Record device_t self.

Not used yet, to be used soon for device_printf and to allow making
some of the internal functions a little more type-safe later.

(riastradh)

device_printf(9): Lock to avoid interleaving output.

XXX pullup-9
XXX pullup-10

(riastradh)

timecounter(9): Sprinkle membar_consumer around th->th_generation.

This code was apparently written under the misapprehension that
membar_producer on one side is good enough, but that doesn't
accomplish anything other than making the code unnecessarily obscure.

For semantics, you always always always need memory barriers to come
in pairs, with membar_consumer on the reading side if you want the
membar_producer to have on the writing side to have any useful
effect.

It is unfortunate that this might hurt performance, but that's an
unfortunate consequence of the design made without understanding
memory barriers, not an unfortunate consequence of the memory
barriers.

If it is really critical for the read side to avoid memory barriers,
then the write side needs to issue an IPI or xcall to effect memory
barriers -- at higher cost to the write side, of course.

(riastradh)

timecounter(9): Use atomic_store_release/load_consume for timehands.

This probably fixes real bugs on Alpha and makes the synchronization
pattern clearer everywhere.

(riastradh)

timecounter(9): Use seqlock for atomic snapshots of timebase.

(riastradh)

i386/LEGACY: Enable ATA_DOWNGRADE_MODE.

PR kern/57362

(riastradh)

ata(4): Add ATA_DOWNGRADE_MODE to opt_ata.h.

This way adding it to kernel config will trigger recompilation.

(riastradh)

Revert "timecounter(9): Use an ipi barrier on time_second/uptime rollover."

Evidently rump doesn't have ipi, so this won't work unless we have an
alternate approach for rump.

(riastradh)

signal(7): Clarify semantics of SIGCHLD with SIG_IGN or SA_NOCLDWAIT.

The semantics is not just a nonportable hack for SysV compatibility;
it is enshrined in POSIX.

Related: PR 57527

(riastradh)

timecounter(9): No static; committed wrong version of patch.

(riastradh)

timecounter(9): Limit scope of time__second/uptime.

Relevant only if __HAVE_ATOMIC64_LOADSTORE -- not updated otherwise.

(riastradh)

timecounter(9): Use an ipi barrier on time_second/uptime rollover.

This way we only need __insn_barrier, not membar_consumer, on the
read side.

(riastradh)

timecounter(9): Revert last -- timecounter_lock is already IPL_HIGH.

(riastradh)

timecounter(9): Ward off interrupts during time_second/uptime update.

Only relevant during 32-bit wraparound, so the potential performance
impact of using splhigh here is negligible; indeed, we would have to
go out of our way to exercise this in tests before it will ever
happen in the next century.

(riastradh)

timecounter(9): Fix thinko in previous.

Swapped the wrong variable in this mental macro expansion!

(riastradh)

sys/param.h: Welcome to NetBSD 10.99.6!

Courtesy of:

- one uvmfault rndsource, not one per CPU
- removal of non-atomic time_second and time_uptime symbols
- struct syncobj::sobj_name

(riastradh)

uvm(9): One rndsource for faults -- not one per CPU.

All relevant state is per-CPU anyway; the only substantive difference
this makes is how many entries appear in `rndctl -l' output and what
they are called -- formerly the somewhat confusing `cpuN', meaning
`page faults on cpuN', and now just `uvmfault'.  I don't think
there's any real value in being able to enable or disable measurement
or counting of page faults on one CPU vs others, so although this
could be a minor compatibility change, it's hard to imagine it
matters much.

XXX kernel ABI change in struct cpu_info

(riastradh)

kern: Make time_second and time_uptime macros that work atomically.

These use atomic load on platforms with atomic 64-bit load, and
seqlocks on platforms without.

This has the unfortunate side effect of slightly reducing the real
times available on 32-bit platforms, from ending some time in the
year 584942417218 AD, available on 64-bit platforms, to ending some
time in the year 584942417355 AD.  But during that slightly shorter
time, 32-bit platforms can avoid bugs arising from non-atomic access
to time_uptime and time_second.

Note: All platforms still have non-atomic access problems for
bintime, binuptime, nanotime, nanouptime, &c.  This can be addressed
by putting a seqlock around timebasebin and possibly some other
variable -- to be done in a later change.

XXX kernel ABI change -- deleting symbols

(riastradh)

ddb: Teach `show all tstiles' to show the type of each lock.

(riastradh)

sys/syncobj.h: Expose this to _KMEMUSER for future use by crash(8).

This will allow crash(8) to get at the syncobj name.

(riastradh)

kern: New struct syncobj::sobj_name member for diagnostics.

XXX potential kernel ABI change -- not sure any modules actually use
struct syncobj but it's hard to rule that out because sys/syncobj.h
leaks into sys/lwp.h

(riastradh)

kthread(9): Fix nested kthread_join.

No reason for one kthread_join to interfere with another, or to cause
non-cyclic dependencies to get stuck.

Uses struct lwp::l_private for this purpose, which for user threads
stores the tls pointer.  I don't think anything in kthread(9) uses
l_private -- generally kernel threads will use lwp specificdata.  But
maybe this should use a new member instead, or a union member with an
existing pointer for the purpose.

(riastradh)

rump: Fix comment to reflect l_private -> l_sched.info.

Comment was misspelled `l->private', hence didn't come up in grep.

(riastradh)

rump: Use l_sched.info, not l_private, for cv waits.

- l_sched is scheduler-private, used only by sched_m2.c, should be safe
- l_private is lwp-private, used by tls in user threads, would like to
  reuse for kthreads too

(riastradh)

aarch64: Omit needless xcfunc_t casts by using xcfunc_t correctly.

No functional change intended, except for avoiding possible undefined
behaviour that could have made demons come flying out your nose.

(riastradh)

x86: Sprinkle extensive commentary about %fs/%gs initialization.

Plus some other side quests like the three-stage GDT metamorphosis
lifecycle.

No functional change intended.

(riastradh)

sys/rndio.h: Fix whitespace.

No functional change intended.

(riastradh)

sys/rndsource.h: Fix whitespace.

No functional change intended.

XXX pullup-10

(riastradh)

sys/rndsource.h: Update comment to reflect new use of delta members.

No functional change intended.

XXX pullup-10

(riastradh)

i386: Re-enable HEARTBEAT.

(riastradh)

heartbeat(9): For now, use time_uptime without atomic_load_relaxed.

A later commit will change time_uptime to a macro so it is atomic,
using atomc_load_relaxed if possible or seqlock if not.

(riastradh)

heartbeat(9): Avoid xcall(9) while cold.

(riastradh)

evbarm: Enable HEARTBEAT option in GENERIC and GENERIC64 kernels.

(riastradh)

amd64: Enable HEARTBEAT option, not just a HEARTBEAT option comment.

(riastradh)

x86: Enable HEARTBEAT by default in GENERIC.

(riastradh)

i386/ALL: Enable HEARTBEAT.

(riastradh)

signal(7): Clarify quirky SysV-inspired SIGCHLD semantics.

Suggest a portable alternative approach for detaching subprocesses.

Break wall of text into paragraphs while here.

(riastradh)

amdgpu: Define CONFIG_DRM_AMD_DC to enable display core logic.

Probably resolves a host of issues with amdgpu not detecting
displays!

Noticed by rjs@.

PR kern/57059
PR kern/57325
PR kern/57452

XXX pullup-10

(riastradh)

evbarm/g42xxeb: Fix off-by-one in previous.

The original find_first_bit(x) was 31 - __builtin_clz((uint32_t)x),
which is equivalent to fls32(x) - 1, not to fls32(x).

Note that fls32 is 1-based and returns 0 for x=0.

(riastradh)

arm/xscale: Missed a spot with previous find_first_bit commit.

(riastradh)

t_bitops: Show the bad results on failure.

(riastradh)

arm/xscale: Use sys/bitops.h fls32 - 1 instead of 31 - __builtin_clz.

Sidesteps namespace collision with `#define bits ...' in net/zlib.c.

(riastradh)

ia64: Need sys/types.h for u_int, vaddr_t; sys/mutex.h for kmutex_t.

(riastradh)

marvell: Reconcile arm vs powerpc marvell_intr_establish.

If there's a better way to do this without #ifdef __powerpc__, please
have at it!

(riastradh)

xen: Record event when local view of timecounter is behind global.

(riastradh)

kern: Print more detailed monotonic-clock-went-backwards messages.

Let's try harder to track this down.

XXX Should add dtrace probes.

(riastradh)

Break cycle by using `struct kmutex *' instead of `kmutex_t *'.

sys/sched.h included sys/mutex.h
which includes sys/intr.h
which includes machine/intr.h
which on cats includes arm/footbridge/footbridge_intr.h
which includes arm/cpu.h
which includes sys/cpu_data.h
which includes sys/sched.h

But there was never any real need for sys/mutex.h in sys/sched.h,
because it only uses pointers to the opaque struct kmutex.  Cycle
broken by using `struct kmutex *' instead of pulling in sys/mutex.h
for the definition of kmutex_t.

Side effect: This revealed that sys/cpu_data.h needed sys/intr.h
(which was pulled in accidentally by sys/mutex.h via sys/sched.h) for
SOFTINT_COUNT.  Also revealed some other machine/cpu.h header files
were missing includes of sys/mutex.h for kmutex_t.

(riastradh)

ddb: Cast pointer to uintptr_t, then to db_expr_t.

Avoids warnings about conversion between pointer and integer of
different size on some architectures.

(riastradh)

machine/mutex.h: Sprinkle sys/types.h, omit machine/lock.h.

Turns out machine/lock.h is not needed for __cpu_simple_lock_t, which
always comes from sys/types.h.  And, really, sys/types.h (or at least
sys/stdint.h) is needed for uintN_t and uintptr_t.

(riastradh)

mac68k/intr.h: Expose ipl_cookie_t to _KMEMUSER for crash(8).

(riastradh)

hp300/intr.h: Put most of this under #ifdef _KERNEL.

Only ipl_cookie_t really needs to be exposed now, for crash(8).

(riastradh)

x68k/intr.h: Put functions under _KERNEL so crash(8) can use this.

(riastradh)

vax/intr.h: Expose ipl_cookie_t to _KMEMUSER for crash(8).

(riastradh)

sun68k/intr.h: Expose ipl_cookie_t to _KMEMUSER for crash(8).

(riastradh)

sys/ksem.h: Hack around fstat(8) abuse of _KERNEL.

(riastradh)

next68k/intr.h: Expose ipl_cookie_t to _KMEMUSER for crash(8).

(riastradh)

news68k/intr.h: Fix includes.  Put some definitions under _KERNEL.

(riastradh)

mvme68k/intr.h: Expose ipl_cookie_t to _KMEMUSER for crash(8).

(riastradh)

luna68k/intr.h: Expose ipl_cookie_t to _KMEMUSER for crash(8).

(riastradh)

hppa/intr.h: Expose ipl_cookie_t to _KMEMUSER for crash(8).

(riastradh)

ddb: Cast pointer to uintptr_t first before db_expr_t.

(riastradh)

arm/cpu.h: Need sys/param.h for COHERENCY_UNIT.

Nix machine/param.h -- not meant to be used directly, pulled in by
sys/param.h.

(riastradh)

atari/intr.h: Expose ipl_cookie_t with _KMEMUSER for crash(8).

(riastradh)

cesfic/intr.h: Expose ipl_cookie_t to userland only with _KMEMUSER.

Probably not necessary but let's be a little more cautious about
this.

(riastradh)

cesfic/intr.h: Expose ipl_cookie_t to userland for crash(8).

(riastradh)

amiga/intr.h: Don't define spl*() functions if !_KERNEL.

This is used by crash(8) now, and what's important is ipl_cookie_t.

(riastradh)

linux/idr.h: Need <sys/mutex.h> for kmutex_t.

(riastradh)

hp300/intr.h: Fix missing includes.

(riastradh)

opencrypto/cryptodev.h: Fix includes.

- Move sys/condvar.h under #ifdef _KERNEL.
- Add some other necessary includes and forward declarations.
- Sort.

(riastradh)

evbarm/intr.h: Define ipl_cookie_t before including ARM_INTR_IMPL.

Otherwise arm/mutex.h doesn't work, due to a cyclic dependency which
should really be fixed.

(riastradh)

arm/mutex.h: Need machine/intr.h, machine/lock.h.

For ipl_cookie_t and __cpu_simple_lock_t.

(riastradh)

sys: Rip <sys/resourcevar.h> out of <uvm/uvm_param.h>.

And thus out of <sys/param.h>, which is exceedingly overused and
fragile and delenda est.

Should fix (some) issues with the recent inclusion of machine/lock.h
in various machine/mutex.h files.

(riastradh)

i915: Fail gracefully, don't panic, with unsupported ggtt views.

Workaround for part of PR kern/56648.

XXX pullup-10

(riastradh)

t_clock_gettime: Avoid signed/unsigned comparison.

(riastradh)

ddb: New `show all tstiles' command.

Shows who's waiting for which locks and what the owner is up to.

XXX pullup-10

(riastradh)

machine/mutex.h: Sprinkle includes so this can be used by crash(8).

XXX pullup-10

(riastradh)

sys/rwlock.h: Make this more self-contained for bool.

XXX pullup-10

(riastradh)

clock_gettime(2): Fix CLOCK_PROCESS/THREAD_CPUTIME_ID.

Use same calculation as getrusage, not some ad-hoc arithmetic of
internal scheduler parameters that are periodically rewound.

PR kern/57512

XXX pullup-8
XXX pullup-9
XXX pullup-10

(riastradh)

t_timerfd: Sprinkle slightly more diagnostics.

Might help us to see if we're off by just a little bit (maybe a tiny
jitter between the hardclock timer and the monotonic clock), or if
something is seriously amiss when the timerfd_block test fails
sporadically.

(riastradh)

t_clock_gettime: Add test for PR kern/57512.

(riastradh)

curcpu_stable(9): New function for asserting curcpu() is stable.

(riastradh)

kern_resource.c: Fix brace placement.

No functional change intended.

(riastradh)

Revert unintentional changes to kern_lock.c in previous commit.

(riastradh)

heartbeat(9): Test whether curcpu is stable, not kpreempt_disabled.

kpreempt_disabled worked for my testing because I tested on aarch64,
which doesn't have kpreemption.

XXX Should move curcpu_stable() to somewhere that other things can
use it.

(riastradh)

xcall(9): If !mp_online, raise spl or set LP_BOUND to call func.

High-priority xcalls may reasonably assume that the spl is raised to
splsoftserial, so make sure to do that in xc_broadcast.

Low-priority xcalls may reasonably enter paths that assume the lwp is
bound to a CPU, so let's make it assertable even if it doesn't have
any other consequences when !mp_online.

XXX pullup-8
XXX pullup-9
XXX pullup-10

(riastradh)

sys/param.h: Welcome to 10.99.5, courtesy of heartbeat(9).

(riastradh)

heartbeat(9): Fix grammaro in man page.

(riastradh)

amd64/ALL: Turn on heartbeat checks.

(riastradh)

heartbeat(9): New mechanism to check progress of kernel.

This uses hard interrupts to check progress of low-priority soft
interrupts, and one CPU to check progress of another CPU.

If no progress has been made after a configurable number of seconds
(kern.heartbeat.max_period, default 15), then the system panics --
preferably on the CPU that is stuck so we get a stack trace in dmesg
of where it was stuck, but if the stuckness was detected by another
CPU and the stuck CPU doesn't acknowledge the request to panic within
one second, the detecting CPU panics instead.

This doesn't supplant hardware watchdog timers.  It is possible for
hard interrupts to be stuck on all CPUs for some reason too; in that
case heartbeat(9) has no opportunity to complete.

Downside: heartbeat(9) relies on hardclock to run at a reasonably
consistent rate, which might cause trouble for the glorious tickless
future.  However, it could be adapted to take a parameter for an
approximate number of units that have elapsed since the last call on
the current CPU, rather than treating that as a constant 1.

XXX kernel revbump -- changes struct cpu_info layout

(riastradh)

crashme(9): New crash methods with raised ipl or kpreempt disabled.

(riastradh)

select(2), poll(2): Clarify return values.

(riastradh)

t_sig_backtrace: Flush stdout before writing to STDOUT_FILENO.

Avoids confusing ordering of output.

(riastradh)

tests/libexec/ld.elf_so: Fix helper library makefiles.

1. Consolidate logic into a single helper.mk to reduce duplication.
2. Set NO* variables, not MK* variables which are reserved for user.
3. Avoid eager X!= in favour of lazy ${X:sh}.
4. Mark _g.a set list entries obsolete.  Never should've been built!

PR misc/57462

(riastradh)

libbsdmalloc: Use a multiply-overflow that clang supports too.

XXX pullup-10

(riastradh)

t_posix_memalign: Simplify.

No functional change intended.

(riastradh)

libbsdmalloc: Make aligned_alloc(A, S) actually align when A > S.

XXX pullup-10

(riastradh)

t_posix_memalign: Fix this to reflect restriction lifted in C17.

(riastradh)

Revert "fsck_ffs(8): Ensure A divides S before aligned_alloc(A, S)."

C17 lifted this restriction.

(riastradh)

Revert "newfs(8): Ensure A divides S before aligned_alloc(A, S)."

C17 lifted this restriction.

(riastradh)

Revert "nvmectl(8): Ensure A divides S before aligned_alloc(A, S)."

C17 lifted this restriction.

(riastradh)

libbsdmalloc: Lift C11 divisibility restriction on aligned_alloc.

Restriction was lifted in C17.

XXX pullup-10

(riastradh)

Revert "jemalloc: Enforce alignment-must-divide-size rule of aligned_alloc."

Apparently this restriction was lifted in C17, and this was even
documented in a part of the man page I didn't notice because I got
stuck at the incomplete sentence `The aligned_alloc function conforms
to.'.  Sorry for the noise, folks.

(riastradh)

libbsdmalloc: Fix build with DEBUG.

Nix __P while here.

XXX pullup-10

(riastradh)

fsck_ffs(8): Fix whitespace issues.

- Nix trailing whitespace.
- Omit excessive blank lines.
- Insert missing blank lines between $NetBSD$ and copyright.

No functional change intended.

(riastradh)

nvmectl(8): Ensure A divides S before aligned_alloc(A, S).

Required by C11 Sec. 7.22.3.1 The aligned_alloc function, para. 2,
p. 348:

  The value of alignment shall be a valid alignment supported by the
  implementation and the value of size shall be an integral multiple
  of alignment.

XXX pullup-10

(riastradh)

newfs(8): Ensure A divides S before aligned_alloc(A, S).

Required by C11 Sec. 7.22.3.1 The aligned_alloc function, para. 2,
p. 348:

  The value of alignment shall be a valid alignment supported by the
  implementation and the value of size shall be an integral multiple
  of alignment.

XXX pullup-10

(riastradh)

fsck_ffs(8): Ensure A divides S before aligned_alloc(A, S).

Required by C11 Sec. 7.22.3.1 The aligned_alloc function, para. 2,
p. 348:

  The value of alignment shall be a valid alignment supported by the
  implementation and the value of size shall be an integral multiple
  of alignment.

XXX pullup-10

(riastradh)

libc: Fix missing membar_consumer in dl_iterate_phdr.

Pairs with the existing membar_producer.

XXX pullup-8
XXX pullup-9
XXX pullup-10

(riastradh)

libbsdmalloc: Nix K&R definitions.  Bump WARNS to 3.

(riastradh)

libbsdmalloc: New man page.

XXX pullup-10

(riastradh)

libbsdmalloc: Bump shlib minor.

New symbols, new minor.

XXX pullup-10

(riastradh)

libbsdmalloc: Nix trailing whitespace.

No functional change intended.

XXX pullup-10

(riastradh)

libbsdmalloc: Provide all allocator front ends and fork hooks.

Front ends:

- aligned_alloc
- calloc
- posix_memalign

Fork hooks:

- _malloc_prefork
- _malloc_postfork
- _malloc_postfork_child

Otherwise these will pull in the jemalloc definitions from libc,
which (a) defeats the purpose, and (b) won't work correctly with
fork and threads.

Thanks to kre@ and the thread on tech-userlevel for pointing me in
the right direction to making this actually work to override
jemalloc:

https://mail-index.netbsd.org/tech-userlevel/2023/06/30/msg013957.html

Note: libbsdmalloc doesn't set errno=ENOMEM on malloc failure, but
these front ends do (even aligned_alloc, which is from C11, which
doesn't define ENOMEM at all, but this pacifies our aligned_alloc
tests in t_posix_memalign.c).  Might want to fix that.

XXX pullup-10

(riastradh)

t_posix_memalign: Expand test cases and properties.

- Test cartesian product of a sampling of sizes and a sampling of
  alignments.

- Verify all the edge cases I could find in posix_memalign and
  aligned_alloc, including failure modes.

- Test an unreasonably large (but aligned) allocation size.

- Use ATF_CHECK_* instead of ATF_REQUIRE_* so all failures will be
  reported, not just the first one.

- While here, build with -fno-builtin-aligned_alloc and with
  -fno-builtin-posix_memalign to make sure the compiler doesn't try
  any shenanigans.

XXX pullup-10

(riastradh)

jemalloc: Enforce alignment-must-divide-size rule of aligned_alloc.

C11, Sec. 7.22.3.1 The aligned_alloc function, paragraph 2, p. 348:

  The value of alignment shall be a valid alignment supported by the
  implementation and the value of size shall be an integral multiple
  of alignment.

posix_memalign does not appear to have any corresponding constraint.

XXX pullup-10

(riastradh)

riscv: Fix (U)INT64_C suffix to match gcc's built-in idea of types.

XXX pullup-10

(riastradh)

tpm(4): Switch tsleep to kpause.

Nothing is interrupt-driven here, so no need for condvars to allow
more prompt wakeups.

(riastradh)

getrandom(2): Reduce editorializing tone in man page.

XXX pullup-10

(riastradh)

getrandom(2), getentropy(3): Rework man pages.

Make supportable promises.  Omit needless verbiage.  Give caveats
with cross-references to entropy(7).  Emphasize that security is
necessarily relative to system configuration.

XXX pullup-10

(riastradh)

rc.conf(5): Set entropy=wait by default.

We no longer block indefinitely -- if nothing else, the hardclock
timer should yield enough samples to unblock /dev/random on all but
the most severely deterministic machines -- so it should be generally
safe for availability to set entropy=wait.

This doesn't guarantee that HWRNG/seed has been provided before you
run ssh-keygen or call getentropy(3) in a user application, but it
does raise the security above netbsd<=9.

PR security/55659
PR lib/56905

XXX pullup-10

(riastradh)

security(5): Check kern.entropy.needed for confident entropy.

Don't test whether a non-blocking read from /dev/random would return
data.

For the sake of availability, /dev/random will unblock based on sources
like timer interrupts, which we can't confidently assert anything about
the actual unpredictability of.

Here, the goal is to highlight systems that have neither obtained
entropy from an HWRNG with a confident entropy assessment, nor been
seeded from a source the operator knows about.

XXX pullup-10

(riastradh)

entropy(9): Reintroduce netbsd<=9 time-delta estimator for unblocking.

The system will (in a subsequent change) by default block for this
condition before almost all of userland is running (including
/etc/rc.d/sshd key generation).  That way, a never-blocking
getentropy(3) API will never return any data without at least
best-effort entropy like netbsd<=9 did to applications except in
single-user mode (where you have to be careful about everything
anyway) or in the few processes that run before a seed can even be
loaded (where blocking indefinitely, e.g. when generating a stack
protector cookie in libc, could pose a severe availability problem
that can't be configured away, but where the security impact is low).

However, (in another subsequent change) we will continue to use
_only_ HWRNG driver estimates and seed estimates, and _not_
time-delta estimator, for _warning_ about security in motd, daily
security report, etc.  And if HWRNG/seed provides enough entropy
before time-delta estimator does, that will unblock /dev/random too.

The result is:

- Machines with HWRNG or seed won't warn about entropy and will
  essentially never block -- even on first boot without a seed, it
  will take only as long as the fastest HWRNG to unblock.

- Machines with neither HWRNG nor seed:
  . will warn about entropy, giving feedback about security;
    and
  . will avoid returning anything more predictable than netbsd<=9;
    but
  . won't block (much) longer than netbsd<=9 would (and won't block
    again after blocking once, except with kern.entropy.depletion=1 for
    testing).

  (The threshold for unblocking is now somewhat higher than before:
  512 samples that pass the time-delta estimator, rather than 80 as
  it used to be.)

  And, of course, adding a seed (or HWRNG) will prevent both warnings
  and blocking.

The mechanism is:

1. /dev/random will block until _either_

  (a) enough bits of entropy (256) from reliable sources have been
      added to the pool, _or_

  (b) enough samples have been added from any sources (512), passing
      the old time-delta entropy estimator, that the possible
      security benefit doesn't justify holding up availability any
      longer (`best effort'), except on systems with higher security
      requirements like securelevel=2 which can disable non-HWRNG,
      non-seed sources with rndctl_flags in rc.conf(5).

2. dmesg will report `entropy: ready' when 1(a) is satisfied, but if
  1(b) is satisfied first, it will report `entropy: best effort', so
  the concise log messages will reflect the timing and whether in
  any period of time any of the system might be relying on best
  effort entropy.

3. The sysctl knob kern.entropy.needed (and the ioctl RNDGETPOOLSTAT
  variable rndpoolstat_t::added) still reflects the number of bits
  of entropy from reliable sources, so we can still use this to
  suggest regenerating ssh keys.

  This matters on platforms that can only be reached, after flashing
  an installation image, by sshing in over a (private) network, like
  small network appliances or remote virtual machines without
  (interactive) serial consoles.  If we blocked indefinitely at boot
  when generating ssh keys, such platforms would be unusable.  This
  way, platforms are usable, but operators can still be advised at
  login time to regenerate keys as soon as they can actually load
  entropy onto the system, e.g. with rndctl(8) on a seed file copied
  from a local machine over the (private) network.

4. On machines without HWRNG, using a seed file still suppresses
  warnings for users who need more confident security.  But it is no
  longer necessary for availability.

This is a compromise between availability and security:

- The security mechanism of blocking indefinitely on machines without
  HWRNG hurts availability too much, as painful experience over the
  multiple years since I made the mistake of introducing it have
  shown.  (Sorry!)

- The other main alternative, not having a blocking path at all (as I
  pushed for, and as OpenBSD has done for a long time) could
  potentially reduce security vs netbsd<=9, and would run against the
  expectations set by many popular operating systems to the severe
  detriment of public perception of NetBSD security.

Even though we can't _confidently_ assess enough entropy from, e.g.,
sampling interrupt timings, this is the traditional behaviour that
most operating systems provide -- and the result here is a net
nondecrease in security over netbsd<=9, because all paths from the
entropy pool to userland now have at least as high a standard before
returning data as they did in netbsd<=9.

PR kern/55641
PR pkg/55847
PR kern/57185
https://mail-index.netbsd.org/current-users/2020/09/02/msg039470.html
https://mail-index.netbsd.org/current-users/2020/11/21/msg039931.html
https://mail-index.netbsd.org/current-users/2020/12/05/msg040019.html

XXX pullup-10

(riastradh)

vis(3): Avoid nonportable MIN in portable code.

(riastradh)

tsleep: Comment out kernel lock assertion for now.

Breaks tpm(4) which breaks boot on a lot of systems.  tpm(4)
shouldn't be using tsleep; it doesn't appear to even have an
interrupt handler for wakeups, so it could get by with kpause.  If it
ever did sprout an interrupt handler it should use condvar(9) anyway.
But for now I don't have time to fix it tonight.

(riastradh)

tsleep(9): Assert kernel lock held.

This is never safe to use without the kernel lock.  It should only
appear in legacy subsystems that still run with the kernel lock.

(riastradh)

rump: KASSERT(x && y) -> KASSERT(x); KASSERT(y)

Add some KASSERTMSG while here.

(riastradh)

pam_krb5: Refuse to operate without a key to verify tickets.

New allow_kdc_spoof overrides this to restore previous behaviour
which was vulnerable to KDC spoofing, because without a host or
service key, pam_krb5 can't distinguish the legitimate KDC from a
spoofed one.

This way, having pam_krb5 enabled isn't dangerous even if you create
an empty /etc/krb5.conf to use client SSO without any host services.

Perhaps this should use krb5_verify_init_creds(3) instead, and
thereby respect the rather obscurely named krb5.conf option
verify_ap_req_nofail like the Linux pam_krb5 does, but:

- verify_ap_req_nofail is default-off (i.e., vulnerable by default),
- changing verify_ap_req_nofail to default-on would probably affect
  more things and therefore be riskier,
- allow_kdc_spoof is a much clearer way to spell the idea,
- this patch is a smaller semantic change and thus less risky, and
- a security change with compatibility issues shouldn't have a
  workaround that might introduce potentially worse security issues
  or more compatibility issues.

Perhaps this should use krb5_verify_user(3) with secure=1 instead,
for simplicity, but it's not clear how to do that without first
prompting for the password -- which we shouldn't do at all if we
later decide we won't be able to use it anyway -- and without
repeating a bunch of the logic here anyway to pick the service name.

References about verify_ap_req_nofail:
- mit-krb5 discussion about verify_ap_req_nofail:
  https://mailman.mit.edu/pipermail/krbdev/2011-January/009778.html
- Oracle has the default-secure setting in their krb5 system:
  https://docs.oracle.com/cd/E26505_01/html/E27224/setup-148.html
  https://docs.oracle.com/cd/E26505_01/html/816-5174/krb5.conf-4.html#REFMAN4krb5.conf-4
  https://docs.oracle.com/cd/E19253-01/816-4557/gihyu/
- Heimdal issue on verify_ap_req_nofail default:
  https://github.com/heimdal/heimdal/issues/1129

(riastradh)

pam_ksu: No need for homedir access.

(riastradh)

pam: Disable pam_krb5, pam_ksu by default.

These are not useful unless you also set up /etc/krb5.conf and a
keytab for the host from the Kerberos KDC.  But having them enabled
by default means that creating /etc/krb5.conf just to enable use of
Kerberos for _client-side_ single sign-on creates usability issues.

As proposed on tech-security:
https://mail-index.netbsd.org/tech-security/2023/06/16/msg001160.html

(riastradh)

linux32_rt_sendsig: Memset zero before copyout.

Not sure if there's any padding here, but it's a pretty big
structure, fairly likely, so let's be rather safe than sorry.

XXX pullup-8
XXX pullup-9
XXX pullup-10

(riastradh)

compat_ossaudio: Zero-initialize idat before copyout.

Unclear if there are any paths to the copyout without initialization,
but let's play it safe to keep the auditing effort low.

XXX pullup-8
XXX pullup-9
XXX pullup-10

(riastradh)

compat_sunos32: Memset zero before copyout.

Unclear if this can leak anything but let's be on the safe side.

XXX pullup-8
XXX pullup-9
XXX pullup-10

(riastradh)

sysctl(7): Note nodev,nosuid requirement for user-mounted fs.

While here, tidy up language and markup a smidge.

(riastradh)

socket(2): Move prose list of socket type summaries into a table.

(riastradh)

socket(2): Tidy up markup a bit.

Use a tag list, not a literal block with hand-formatted text, for
protocols and socket type flags.

(riastradh)

/etc/rc.d/sshd: Use default curve for ECDSA keygen, not NIST P-521.

The default is NIST P-256, which:

(a) has plenty of cryptanalytic security,
(b) performs better on essentially all platforms (smaller enough that
    even the advantage of the Mersenne prime structure of P-521 can't
    compete), and
(c) likely gets more scrutiny on implementations than P-521 since it's
    more widespread.

(riastradh)

/etc/rc.d/sshd: Stop generating DSA host keys by default.

If you want them you can generate them yourself, but in this day and
age (Monday and 2023, specifically) there's no reason to be using DSA
except for compatibility with ancient legacy software.

(riastradh)

ld.elf_so: Sprinkle more debug messages on dlopen and error.

PR pkg/57445

(riastradh)

mesa: Remove massive kludge that worked around PR toolchain/50277.

Now that the underlying bug in ld.elf_so is fixed, the workaround is
causing harm.

XXX pullup-10

(riastradh)

ld.elf_so: Fix missing tab in debug/shl.mi in last change.

XXX pullup-10

(riastradh)

ld.elf_so: Test dynamic-then-static abuse via ctor.

XXX pullup-10

(riastradh)

ld.elf_so: Test another edge case of mismatched TLS models.

One library defines a symbol and _doesn't_ use it, so it has no
indication of whether the symbol is for static TLS or dynamic TLS,
and then two other libraries use it in different ways.

XXX pullup-10

(riastradh)

ld.elf_so: tls_extern dynamic_defabuse_eager must xfail differently.

If a symbol has already been resolved as dynamic TLS, any library
that tries to use it as static TLS cannot be dlopened.

XXX pullup-10

(riastradh)

ld.elf_so: Add new files to debug/shl.mi.

XXX pullup-10

(riastradh)

ld.elf_so: Separately test eager and lazy resolution of def tls ptr.

eager: before loading use library
lazy: after loading use library

XXX pullup-10

(riastradh)

ld.elf_so: Shorter test names.

No functional non-cosmetic change intended.

XXX pullup-10

(riastradh)

ld.elf_so: Test TLS abuse of static def, dynamic use and vice versa.

XXX pullup-10

(riastradh)

ld.elf_so: Factor out logic in TLS tests to make writing more easier.

No functional change intended.

XXX pullup-10

(riastradh)

ld.elf_so: Test extern dynamic TLS too.

XXX pullup-10

(riastradh)

ld.elf_so: Test variations on PR toolchain/50277.

XXX pullup-10

(riastradh)

ld.elf_so: Make tls alloc debug messages more detailed and greppable.

XXX pullup-10

(riastradh)

ld.elf_so: Sprinkle tls debug messages.

XXX pullup-10

(riastradh)

ld.elf_so: Fix set lists for MKDEBUG=yes builds with t_tls_extern.

XXX pullup-10

(riastradh)

ld.elf_so: Nix inadvertently committed private test program.

(riastradh)

ld.elf_so: Fix extern TLS test to match PR toolchain/50277.

Now it's actually testing the problem.

(riastradh)

ld.elf_so: New test for extern initial-exec TLS, PR toolchain/50277.

XXX pullup-10

(riastradh)

openssl: Add missing sparc compat openssl engines to set list.

(riastradh)

openssl: Delete capi makefiles -- not likely to ever see use here.

(riastradh)

openssl: Nix capi.so engine -- it's for the Windows crypto API.

Not likely to work in NetBSD base!

(riastradh)

openssl: Fix typo in mips set list.

(riastradh)

openssl: padlock.so engine for VIA CPUs makes sense only on x86.

(riastradh)

openssl: Link engines against libcrypto so they can be loaded.

(riastradh)

openssl: Wire up dynamic engine build goo.

This restores the devcrypto engine, which was converted from static
to dynamic since the previous OpenSSL update.

XXX Pretty sure I got some of the set list package names wrong, but
it's not clear what the right ones are or what consequences any of
this has.

XXX Needs testing.

(riastradh)

openssl: Add /usr/lib/${ARCHDIR}/openssl/engines to aarch64 build.

(riastradh)

Revert "openssl: Nix build goo for dynamically loaded engines."

Turns out there is a regression lurking here: the devcrypto engine
was made dynamic-only (https://github.com/openssl/openssl/pull/7859),
so to maintain that functionality, we need to build dynamic engines.
Will fix the aarch64 build issue and wire things up separately.

(riastradh)

openssl: Nix build goo for dynamically loaded engines.

- Engines are deprecated in openssl 3.

- We never actually had the .so engines wired up to be built in
  openssl 1.1, and judging by the lack of obsolete entries in the set
  lists, I don't think we ever had them wired up to be built at all.

(riastradh)

/etc/rc.d/sshd: New check cmd and reload precmd.

- check cmd: run `sshd -t' to check sshd_config file

- reload precmd: run check cmd before reloading so we don't nuke sshd
  if there's an error in the sshd_config file

(It is still possible to effectively nuke sshd by changing the
configuration tosomething that won't work on your network, but at
least we avoid making sshd just exit on reload when you make a typo
in a config option.)

XXX pullup-9
XXX pullup-10

(riastradh)

openssl: CWARNFLAGS, not COPTS, for -Wno-error=...

COPTS gets put too early, so these are cancelled out by -W flags
added later by bsd.sys.mk.  (How did this ever work???)

(riastradh)

openssl: Note that -Wno-error=char-subscripts likely masks real bugs.

(riastradh)

openssl: Consolidate new CWARNFLAGS.clang stuff in Makefile.openssl.

(riastradh)

openssl: Suppress clang warnings for using string literals as char *.

Not sure this is actually forbidden by the C standard.  There may be
bugs lurking here, but OpenSSL uses this extensively, so let's just
get the build going again.

(riastradh)

openssl: Nix trailing whitespace in build goo.

No functional change intended.

(riastradh)

libpthread: New pthread__smt_wait to put CPU in low power for spin.

This is now distinct from pthread__smt_pause, which is for spin lock
backoff with no paired wakeup.

On Arm, there is a single-bit event register per CPU, and there are two
instructions to manage it:

- wfe, wait for event -- if event register is clear, enter low power
  mode and wait until event register is set; then exit low power mode
  and clear event register

- sev, signal event -- sets event register on all CPUs (other
  circumstances like interrupts also set the event register and cause
  wfe to wake)

These can be used to reduce the power consumption of spinning for a
lock, but only if they are actually paired -- if there's no sev, wfe
might hang indefinitely.  Currently only pthread_spin(3) actually
pairs them; the other lock primitives (internal lock, mutex, rwlock)
do not -- they have spin lock backoff loops, but no corresponding
wakeup to cancel a wfe.

It may be worthwhile to teach the other lock primitives to pair
wfe/sev, but that requires some performance measurement to verify
it's actually worthwhile.  So for now, we just make sure not to use
wfe when there's no sev, and keep everything else the same -- this
should fix severe performance degredation in libpthread on Arm
without hurting anything else.

No change in the generated code on amd64 and i386.  No change in the
generated code for pthread_spin.c on arm and aarch64 -- changes only
the generated code for pthread_lock.c, pthread_mutex.c, and
pthread_rwlock.c, as intended.

PR port-arm/57437

XXX pullup-10

(riastradh)

libpthread: Use __nothing, not /* nothing */, for empty macros.

No functional change intended -- just safer to do it this way in case
the macros are used in if branches or comma expressions.

PR port-arm/57437 (pthread__smt_pause/wake issue)

XXX pullup-10

(riastradh)

amdgpu: Fix mostly harmless merge botch.

Avoids confusing error message that should have been confined to an
error branch.

(riastradh)

entropy(9): Avoid race between rnd_add_data and ioctl(RNDCTL).

XXX pullup-10

(riastradh)

entropy(9): On flags change, cancel any scheduled consolidation.

We've been instructed to lose confidence in existing entropy sources,
so let's make sure to re-gather enough entropy before the next
consolidation can happen, in case some of what would be counted in
consolidation is from those entropy sources.

XXX pullup-10

(riastradh)

efi(4): Fix logic to handle buffer sizing.

Can't KASSERT(datasize <= databufsize) because the caller is allowed
to pass in a too-small size and get ERR_BUFFER_TOO_SMALL back, with
the actual size returned so it can resize its buffer.  So just clamp
the size to the smaller of what the caller provided and what the
firwmare provided, instead of asserting anything.

PR kern/57076

XXX pullup-10

(riastradh)

autoconf(9): Omit config_detach kernel lock assertion too for now.

like in config_attach_pseudo, this assertion almost certainly
indicates real bugs, but let's try to get the tests back and running
again before addressing those.

(riastradh)

autoconf(9): Omit config_attach_pseudo kernel lock assertion for now.

Breaks too many things that I didn't test in the branch (cgd, fss,
&c.); let's address all forty-odd cases before turning it on.

(riastradh)

i386/GENERIC: Enable EFI runtime support.

PR kern/57076

XXX pullup-10

(riastradh)