Ifdef out fast path that relies on atomic 64-bit load/store.

(Really this sliding window business could probably be done with
32-bit sequence numbers and careful detection of wraparound, but
that's a little more effort to work out -- let's just unbreak the
builds for now.)

(riastradh)

make(1): clean up list handling

Lst_Init never returns NULL.  Casting postfix increment to void is
unnecessary since that is quite common.  Found a last instance of a
local variable named 'nlnode'.

(rillig)

make(1): remove type information from local variables in list library

Every node in this file is of type LstNode, which makes the 'l' in the
name 'ln' redundant.  The name 'ln' is quite close to the name 'l',
which in turn can easily be confused with the digit '1'.  Therefore,
rename 'l' to 'list' and 'ln' to 'node'.

(rillig)

make(1): use shorter field names in Lst and LstNode

In a doubly linked list, it is commonly known that the 'prev' and 'next'
fields of the nodes are pointers to other nodes, therefore this does not
need to be repeated in the variable name.

(rillig)

Disable libsodium HAVE_TI_MODE for now.

This may reduce performance by not taking advantage of 64x64->128
multiplications on some platforms, but let's worry about that later
and fix the build on the other platforms instead.

(riastradh)

Split flags onto separate lines, sorted, to make diffs easier.

(riastradh)

make(1): condense the API comments in the list library

Most mentioned "side effects" were either implementation details or
rather "main effects".  The wording of similar functions has been
aligned.

(rillig)

Disable -Wshadow for libsodium.

Evidently ed25519_ref10.c has a global and a local both named `d'.

(riastradh)

make(1): remove unnecessary type duplication in list implementation

Having both the interface types Lst/LstNode and the implementation types
List/ListNode does not make the code clearer, it is an unnecessary
abstraction.  Eliminating the implementation type means that some of the
local variables can be merged, which will make the code simpler than
before.

(rillig)

make(1): extract creation of a list node into a separate function

(rillig)

make(1): remove unnecessary macro PAlloc

The ptype parameter was never used, which made it redundant.  The
remaining text is so simple that it's not worth having a macro for this
purpose.  This makes it easier to read the code since the previously
implicit variable assignment is now clearly visible.

(rillig)

make(1): use stricter list API for sequential access

In several places, it just doesn't make sense to have a null pointer
when a list is expected.

In the existing unit tests, the list passed to Lst_Open is always valid,
but that's not a guarantee for real-world usage.  Therefore, Lst_Open
has been left for now, and Lst_OpenS is only the preferred alternative
to it.

(rillig)

make(1): assert correct usage of the Lst_Open API

All calls to Lst_Next are properly protected by Lst_Open, so there is no
possible assertion failure here.

(rillig)

Mark up a few missed equal signs as literal.

(uwe)

make(1): make list library code stricter

Up to now, the list library didn't distinguish between programming
mistakes (violations of invariants, illegal parameter values) and
actually interesting situations like "element not found in list".

The current code contains many branches for conditions that are neither
exercised by the unit tests nor by real-world usage.  There is no point
in keeping this unnecessary code.

The list functions will be migrated from their lenient variants to the
stricter variants in small parts, each function getting the S suffix
when it is made strict, to avoid any confusion about how strict a
particular function is.  When all functions have been migrated, they
will be renamed back to their original names.

While here, the comments of the functions are cleaned up since they
mention irrelevant implementation details in the API comments, as well
as "side effects" that are really main effects.

(rillig)

Markup fixes.

(uwe)

make(1): don't use bitfields in list processing

There is no need to squeeze unrelated fields of the struct into a single
object.  A bitset with a single flag is the same as a simple boolean
variable.

(rillig)

make(1): properly clean up the remaining code mentioning circular lists

(rillig)

Fix synopsis.

(uwe)

make(1): remove unused code for circular lists

The list library had probably been imported from a general-purpose
library that also supported circular lists.  These are not used by make
though.

After replacing Lst_Init(FALSE) with Lst_Init(), only a single call to
Lst_Init remained with a non-constant argument, and that was in
Lst_Concat, which was to be expected.

(rillig)

Fix markup around optional command name.

Avoids emboldened brackets.  I remain fuzzy on how grouping in roff
works, or doesn't work.

(riastradh)

fix hpcmips and evbppc builds (wrong type in panic()).

(mrg)

Man page enhancements.

Better describe the command search procedure.
Document "trap -P"
Describe what works as a function name.
More accurate description of reserved word recognition.
Be more accurate about when field splittng happens after
expansions (and in particular note that tilde expansions are
not subject to field splitting).  Be clear that "$@" is
not field split, it simply produces multiple fields as part
of its expansion (hence IFS is irrelevant to this), but if
used as $@ (unquoted) each field produced is potentially subject
to field splitting.  Other minor wording changes.

(kre)

Be less conservative about when we do clear_traps() when we have
traps_invalid (that is, when we actually nuke the parent shell's
caught traps in a subshell).  This allows more reasonable use of
"trap -p" (and similar) in subshells than existed before (and in
particular, that command can be in a function now - there can also
be several related commands like
traps=$(trap -p INT; trap -p QUIT; trap -p HUP)
A side effect of all of this is that
(eval "$(trap -p)"; ...)
now allows copying caught traps into a subshell environment, if desired.

Also att the ksh93 variant (the one not picked by POSIX as it isn't
generally as useful) of "trap -p" (but call it "trap -P" which extracts
just the trap action for named signals (giving more than one is usually
undesirable).  This allows
eval "$(trap -P INT)"
to run the action for SIGINT traps, without needing to attempt to parse
the "trap -p" output.

(kre)

Add lots of comments explaining what is happening in here.

Also enhance some of the DEBUG mode trace output (nothing visible
in a normal shell build).

A couple of very minor code changes that no-one should ever notice
(eg: one less wait() call in the case that there is nothing pending).

(kre)

Add missing description for wgconfig `set private-key' command.

(riastradh)

Mark up argument as such and write out wgconfig command correctly.

(riastradh)

Tweak markup so the square brackets don't become bold.

(riastradh)

When not compiling -DSMALL permit use of names RTMIN[+n] and RTMAX[-n]
(where n is a decimal integer in the range [0 .. SIGRTMAX-SIGRTMIN].
As usual a leading "sig" is ignored and the strings are case independent.

Some implementations do not name the real time signals, and using
labels like RTMIN+3 can be the only way they can be manipulated,
so allow that technique (we still return the RTnn names on the inverse
translation though).

Because this is used by both kill(1) and sh(1) the kill and trap
commands both gain access to the new notation (when !SMALL).

(kre)

Slightly less indentation.

(riastradh)

Fix self-xref.  Indent example displays.

(riastradh)

Fix up wg-userspace(8) man page.

(riastradh)

3RDPARTY janitorial service

(riastradh)

Note WireGuard.

(ozaki-r did the vast majority of the actual implementation in 2018;
I just dusted it off and imported it.)

(riastradh)

Mark KASSERT-only variable as __diagused.

(riastradh)

Add wg(4) to amd64/ALL.

(riastradh)

Fill out WireGuard man pages.

(riastradh)

Avoid callout_halt under lock.

- We could pass the lock in, except we hold another lock too.

- We could halt before taking the other lock, but it's not safe to
  sleep after getting the session pointer before taking its lock.

- We could halt before getting the session pointer, but then there's
  no point in doing it under the lock.

So just halt a little earlier instead.

(riastradh)

Sprinkle const.

(riastradh)

Use container_of rather than casts via void *.

(riastradh)

Use be32enc, rather than possibly unaligned uint32_t cast and htonl.

(riastradh)

KNF

(riastradh)

Update wgconfig(8) for proplib API changes.

Also check type tags before conversion.

(riastradh)

Use consttime_memequal, not memcmp, to compare secrets for equality.

(riastradh)

Take advantage of prop_dictionary_util(3).

(riastradh)

Split up wg_process_peer_tasks into bite-size functions.

(riastradh)

Fix race in wg_worker kthread destruction.

Also allow the thread to migrate between CPUs -- just not while we're
in the middle of processing and holding onto things with psrefs.

(riastradh)

Update for proplib API changes.

(riastradh)

Use SYSCTL_SETUP for net.wireguard subtree.

(riastradh)

Missed a spot -- add sys/crypto/blake2 to .PATH here.

(riastradh)

Fix in-kernel debug build.

(riastradh)

Implement sliding window for wireguard replay detection.

(riastradh)

Make `wgconfig --help' and variations work.

(riastradh)

Don't falsely assert cpu_softintr_p().

Will fail in the following stack trace:

wg_worker (kthread)
wg_receive_packets
wg_handle_packet
wg_handle_msg_data
KASSERT(cpu_softintr_p())

Instead, use kpreempt_disable/enable around softint_schedule.

XXX Not clear that softint is the right place to do this!

(riastradh)

Convert wg(4) to if_stat.

(riastradh)

Use cprng_strong, not cprng_fast, for ephemeral key.

(riastradh)

Descend into wg-userspace.

(riastradh)

Fix vestiges of libb2.

(riastradh)

[ozaki-r] Fix bugs found by maxv's audits

(riastradh)

[ozaki-r] Add wg files

(riastradh)

[ozaki-r] Changes to the kernel core for wireguard

(riastradh)

Import small BLAKE2s implementation.

(riastradh)

Split ge25519_scalarmult up in order to reduce stack usage.

(riastradh)

Reuse temporaries in ge25519_scalarmult to reduce stack usage.

(riastradh)

[ozaki-r] libsodium glue

(riastradh)

Don't cache id's for vnodes that have ACLs. ok chs@

(christos)

make(1): add test for variable expressions in conditions

(rillig)

make(1): move complicated boolean expression out of the function call

It's easier to inspect in a debugger this way.

(rillig)

make(1): add test for string literals in comparisons

(rillig)

make(1): enable debug logging for archive test

This test succeeds locally on NetBSD 8.0 but not in the official test
runs on https://releng.netbsd.org/test-results.html.  To see the
difference, run the test with full debug information.

This test is commented out in usr.bin/make/unit-tests/Makefile, but that
doesn't stop tests/usr.bin/make/t_make.sh from running it nevertheless,
since over there, all *.mk files are considered tests, be they commented
out or not.

(rillig)

make(1): add test for parsing functions in .if conditions

(rillig)

make(1): add test for the function defined(...)

(rillig)

make(1): remove unused function declarations

(rillig)

make(1): fix type of string length variables

(rillig)

make(1): fix wrong or outdated comments

(rillig)

Ticket #1059

(martin)

Whitespace.  NFCI.

(kre)

Pull up following revision(s) (requested by kim in ticket #1059):

share/mk/bsd.x11.mk: revision 1.133
share/mk/bsd.x11.mk: revision 1.136
external/mit/xorg/server/xorg-server/hw/xfree86/doc/Makefile: revision 1.8
external/mit/xorg/server/xorg-server/doc/Makefile: revision 1.6

Allow overriding the default man substitutions with X11EXTRAMANTRANSFORMS
fix various missing transforms for man page, ala PR#55422.
add method to transform both the "__foo__" and "@foo@"
version of various transforms, while upstream are converting
from the former to the latter it seems.  convert all the
common transforms to this method.

expand the grep for .pc files to look for missing @foo@

fixes to the man page genration, and ignore a few things
(thanks uwe@ - .IN lines, and stuff between tab(@) and .TE.)

xorg-server __default_font_path__ -> @default_font_path@,
and xfree86 transform xconfigdir, xkbdir, and modulepath
using new both method.

(martin)

mmap MAP_FAILED audit.

(riastradh)

Nix trailing whitespace.

(riastradh)

clang can't handle __aligned on anonymous structure initializers.

(riastradh)

nvmm-x86: improve the CPUID emulation

- x86-svm: explicitly handle 0x80000007 and 0x80000008. The latter
  contains extended features we must filter out. Apply the same in
  x86-vmx for symmetry.
- x86-svm: explicitly handle extended leaves until 0x8000001F, and
  truncate to it.

(maxv)

nvmm-x86: advertise the SERIALIZE instruction, available on future CPUs

(maxv)

Sprinkle some const

(roy)

call this bozohttpd/20200820

(mrg)

compare mmap return again MAP_FAILED not -1 or 0.

(mrg)

update for recent changes.

(mrg)

Add cleanup of possible leftover rump processes, replacing the
non-working cleanup code just removed from ffs_common.sh.  Fixes
PR bin/48892 with respect to the t_rquotad test.

(gson)

Remove non-functional cleanup code from test_case() and test_case_root().
It had no effect because RUMP_SOCKETS_LIST is not set in the shell
running the cleanup phase.  Even if RUMP_SOCKETS_LIST had been set,
the code would still not have worked correctly because it ran
rump.halt via "atf_check -s exit:1", which would cause the first
successful halting of a rump processes to be treated as a failure
and abort the cleanup without halting any other rump processes still
running.

(gson)

make(1): make a few comments more precise

(rillig)

make(1): use more descriptive variable name in ModifyWords

(rillig)

make(1): remove VARP_SUB_MATCHED

This flag didn't really belong to the other flags.  The other flags are
set during parsing and are then left as-is by ModifyWord_Subst and
ModifyWord_SubstRegex.

It's clearer to use a separate variable for storing whether there was a
match already.

(rillig)

make(1): consistently access args->pflags in ModifyWord_Subst

It was confusing that some accesses were via pflags and some via
args->pflags.

(rillig)

make(1): remove unreached code from bmake_strndup

The "at most" branch was never taken since all call sites in var.c only
ever need a substring, and the target buffer is not limited.  Therefore
rename the function and make it simpler.

It's ok that bmake_strldup is defined as estrndup in case of USE_EMALLOC
since that function's implementation is compatible to the "copy
exactly", it just contains some extra null checks that will never match
since the variable values cannot (well, or should not) contain null
bytes.  Theoretically they can, but the behavior then depends on the
exact implementation and is unreliable, therefore nobody does this.
After all, Makefiles are used for text processing, not for binary data.

(rillig)

move pmap segtab history into a new history of only 1000 entries,
but will overflow much slower than the main pmap history.

move various debug info into kernhist.  make pte array checker
into an array and use it in pmap_segtab_release() and
pmap_pte_reserve().  move check before MD callback(), incase it
wants to change ptes for some reason (they're passed in, but
this callback is currently always NULL.)

clean up some history logs to reduce the number of lines required.

(mrg)

send close_notify for the ssl connection before closing the TCP connection
Thanks to Dr. Thomas Orgis for reporting the issue.

(spz)

update the latest note to talk about all 3 potential ways that
failure can occur now (1 still upcoming.)

(mrg)

make GCC 8 consumers to use gcc.old.

(mrg)

make(1): add empty lines to separate the test cases in cond-short

(rillig)

For now, probably forever, prohibit unquoted $ and ` in the names of
functions being defined (they can still be included if quoted).

If we parsed the way POSIX specifies (leaving the exact input text of
$ and ` expansions unaltered, until required to be expanded) this would
not be needed, as the name of a function being defined does not underbo
parameter, command, or arith expansions, so xxx$3() { : ; } would just
work.  But for many reasons we don't do that (and are unlikely to ever,
though maintaing both forms might be an option someday) - which led to
very obscure behaviour (if sh were compiled in DEBUG mode, even an abort())
and certainly nothing useful.  So just prohibit these uses for now.
(A portable function name must be a "name" so this makes no difference
at all to posix compat applications/scripts).

A doc update is pending (the updated sh.1 also contains updates in other
areas not yet appropriate to commit).

(kre)

Ticket #1598

(martin)

Pull up following revision(s) (requested by chs in ticket #1598):

sys/uvm/uvm_amap.c: revision 1.123 (via patch)

fix amap_extend() to handle amaps where we previously failed to allocate
the ppref memory.

(martin)

Ticket #1057

(martin)

Pull up following revision(s) (requested by chs in ticket #1057):

sys/uvm/uvm_amap.c: revision 1.123 (via patch)

fix amap_extend() to handle amaps where we previously failed to allocate
the ppref memory.

(martin)

in uao_get(), if we unlock the uobj to read a page from swap,
we must clear the cached page array because it is now stale.
also add a missing call to uvm_page_array_fini() if the I/O fails.
fixes PR 55493.

(chs)

Make sure pmap_kenter_pa(9) handles uncached mappings properly.

Fixes "cgfour(4) is mis-probed as bwtwo(4)" problem on 3/80
that has been broken since NetBSD 1.6.
Now Xorg 1.20 based Xsun 8bpp color server is confirmed working
on the cgfour(4).

Should be pulled up to netbsd-9.

XXX: all MD PMAP_NC flags should be replaced with MI PMAP_NOCACHE flag.

(tsutsui)

Make descriptor_num in ixl(4) readonly
because ixl(4) does not support reallocating related resources

pointed out by knakahara@, thanks.

(yamaguchi)

Adjust the default descriptor sizes for ixl(4)

These decreasements has no impact for throughput
while forwarding 64-1518 byte packets.

(yamaguchi)

whitespace fix

>From msaitoh@, thanks.

(yamaguchi)

merge GCC 8.4 into gcc.old.

(mrg)

Remove trailing \n from UVMHIST_LOG() format strings.

(simonb)

make(1): don't optimize when measuring the code coverage

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=96622

(rillig)

KNF.  Add some whitespace to the TLBINV_MAP macro and tlb_invalidate_op
enum.

(skrll)

make(1): fix a few inconsistencies in the manual page

(rillig)

Unwrap short line KASSERT

(skrll)

Fix inverted logic test in pmap_tlb_shootdown_process for if the victim
is onproc.

(skrll)

make(1): add test for the .newline variable

(rillig)

make(1): add test for the $$ expression

(rillig)

make(1): add test for the -n option

(rillig)

make(1): add test for the -q option

(rillig)

s/ be be / be /

(msaitoh)

use opendisk()/dehumanize_number

(christos)

make sure errno is always set when we return -1 (Anon Ymous)

(christos)

nvmm-x86-svm: improve the CPUID emulation

Limit the hypervisor range, and properly handle each basic leaf until 0xD.

(maxv)

nvmm: use relaxed atomics to read nmachines

(maxv)

nvmm: localify a variable that doesn't need to be global

(maxv)

nvmm-x86: also flush the guest TLB when CR4.{PCIDE,SMEP} changes

(maxv)

Fix ugen detach after partial attach.

While here, register null pmf handler even for partially attached
devices so they don't needlessly interfere with suspend.

Reported-by: syzbot+5a091d2e62da20b77259@syzkaller.appspotmail.com

(riastradh)

Fix small tyop in a comment.

(simonb)

Remove an extra word from a comment to make it more readable.

(simonb)

fix amap_extend() to handle amaps where we previously failed to allocate
the ppref memory.

(chs)

We don't need to call the POW and FGA bootstrap functions from the FDT
mainbus attach, the iobus attach code does this already and is called
for both iobus-only and FDT cases anyway.

After discussion with jmcneill@.

(simonb)

Operation union_readdirhook() stores the lower directory as un_uppervp.
This breaks the assumption that un_uppervp->v_mount is the upper mount.

Fix by storing the directory as un_lowervp and adapt union_readdir().

Should fix PR kern/55552: panic with union mount

(hannken)

Ticket #1597

(martin)

Pull up following revision(s) (requested by nonaka in ticket #1597):

sys/dev/ipmi.c: revision 1.5
(applied to sys/arch/x86/x86/ipmi.c)

ipmi(4): Fixed a bug that incorrect condition is notified.

When the value obtained from the sensor is below the lower limit of
the critical threshold, it is notified that the value is below the lower
limit of the warning threshold.

(martin)

Tickets #1055 and #1056

(martin)

Pull up following revision(s) (requested by nonaka in ticket #1056):

sys/dev/ipmi.c: revision 1.5

ipmi(4): Fixed a bug that incorrect condition is notified.

When the value obtained from the sensor is below the lower limit of
the critical threshold, it is notified that the value is below the lower
limit of the warning threshold.

(martin)

Pull up following revision(s) (requested by maxv in ticket #1055):

sys/dev/nvmm/nvmm.h: revision 1.13
sys/dev/nvmm/nvmm.h: revision 1.14
sys/dev/nvmm/nvmm.c: revision 1.33
sys/dev/nvmm/x86/nvmm_x86_vmx.c: revision 1.67
sys/dev/nvmm/nvmm_internal.h: revision 1.17
sys/dev/nvmm/x86/nvmm_x86_svm.c: revision 1.67
sys/dev/nvmm/x86/nvmm_x86.c: revision 1.10

Put the few x86-specific structures under #ifdef __x86_64__, for clarity.

Make it easier to understand what's going on, no functional change.

Add new field definitions.

Add new field definitions, and intercept everything, for future-proofness.

Add CTASSERT.

(martin)

Add missing cases, to prevent memory corruption.

Reported-by: syzbot+f8b8a689a3560dda27f7@syzkaller.appspotmail.com

(maxv)

Fix MULTIPROCESSOR build

(skrll)

Improve a panic message ever so slightly

(skrll)

Quoted from the top of the file:
# Note: don't delete entries from here - mark them as "obsolete" instead.

(nakayama)

Add skipped test for in-kernel roll-forward agent

(perseant)

Regen.

(simonb)

Add Moonbase Otago OneRNG Random Number Generator.

(simonb)

avoid build failure on !DBB kernels.

should fix arc, emips, ews4800mips, mipsco, newsmips and pmax builds.

(mrg)

Attach an iobus with octrnm even if using devicetree (there is no
corresponding node for this device in the DT).

(jmcneill)

IPI_SHOOTDOWN needs to be IPL_SCHED. Spotted by nick.

(jmcneill)

remove commented out MEMORY_DISK_* entries

(christos)

correct MEMORY_DISK_* option docs

(christos)

Move wayward clause to where it belongs and makes sense.

Note how blowfish-cbc chooses the CBC IV.

(riastradh)

be host friendly

(christos)

Make the AES and ChaCha NEON tests work in softfloat userland.

(`Softfloat' here refers to the ABI, which of course may be running
on a CPU with NEON.)

(riastradh)

Give a hint to what the IPI numbers are, i.e. "(A/R)" meaning active and
requested respectively

(skrll)

Fix entry for ticket #1051

(martin)

Disable __HAVE_PREEMPTION.  It is currently marked

#if defined(MULTIPROCESSOR) && defined(__HAVE_FAST_SOFTINTS)

but has no chance of working on OCTEON due to at least the spl functions

(skrll)

Tickets #1050 - #1054

(martin)

Pull up following revision(s) (requested by jdolecek in ticket #1054):

sys/dev/pci/cmdide.c: revision 1.46

remove explicit irqack() call for unexpected interrupt; this happens
normally during boot when running identify for the drive as polled
command, and fixes hard hang caused by the irqack() on a sun machine
the unexpected interrupt happens because cmdide doesn't respect
WDCTL_IDS bit and triggers interrupt also for polled commands

also reclassify the "bogus intr" as aprint_verbose() to avoid noise
on boot

discussed and tested by Martin Husemann

(martin)

Pull up following revision(s) (requested by riastradh in ticket #1053):

sys/secmodel/securelevel/secmodel_securelevel.c: revision 1.36

Accept ioctl(RNDADDDATA) estimates at securelevel 1 (but not 2).
securelevel=1 is supposed to be a reasonable default for normal
computers.  This got in the way of ever getting entropy from a seed
on a machine with no HWRNG -- e.g., from another machine, or by
making the executive decision that what has been sampled is good
enough and issuing `head -c 32 < /dev/urandom > /dev/random'.

(martin)