 @@ -1,14 +1,14 @@
-.\"	$NetBSD: membar_ops.3,v 1.5 2017/10/24 18:19:17 abhinav Exp $
+.\"	$NetBSD: membar_ops.3,v 1.6 2020/09/03 00:00:06 riastradh Exp $
 .\"
 .\" Copyright (c) 2007, 2008 The NetBSD Foundation, Inc.
 .\" All rights reserved.
 .\"
 .\" This code is derived from software contributed to The NetBSD Foundation
 .\" by Jason R. Thorpe.
 .\"
 .\" Redistribution and use in source and binary forms, with or without
 .\" modification, are permitted provided that the following conditions
 .\" are met:
 .\" 1. Redistributions of source code must retain the above copyright
 .\" notice, this list of conditions and the following disclaimer.
 .\" 2. Redistributions in binary form must reproduce the above copyright
 @@ -17,121 +17,342 @@
 .\"
 .\" THIS SOFTWARE IS PROVIDED BY THE NETBSD FOUNDATION, INC. AND CONTRIBUTORS
 .\" ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
 .\" TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
 .\" PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL THE FOUNDATION OR CONTRIBUTORS
 .\" BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
 .\" CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
 .\" SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
 .\" INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
 .\" CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
 .\" ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
 .\" POSSIBILITY OF SUCH DAMAGE.
 .\"
-.Dd November 20, 2014
+.Dd September 2, 2020
 .Dt MEMBAR_OPS 3
 .Os
 .Sh NAME
 .Nm membar_ops ,
 .Nm membar_enter ,
 .Nm membar_exit ,
 .Nm membar_producer ,
 .Nm membar_consumer ,
 .Nm membar_datadep_consumer ,
 .Nm membar_sync
-.Nd memory access barrier operations
+.Nd memory ordering barriers
 .\" .Sh LIBRARY
 .\" .Lb libc
 .Sh SYNOPSIS
 .In sys/atomic.h
 .\"
 .Ft void
 .Fn membar_enter "void"
 .Ft void
 .Fn membar_exit "void"
 .Ft void
 .Fn membar_producer "void"
 .Ft void
 .Fn membar_consumer "void"
 .Ft void
 .Fn membar_datadep_consumer "void"
 .Ft void
 .Fn membar_sync "void"
 .Sh DESCRIPTION
 The
-.Nm membar_ops
+.Nm
-family of functions provide memory access barrier operations necessary
+family of functions prevent reordering of memory operations, as needed
 for synchronization in multiprocessor execution environments that have
 relaxed load and store order.
-.Bl -tag -width "mem"
+.Pp
 In general, memory barriers must come in pairs \(em a barrier on one
 CPU, such as
 .Fn membar_exit ,
 must pair with a barrier on another CPU, such as
 .Fn membar_enter ,
 in order to synchronize anything between the two CPUs.
 Code using
 .Nm
 should generally be annotated with comments identifying how they are
 paired.
 .Pp
 .Nm
 affect only operations on regular memory, not on device
 memory; see
 .Xr bus_space 9
 and
 .Xr bus_dma 9
 for machine-independent interfaces to handling device memory and DMA
 operations for device drivers.
 .Pp
 Unlike C11,
 .Em all
 memory operations \(em that is, all loads and stores on regular
 memory \(em are affected by
 .Nm ,
 not just C11 atomic operations on
 .Vt _Atomic Ns -qualified
 objects.
 .Bl -tag -width abcd
 .It Fn membar_enter
 Any store preceding
 .Fn membar_enter
-will reach global visibility before all loads and stores following it.
+will happen before all memory operations following it.
 .Pp
 An atomic read/modify/write operation
 .Pq Xr atomic_ops 3
 followed by a
 .Fn membar_enter
 implies a
 .Em load-acquire
 operation in the language of C11.
 .Pp
 .Sy WARNING :
 A load followed by
 .Fn membar_enter
 .Em does not
 imply a
 .Em load-acquire
 operation, even though
 .Fn membar_exit
 followed by a store implies a
 .Em store-release
 operation; the symmetry of these names and asymmetry of the semantics
 is a historical mistake.
 In the
 .Nx
 kernel, you can use
 .Xr atomic_load_acquire 9
 for a
 .Em load-acquire
 operation without any atomic read/modify/write.
 .Pp
 .Fn membar_enter
 is typically used in code that implements locking primitives to ensure
-that a lock protects its data.
+that a lock protects its data, and is typically paired with
 .Fn membar_exit ;
 see below for an example.
 .It Fn membar_exit
-All loads and stores preceding
+All memory operations preceding
 .Fn membar_exit
-will reach global visibility before any store that follows it.
+will happen before any store that follows it.
 .Pp
+A
 .Fn membar_exit
 followed by a store implies a
 .Em store-release
 operation in the language of C11.
 For a regular store, rather than an atomic read/modify/write store, you
 should use
 .Xr atomic_store_release 9
 instead of
 .Fn membar_exit
 followed by the store.
 .Pp
 .Fn membar_exit
 is typically used in code that implements locking primitives to ensure
-that a lock protects its data.
+that a lock protects its data, and is typically paired with
 .Fn membar_enter .
 For example:
 .Bd -literal -offset abcdefgh
 /* thread A */
 obj->state.mumblefrotz = 42;
 KASSERT(valid(&obj->state));
 membar_exit();
 obj->lock = 0;
 /* thread B */
 if (atomic_cas_uint(&obj->lock, 0, 1) != 0)
 	return;
 membar_enter();
 KASSERT(valid(&obj->state));
 obj->state.mumblefrotz--;
 .Ed
 .Pp
 In this example,
 .Em if
 the
 .Fn atomic_cas_uint
 operation in thread B witnesses the store
 .Li "obj->lock = 0"
 from thread A,
 .Em then
 everything in thread A before the
 .Fn membar_exit
 is guaranteed to happen before everything in thread B after the
 .Fn membar_enter ,
 as if the machine had sequentially executed:
 .Bd -literal -offset abcdefgh
 obj->state.mumblefrotz = 42;	/* from thread A */
 KASSERT(valid(&obj->state));
 \&...
 KASSERT(valid(&obj->state));	/* from thread B */
 obj->state.mumblefrotz--;
 .Ed
 .Pp
 .Fn membar_exit
 followed by a store, serving as a
 .Em store-release
 operation, may also be paired with a subsequent load followed by
 .Fn membar_sync ,
 serving as the corresponding
 .Em load-acquire
 operation.
 However, you should use
 .Xr atomic_store_release 9
 and
 .Xr atomic_load_acquire 9
 instead in that situation, unless the store is an atomic
 read/modify/write which requires a separate
 .Fn membar_exit .
 .It Fn membar_producer
-All stores preceding the memory barrier will reach global visibility
+All stores preceding
-before any stores after the memory barrier reach global visibility.
+.Fn membar_producer
 will happen before any stores following it.
 .Pp
 .Fn membar_producer
 has no analogue in C11.
 .Pp
 .Fn membar_producer
 is typically used in code that produces data for read-only consumers
 which use
 .Fn membar_consumer ,
 such as
 .Sq seqlocked
 snapshots of statistics; see below for an example.
 .It Fn membar_consumer
-All loads preceding the memory barrier will complete before any loads
+All loads preceding
-after the memory barrier complete.
+.Fn membar_consumer
 will complete before any loads after it.
 .Pp
 .Fn membar_consumer
 has no analogue in C11.
 .Pp
 .Fn membar_consumer
 is typically used in code that reads data from producers which use
 .Fn membar_producer ,
 such as
 .Sq seqlocked
 snapshots of statistics.
 For example:
 .Bd -literal
 struct {
 	/* version number and in-progress bit */
 	unsigned	seq;
 	/* read-only statistics, too large for atomic load */
 	unsigned	foo;
 	int		bar;
 	uint64_t	baz;
 } stats;
 	/* producer (must be serialized, e.g. with mutex(9)) */
 	stats->seq |= 1;	/* mark update in progress */
 	membar_producer();
 	stats->foo = count_foo();
 	stats->bar = measure_bar();
 	stats->baz = enumerate_baz();
 	membar_producer();
 	stats->seq++;		/* bump version number */
 	/* consumer (in parallel w/ producer, other consumers) */
 restart:
 	while ((seq = stats->seq) & 1)	/* wait for update */
 		SPINLOCK_BACKOFF_HOOK;
 	membar_consumer();
 	foo = stats->foo;	/* read out a candidate snapshot */
 	bar = stats->bar;
 	baz = stats->baz;
 	membar_consumer();
 	if (seq != stats->seq)	/* try again if version changed */
 		goto restart;
 .Ed
 .It Fn membar_datadep_consumer
 Same as
 .Fn membar_consumer ,
 but limited to loads of addresses dependent on prior loads, or
 .Sq data-dependent
 loads:
 .Bd -literal -offset indent
 int **pp, *p, v;
 p = *pp;
 membar_datadep_consumer();
 v = *p;
 consume(v);
 .Ed
 .Pp
-Does not guarantee ordering of loads in branches, or
+.Fn membar_datadep_consumer
 is typically paired with
 .Fn membar_exit
 by code that initializes an object before publishing it.
 However, you should use
 .Xr atomic_store_release 9
 and
 .Xr atomic_load_consume 9
 instead, to avoid obscure edge cases in case the consumer is not
 read-only.
 .Pp
 .Fn membar_datadep_consumer
 does not guarantee ordering of loads in branches, or
 .Sq control-dependent
-loads -- you must use
+loads \(em you must use
 .Fn membar_consumer
 instead:
 .Bd -literal -offset indent
 int *ok, *p, v;
 if (*ok) {
 	membar_consumer();
 	v = *p;
 	consume(v);
+}
 .Ed
 .Pp
 Most CPUs do not reorder data-dependent loads (i.e., most CPUs
 guarantee that cached values are not stale in that case), so
 .Fn membar_datadep_consumer
 is a no-op on those CPUs.
 .It Fn membar_sync
-All loads and stores preceding the memory barrier will complete and
+All memory operations preceding
-reach global visibility before any loads and stores after the memory
+.Fn membar_sync
-barrier complete and reach global visibility.
+will happen before any memory operations following it.
 .Pp
 .Fn membar_sync
 is a sequential consistency acquire/release barrier, analogous to
 .Li "atomic_thread_fence(memory_order_seq_cst)"
 in C11.
 .Pp
 .Fn membar_sync
 is typically paired with
 .Fn membar_sync .
 .Pp
 A load followed by
 .Fn membar_sync ,
 serving as a
 .Em load-acquire
 operation, may also be paired with a prior
 .Fn membar_exit
 followed by a store, serving as the corresponding
 .Em store-release
 operation.
 However, you should use
 .Xr atomic_load_acquire 9
 instead of
 .No load-then- Ns Fn membar_sync
 if it is a regular load, or
 .Fn membar_enter
 instead of
 .Fn membar_sync
 if the load is in an atomic read/modify/write operation.
 .El
 .Sh SEE ALSO
-.Xr atomic_ops 3
+.Xr atomic_ops 3 ,
 .Xr atomic_loadstore 9
 .Sh HISTORY
 The
 .Nm membar_ops
 functions first appeared in
 .Nx 5.0 .
 The data-dependent load barrier,
 .Fn membar_datadep_consumer ,
 first appeared in
 .Nx 7.0 .