Keeping the namecache lock next to the vnode lock has a repeatable
detrimental effect.  So shuffle some stuff around in vnode_impl to save
a bit of space, and give the namecache locks their own cache line.

(ad)

Resurrect the counter stuff from HEAD, which tries to deal with 32-bit
counter rollover.  Drive it by sysctl and/or callout every 5 minutes,
since there's no garbage collection kthread any more.

(ad)

- rb_tree_insert_node() refuses to insert duplicate nodes.  It always
  returns a pointer to the current node in the tree for the given key.
  Therefore the call to rb_tree_lookup_node() in cache_enter() is not
  needed.

- Adjust a comment and move some code for clarity.

(ad)

- Fold the "doingcache" conditional into the max length conditional.
- Mark the tunables __read_mostly.

(ad)

Move nc_key up, so cache_lookup_entry() only needs to touch one cache line
at each step.

(ad)

- Include sys/types.h for vax.
- Use macro versions of kpreempt_disable()/enable().
- Use atomic_load_relaxed().
- Make cache_key() inline.

(ad)

__read_mostly -> const in one place

(ad)

Regen.

(ad)

Fix a comment.

(ad)

lookup_fastforward(): bail out on mount -o union

(ad)

Back out experimental change - not ready for LK_SHARED on VOP_OPEN() just yet.

(ad)

Sync with head.

(ad)

Sync with head.

(ad)

Sync with head.

(ad)

Sync with head.

(ad)

Sync with head.

(ad)

Sync with head.

(ad)

Turns out there's no point adjusting v_holdcnt with atomics.

(ad)

Comments.

(ad)

Sync with head.

(ad)

Remove unintentional differences to base.

(ad)

Add a missing membar to previous.

(ad)

Make cwdinfo use mostly lockless, and largely hide the details in vfs_cwd.c.

(ad)

Regen.

(ad)

- Put all the namecache stuff back into vnode_impl_t.
- Tidy vfs_cache.c up, finish the comments.
- Finalise how ID information is entered to the cache.
- Handle very small/old systems.

(ad)

Add a comment.

(ad)

vnodes:

- Have own v_usecount again, don't share the uvm_object's refcount.
- Cluster the members of vnode_t and vnode_impl_t in a cache-concious way.
- Go back to having vi_lock directly in vnode_impl_t.
- Go back to having v_usecount adjusted with atomics.
- Start adjusting v_holdcnt with atomics, too.
- Put all the namecache stuff back into vnode_impl_t.

(ad)

Add an IMNT_NCLOOKUP flag to indicate the FS is OK with lookup directly
from cache.

(ad)

Improve an assertion.

(ad)

vcache_reclaim(): purge namecache immediately after setting vnode to
VS_RECLAIMED.

(ad)

Update comments.

(ad)

- Change style of new code slightly to match rest of file.
- NFS lookup needs to cross mountpoint too.
- Update comments.

(ad)

Back out previous.

(ad)

Fast-forward through the namecache was stopping one component too soon when
there was an obstacle, e.g. a mountpoint.  The obstacle should be returned
not the parent directory.

(ad)

Copy the IMNT_SHRLOOKUP flag from lowerrootvp's mount.

(ad)

Make sure LK_UPGRADE always comes with LK_NOWAIT; dropping the lock in here
is unclean and I wonder if it could screw over fstrans.

(ad)

Back out previous.

(ad)

Regen.

(ad)

Use LOCKLEAF in the few cases it's useful for ffs/tmpfs/nullfs.  Others need
to be checked.

(ad)

Set IMNT_SHRLOOKUP and use it for the in-cache case.  Need to check what
more can be done with tmpfs though, it can probably do the whole lookup.

(ad)

- Add a LOCKSHARED flag to namei (matching FreeBSD) indicating that we want
  the leaf locked with LK_SHARED.

- Add an IMNT_SHRLOOKUP flag to struct mount indicating that the file
  system can do VOP_LOOKUP() with an shared lock.  If it encounters
  something tricky, VOP_LOOKUP() is free to return ENOLCK and namei() will
  retry the lookup with an exclusive lock.  If the file system has this flag
  set, namei() will try with shared locks for all of the "read only"
  lookups, i.e. nameiop=LOOKUP or !ISLASTCN.

- vfs_getcwd: only take vnode locks when really needed, take shared locks if
  possible, and where the namecache has identify info for the directories,
  do it all in the namecache.

- vfs_lookup: when crossing mountpoints take only a shared lock on the
  covered vnode; don't need anything else.

(ad)

empty these; remove later.

(ad)

Adaptive rwlocks proposed on tech-kern and working well on this branch
with vnode locks.

(ad)

Sync with head.

(ad)

Fix namecache_print().

(ad)

Regen.

(ad)

Regen.

(ad)

Allow VOP_LOCK(LK_NONE).

(ad)

- Adjust comments & assertions.
- Ditch memcmp().

(ad)

vfs_lookup:

- Do the easy component name lookups directly in the namecache without
  taking vnode locks nor vnode references (between the start and the leaf /
  parent), which seems to largely solve the lock contention problem with
  namei().  It needs support from the file system, which has to tell the
  name cache about directory permissions (only ffs and tmpfs tried so far),
  and I'm not sure how or if it can work with layered file systems yet.
  Work in progress.

vfs_cache:

- Make the rbtree operations more efficient: inline the lookup, and key on a
  64-bit hash value (32 bits plus 16 bits length) rather than names.

- Take namecache stuff out of vnode_impl, and take the rwlocks, and put them
  all together an an nchnode struct which is mapped 1:1: with vnodes.  Saves
  memory and nicer cache profile.

- Add a routine to help vfs_lookup do its easy component name lookups.

- Report some more stats.

- Tidy up the file a bit.

(ad)

vrelel: don't change the vnode state to VS_BLOCKED for VOP_INACTIVE(), it's
not needed (at least not for the usual case).  Will revist before merge.

(ad)

- Get shared locks on the directories, except for VOP_GETATTR() which
  unfortunately still needs LK_EXCLUSIVE due to itimes handling in UFS.

- vnode locks are not needed to traverse the namecache.

(ad)

LK_SHARED for VOP_ACCESS().

(ad)

Sync with head.

(ad)

Push the vnode locking in namei() about as far back as it will go.

(ad)

Regen.

(ad)

namecache:

This is working better than expected.  It seems to cut system time for
build.sh by ~10% on my test machine and joerg@ is seeing better results with
pbulk.  Improve it a bit more without changing the basic idea:

- Split cache_list_lock into a per-vnode rwlock for reverse lookup, and a
  lightly contended global lock on LRU state (cache_lru_lock),

- For LRU replacement, imitate the VM system's page replacement algorithm.
  This eliminates the writebacks to struct namecache (to track time of last
  hit).

- Dynamically allocate the per-directory lock, preparing the way for having
  a "struct nchdir" or similar which could contain stuff like different
  structures for lookup, cached info to do the equivalent of VOP_ACCESS() in
  cache, and so on.

(ad)

Make the per-directory lock a rwlock.

(ad)

Fix a comment.

(ad)

cache_enter(): check namelen before touching name.

(ad)

Regen.

(ad)

Redo the namecache to focus on per-directory data structures, removing the
huge hashtable and nasty locking scheme.

Initially this uses rbtrees (because that's what's there).  The intent is
experiment with other data structures.

(ad)