libarchive: fail if name_max is 0

Add error handling to the USE_READDIR_R code paths that set name_max
from struct statfs or statvfs; if the determined name_max == 0
then return an error.

Avoids a crash in tree_dir_next_posix() when the calculation of
dirent_size from name_max is too small for the memory allocated
for struct dirent.

Submitted to upstream in pull request
https://github.com/libarchive/libarchive/pull/1903

Should fix PR bin/56080

(lukem)