lint: add type information to unportable bit-field type

Seeing the message "unportable bit-field type 'int'" may sound strange
at first, but that's a strict interpretation of the wording in C99
6.7.2.1p4, which requires that the bit-field type is "'_Bool', 'unsigned
int' or 'signed int', or some other implementation-defined type".

The rationale for C99 6.7.2.1 explicitly lists plain 'int' among the
allowed types for bit-fields, regardless of any additional
implementation-defined types.  This means that lint had interpreted this
paragraph wrong, and it should be fixed to allow plain int as well.

(rillig)

lint: add quotes to message 309 about constant conversion

(rillig)

tests/lint: add test for warning about zero-bits in '&'

(rillig)

memmem: remove unreachable return statement

(rillig)

tests/lint: add expected output for testing '&' in switch statement

(rillig)

xterm: fix Clang build on i386

xsrc/external/mit/xterm/dist/misc.c:3250:47: error: format specifies
type 'unsigned long' but the argument has type 'size_t' (aka 'unsigned
int') [-Werror,-Wformat]

(rillig)

wpa: fix Clang build

src/external/bsd/wpa/bin/hostapd/../../dist/src/common/dpp.c:5377:7:
error: format specifies type 'unsigned long' but the argument has type
'os_time_t' (aka 'long long') [-Werror,-Wformat]

(rillig)

lint: warn about unreachable case labels for '&&'

See octeon_gmxreg.h 1.2 from 2020-06-18 for an example, where
RXN_RX_INBND_SPEED was cleaned up without adjusting the corresponding
code in octeon_gmx.c.

(rillig)

tests/lint: test bitwise mismatch in switch statement

(rillig)

tests/make: show that in ':S', '.' and '*' are ordinary characters

(rillig)

tests/lint: make test for message 118 platform-independent

(rillig)

libXaw: fix Clang build

--- Pixmap.pico ---
error: unknown warning option '-Wno-discarded-qualifiers'; did you mean
'-Wno-ignored-qualifiers'? [-Werror,-Wunknown-warning-option]

(rillig)

compiler_rt: build .a.syms files as part of dependall

The previous target 'all' is a top-level target intended to be specified
by the user.  During the regular build, 'dependall' is used instead of
'all'.  The target 'realall' is common to those two.

(rillig)

bind: remove unnecessary CONSTCOND

Lint does not need this anymore, since 2021-01-31.

(rillig)

bind: fix Clang build

I had already fixed this on 2021-03-27, but the fix was accidentally
reverted on 2021-04-29 when updating to bind 9.16.15.

(rillig)

lint: fix assertion failure when promoting a bit-field larger than int

(rillig)

tests/lint: demonstrate assertion failure "len == size_in_bits(INT)"

Seen on sparc64 in hdtoa.c:341 since sparc64 is one of the platforms
that has 128-bit long double and defines struct ieee_ext.ext_frach:48
based on uint64_t, which is a GCC extension.  Plain C99 only allows
_Bool, signed int and unsigned int as base type for bit-fields.

(rillig)

lint: undefine macros for defining keywords after use

No functional change.

(rillig)

if_wm: do not return a void value from a void function

This is a GCC extension.

(rillig)

lint: remove redundancy when defining classic keywords

No functional change.

(rillig)

lint: reduce redundancy when defining GCC attributes

This macro would have prevented the unintended inconsistency in the
attribute 'pcs' (for ARM).

No functional change.  The generated code is the same, except for the
line numbers in lint_assert.

(rillig)

lint: allow variables to be named 'pcs'

(rillig)

tests/lint: demonstrate bug in the lexer for GCC attributes

(rillig)

lint: fix parsing of __typeof__ after statement in ({ ... })

Since C99, declarations and statements can be freely mixed, and GCC
supported this even before 1999.

(rillig)

tests/lint: remove outdated comment

The test d_gcc_extension is already activated in line 121.

(rillig)

lint: fix compiler warning about signedness mismatch

No functional change.

(rillig)

lint: allow large integer types for bit-fields in GCC mode

These types are explicitly allowed by GCC.

I'm not sure which of the flags -g and -p should be stronger.  That is,
if both -g and -p are given, should 'unsigned char' be allowed as a
bit-field type since -g would allow it, or should it be warned about
since -p warns about it?  For now, continue to warn about these.

(rillig)

lint: extract check_bit_field_type from declare_bit_field

No functional change.

(rillig)

tests/lint: fix test for GCC bit-field types

The whole purpose of this test is to try the message about invalid
bit-field types in GCC mode.  Therefore, use the default lint1-flags
that include -g.

(rillig)

tests/lint: add test for bit-field types in GCC mode

(rillig)

lint: on request, keep the preprocessor output

Previously, the simplest way of getting the preprocessed translation
unit was to run lint with the additional flag -V, which outputs the
command line of the C preprocessor, among other things.  That command
line does not include the proper quoting though, so it cannot be used
verbatim as a shell command if the command line contains spaces such as
in -Du64="unsigned long long".

In the common situation where lint is run via a Makefile, the option -V
had to be added in the Makefile itself since there is no make variable
for additional user-settable lint flags.  This is not straight-forward
enough.

Adding another command line option for this purpose would reduce the
remaining namespace for options.  Most of the 52 letters are already
used up.

To make this situation as simple as possible, preserve the output of the
C preprocessor depending on an environment variable.

(rillig)

lint: sort GCC attributes alphabetically in the grammar

No functional change.

(rillig)

tests/lint: demonstrate missing support for __packed __aligned

(rillig)

sys/conf/lint.mk: fix order of command line options

The option -o must come before the first filename.

Now it is possible to lint kern_exec.c at least.  The next failure is in
netbsd32_machdep.c:

netbsd32_machdep.c(395): error: illegal bit-field size: 255 [36]

That line in netbsd32_process_read_fpregs reads:

__CTASSERT(sizeof(*regs) == sizeof(struct save87));

This is probably a bug in lint.  The struct save87 contains 3 uint16_t,
followed by a union containing a uint64_t, followed by a packed struct
with size 10.  The combination of packed and padding is suspicious.

(rillig)

sys/conf/lint.mk: allow GCC extensions in the kernel

The syntax errors due to the __asm statements are now gone.  The file
kern_exec.c only produces a few warnings now.  But after running that
file through lint1, the main lint complains about wrong usage.  This is
due to this call:

${LINT} ... -i $< -o $@

The main lint has never supported -o after the first filename, so it
complains.  It would have been helpful if lint had given any hint as to
which option was invalid.  Anyway, the next step is to reorder the
arguments.  As it is now, the code can never have worked.

(rillig)

sys/conf/lint.mk: a small step for linting the kernel

Due to the missing path, the following commands had failed:

$ cd src/sys/arch/amd64/compile/GENERIC
$ nbmake-amd64-lint kern_exec.ln
nbmake: don't know how to make kern/kern_exec.c. Stop

After fixing the path, "make kern_exec.ln" fails with:

sys/kern/kern_exec.c(65): error: syntax error '"' [249]

The affected line contains:

__KERNEL_RCSID(0, "...");

The macro __KERNEL_RCSID expands to __asm("some strings").  Since
KERNLINTFLAGS is missing the -g, lint does not recognize __asm as
keyword and tries to parse it as an identifier instead, expecting a
variable or function declaration.

(rillig)

countmail: remove outdated comments

Using the shell builtin 'read' command was necessarily slow because it
is specified to read a single byte at a time.

(rillig)

games: remove trailing whitespace in *.c and *.h

(rillig)

fish: remove trailing whitespace

(rillig)

fish: use arc4random_uniform for drawing random numbers

Thanks nia@ for the hint.

(rillig)

fish: remove modulo bias from random number generation

It probably doesn't matter in practice, but omitting this piece of code
always looks like an oversight.

(rillig)

cgram: rename local functions

The word 'cleanup' should have been named clean_up all along, but 'done'
is even shorter.

No functional change.

(rillig)

caesar: WARNS=6, strict bool mode

The rotation is validated to be nonnegative, therefore use unsigned int
for it.

(rillig)

ktruss: fix typo in comment

(rillig)

tests/lint: add test for __attribute__((nonnull()))

(rillig)

lint: support all documented variants of __attribute__((nonnull))

(rillig)

tests/lint: add very basic tests for GCC __attribute__

(rillig)

compiler_rt: generate symbol table early

These libraries have the special feature that they install both the .a
library as well as the .a.syms that is generated from them.  To avoid
the transformation rule .a.a.syms to apply to the files in ${DESTDIR},
that rule has to be disabled during the installation phase.

If the transformation rule were active during installation, the .a.syms
file in ${DESTDIR} would have more than one possible source file, which
eventually leads to an error during installation since in the command
"install a b c", the last argument must be a directory, not a file.

The installation phase should be as minimal as possible anyway,
therefore move the generation of the .syms files to the build phase.

This hopefully fixes the build for HAVE_LLVM=yes, which has been broken
for several weeks now, at least for the official builds.

(rillig)

cgram: in hinting mode, show all correct characters in bold face

For almost solved cryptograms, this makes the punctuation stand out
less, helping to focus on the missing letters.

(rillig)

make: use consistent variable names, types and braces in DebugFailedJob

No functional change.

(rillig)

make: remove stray space in -de output in jobs mode

In compat mode, having a space in this place makes sense to align the
target name with the command.  In jobs mode, since each command is
listed in a separate line, there is no need for the double space.

(rillig)

tests/make: test the combination of -de with -j1

The test cases are the same as in opt-debug-errors.mk.  The output
differs in several details though.

Even though the option '-k' is given (which is the default for any tests
that don't override it in unit-tests/Makefile), there is no message
"(continuing)" anywhere.

The failed target is printed twice.  Once before the failed commands,
once after.  This redundancy is not necessary and may be removed in a
follow-up commit.

The printed commands are in their unexpanded form, which may or may not
be more helpful than the expanded and space-normalized form of compat
mode.  Either way, this is an unnecessary inconsistency between compat
mode and jobs mode.

In jobs mode, the message "make: stopped in $dir" is printed for each
failure, which is helpful since each of the jobs may have started in a
separate directory.

(rillig)

tests/make: add tests for ${...} without operator in conditions

These tests are already covered in cond-cmp-unary.mk, but my first guess
was to search for these tests in cond-token-var.mk, so keep them in both
tests.

(rillig)

cgram: use ASCII-only implementation of <ctype.h> functions

The function 'encode' already assumes that all letter characters are
contiguous and that there are only 26 letters of each case.  At the
moment, cgram cannot handle UTF-8 anyway since it reads the input
byte-wise, assuming that each byte is exacty one character.

(rillig)

cgram: provide usage on incorrect invocation

(rillig)

cgram: conform to lint's strict bool mode, KNF

(rillig)

lint: merge duplicate code in grammar for GCC compound expressions

This increases the number of shift/reduce conflicts, once again.  I
don't know why since the previous two grammar rules only differed in a
single detail.  One of them had a declaration_list while the other
didn't.  This difference is exactly what declaration_list_opt is for.

Anyway, the tests for both cases (with and without declarations) work
exactly as before.

Merging this duplicate code will probably make it easier to fix the
current limitation in lint regarding GCC compound expressions that a
statement followed by a declaration generates a syntax error.

Fixing this is not trivial, my first attempt failed either with
segmentation faults or with sym->s_type becoming NOSPEC during the
expression.

(rillig)

tests/lint: test GCC compound expression without declaration

(rillig)

tests/lint: add test for typeof after statement

Found by christos@.

(rillig)

tests/lint: fix the hack for emit.ln to also work if MKUPDATE != "no"

(rillig)

tests/lint: fix inconsistent operator for emit.ln

The build failed with:

nbmake[8]: "/.../src/share/mk/bsd.files.mk" line 47:
    Inconsistent operator for /.../emit.ln

(rillig)

clang: fix build for installing libclang_rt.ubsan_minimal-x86_64.a

The build failed with:

--- install-ubsan_minimal-m64 ---
x86_64--netbsd-install: the last argument
    (/.../libclang_rt.ubsan_minimal-x86_64.a.syms)
    must name an existing directory

The cause of this is that the suffix transformation rule for .a.a.sym
was active even during 'make install'.

At that point, the make variable .ALLSRC for the target
${DESTDIR}/.../libclang_rt.ubsan_minimal-x86_64.a.sym included not only
the corresponding file from the source directory, but also the potential
source file from the transformation rule, in this case
libclang_rt.ubsan_minimal-x86_64.a.

This led to several file parameters for the command 'install', which is
only allowed if the last parameter names an existing directory.

(rillig)

tests/lint1: fix build error "must name an existing directory"

The build failed with this error message:

x86_64--netbsd-install: the last argument (/.../emit.ln) must
name an existing directory

The cause for this message was that ${DESTDIR}/.../emit.ln has
${DESTDIR}/.../emit.c as implicit target, which is kind of correct but
unintended in this case.  Because of this, the command 'install' was run
like this:

install ${NETBSDSRCDIR}/.../emit.ln ${DESTDIR}/.../emit.c \
    ${DESTDIR}/.../emit.ln

(rillig)

tmux: move __printflike to the declaration in the header

Noted by Akihiko HAYASHI.

(rillig)

make: do not complain when skipping the condition 'no >= 10'

Seen in external/bsd/tmux when building with Clang.  See
varmod-ifelse.mk for the detailed story.

(rillig)

tests/make: add another example for parsing of the modifier ':?'

(rillig)

tests/make: add detailed explanation for error message in conditional

(rillig)

make: use straight quotes for error 'Bad conditional expression'

This diagnostic was supposed to be an error, see ApplyModifier_IfElse.

When such an error occurs while the makefiles are read, make stops with
an error, as can be expected.  But when such an error occurs later,
after all makefiles have been read, the message is printed but make does
not stop.

In lint mode (-dL), make stops in such a case.  I didn't dare to make
this the default behavior, out of fear of breaking existing build
infrastructure, not only in NetBSD or pkgsrc, but also FreeBSD and other
operating systems that use the bmake distribution, generated from the
same source code.

(rillig)

tests/make: demonstrate unexpected behavior of the modifier ':?'

(rillig)

tmux: fix -Wformat-nonliteral for Clang

dist/control.c:394:17: error: format string is not a string literal
    [-Werror,-Wformat-nonliteral]
        xvasprintf(&s, fmt, ap);
                      ^~~

dist/status.c:436:33: error: format string is not a string literal
    [-Werror,-Wformat-nonliteral]
        xvasprintf(&c->message_string, fmt, ap);
                                      ^~~

(rillig)

make: avoid double slash in name of temporary directory

If the environment variable TMPDIR is not set, make uses a default path
that includes a trailing '/'.

For extra correctness it always appended an extra '/', leading to paths
of the form '/tmp//makeXXXXXX'.  This looked suspicious, as if there had
been a forgotten empty part between the two '/'.  Avoid this ambiguity
by replacing '//' with '/'.

(rillig)

lint: add debug logging for reachability

(rillig)

tests/lint: record all files from FILES in distrib/sets/lists/tests/mi

(rillig)

tests/lint: add emit.ln to the release files

(rillig)

lint: remove WARNS=3, falling back to the default WARNS=5

It's strange that GCC does not warn about the nonliteral format strings
in lint1/err.c, lint2/msg.c and lint2/read.c, despite -Wformat=2, but
Clang does.

(rillig)

lint: fix strict bool mode errors in cgram.y

The code generated by yacc already adheres to strict bool mode, in
default mode as well as in debug mode.

Running lint on the generated cgram.c as well avoids most of the
"declared but not used" warnings from lint2.

The code generated by lex does not adhere to strict bool mode though.
Suppressing the errors from strict bool mode works, but then lint1 runs
into an assertion failure:

assertion "tn != NULL || nerr != 0" failed
    in expr at tree.c:3610 near scan.c:822

This leaves several warnings about "declared but not used" for the
functions from lex.c.

(rillig)

lint: reduce duplicate code for parsing .ln files

(rillig)

lint: fix error message in lint2 for missing record type in .ln file

(rillig)

lint: clean up option parsing

(rillig)

lint: test emitting of symbol information in the .ln files

Even though the new test is quite large, it didn't find any bugs in the
code.  The only thing I'm unsure about is why static functions are
exported as well, since they are supposed to be local to the translation
unit.

(rillig)

lint: rename parameter to expr

That parameter used to be used for a single purpose, later it got used
for checking the reachability as well, which made the name misleading.

(rillig)

lint: remove redundant CONSTCOND

In do-while-0 loops, these are no longer needed since tree.c 1.202 from
2021-01-31.

(rillig)

lint: pass pos_t via const pointer

Thanks for the suggestion, christos@.

(rillig)

lint: rename parameter to diagnostic functions and macros

The first parameter is not just an arbitrary number, it's a message ID.

(rillig)

lint: align the member access macros for tnode_t

(rillig)

lint: flip parameters in verror_at, vwarning_at, vmessage_at

The parameters are in the same order now as in the corresponding macros.

(rillig)

lint: extend documentation about handling initializations

(rillig)

lint: remove redundant test for tflag in initialization handling

(rillig)

lint: migrate global_clean_up_decl to warning_at

(rillig)

lint: preserve curr_pos when checking whether a symbol is used

At the end of a function definition (funcend), curr_pos was not
restored, probably because the very next action is to read the next
token, which will then set curr_pos.

Again, this may affect the location reported in assertion failures,
everything else is unchanged.

(rillig)

lint: do not modify curr_pos in check_global_symbols

No functional change.

(rillig)

lint: in check_global_variable_size, restore curr_pos

This affects only calls to lint_assert, all other code stays
functionally the same.

(rillig)

lint: migrate check_tag_usage to warning_at

This one is already a little trickier since it does not restore the
original curr_pos.  This function is only called in a few places, and
these either restore curr_pos themselves or only call functions
afterwards that immediately overwrite curr_pos.

Except for lint_assert, which may now report a different location.

The call to check_global_variable that is related to check_tag_usage was
the function that started this whole refactoring since its lint_assert
reported a completely wrong location when using a compound literal in an
initializer expression at block level 0.

(rillig)

lint: migrate print_previous_declaration to message_at

(rillig)

lint: migrate the getopt check to warning_at

(rillig)

lint: add error_at, warning_at, message_at

Right now there are several places in the code that use the global
variable curr_pos for passing a parameter to the diagnostic functions.
That's not what global variables are for.

Make it easy for the code to migrate to the parameter-passing style.

No functional change.

(rillig)

lint: do not allow the diagnostics to be modified

(rillig)

tests/lint: remove work-in-progress code

The only purpose of the additional initializers was to generate an error
message, to make the .exp file non-empty.  This is no longer needed.

(rillig)

lint: fix storage class of compound literal in initializer

A compound literal that occurs at block_level 0 does not have storage
class AUTO.  Instead, its storage class is either EXTERN or STATIC.

While removing the temporary objects from the symbol table had prevented
the assertion, it did not properly fix the underlying problem, which was
that since C99 the initializers can contain references to unnamed
objects that are created on-the-fly.  For C90 it was correct to always
use AUTO as the storage class of a temporary symbol.

(rillig)

lint: document wrong location information in diagnostics

(rillig)

lint: fix assertion failure for temporary objects in initialization

(rillig)

tests/lint: fix analysis from previous commit, add another example

This is not a GCC feature, it's required by C99 already.

(rillig)

tests/lint: demonstrate assertion failure in initialization

(rillig)

sys/arch/amd64: remove trailing whitespace

(rillig)

lint: do not warn about alignment when casting from incomplete struct

This removes a bunch of technically correct but practically useless
warnings from the regular NetBSD build.

(rillig)

tests/lint: add test for pointer cast from incomplete struct

(rillig)

tests/make: remove ATF descr

The description of the tests is not going to be added anytime soon, as
that would either create redundancy or additional run-time cost.
Parsing and evaluating unit-tests/Makefile is already O(n^2) with regard
to the number of tests, and there are quite a few tests.  Therefore don't
add any overhead for now.

(rillig)

tests/make: indent consistently

(rillig)

tests/make: simplify selection of the files to be copied

No functional change.

(rillig)

lint1: remove redundant dependency

This is already handled by "make depend".

(rillig)

make: use distinct wording for writing to the shell commands file

The word 'write' now means to write to the file that holds the shell
commands to be run later.

The word 'print' is now used exclusively for handling the output of the
child commands and printing them to make's stdout.

No functional change.

(rillig)

make: align name of jobs_table_dump to the other functions

(rillig)

make: remove type name for the abort status in job handling

(rillig)

tests/make: demonstrate handling of null bytes

(rillig)

make: rename PrintOutput to PrintFilteredOutput to avoid confusion

In the default configuration, the function PrintOutput did nothing.
Only if the shell has defined an output filter, something happens at
all.

(rillig)

make: document two previously unknown bugs in job mode

(rillig)

lint: remove stray semicolon from grammar, make empty rule visible

No functional change.

(rillig)

tests/lint: adjust test for C11 to the change in grammar and options

(rillig)

lint: un-abbreviate code for passing options to cpp, lint1 and lint2

No functional change.

(rillig)

lint: add option to accept C11 features

The list of available letters for the command line options gets shorter
and shorter.  Most of the interesting letters are already used for some
warning categories.  Curiously, -A, -W and -E were all still available.

The option -A nicely matches the intention of the option, which is to
allow a certain set of language features.  To keep the option available
for further extensions, define -Ac11 as the currently only valid option
of that kind.  This allows straight-forward extension for C17 and future
language standards, as well as independent feature-sets.  The options -W
and -E may someday complement the -A option, using the allow/warn/error
categories.

(rillig)

lint: arrange options in the usage messages

* lowercase the word 'usage'
* group the preprocessor flags on a line of their own
* add missing closing ']' for '-X'
* properly indent the lines

(rillig)

lint: remove redundant condition in update_location

(rillig)

lint: add support for C11-isms such as int[static 3]

(rillig)

lint: add test for newly added message about static array size

(rillig)

make: let the compiler decide whether to inline string functions

On x86_64, this reduces the binary size by 2 kB.

(rillig)

make: turn run-time string concatenation into compile-time

No functional change.

(rillig)

make: remove unnecessary modifier ':U' for certain fixed expressions

No functional change, since the expression is evaluated using
VARE_WANTRES, not using VARE_UNDEFERR.

(rillig)

make: reduce memory allocations in the modifiers ':D' and ':U'

(rillig)

make: rename members of ModifyWord_LoopArgs

No functional change.

(rillig)

make: clean up pattern flags for the modifiers ':S' and ':C'

No special handling is necessary for C90 since none of the struct
members is a const_member.

The prefix 'Var' is not necessary since this type does not apply to a
variable but only to a modifier.

No functional change.

(rillig)

make: spell SysV in mixed case

No functional change.

(rillig)

make: reduce memory allocation and strlen calls in modifier ':from=to'

Previously, SysVMatch was quite verbose and felt like hand-optimized
assembler code, which made it difficult to discover the underlying idea
of the code.

All this code was replaced with two simple calls to Substring_HasPrefix
and Substring_HasSuffix.  Now that the operands of that modifier are no
longer passed as C strings, there is no need to collect all information
in a single scan through the word and the pattern.

It was not necessary to call Var_Subst unconditionally.  Calling it only
when the string contains a '$' saves another memory allocation and two
string copies (because of the Buf_DoneDataCompact).

No functional change.

(rillig)

tests/make: fix test for modifier ':from=to'

I had forgotten to remove the single quotes, to produce a really empty
word.  Plus I had forgotten the modifier ':Q' for the result, which
meant that the single quotes did not show up in the output.

(rillig)

tests/make: add table test for the modifier ':from=to'

Just to prevent any accidental change in the upcoming refactoring to
reduce memory allocation in ApplyModifier_SysV.

(rillig)

make: in the ':Q' modifier, only allocate memory if necessary

(rillig)

make: improve performance for LazyBuf

The previous O(n^2) time complexity for parsing a long string with many
variable expressions was not meant to last for long.  I had hoped to fix
it within a few minutes, but that will take more time.

For now, make LazyBuf simpler by using a traditional C string for the
expected part instead of a Substring.  This avoids a strlen call per
Var_Parse.

No functional change, only performance.

(rillig)

make: remove redundant parameter from ParseVarnameLong

No functional change.

(rillig)

make: migrate ParseModifierPart to use Substring

This will reduce memory allocation for modifier parts without the escape
characters '$' or '\'.

No functional change.

(rillig)

make: avoid unnecessary calls to strlen when evaluating modifiers

No functional change.

(rillig)

make: migrate ModifyWord functions to use Substring

This benefits the modifiers ':T' and ':H' since these scan the word from
the end.  The SysV modifier '.c=.o' does not benefit yet, this will be
done in a follow-up commit.

Currently ModifyWords calls strlen for each single word, which degrades
performance.  This will be cleaned up in a follow-up commit as well.

No functional change.

(rillig)

make: migrate handling of the modifier ':S,from,to,' to Substring

Right now this does not gain any performance, it only makes the code in
ModifyWord_Subst a little simpler since it only uses
Buf_AddBytesBetween, not a mixture with Buf_AddBytes.

When the word passed to the ModifyWord functions is converted to
Substring as well, the call to strlen will become redundant.

No functional change.

(rillig)

make: clean up remaining references to VarEvalFlags

VarEvalFlags has been replaced with VarEvalMode.  There were some
comments and tests that still referred to the old names.

No functional change.

(rillig)

make: avoid allocating memory for simple variable names

The main change is in ParseVarname, where a Buffer is replaced with the
newly introduced LazyBuf.  LazyBuf is inspired by
https://golang.org/src/path/path.go.

In CanonicalVarname, the pre-comparison of the first letter of the
variable name is no longer necessary.  GCC 9 optimizes a fixed-length
memcmp so well that the code can finally be written to target human
readers, leaving the optimization to the compiler.

(rillig)

make: add types Substring and LazyBuf

These will be used for making the string handling more efficient,
avoiding allocations, especially when evaluating variable expressions.

Since the string handling has grown quite a bit in the last months,
extract it into its own header file.

No functional change.

(rillig)

make: clean up Makefile

The dependencies on the header files are handled by 'make depend'.  They
do not need to be spelled out.

(rillig)

lint: fix build (missing include directory)

(rillig)

make: fix out-of-bounds read in Var_Parse_FastLane (since 30 minutes)

This bug made the test varmod-edge.mk fail sometimes with varying error
messages, as can be expected for an out-of-bounds read.

(rillig)

make: reword debug log message for empty variable name

The function names did not match anymore, after one of the many
refactorings in the last few months.

(rillig)

make: reduce debug logging and memory allocation for ${:U...}

Expressions of the form ${:U...} are often generated by .for loops.
Since these expressions are not generated knowingly by the make user, do
not fill the debug log with them since that would interrupt the normal
reading flow of the -dv log for nested expressions.

(rillig)

tests/libevent: fix Clang build

No idea why GCC did not complain about this as well.

(rillig)

libevent: remove remaining CONSTCOND comments

Since lint1/tree.c 1.202 from 2021-01-31, lint no longer needs the
/*CONSTCOND*/ for do-while-0 "loops".

The comments from this commit were in do-while-0 loops that do not
follow the canonical form, therefore the previous commit did not catch
them.  Some of them have a trailing semicolon (which defies the whole
purpose of the do-while-0 loop), some are missing the space between
'while (0)', some start in column 1 instead of 9.

This reduces the local modifications, compared to the upstream code.

No functional change.

(rillig)

libevent: remove unnecessary CONSTCOND comments

Since lint1/tree.c 1.202 from 2021-01-31, lint no longer needs the
/*CONSTCOND*/ for do-while-0 "loops".

This reduces the local modifications.

No functional change.

(rillig)

lint: use distinct struct tags for type_t in lint1 and lint2

Having two similar but still different definitions of 'struct type' is
unnecessarily confusing.  Exchange this confusion for 8 lines of
straight-forward preprocessing code.

(rillig)

lint: prepare renaming of 'struct type'

It's confusing to have the same struct tag in both lint1 and lint2, with
mostly the same members, but also some differences.  Before actually
changing this, I reviewed all occurrences of the word 'type' in the
code.

No functional change.

(rillig)

lint: fix initialization with brace-enclosed string literal

C99 allows this form in 6.7.8p14 and p15.

The previous lint tests did not cover the case of an array at the top
level of the object to be initialized, they only covered the error cases
(d_c99_init.c, variables 'prefixed_message' and 'message_with_suffix').

Lint is now more generous than strictly required by C99, but since GCC
and Clang already cover the case of 'message_with_suffix', this is ok.

The test d_init_array_using_string.c was wrong before in rejecting the
initializer for 'extra_braces'.  I had tested that Clang generated a
warning for this, but I had not inspected its warning carefully enough.
Clang had not warned about the extra braces but only about a type
mismatch since I tested on a platform where wchar_t was 16 bit.

(rillig)

tests/lint: demonstrate wrong warning when initializing a string

(rillig)

lint: fix wrong warning about uninitialized _Complex variable

Seen in divxc3.c.

(rillig)

tests/lint: demonstrate wrong lint warning about complex variables

(rillig)

lint: quote placeholders in messages for unused variables

(rillig)

lint: do not warn about pointer conversion to or from incomplete type

This cuts down the warnings in a NetBSD release build by about 38,000.

(rillig)

lint: clean up the check for pointer conversions

No functional change.

(rillig)

lint: for structs and unions, include incompleteness in the type name

This mainly helps to assess the situation where lint warns that a
pointer cast "may be troublesome", see msg_247.exp.

(rillig)

tests/lint: document where the 38,000 lint warnings come from

(rillig)

tests/lint: test bit-shift amount equal to the type's width

(rillig)

lint: clean up handling of preprocessing output lines

No functional change.

(rillig)

lint: in code from included files, print stack trace

Previously, the standard NetBSD build generated several lint warnings in
lhash.h from OpenSSL, without providing any hint as to which file
actually included that header.  In cases like these, lint now interprets
the line number information in the preprocessor output from GCC to
reconstruct the exact include path to the file in question.

The program check-expect.lua had to be rewritten almost completely since
it assumed that all diagnostics would come from the main file.  In all
existing tests, this was true, but these tests did not cover all cases
that occurred in practice.  Now it records the complete location of the
diagnostic instead of just the line number.

(rillig)

tests/lint: refine comment and function name in test for pointer cast

In a typical NetBSD build, there are about 100,000 lint warnings.  About
50,000 of them are warning 247.  About 38,000 of these are from OpenSSL
and may be fixed by now.

(rillig)

lint: don't warn about cast between pointers to compatible structs

(rillig)

tests/lint: add test for struct pointer cast

(rillig)

bozohttpd: fix argument type for functions from <ctype.h>

Found by the recently added check to lint (message 342).

ok mrg@

(rillig)

bozohttpd-small: fix typo in SRCS, fix targets dependall and lint

The additional ':' after 'main.c' had no effect on the compilation since
the suffix is removed when computing the list of object files.  The ':'
excluded main.c from the dependency computation and from lint.

By defining CPPFLAGS instead of CFLAGS, the default "-O2" steps in
again, and the proper flags are passed to lint, which did not work
before at all because it could not find an included file.

(rillig)

bozohttpd-small: fix error handling for unifdef

The second of the '$$?' was spelled only '$?'.  At the time where this
shell expression was evaluated, it had already been overwritten by the
preceding command '['.

(rillig)

lint: sync comment with reality after renaming several functions

No functional change.

(rillig)

lint: for shift in C99 mode, do not warn about difference to pre-C90

C99 is too far away from traditional C to make this warning useful.
There are 3 different situations in which this warning is generated:

For '1 << (unsigned char)1', the result type is 'unsigned int' in
traditional C.  The result type is unsigned because at least 1 of the
operators is unsigned, and it is 'unsigned int' because the usual
arithmetic promotions are applied.

For '1 >> (long)1', as well as for '1 << (long)1', the result type is
'long' in traditional C since the usual arithmetic promotions are
applied.

Omitting this warning in C99 mode reduces the amount of lint warnings in
a typical NetBSD release build by approximately 6800 of 107000 total.

(rillig)

lint: test what happens for 'int << double'

I just wanted to see whether typeok_shift would be reached in this case,
to ensure that is_uinteger applies as expected.  It does.

(rillig)

lint: reduce indentation of typeok_shift

No functional change.

(rillig)

lint: add details to warning about too large shift amount

The previous message 'shift greater than size of object' was too short
to give reasonable hints, especially when the expressions involve
typedefs or macros.

(rillig)

lint: fix wrong warning about losing accuracy when converting to _Bool

(rillig)

tests/lint: demonstrate wrong warning about losing accuracy

(rillig)

lint: move check for strict bool mode into separate file

No functional change.

(rillig)

make: reduce verbosity of the -dv debug logging for standard cases

The verbosity was already removed from LogBeforeApply, now it is
consistent between LogBeforeApply and LogAfterApply.

(rillig)

tests/libcurses: fix names of over-the-wire data types

The test 'mvscanw' reported that it would send '%s' as 'numeric', which
was rather suspicious.

(rillig)

tests/libcurses: clean up table of input functions

The previous "table" was an insult to any reader.  It was unsorted,
listed the functions shuffled, and was not even formatted consistently.

No functional change.

(rillig)

tests/libcurses: don't waste time calling strlen needlessly

(rillig)

make: in debug log, add space between scope and variable name

Without this space, the debug log looked more like line noise, even
though the only punctuation was a single innocent ':'.  From a make
user's perspective, the variable name is a word of its own and should
not be visually glued to its namespace.

(rillig)

make: omit unnecessary details from -dv debug log

When an expression is based on a defined variable, it does not matter
whether the evaluation mode is "eval" or "eval-defined", therefore omit
these details to reduce confusion.

(rillig)

make: be more verbose in -dv debug logging

The previous log output was too brief to be understandable.  Give more
hints by describing each part of the expression when evaluating a
modifier.  Distinguish between parse-only mode and eval mode since in
parse-only mode most of the details are irrelevant.

(rillig)

make: clean up debug logging for ':M' and ':N'

Using square brackets as quotes was confusing since patterns can contain
square brackets themselves.

The debug logging for VarMatch was a bit too detailed.  Having the
"before" and "after" states is enough for all practically relevant
cases.

(rillig)

bind: remove unnecessary local modifications

No functional change.

(rillig)

bind: remove non-canonical redundant CONSTCOND comments

Since lint1/tree.c 1.202 from 2021-01-31, lint no longer needs the
/*CONSTCOND*/ for do-while-0 "loops".

No functional change.

(rillig)

bind: remove unnecessary CONSTCOND comments

Since lint1/tree.c 1.202 from 2021-01-31, lint no longer needs the
/*CONSTCOND*/ for do-while-0 "loops".

No functional change.

(rillig)

bind: remove obsolete local modifications

The duplicate include has been added upstream, further up in the file.

Since lint1/tree.c 1.202 from 2021-01-31, lint no longer needs the
/*CONSTCOND*/ for do-while-0 "loops".

(rillig)

bind: remove workaround for bugs in lint

The bugs have been fixed in lint1/init.c 1.179 from 2021-03-30.

(rillig)

lint: inline macro in check for <ctype.h> functions

No functional change.

(rillig)

lint: warn about for wrong type cast in argument to ctype.h functions

The argument to most of the functions from <ctype.h> "shall either be
representable as an 'unsigned char' or shall equal the value of the
macro EOF".

When confronted with the infamous warning 'array subscript has type
char', there are enough programmers who don't know the background of
that warning and thus fix it in a wrong way.  Neither GCC nor Clang
explain its warning to target these programmers.

Both GCC and Clang warn about 'array subscript has type char', but they
ignore the other requirements of the <ctype.h> functions, even though
these are in the C standard library.

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=94182
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=95177
https://stackoverflow.com/a/60696378

(rillig)