 @@ -1,6 +1,5 @@
 The one, true implementation of the AWK pattern-directed scanning and
 processing language, by one of the language's creators, Brian Kernighan.
-This is the version of awk described in The AWK Programming Language,
+This is the version of awk described in "The AWK Programming Language",
-Second Edition, by Al Aho, Brian Kernighan, and Peter Weinberger
+by Al Aho, Brian Kernighan, and Peter Weinberger (Addison-Wesley,
-(Addison-Wesley, 2024, ISBN-13 978-0138269722, ISBN-10 0138269726).
+, ISBN 0-201-07981-X).  It is also known as new awk, or nawk.
 It is also known as new awk, or nawk.

 @@ -1,16 +1,16 @@
-# $NetBSD: Makefile,v 1.45 2023/09/12 19:16:52 vins Exp $
+# $NetBSD: Makefile,v 1.46 2023/09/17 10:32:05 vins Exp $
-DISTNAME=	nawk-20230911
+DISTNAME=	nawk-20230909
 CATEGORIES=	lang
 MASTER_SITES=	# empty
 DISTFILES=	# empty
 MAINTAINER=	pkgsrc-users@NetBSD.org
 HOMEPAGE=	https://www.cs.princeton.edu/~bwk/btl.mirror/
 COMMENT=	Brian Kernighan's pattern-directed scanning and processing language
 LICENSE=	mit
 BOOTSTRAP_PKG=	yes
 CFLAGS+=	${CPPFLAGS} -DYYMAXDEPTH=300
 MAKE_FLAGS+=	CC=${CC:Q} CFLAGS=${CFLAGS:M*:Q}

 @@ -15,47 +15,26 @@ permission.
 LUCENT DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS SOFTWARE,
 INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS.
 IN NO EVENT SHALL LUCENT OR ANY OF ITS ENTITIES BE LIABLE FOR ANY
 SPECIAL, INDIRECT OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
 WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER
 IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION,
 ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF
 THIS SOFTWARE.
 ****************************************************************/
 This file lists all bug fixes, changes, etc., made since the AWK book
 was sent to the printers in August 1987.
 Sep 11, 2023:
 	Added --csv option to enable processing of comma-separated
 	values inputs.  When --csv is enabled, fields are separated
 	by commas, fields may be quoted with " double quotes, fields
 	may contain embedded newlines.
 	If no explicit separator argument is provided, split() uses
 	the setting of --csv to determine how fields are split.
 	Strings may now contain UTF-8 code points (not necessarily
 	characters).  Functions that operate on characters, like
 	length, substr, index, match, etc., use UTF-8, so the length
 	of a string of 3 emojis is 3, not 12 as it would be if bytes
 	were counted.
 	Regular expressions are processes as UTF-8.
 	Unicode literals can be written as \u followed by one
 	to eight hexadecimal digits.  These may appear in strings and
 	regular expressions.
 Sep 06, 2023:
 	Fix edge case where FS is changed on commandline. Thanks to
 	Gordon Shephard and Miguel Pineiro Jr.
 	Fix regular expression clobbering in the lexer, where lexer does
 	not make a copy of regexp literals. also makedfa memory leaks have
 	been plugged. Thanks to Miguel Pineiro Jr.
 Dec 15, 2022:
 	Force hex escapes in strings to be no more than two characters,
 	as they already are in regular expressions. This brings internal
 	consistency, as well as consistency with gawk. Thanks to
 	Arnold Robbins.

 @@ -1,48 +1,18 @@
 # The One True Awk
 This is the version of `awk` described in _The AWK Programming Language_,
-Second Edition, by Al Aho, Brian Kernighan, and Peter Weinberger
+by Al Aho, Brian Kernighan, and Peter Weinberger
-(Addison-Wesley, 2024, ISBN-13 978-0138269722, ISBN-10 0138269726).
+(Addison-Wesley, 1988, ISBN 0-201-07981-X).
 ## What's New? ##
 This version of Awk handles UTF-8 and comma-separated values (CSV) input.
 ### Strings ###
 Functions that process strings now count Unicode code points, not bytes;
 this affects `length`, `substr`, `index`, `match`, `split`,
 `sub`, `gsub`, and others.  Note that code
 points are not necessarily characters.
 UTF-8 sequences may appear in literal strings and regular expressions.
 Aribtrary characters may be included with `\u` followed by 1 to 8 hexadecimal digits.
 ### Regular expressions ###
 Regular expressions may include UTF-8 code points, including `\u`.
 Character classes are likely to be limited to about 256 characters
 when expanded.
 ### CSV ###
 The option `--csv` turns on CSV processing of input:
 fields are separated by commas, fields may be quoted with
 double-quote (`"`) characters, fields may contain embedded newlines.
 In CSV mode, `FS` is ignored.
 If no explicit separator argument is provided,
 field-splitting in `split` is determined by CSV mode.
 ## Copyright
 Copyright (C) Lucent Technologies 1997<br/>
 All Rights Reserved
 Permission to use, copy, modify, and distribute this software and
 its documentation for any purpose and without fee is hereby
 granted, provided that the above copyright notice appear in all
 copies and that both that the copyright notice and this
 permission notice and warranty disclaimer appear in supporting
 documentation, and that the name Lucent Technologies or any of
 its entities not be used in advertising or publicity pertaining
 @@ -86,61 +56,69 @@ posting the pull request. To do so:
 * Please create the pull request with a request
 to merge into the `staging` branch instead of into the `master` branch.
 This allows us to do testing, and to make any additional edits or changes
 after the merge but before merging to `master`.
 ## Building
 The program itself is created by
 	make
 which should produce a sequence of messages roughly like this:
-	yacc -d -b awkgram  awkgram.y
+	yacc -d awkgram.y
-	yacc: 44 shift/reduce conflicts, 85 reduce/reduce conflicts.
+	conflicts: 43 shift/reduce, 85 reduce/reduce
-	cc -g -Wall -pedantic -Wcast-qual   -O2   -c -o awkgram.tab.o awkgram.tab.c
+	mv y.tab.c ytab.c
-	cc -g -Wall -pedantic -Wcast-qual   -O2   -c -o b.o b.c
+	mv y.tab.h ytab.h
-	cc -g -Wall -pedantic -Wcast-qual   -O2   -c -o main.o main.c
+	cc -c ytab.c
-	cc -g -Wall -pedantic -Wcast-qual   -O2   -c -o parse.o parse.c
+	cc -c b.c
-	cc -g -Wall -pedantic -Wcast-qual -O2 maketab.c -o maketab
+	cc -c main.c
-	./maketab awkgram.tab.h >proctab.c
+	cc -c parse.c
-	cc -g -Wall -pedantic -Wcast-qual   -O2   -c -o proctab.o proctab.c
+	cc maketab.c -o maketab
-	cc -g -Wall -pedantic -Wcast-qual   -O2   -c -o tran.o tran.c
+	./maketab >proctab.c
-	cc -g -Wall -pedantic -Wcast-qual   -O2   -c -o lib.o lib.c
+	cc -c proctab.c
-	cc -g -Wall -pedantic -Wcast-qual   -O2   -c -o run.o run.c
+	cc -c tran.c
-	cc -g -Wall -pedantic -Wcast-qual   -O2   -c -o lex.o lex.c
+	cc -c lib.c
-	cc -g -Wall -pedantic -Wcast-qual   -O2 awkgram.tab.o b.o main.o parse.o proctab.o tran.o lib.o run.o lex.o   -lm
+	cc -c run.c
 	cc -c lex.c
 	cc ytab.o b.o main.o parse.o proctab.o tran.o lib.o run.o lex.o -lm
 This produces an executable `a.out`; you will eventually want to
 move this to some place like `/usr/bin/awk`.
 If your system does not have `yacc` or `bison` (the GNU
 equivalent), you need to install one of them first.
 NOTE: This version uses ISO/IEC C99, as you should also.  We have
 compiled this without any changes using `gcc -Wall` and/or local C
 compilers on a variety of systems, but new systems or compilers
 may raise some new complaint; reports of difficulties are
 welcome.
 This compiles without change on Macintosh OS X using `gcc` and
 the standard developer tools.
 You can also use `make CC=g++` to build with the GNU C++ compiler,
 should you choose to do so.
 The version of `malloc` that comes with some systems is sometimes
 astonishly slow.  If `awk` seems slow, you might try fixing that.
 More generally, turning on optimization can significantly improve
 `awk`'s speed, perhaps by 1/3 for highest levels.
 ## A Note About Releases
 We don't usually do releases.
 ## A Note About Maintenance
 NOTICE! Maintenance of this program is on a ''best effort''
 basis.  We try to get to issues and pull requests as quickly
 as we can.  Unfortunately, however, keeping this program going
 is not at the top of our priority list.
 #### Last Updated
-Sun Sep  3 09:26:43 EDT 2023
+Sun 23 Jan 2022 03:48:01 PM EST

 @@ -358,28 +358,26 @@ int yylex(void)
 		case '(':
 			parencnt++;
 			RET('(');
 		case '"':
 			return string();	/* BUG: should be like tran.c ? */
 		default:
 			RET(c);
+		}
+	}
+}
 extern int runetochar(char *str, int c);
 int string(void)
+{
 	int c, n;
 	char *s, *bp;
 	static char *buf = NULL;
 	static int bufsz = 500;
 	if (buf == NULL && (buf = (char *) malloc(bufsz)) == NULL)
 		FATAL("out of space for strings");
 	for (bp = buf; (c = input()) != '"'; ) {
 		if (!adjbuf(&buf, &bufsz, bp-buf+2, 500, &bp, "string"))
 			FATAL("out of space for string %.10s...", buf);
 		switch (c) {
 @@ -407,73 +405,52 @@ int string(void)
 			case '\\': *bp++ = '\\'; break;
 			case '0': case '1': case '2': /* octal: \d \dd \ddd */
 			case '3': case '4': case '5': case '6': case '7':
 				n = c - '0';
 				if ((c = peek()) >= '0' && c < '8') {
 					n = 8 * n + input() - '0';
 					if ((c = peek()) >= '0' && c < '8')
 						n = 8 * n + input() - '0';
+				}
 				*bp++ = n;
 				break;
-			case 'x':	/* hex  \x0-9a-fA-F (exactly two) */
+			case 'x':	/* hex  \x0-9a-fA-F + */
+			    {
 				int i;
 				n = 0;
 				for (i = 1; i <= 2; i++) {
 					c = input();
 					if (c == 0)
 						break;
 					if (isxdigit(c)) {
 						c = tolower(c);
 						n *= 16;
 						if (isdigit(c))
 							n += (c - '0');
 						else
 							n += 10 + (c - 'a');
 					} else
 						break;
+				}
 				if (n)
 					*bp++ = n;
 				else
 					unput(c);
 				break;
+			    }
 			case 'u':	/* utf  \u0-9a-fA-F (1..8) */
 				int i;
 				n = 0;
 				for (i = 0; i < 8; i++) {
 					c = input();
 					if (!isxdigit(c) || c == 0)
 						break;
 					c = tolower(c);
 					n *= 16;
 					if (isdigit(c))
 						n += (c - '0');
 					else
 						n += 10 + (c - 'a');
 				unput(c);
 				bp += runetochar(bp, n);
 				break;
 			default:
 				*bp++ = c;
 				break;
+			}
 			break;
 		default:
 			*bp++ = c;
 			break;
+		}
+	}
 	*bp = 0;
 	s = tostring(buf);
 	*bp++ = ' '; *bp++ = '\0';

 @@ -12,55 +12,53 @@ its entities not be used in advertising
 to distribution of the software without specific, written prior
 permission.
 LUCENT DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS SOFTWARE,
 INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS.
 IN NO EVENT SHALL LUCENT OR ANY OF ITS ENTITIES BE LIABLE FOR ANY
 SPECIAL, INDIRECT OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
 WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER
 IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION,
 ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF
 THIS SOFTWARE.
 ****************************************************************/
-const char	*version = "version 20230911";
+const char	*version = "version 20230909";
 #define DEBUG
 #include <stdio.h>
 #include <ctype.h>
 #include <locale.h>
 #include <stdlib.h>
 #include <string.h>
 #include <signal.h>
 #include "awk.h"
 extern	char	**environ;
 extern	int	nfields;
 int	dbg	= 0;
 Awkfloat	srand_seed = 1;
 char	*cmdname;	/* gets argv[0] for error messages */
 extern	FILE	*yyin;	/* lex input file */
 char	*lexprog;	/* points to program argument if it exists */
 extern	int errorflag;	/* non-zero if any syntax errors; set by yyerror */
 enum compile_states	compile_time = ERROR_PRINTING;
 static char	**pfile;	/* program filenames from -f's */
 static size_t	maxpfile;	/* max program filename */
 static size_t	npfile;		/* number of filenames */
 static size_t	curpfile;	/* current filename */
 bool	CSV = false;	/* true for csv input */
 bool	safe = false;	/* true => "safe" mode */
 static noreturn void fpecatch(int n
 #ifdef SA_SIGINFO
 	, siginfo_t *si, void *uc
 #endif
+)
+{
 #ifdef SA_SIGINFO
 	static const char *emsg[] = {
 		[0] = "Unknown error",
 		[FPE_INTDIV] = "Integer divide by zero",
 		[FPE_INTOVF] = "Integer overflow",
 @@ -142,32 +140,26 @@ int main(int argc, char *argv[])
 	yyin = NULL;
 	symtab = makesymtab(NSYMTAB/NSYMTAB);
 	while (argc > 1 && argv[1][0] == '-' && argv[1][1] != '\0') {
 		if (strcmp(argv[1], "-version") == 0 || strcmp(argv[1], "--version") == 0) {
 			printf("awk %s\n", version);
 			return 0;
+		}
 		if (strcmp(argv[1], "--") == 0) {	/* explicit end of args */
 			argc--;
 			argv++;
 			break;
+		}
 		if (strcmp(argv[1], "--csv") == 0) {	/* turn on csv input processing */
 			CSV = true;
 			argc--;
 			argv++;
 			continue;
 		switch (argv[1][1]) {
 		case 's':
 			if (strcmp(argv[1], "-safe") == 0)
 				safe = true;
 			break;
 		case 'f':	/* next argument is program filename */
 			fn = getarg(&argc, &argv, "no program filename");
 			if (npfile >= maxpfile) {
 				maxpfile += 20;
 				pfile = (char **) realloc(pfile, maxpfile * sizeof(*pfile));
 				if (pfile == NULL)
 					FATAL("error allocating space for -f options");
+ 			}

 @@ -68,28 +68,26 @@ extern char	EMPTY[];	/* this avoid -Wwri
 extern char	**FS;
 extern char	**RS;
 extern char	**ORS;
 extern char	**OFS;
 extern char	**OFMT;
 extern Awkfloat *NR;
 extern Awkfloat *FNR;
 extern Awkfloat *NF;
 extern char	**FILENAME;
 extern char	**SUBSEP;
 extern Awkfloat *RSTART;
 extern Awkfloat *RLENGTH;
 extern bool	CSV;		/* true for csv input */
 extern char	*record;	/* points to $0 */
 extern int	lineno;		/* line number in awk program */
 extern int	errorflag;	/* 1 if error has occurred */
 extern bool	donefld;	/* true if record broken into fields */
 extern bool	donerec;	/* true if record is valid (no fld has changed */
 extern int	dbg;
 extern const char *patbeg;	/* beginning of pattern matched */
 extern	int	patlen;		/* length of pattern matched.  set in b.c */
 /* Cell:  all information about a variable or constant */
 typedef struct Cell {
 @@ -217,51 +215,43 @@ extern	int	pairstack[], paircnt;
 #define isfld(n)	((n)->tval & FLD)
 #define isstr(n)	((n)->tval & STR)
 #define isnum(n)	((n)->tval & NUM)
 #define isarr(n)	((n)->tval & ARR)
 #define isfcn(n)	((n)->tval & FCN)
 #define istrue(n)	((n)->csub == BTRUE)
 #define istemp(n)	((n)->csub == CTEMP)
 #define	isargument(n)	((n)->nobj == ARG)
 /* #define freeable(p)	(!((p)->tval & DONTFREE)) */
 #define freeable(p)	( ((p)->tval & (STR|DONTFREE)) == STR )
 /* structures used by regular expression matching machinery, mostly b.c: */
-#define NCHARS	(1256+3)		/* 256 handles 8-bit chars; 128 does 7-bit */
+#define NCHARS	(256+3)		/* 256 handles 8-bit chars; 128 does 7-bit */
 				/* BUG: some overflows (caught) if we use 256 */
 				/* watch out in match(), etc. */
 #define	HAT	(NCHARS+2)	/* matches ^ in regular expr */
 #define NSTATES	32
 typedef struct rrow {
 	long	ltype;	/* long avoids pointer warnings on 64-bit */
 	union {
 		int i;
 		Node *np;
 		uschar *up;
 		int *rp; /* rune representation of char class */
 	} lval;		/* because Al stores a pointer in it! */
 	int	*lfollow;
 } rrow;
 typedef struct gtt { /* gototab entry */
 	unsigned int ch;
 	unsigned int state;
 } gtt;
 typedef struct fa {
-	gtt	**gototab;
+	unsigned int	**gototab;
 	int	gototab_len;
 	uschar	*out;
 	uschar	*restr;
 	int	**posns;
 	int	state_count;
 	bool	anchor;
 	int	use;
 	int	initstat;
 	int	curstat;
 	int	accept;
 	struct	rrow re[1];	/* variable: actual size set by calling malloc */
 } fa;

 @@ -70,119 +70,83 @@ static const uschar 	*basestr;	/* starts
 				   repetition processing */
 static const uschar 	*firstbasestr;
 static	int setcnt;
 static	int poscnt;
 const char	*patbeg;
 int	patlen;
 #define	NFA	128	/* cache this many dynamic fa's */
 fa	*fatab[NFA];
 int	nfatab	= 0;	/* entries in fatab */
 /* utf-8 mechanism:
    For most of Awk, utf-8 strings just "work", since they look like
    null-terminated sequences of 8-bit bytes.
    Functions like length(), index(), and substr() have to operate
    in units of utf-8 characters.  The u8_* functions in run.c
    handle this.
    Regular expressions are more complicated, since the basic
    mechanism of the goto table used 8-bit byte indices into the
    gototab entries to compute the next state.  Unicode is a lot
    bigger, so the gototab entries are now structs with a character
    and a next state, and there is a linear search of the characters
    to find the state.  (Yes, this is slower, by a significant
    amount.  Tough.)
    Throughout the RE mechanism in b.c, utf-8 characters are
    converted to their utf-32 value.  This mostly shows up in
    cclenter, which expands character class ranges like a-z and now
    alpha-omega.  The size of a gototab array is still about 256.
    This should be dynamic, but for now things work ok for a single
    code page of Unicode, which is the most likely case.
    The code changes are localized in run.c and b.c.  I have added a
    handful of functions to somewhat better hide the implementation,
    but a lot more could be done.
  */
 static int get_gototab(fa*, int, int);
 static int set_gototab(fa*, int, int, int);
 extern int u8_rune(int *, const uschar *);
 static int *
 intalloc(size_t n, const char *f)
+{
 	int *p = (int *) calloc(n, sizeof(int));
 	if (p == NULL)
 		overflo(f);
 	return p;
+}
 static void
 resizesetvec(const char *f)
+{
 	if (maxsetvec == 0)
 		maxsetvec = MAXLIN;
 	else
 		maxsetvec *= 4;
 	setvec = (int *) realloc(setvec, maxsetvec * sizeof(*setvec));
 	tmpset = (int *) realloc(tmpset, maxsetvec * sizeof(*tmpset));
 	if (setvec == NULL || tmpset == NULL)
 		overflo(f);
+}
 static void
 resize_state(fa *f, int state)
+{
-	gtt **p;
+	unsigned int **p;
 	uschar *p2;
 	int **p3;
 	int i, new_count;
 	if (++state < f->state_count)
 		return;
 	new_count = state + 10; /* needs to be tuned */
-	p = (gtt **) realloc(f->gototab, new_count * sizeof(f->gototab[0]));
+	p = (unsigned int **) realloc(f->gototab, new_count * sizeof(f->gototab[0]));
 	if (p == NULL)
 		goto out;
 	f->gototab = p;
 	p2 = (uschar *) realloc(f->out, new_count * sizeof(f->out[0]));
 	if (p2 == NULL)
 		goto out;
 	f->out = p2;
 	p3 = (int **) realloc(f->posns, new_count * sizeof(f->posns[0]));
 	if (p3 == NULL)
 		goto out;
 	f->posns = p3;
 	for (i = f->state_count; i < new_count; ++i) {
-		f->gototab[i] = (gtt *) calloc(NCHARS, sizeof(**f->gototab));
+		f->gototab[i] = (unsigned int *) calloc(NCHARS, sizeof(**f->gototab));
 		if (f->gototab[i] == NULL)
 			goto out;
 		f->out[i]  = 0;
 		f->posns[i] = NULL;
+	}
 	f->gototab_len = NCHARS; /* should be variable, growable */
 	f->state_count = new_count;
 	return;
 out:
 	overflo(__func__);
+}
 fa *makedfa(const char *s, bool anchor)	/* returns dfa for reg expr s */
+{
 	int i, use, nuse;
 	fa *pfa;
 	static int now = 1;
 	if (setvec == NULL) {	/* first time through any RE */
 @@ -257,27 +221,27 @@ int makeinit(fa *f, bool anchor)
 	int i, k;
 	f->curstat = 2;
 	f->out[2] = 0;
 	k = *(f->re[0].lfollow);
 	xfree(f->posns[2]);
 	f->posns[2] = intalloc(k + 1,  __func__);
 	for (i = 0; i <= k; i++) {
 		(f->posns[2])[i] = (f->re[0].lfollow)[i];
+	}
 	if ((f->posns[2])[1] == f->accept)
 		f->out[2] = 1;
 	for (i = 0; i < NCHARS; i++)
-		set_gototab(f, 2, 0, 0); /* f->gototab[2][i] = 0; */
+		f->gototab[2][i] = 0;
 	f->curstat = cgoto(f, 2, HAT);
 	if (anchor) {
 		*f->posns[2] = k-1;	/* leave out position 0 */
 		for (i = 0; i < k; i++) {
 			(f->posns[0])[i] = (f->posns[2])[i];
+		}
 		f->out[0] = f->out[2];
 		if (f->curstat != 2)
 			--(*f->posns[f->curstat]);
+	}
 	return f->curstat;
+}
 @@ -326,151 +290,128 @@ void freetr(Node *p)	/* free parse tree
 		freetr(left(p));
 		freetr(right(p));
 		xfree(p);
 		break;
 	default:	/* can't happen */
 		FATAL("can't happen: unknown type %d in freetr", type(p));
 		break;
+	}
+}
 /* in the parsing of regular expressions, metacharacters like . have */
 /* to be seen literally;  \056 is not a metacharacter. */
-int hexstr(const uschar **pp, int max)	/* find and eval hex string at pp, return new p */
+int hexstr(const uschar **pp)	/* find and eval hex string at pp, return new p */
 {			/* only pick up one 8-bit byte (2 chars) */
 	const uschar *p;
 	int n = 0;
 	int i;
-	for (i = 0, p = *pp; i < max && isxdigit(*p); i++, p++) {
+	for (i = 0, p = *pp; i < 2 && isxdigit(*p); i++, p++) {
 		if (isdigit(*p))
 			n = 16 * n + *p - '0';
 		else if (*p >= 'a' && *p <= 'f')
 			n = 16 * n + *p - 'a' + 10;
 		else if (*p >= 'A' && *p <= 'F')
 			n = 16 * n + *p - 'A' + 10;
+	}
 	*pp = p;
 	return n;
+}
 #define isoctdigit(c) ((c) >= '0' && (c) <= '7')	/* multiple use of arg */
 int quoted(const uschar **pp)	/* pick up next thing after a \\ */
 			/* and increment *pp */
+{
 	const uschar *p = *pp;
 	int c;
-/* BUG: should advance by utf-8 char even if makes no sense */
+	if ((c = *p++) == 't')
 	if ((c = *p++) == 't') {
 		c = '\t';
-	} else if (c == 'n') {
+	else if (c == 'n')
 		c = '\n';
-	} else if (c == 'f') {
+	else if (c == 'f')
 		c = '\f';
-	} else if (c == 'r') {
+	else if (c == 'r')
 		c = '\r';
-	} else if (c == 'b') {
+	else if (c == 'b')
 		c = '\b';
-	} else if (c == 'v') {
+	else if (c == 'v')
 		c = '\v';
-	} else if (c == 'a') {
+	else if (c == 'a')
 		c = '\a';
-	} else if (c == '\\') {
+	else if (c == '\\')
 		c = '\\';
-	} else if (c == 'x') {	/* 2 hex digits follow */
+	else if (c == 'x') {	/* hexadecimal goo follows */
-		c = hexstr(&p, 2);	/* this adds a null if number is invalid */
+		c = hexstr(&p);	/* this adds a null if number is invalid */
 	} else if (c == 'u') {	/* unicode char number up to 8 hex digits */
 		c = hexstr(&p, 8);
 	} else if (isoctdigit(c)) {	/* \d \dd \ddd */
 		int n = c - '0';
 		if (isoctdigit(*p)) {
 			n = 8 * n + *p++ - '0';
 			if (isoctdigit(*p))
 				n = 8 * n + *p++ - '0';
+		}
 		c = n;
 	} /* else */
 		/* c = c; */
 	*pp = p;
 	return c;
+}
-int *cclenter(const char *argp)	/* add a character class */
+char *cclenter(const char *argp)	/* add a character class */
+{
 	int i, c, c2;
-	int n;
+	const uschar *op, *p = (const uschar *) argp;
-	const uschar *p = (const uschar *) argp;
+	uschar *bp;
-	int *bp, *retp;
+	static uschar *buf = NULL;
 	static int *buf = NULL;
 	static int bufsz = 100;
-	if (buf == NULL && (buf = (int *) calloc(bufsz, sizeof(int))) == NULL)
+	op = p;
 	if (buf == NULL && (buf = (uschar *) malloc(bufsz)) == NULL)
 		FATAL("out of space for character class [%.10s...] 1", p);
 	bp = buf;
-	for (i = 0; *p != 0; ) {
+	for (i = 0; (c = *p++) != 0; ) {
 		n = u8_rune(&c, p);
 		p += n;
 		if (c == '\\') {
 			c = quoted(&p);
 		} else if (c == '-' && i > 0 && bp[-1] != 0) {
 			if (*p != 0) {
 				c = bp[-1];
-				/* c2 = *p++; */
+				c2 = *p++;
 				n = u8_rune(&c2, p);
 				p += n;
 				if (c2 == '\\')
-					c2 = quoted(&p); /* BUG: sets p, has to be u8 size */
+					c2 = quoted(&p);
 				if (c > c2) {	/* empty; ignore */
 					bp--;
 					i--;
 					continue;
+				}
 				while (c < c2) {
-					if (i >= bufsz) {
+					if (!adjbuf((char **) &buf, &bufsz, bp-buf+2, 100, (char **) &bp, "cclenter1"))
-						bufsz *= 2;
+						FATAL("out of space for character class [%.10s...] 2", p);
 						buf = (int *) realloc(buf, bufsz * sizeof(int));
 						if (buf == NULL)
 							FATAL("out of space for character class [%.10s...] 2", p);
 						bp = buf + i;
 					*bp++ = ++c;
 					i++;
+				}
 				continue;
+			}
+		}
-		if (i >= bufsz) {
+		if (!adjbuf((char **) &buf, &bufsz, bp-buf+2, 100, (char **) &bp, "cclenter2"))
-			bufsz *= 2;
+			FATAL("out of space for character class [%.10s...] 3", p);
 			buf = (int *) realloc(buf, bufsz * sizeof(int));
 			if (buf == NULL)
 				FATAL("out of space for character class [%.10s...] 2", p);
 			bp = buf + i;
 		*bp++ = c;
 		i++;
+	}
 	*bp = 0;
-	/* DPRINTF("cclenter: in = |%s|, out = |%s|\n", op, buf); BUG: can't print array of int */
+	DPRINTF("cclenter: in = |%s|, out = |%s|\n", op, buf);
-	/* xfree(op);  BUG: what are we freeing here? */
+	xfree(op);
-	retp = (int *) calloc(bp-buf+1, sizeof(int));
+	return (char *) tostring((char *) buf);
 	for (i = 0; i < bp-buf+1; i++)
 		retp[i] = buf[i];
 	return retp;
+}
 void overflo(const char *s)
+{
 	FATAL("regular expression too big: out of space in %.30s...", s);
+}
 void cfoll(fa *f, Node *v)	/* enter follow set of each leaf of vertex v into lfollow[leaf] */
+{
 	int i;
 	int *p;
 	switch (type(v)) {
 @@ -573,284 +514,195 @@ void follow(Node *v)	/* collects leaves
 	case CAT:
 		if (v == left(p)) {	/* v is left child of p */
 			if (first(right(p)) == 0) {
 				follow(p);
 				return;
+			}
 		} else		/* v is right child */
 			follow(p);
 		return;
+	}
+}
-int member(int c, int *sarg)	/* is c in s? */
+int member(int c, const char *sarg)	/* is c in s? */
+{
-	int *s = (int *) sarg;
+	const uschar *s = (const uschar *) sarg;
 	while (*s)
 		if (c == *s++)
 			return(1);
 	return(0);
+}
 static int get_gototab(fa *f, int state, int ch) /* hide gototab inplementation */
 	int i;
 	for (i = 0; i < f->gototab_len; i++) {
 		if (f->gototab[state][i].ch == 0)
 			break;
 		if (f->gototab[state][i].ch == ch)
 			return f->gototab[state][i].state;
 	return 0;
 static int set_gototab(fa *f, int state, int ch, int val) /* hide gototab inplementation */
 	int i;
 	for (i = 0; i < f->gototab_len; i++) {
 		if (f->gototab[state][i].ch == 0 || f->gototab[state][i].ch == ch) {
 			f->gototab[state][i].ch = ch;
 			f->gototab[state][i].state = val;
 			return val;
 	overflo(__func__);
 	return val; /* not used anywhere at the moment */
 int match(fa *f, const char *p0)	/* shortest match ? */
+{
 	int s, ns;
 	int n;
 	int rune;
 	const uschar *p = (const uschar *) p0;
 	/* return pmatch(f, p0); does it matter whether longest or shortest? */
 	s = f->initstat;
 	assert (s < f->state_count);
 	if (f->out[s])
 		return(1);
 	do {
 		/* assert(*p < NCHARS); */
-		n = u8_rune(&rune, p);
+		if ((ns = f->gototab[s][*p]) != 0)
 		if ((ns = get_gototab(f, s, rune)) != 0)
 			s = ns;
 		else
-			s = cgoto(f, s, rune);
+			s = cgoto(f, s, *p);
 		if (f->out[s])
 			return(1);
-		if (*p == 0)
+	} while (*p++ != 0);
 			break;
 		p += n;
 	} while (1);  /* was *p++ != 0 */
 	return(0);
+}
 int pmatch(fa *f, const char *p0)	/* longest match, for sub */
+{
 	int s, ns;
 	int n;
 	int rune;
 	const uschar *p = (const uschar *) p0;
 	const uschar *q;
 	s = f->initstat;
 	assert(s < f->state_count);
 	patbeg = (const char *)p;
 	patlen = -1;
 	do {
 		q = p;
 		do {
 			if (f->out[s])		/* final state */
 				patlen = q-p;
 			/* assert(*q < NCHARS); */
-			n = u8_rune(&rune, q);
+			if ((ns = f->gototab[s][*q]) != 0)
 			if ((ns = get_gototab(f, s, rune)) != 0)
 				s = ns;
 			else
-				s = cgoto(f, s, rune);
+				s = cgoto(f, s, *q);
 			assert(s < f->state_count);
 			if (s == 1) {	/* no transition */
 				if (patlen >= 0) {
 					patbeg = (const char *) p;
 					return(1);
+				}
 				else
 					goto nextin;	/* no match */
+			}
-			if (*q == 0)
+		} while (*q++ != 0);
 				break;
 			q += n;
 		} while (1);
 		q++;  /* was *q++ */
 		if (f->out[s])
 			patlen = q-p-1;	/* don't count $ */
 		if (patlen >= 0) {
 			patbeg = (const char *) p;
 			return(1);
+		}
 	nextin:
 		s = 2;
-		if (*p == 0)
+	} while (*p++);
 			break;
 		n = u8_rune(&rune, p);
 		p += n;
 	} while (1); /* was *p++ */
 	return (0);
+}
 int nematch(fa *f, const char *p0)	/* non-empty match, for sub */
+{
 	int s, ns;
         int n;
         int rune;
 	const uschar *p = (const uschar *) p0;
 	const uschar *q;
 	s = f->initstat;
 	assert(s < f->state_count);
 	patbeg = (const char *)p;
 	patlen = -1;
 	while (*p) {
 		q = p;
 		do {
 			if (f->out[s])		/* final state */
 				patlen = q-p;
 			/* assert(*q < NCHARS); */
-			n = u8_rune(&rune, q);
+			if ((ns = f->gototab[s][*q]) != 0)
 			if ((ns = get_gototab(f, s, rune)) != 0)
 				s = ns;
 			else
-				s = cgoto(f, s, rune);
+				s = cgoto(f, s, *q);
 			if (s == 1) {	/* no transition */
 				if (patlen > 0) {
 					patbeg = (const char *) p;
 					return(1);
 				} else
 					goto nnextin;	/* no nonempty match */
+			}
-			if (*q == 0)
+		} while (*q++ != 0);
 				break;
 			q += n;
 		} while (1);
 		q++;
 		if (f->out[s])
 			patlen = q-p-1;	/* don't count $ */
 		if (patlen > 0 ) {
 			patbeg = (const char *) p;
 			return(1);
+		}
 	nnextin:
 		s = 2;
 		p++;
+	}
 	return (0);
+}
 static int getrune(FILE *fp, char **pbuf, int *pbufsize, int quantum,
 		   int *curpos, int *lastpos)
 	int c = 0;
 	char *buf = *pbuf;
 	static const int max_bytes = 4;	// max multiple bytes in UTF-8 is 4
 	int i, rune;
 	uschar private_buf[max_bytes + 1];
 	for (i = 0; i <= max_bytes; i++) {
 		if (++*curpos == *lastpos) {
 			if (*lastpos == *pbufsize)
 				if (!adjbuf((char **) pbuf, pbufsize, *pbufsize+1, quantum, 0, "getrune"))
 					FATAL("stream '%.30s...' too long", buf);
 			buf[(*lastpos)++] = (c = getc(fp)) != EOF ? c : 0;
 			private_buf[i] = c;
 		if (c == 0 || c < 128 ||  (c >> 6) == 4) { // 10xxxxxx starts a new character
 			ungetc(c, fp);
 			private_buf[i] = 0;
 			break;
 	u8_rune(& rune, private_buf);
 	return rune;
 /*
  * NAME
  *     fnematch
+ *
  * DESCRIPTION
  *     A stream-fed version of nematch which transfers characters to a
  *     null-terminated buffer. All characters up to and including the last
  *     character of the matching text or EOF are placed in the buffer. If
  *     a match is found, patbeg and patlen are set appropriately.
+ *
  * RETURN VALUES
  *     false    No match found.
  *     true     Match found.
  */
 bool fnematch(fa *pfa, FILE *f, char **pbuf, int *pbufsize, int quantum)
+{
 	char *buf = *pbuf;
 	int bufsize = *pbufsize;
 	int c, i, j, k, ns, s;
 	int rune;
 	s = pfa->initstat;
 	patlen = 0;
 	/*
 	 * All indices relative to buf.
 	 * i <= j <= k <= bufsize
+	 *
 	 * i: origin of active substring
 	 * j: current character
 	 * k: destination of next getc()
 	 */
 	i = -1, k = 0;
         do {
 		j = i++;
 		do {
 			if (++j == k) {
 				if (k == bufsize)
 					if (!adjbuf((char **) &buf, &bufsize, bufsize+1, quantum, 0, "fnematch"))
 						FATAL("stream '%.30s...' too long", buf);
 				buf[k++] = (c = getc(f)) != EOF ? c : 0;
+			}
 			c = (uschar)buf[j];
-			if (c < 128)
+			/* assert(c < NCHARS); */
 				rune = c;
 			else {
 				j--;
 				k--;
 				ungetc(c, f);
 				rune = getrune(f, &buf, &bufsize, quantum, &j, &k);
-			if ((ns = get_gototab(pfa, s, rune)) != 0)
+			if ((ns = pfa->gototab[s][c]) != 0)
 				s = ns;
 			else
-				s = cgoto(pfa, s, rune);
+				s = cgoto(pfa, s, c);
 			if (pfa->out[s]) {	/* final state */
 				patlen = j - i + 1;
 				if (c == 0)	/* don't count $ */
 					patlen--;
+			}
 		} while (buf[j] && s != 1);
 		s = 2;
 	} while (buf[i] && !patlen);
 	/* adjbuf() may have relocated a resized buffer. Inform the world. */
 	*pbuf = buf;
 	*pbufsize = bufsize;
 @@ -1157,51 +1009,43 @@ static int repeat(const uschar *reptok,
 			return replace_repeat(reptok, reptoklen, atom, atomlen,
 					firstnum, secondnum, REPEAT_SIMPLE);
+		}
 	} else if (firstnum < secondnum) {	/* {n,m} -> repeat n-1 times then alternate  */
 		/*  x{n,m}  =>  xx...x{1, m-n+1}  =>  xx...x?x?x?..x?	*/
 		return replace_repeat(reptok, reptoklen, atom, atomlen,
 					firstnum, secondnum, REPEAT_WITH_Q);
 	} else {	/* Error - shouldn't be here (n>m) */
 		FATAL("internal error");
+	}
 	return 0;
+}
 extern int u8_rune(int *, const uschar *); /* run.c; should be in header file */
 int relex(void)		/* lexical analyzer for reparse */
+{
 	int c, n;
 	int cflag;
 	static uschar *buf = NULL;
 	static int bufsz = 100;
 	uschar *bp;
 	const struct charclass *cc;
 	int i;
 	int num, m;
 	bool commafound, digitfound;
 	const uschar *startreptok;
 	static int parens = 0;
 rescan:
 	starttok = prestr;
 	if ((n = u8_rune(&rlxval, prestr)) > 1) {
 		prestr += n;
 		starttok = prestr;
 		return CHAR;
 	switch (c = *prestr++) {
 	case '|': return OR;
 	case '*': return STAR;
 	case '+': return PLUS;
 	case '?': return QUEST;
 	case '.': return DOT;
 	case '\0': prestr--; return '\0';
 	case '^':
 	case '$':
 		return c;
 	case '(':
 		parens++;
 		return c;
 @@ -1219,35 +1063,30 @@ rescan:
 	default:
 		rlxval = c;
 		return CHAR;
 	case '[':
 		if (buf == NULL && (buf = (uschar *) malloc(bufsz)) == NULL)
 			FATAL("out of space in reg expr %.10s..", lastre);
 		bp = buf;
 		if (*prestr == '^') {
 			cflag = 1;
 			prestr++;
+		}
 		else
 			cflag = 0;
-		n = 5 * strlen((const char *) prestr)+1; /* BUG: was 2.  what value? */
+		n = 2 * strlen((const char *) prestr)+1;
 		if (!adjbuf((char **) &buf, &bufsz, n, n, (char **) &bp, "relex1"))
 			FATAL("out of space for reg expr %.10s...", lastre);
 		for (; ; ) {
 			if ((n = u8_rune(&rlxval, prestr)) > 1) {
 				for (i = 0; i < n; i++)
 					*bp++ = *prestr++;
 				continue;
 			if ((c = *prestr++) == '\\') {
 				*bp++ = '\\';
 				if ((c = *prestr++) == '\0')
 					FATAL("nonterminated character class %.20s...", lastre);
 				*bp++ = c;
 			/* } else if (c == '\n') { */
 			/* 	FATAL("newline in character class %.20s...", lastre); */
 			} else if (c == '[' && *prestr == ':') {
 				/* POSIX char class names, Dag-Erling Smorgrav, des@ofug.org */
 				for (cc = charclasses; cc->cc_name; cc++)
 					if (strncmp((const char *) prestr + 1, (const char *) cc->cc_name, cc->cc_namelen) == 0)
 						break;
 				if (cc->cc_name != NULL && prestr[1 + cc->cc_namelen] == ':' &&
 @@ -1394,44 +1233,44 @@ rescan:
 				FATAL("illegal repetition expression: class %.20s",
 					lastre);
+			}
+		}
 		break;
+	}
+}
 int cgoto(fa *f, int s, int c)
+{
 	int *p, *q;
 	int i, j, k;
-	/* assert(c == HAT || c < NCHARS);  BUG: seg fault if disable test */
+	assert(c == HAT || c < NCHARS);
 	while (f->accept >= maxsetvec) {	/* guessing here! */
 		resizesetvec(__func__);
+	}
 	for (i = 0; i <= f->accept; i++)
 		setvec[i] = 0;
 	setcnt = 0;
 	resize_state(f, s);
 	/* compute positions of gototab[s,c] into setvec */
 	p = f->posns[s];
 	for (i = 1; i <= *p; i++) {
 		if ((k = f->re[p[i]].ltype) != FINAL) {
 			if ((k == CHAR && c == ptoi(f->re[p[i]].lval.np))
 			 || (k == DOT && c != 0 && c != HAT)
 			 || (k == ALL && c != 0)
 			 || (k == EMPTYRE && c != 0)
-			 || (k == CCL && member(c, (int *) f->re[p[i]].lval.rp))
+			 || (k == CCL && member(c, (char *) f->re[p[i]].lval.up))
-			 || (k == NCCL && !member(c, (int *) f->re[p[i]].lval.rp) && c != 0 && c != HAT)) {
+			 || (k == NCCL && !member(c, (char *) f->re[p[i]].lval.up) && c != 0 && c != HAT)) {
 				q = f->re[p[i]].lfollow;
 				for (j = 1; j <= *q; j++) {
 					if (q[j] >= maxsetvec) {
 						resizesetvec(__func__);
+					}
 					if (setvec[q[j]] == 0) {
 						setcnt++;
 						setvec[q[j]] = 1;
+					}
+				}
+			}
+		}
+	}
 @@ -1443,42 +1282,42 @@ int cgoto(fa *f, int s, int c)
 			tmpset[j++] = i;
+		}
 	resize_state(f, f->curstat > s ? f->curstat : s);
 	/* tmpset == previous state? */
 	for (i = 1; i <= f->curstat; i++) {
 		p = f->posns[i];
 		if ((k = tmpset[0]) != p[0])
 			goto different;
 		for (j = 1; j <= k; j++)
 			if (tmpset[j] != p[j])
 				goto different;
 		/* setvec is state i */
 		if (c != HAT)
-			set_gototab(f, s, c, i);
+			f->gototab[s][c] = i;
 		return i;
 	  different:;
+	}
 	/* add tmpset to current set of states */
 	++(f->curstat);
 	resize_state(f, f->curstat);
 	for (i = 0; i < NCHARS; i++)
-		set_gototab(f, f->curstat, 0, 0);
+		f->gototab[f->curstat][i] = 0;
 	xfree(f->posns[f->curstat]);
 	p = intalloc(setcnt + 1, __func__);
 	f->posns[f->curstat] = p;
 	if (c != HAT)
-		set_gototab(f, s, c, f->curstat);
+		f->gototab[s][c] = f->curstat;
 	for (i = 0; i <= setcnt; i++)
 		p[i] = tmpset[i];
 	if (setvec[f->accept])
 		f->out[f->curstat] = 1;
 	else
 		f->out[f->curstat] = 0;
 	return f->curstat;
+}
 void freefa(fa *f)	/* free a finite automaton */
+{
 	int i;

 @@ -16,43 +16,42 @@ LUCENT DISCLAIMS ALL WARRANTIES WITH REG
 INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS.
 IN NO EVENT SHALL LUCENT OR ANY OF ITS ENTITIES BE LIABLE FOR ANY
 SPECIAL, INDIRECT OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
 WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER
 IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION,
 ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF
 THIS SOFTWARE.
 ****************************************************************/
 #define DEBUG
 #include <stdio.h>
 #include <ctype.h>
 #include <errno.h>
 #include <wchar.h>
 #include <wctype.h>
 #include <fcntl.h>
 #include <setjmp.h>
 #include <limits.h>
 #include <math.h>
 #include <string.h>
 #include <stdlib.h>
 #include <time.h>
 #include <sys/types.h>
 #include <sys/wait.h>
 #include "awk.h"
 #include "awkgram.tab.h"
 static void stdinit(void);
 static void flush_all(void);
 static char *wide_char_to_byte_str(int rune, size_t *outlen);
 #if 1
 #define tempfree(x)	do { if (istemp(x)) tfree(x); } while (/*CONSTCOND*/0)
 #else
 void tempfree(Cell *p) {
 	if (p->ctype == OCELL && (p->csub < CUNK || p->csub > CFREE)) {
 		WARNING("bad csub %d in Cell %d %s",
 			p->csub, p->ctype, p->sval);
+	}
 	if (istemp(p))
 		tfree(p);
+}
 #endif
 @@ -570,280 +569,54 @@ Cell *intest(Node **a, int n)	/* a[0] is
 		ap->sval = (char *) makesymtab(NSYMTAB);
+	}
 	buf = makearraystring(a[0], __func__);
 	k = lookup(buf, (Array *) ap->sval);
 	tempfree(ap);
 	free(buf);
 	if (k == NULL)
 		return(False);
 	else
 		return(True);
+}
 /* ======== utf-8 code ========== */
 /*
  * Awk strings can contain ascii, random 8-bit items (eg Latin-1),
  * or utf-8.  u8_isutf tests whether a string starts with a valid
  * utf-8 sequence, and returns 0 if not (e.g., high bit set).
  * u8_nextlen returns length of next valid sequence, which is
  * 1 for ascii, 2..4 for utf-8, or 1 for high bit non-utf.
  * u8_strlen returns length of string in valid utf-8 sequences
  * and/or high-bit bytes.  Conversion functions go between byte
  * number and character number.
  * In theory, this behaves the same as before for non-utf8 bytes.
  * Limited checking! This is a potential security hole.
  */
 /* is s the beginning of a valid utf-8 string? */
 /* return length 1..4 if yes, 0 if no */
 int u8_isutf(const char *s)
 	int n, ret;
 	unsigned char c;
 	c = s[0];
 	if (c < 128)
 		return 1; /* what if it's 0? */
 	n = strlen(s);
 	if (n >= 2 && ((c>>5) & 0x7) == 0x6 && (s[1] & 0xC0) == 0x80) {
 		ret = 2; /* 110xxxxx 10xxxxxx */
 	} else if (n >= 3 && ((c>>4) & 0xF) == 0xE && (s[1] & 0xC0) == 0x80
 			 && (s[2] & 0xC0) == 0x80) {
 		ret = 3; /* 1110xxxx 10xxxxxx 10xxxxxx */
 	} else if (n >= 4 && ((c>>3) & 0x1F) == 0x1E && (s[1] & 0xC0) == 0x80
 			 && (s[2] & 0xC0) == 0x80 && (s[3] & 0xC0) == 0x80) {
 		ret = 4; /* 11110xxx 10xxxxxx 10xxxxxx 10xxxxxx */
 	} else {
 		ret = 0;
 	return ret;
 /* Convert (prefix of) utf8 string to utf-32 rune. */
 /* Sets *rune to the value, returns the length. */
 /* No error checking: watch out. */
 int u8_rune(int *rune, const char *s)
 	int n, ret;
 	unsigned char c;
 	c = s[0];
 	if (c < 128) {
 		*rune = c;
 		return 1;
 	n = strlen(s);
 	if (n >= 2 && ((c>>5) & 0x7) == 0x6 && (s[1] & 0xC0) == 0x80) {
 		*rune = ((c & 0x1F) << 6) | (s[1] & 0x3F); /* 110xxxxx 10xxxxxx */
 		ret = 2;
 	} else if (n >= 3 && ((c>>4) & 0xF) == 0xE && (s[1] & 0xC0) == 0x80
 			  && (s[2] & 0xC0) == 0x80) {
 		*rune = ((c & 0xF) << 12) | ((s[1] & 0x3F) << 6) | (s[2] & 0x3F);
 			/* 1110xxxx 10xxxxxx 10xxxxxx */
 		ret = 3;
 	} else if (n >= 4 && ((c>>3) & 0x1F) == 0x1E && (s[1] & 0xC0) == 0x80
 			  && (s[2] & 0xC0) == 0x80 && (s[3] & 0xC0) == 0x80) {
 		*rune = ((c & 0x7) << 18) | ((s[1] & 0x3F) << 12) | ((s[2] & 0x3F) << 6) | (s[3] & 0x3F);
 			/* 11110xxx 10xxxxxx 10xxxxxx 10xxxxxx */
 		ret = 4;
 	} else {
 		*rune = c;
 		ret = 1;
 	return ret; /* returns one byte if sequence doesn't look like utf */
 /* return length of next sequence: 1 for ascii or random, 2..4 for valid utf8 */
 int u8_nextlen(const char *s)
 	int len;
 	len = u8_isutf(s);
 	if (len == 0)
 		len = 1;
 	return len;
 /* return number of utf characters or single non-utf bytes */
 int u8_strlen(const char *s)
 	int i, len, n, totlen;
 	unsigned char c;
 	n = strlen(s);
 	totlen = 0;
 	for (i = 0; i < n; i += len) {
 		c = s[i];
 		if (c < 128) {
 			len = 1;
 		} else {
 			len = u8_nextlen(&s[i]);
 		totlen++;
 		if (i > n)
 			FATAL("bad utf count [%s] n=%d i=%d\n", s, n, i);
 	return totlen;
 /* convert utf-8 char number in a string to its byte offset */
 int u8_char2byte(const char *s, int charnum)
 	int n;
 	int bytenum = 0;
 	while (charnum > 0) {
 		n = u8_nextlen(s);
 		s += n;
 		bytenum += n;
 		charnum--;
 	return bytenum;
 /* convert byte offset in s to utf-8 char number that starts there */
 int u8_byte2char(const char *s, int bytenum)
 	int i, len, b;
 	int charnum = 0; /* BUG: what origin? */
 	/* should be 0 to match start==0 which means no match */
 	b = strlen(s);
 	if (bytenum >= b) {
 		return -1; /* ??? */
 	for (i = 0; i <= bytenum; i += len) {
 		len = u8_nextlen(s+i);
 		charnum++;
 	return charnum;
 /* runetochar() adapted from rune.c in the Plan 9 distributione */
 enum
 	Runeerror = 128, /* from somewhere else */
 	Runemax = 0x10FFFF,
 	Bit1    = 7,
 	Bitx    = 6,
 	Bit2    = 5,
 	Bit3    = 4,
 	Bit4    = 3,
 	Bit5    = 2,
 	T1      = ((1<<(Bit1+1))-1) ^ 0xFF,     /* 0000 0000 */
 	Tx      = ((1<<(Bitx+1))-1) ^ 0xFF,     /* 1000 0000 */
 	T2      = ((1<<(Bit2+1))-1) ^ 0xFF,     /* 1100 0000 */
 	T3      = ((1<<(Bit3+1))-1) ^ 0xFF,     /* 1110 0000 */
 	T4      = ((1<<(Bit4+1))-1) ^ 0xFF,     /* 1111 0000 */
 	T5      = ((1<<(Bit5+1))-1) ^ 0xFF,     /* 1111 1000 */
 	Rune1   = (1<<(Bit1+0*Bitx))-1,	 	/* 0000 0000 0000 0000 0111 1111 */
 	Rune2   = (1<<(Bit2+1*Bitx))-1,	 	/* 0000 0000 0000 0111 1111 1111 */
 	Rune3   = (1<<(Bit3+2*Bitx))-1,	 	/* 0000 0000 1111 1111 1111 1111 */
 	Rune4   = (1<<(Bit4+3*Bitx))-1,	 	/* 0011 1111 1111 1111 1111 1111 */
 	Maskx   = (1<<Bitx)-1,		  	/* 0011 1111 */
 	Testx   = Maskx ^ 0xFF,		 	/* 1100 0000 */
 };
 int runetochar(char *str, int c)
 	/* one character sequence 00000-0007F => 00-7F */
 	if (c <= Rune1) {
 		str[0] = c;
 		return 1;
 	/* two character sequence 00080-007FF => T2 Tx */
 	if (c <= Rune2) {
 		str[0] = T2 | (c >> 1*Bitx);
 		str[1] = Tx | (c & Maskx);
 		return 2;
 	/* three character sequence 00800-0FFFF => T3 Tx Tx */
 	if (c > Runemax)
 		c = Runeerror;
 	if (c <= Rune3) {
 		str[0] = T3 |  (c >> 2*Bitx);
 		str[1] = Tx | ((c >> 1*Bitx) & Maskx);
 		str[2] = Tx |  (c & Maskx);
 		return 3;
 	/* four character sequence 010000-1FFFFF => T4 Tx Tx Tx */
 	str[0] = T4 |  (c >> 3*Bitx);
 	str[1] = Tx | ((c >> 2*Bitx) & Maskx);
 	str[2] = Tx | ((c >> 1*Bitx) & Maskx);
 	str[3] = Tx |  (c & Maskx);
 	return 4;
 /* ========== end of utf8 code =========== */
 Cell *matchop(Node **a, int n)	/* ~ and match() */
+{
 	Cell *x, *y;
 	char *s, *t;
 	int i;
 	int cstart, cpatlen, len;
 	fa *pfa;
 	int (*mf)(fa *, const char *) = match, mode = 0;
 	if (n == MATCHFCN) {
 		mf = pmatch;
 		mode = 1;
+	}
 	x = execute(a[1]);	/* a[1] = target text */
 	s = getsval(x);
 	if (a[0] == NULL)	/* a[1] == 0: already-compiled reg expr */
 		i = (*mf)((fa *) a[2], s);
 	else {
 		y = execute(a[2]);	/* a[2] = regular expr */
 		t = getsval(y);
 		pfa = makedfa(t, mode);
 		i = (*mf)(pfa, s);
 		tempfree(y);
+	}
 	tempfree(x);
 	if (n == MATCHFCN) {
-		int start = patbeg - s + 1; /* origin 1 */
+		int start = patbeg - s + 1;
-		if (patlen < 0) {
+		if (patlen < 0)
-			start = 0; /* not found */
+			start = 0;
 		} else {
 			cstart = u8_byte2char(s, start-1);
 			cpatlen = 0;
 			for (i = 0; i < patlen; i += len) {
 				len = u8_nextlen(patbeg+i);
 				cpatlen++;
 			start = cstart;
 			patlen = cpatlen;
 		setfval(rstartloc, (Awkfloat) start);
 		setfval(rlengthloc, (Awkfloat) patlen);
 		x = gettemp();
 		x->tval = NUM;
 		x->fval = start;
 		return x;
 	} else if ((n == MATCH && i == 1) || (n == NOTMATCH && i == 0))
 		return(True);
 	else
 		return(False);
+}
 @@ -874,49 +647,43 @@ Cell *boolop(Node **a, int n)	/* a[0] ||
 		if (i) return(False);
 		else return(True);
 	default:	/* can't happen */
 		FATAL("unknown boolean operator %d", n);
+	}
 	return 0;	/*NOTREACHED*/
+}
 Cell *relop(Node **a, int n)	/* a[0 < a[1], etc. */
+{
 	int i;
 	Cell *x, *y;
 	Awkfloat j;
 	bool x_is_nan, y_is_nan;
 	x = execute(a[0]);
 	y = execute(a[1]);
 	x_is_nan = isnan(x->fval);
 	y_is_nan = isnan(y->fval);
 	if (x->tval&NUM && y->tval&NUM) {
 		if ((x_is_nan || y_is_nan) && n != NE)
 			return(False);
 		j = x->fval - y->fval;
 		i = j<0? -1: (j>0? 1: 0);
 	} else {
 		i = strcmp(getsval(x), getsval(y));
+	}
 	tempfree(x);
 	tempfree(y);
 	switch (n) {
 	case LT:	if (i<0) return(True);
 			else return(False);
 	case LE:	if (i<=0) return(True);
 			else return(False);
-	case NE:	if (x_is_nan && y_is_nan) return(True);
+	case NE:	if (i!=0) return(True);
 			else if (i!=0) return(True);
 			else return(False);
 	case EQ:	if (i == 0) return(True);
 			else return(False);
 	case GE:	if (i>=0) return(True);
 			else return(False);
 	case GT:	if (i>0) return(True);
 			else return(False);
 	default:	/* can't happen */
 		FATAL("unknown relational operator %d", n);
+	}
 	return 0;	/*NOTREACHED*/
+}
 @@ -965,27 +732,26 @@ Cell *indirect(Node **a, int n)	/* $( a[
 	if (m == 0 && !is_number(s = getsval(x), NULL))	/* suspicion! */
 		FATAL("illegal field $(%s), name \"%s\"", s, x->nval);
 		/* BUG: can x->nval ever be null??? */
 	tempfree(x);
 	x = fieldadr(m);
 	x->ctype = OCELL;	/* BUG?  why are these needed? */
 	x->csub = CFLD;
 	return(x);
+}
 Cell *substr(Node **a, int nnn)		/* substr(a[0], a[1], a[2]) */
+{
 	int k, m, n;
 	int mb, nb;
 	char *s;
 	int temp;
 	Cell *x, *y, *z = NULL;
 	x = execute(a[0]);
 	y = execute(a[1]);
 	if (a[2] != NULL)
 		z = execute(a[2]);
 	s = getsval(x);
 	k = strlen(s) + 1;
 	if (k <= 1) {
 		tempfree(x);
 		tempfree(y);
 @@ -1001,86 +767,62 @@ Cell *substr(Node **a, int nnn)		/* subs
 		m = 1;
 	else if (m > k)
 		m = k;
 	tempfree(y);
 	if (a[2] != NULL) {
 		n = (int) getfval(z);
 		tempfree(z);
 	} else
 		n = k - 1;
 	if (n < 0)
 		n = 0;
 	else if (n > k - m)
 		n = k - m;
 	/* m is start, n is length from there */
 	DPRINTF("substr: m=%d, n=%d, s=%s\n", m, n, s);
 	y = gettemp();
-	mb = u8_char2byte(s, m-1); /* byte offset of start char in s */
+	temp = s[n+m-1];	/* with thanks to John Linderman */
-	nb = u8_char2byte(s, m-1+n);  /* byte offset of end+1 char in s */
+	s[n+m-1] = '\0';
 	setsval(y, s + m - 1);
-	temp = s[nb];	/* with thanks to John Linderman */
+	s[n+m-1] = temp;
 	s[nb] = '\0';
 	setsval(y, s + mb);
 	s[nb] = temp;
 	tempfree(x);
 	return(y);
+}
 Cell *sindex(Node **a, int nnn)		/* index(a[0], a[1]) */
+{
 	Cell *x, *y, *z;
 	char *s1, *s2, *p1, *p2, *q;
 	Awkfloat v = 0.0;
 	x = execute(a[0]);
 	s1 = getsval(x);
 	y = execute(a[1]);
 	s2 = getsval(y);
 	z = gettemp();
 	for (p1 = s1; *p1 != '\0'; p1++) {
 		for (q = p1, p2 = s2; *p2 != '\0' && *q == *p2; q++, p2++)
 			continue;
 		if (*p2 == '\0') {
-			/* v = (Awkfloat) (p1 - s1 + 1);	 origin 1 */
+			v = (Awkfloat) (p1 - s1 + 1);	/* origin 1 */
 		   /* should be a function: used in match() as well */
 			int i, len;
 			v = 0;
 			for (i = 0; i < p1-s1+1; i += len) {
 				len = u8_nextlen(s1+i);
 				v++;
 			break;
+		}
+	}
 	tempfree(x);
 	tempfree(y);
 	setfval(z, v);
 	return(z);
+}
 int has_utf8(char *s)	/* return 1 if s contains any utf-8 (2 bytes or more) character */
 	int n;
 	for (n = 0; *s != 0; s += n) {
 		n = u8_nextlen(s);
 		if (n > 1)
 			return 1;
 	return 0;
 #define	MAXNUMSIZE	50
 int format(char **pbuf, int *pbufsize, const char *s, Node *a)	/* printf-like conversions */
+{
 	char *fmt;
 	char *p, *t;
 	const char *os;
 	Cell *x;
 	int flag = 0, n;
 	int fmtwd; /* format width */
 	int fmtsz = recsize;
 	char *buf = *pbuf;
 	int bufsize = *pbufsize;
 @@ -1103,26 +845,27 @@ int format(char **pbuf, int *pbufsize, c
 	if ((fmt = (char *) malloc(fmtsz)) == NULL)
 		FATAL("out of memory in format()");
 	while (*s) {
 		adjbuf(&buf, &bufsize, MAXNUMSIZE+1+p-buf, recsize, &p, "format1");
 		if (*s != '%') {
 			*p++ = *s++;
 			continue;
+		}
 		if (*(s+1) == '%') {
 			*p++ = '%';
 			s += 2;
 			continue;
+		}
 		/* have to be real careful in case this is a huge number, eg, %100000d */
 		fmtwd = atoi(s+1);
 		if (fmtwd < 0)
 			fmtwd = -fmtwd;
 		adjbuf(&buf, &bufsize, fmtwd+1+p-buf, recsize, &p, "format2");
 		for (t = fmt; (*t++ = *s) != '\0'; s++) {
 			if (!adjbuf(&fmt, &fmtsz, MAXNUMSIZE+1+t-fmt, recsize, &t, "format3"))
 				FATAL("format item %.30s... ran format() out of memory", os);
 			/* Ignore size specifiers */
 			if (strchr("hjLlqtz", *s) != NULL) {	/* the ansi panoply */
 				t--;
 				continue;
+			}
 			if (isalpha((uschar)*s))
 @@ -1175,211 +918,63 @@ int format(char **pbuf, int *pbufsize, c
 			WARNING("weird printf conversion %s", fmt);
 			flag = '?';
 			break;
+		}
 		if (a == NULL)
 			FATAL("not enough args in printf(%s)", os);
 		x = execute(a);
 		a = a->nnext;
 		n = MAXNUMSIZE;
 		if (fmtwd > n)
 			n = fmtwd;
 		adjbuf(&buf, &bufsize, 1+n+p-buf, recsize, &p, "format5");
 		switch (flag) {
-		case '?':
+		case '?':	snprintf(p, BUFSZ(p), "%s", fmt);	/* unknown, so dump it too */
 			snprintf(p, BUFSZ(p), "%s", fmt);	/* unknown, so dump it too */
 			t = getsval(x);
 			n = strlen(t);
 			if (fmtwd > n)
 				n = fmtwd;
 			adjbuf(&buf, &bufsize, 1+strlen(p)+n+p-buf, recsize, &p, "format6");
 			p += strlen(p);
 			snprintf(p, BUFSZ(p), "%s", t);
 			break;
 		case 'a':
 		case 'A':
 		case 'f':	snprintf(p, BUFSZ(p), fmt, getfval(x)); break;
 		case 'd':	snprintf(p, BUFSZ(p), fmt, (intmax_t) getfval(x)); break;
 		case 'u':	snprintf(p, BUFSZ(p), fmt, (uintmax_t) getfval(x)); break;
 		case 's':
 		case 's': {
 			t = getsval(x);
 			n = strlen(t);
-			/* if simple format or no utf-8 in the string, sprintf works */
+			if (fmtwd > n)
-			if (!has_utf8(t) || strcmp(fmt,"%s") == 0) {
+				n = fmtwd;
-				if (fmtwd > n)
+			if (!adjbuf(&buf, &bufsize, 1+n+p-buf, recsize, &p, "format7"))
-					n = fmtwd;
+				FATAL("huge string/format (%d chars) in printf %.30s... ran format() out of memory", n, t);
-				if (!adjbuf(&buf, &bufsize, 1+n+p-buf, recsize, &p, "format7"))
+			snprintf(p, BUFSZ(p), fmt, t);
 					FATAL("huge string/format (%d chars) in printf %.30s..." \
 						" ran format() out of memory", n, t);
 				snprintf(p, BUFSZ(p), fmt, t);
 				break;
 			/* get here if string has utf-8 chars and fmt is not plain %s */
 			/* "%-w.ps", where -, w and .p are all optional */
 			/* '0' before the w is a flag character */
 			/* fmt points at % */
 			int ljust = 0, wid = 0, prec = n, pad = 0;
 			char *f = fmt+1;
 			if (f[0] == '-') {
 				ljust = 1;
 				f++;
 			// flags '0' and '+' are recognized but skipped
 			if (f[0] == '0') {
 				f++;
 				if (f[0] == '+')
 					f++;
 			if (f[0] == '+') {
 				f++;
 				if (f[0] == '0')
 					f++;
 			if (isdigit(f[0])) { /* there is a wid */
 				wid = strtol(f, &f, 10);
 			if (f[0] == '.') { /* there is a .prec */
 				prec = strtol(++f, &f, 10);
 			if (prec > u8_strlen(t))
 				prec = u8_strlen(t);
 			pad = wid>prec ? wid - prec : 0;  // has to be >= 0
 			int i, k, n;
 			if (ljust) { // print prec chars from t, then pad blanks
 				n = u8_char2byte(t, prec);
 				for (k = 0; k < n; k++) {
 					//putchar(t[k]);
 					*p++ = t[k];
 				for (i = 0; i < pad; i++) {
 					//printf(" ");
 					*p++ = ' ';
 			} else { // print pad blanks, then prec chars from t
 				for (i = 0; i < pad; i++) {
 					//printf(" ");
 					*p++ = ' ';
 				n = u8_char2byte(t, prec);
 				for (k = 0; k < n; k++) {
 					//putchar(t[k]);
 					*p++ = t[k];
 			*p = 0;
 			break;
 		case 'c':
                case 'c': {
 			/*
 			 * If a numeric value is given, awk should just turn
 			 * it into a character and print it:
 			 *      BEGIN { printf("%c\n", 65) }
 			 * prints "A".
 			 * But what if the numeric value is > 128 and
 			 * represents a valid Unicode code point?!? We do
 			 * our best to convert it back into UTF-8. If we
 			 * can't, we output the encoding of the Unicode
 			 * "invalid character", 0xFFFD.
 			 */
 			if (isnum(x)) {
-				int charval = (int) getfval(x);
+				if ((int)getfval(x))
 					snprintf(p, BUFSZ(p), fmt, (int) getfval(x));
-				if (charval != 0) {
+				else {
 					if (charval < 128)
 						snprintf(p, BUFSZ(p), fmt, charval);
 					else {
 						// possible unicode character
 						size_t count;
 						char *bs = wide_char_to_byte_str(charval, &count);
 						if (bs == NULL)	{ // invalid character
 							// use unicode invalid character, 0xFFFD
 							bs = "\357\277\275";
 							count = 3;
 						t = bs;
 						n = count;
 						goto format_percent_c;
 				} else {
 					*p++ = '\0'; /* explicit null byte */
 					*p = '\0';   /* next output will start here */
+				}
-				break;
+			} else
 			t = getsval(x);
 			n = u8_nextlen(t);
 		format_percent_c:
 			if (n < 2) { /* not utf8 */
 				snprintf(p, BUFSZ(p), fmt, getsval(x)[0]);
 				break;
 			// utf8 character, almost same song and dance as for %s
 			int ljust = 0, wid = 0, prec = n, pad = 0;
 			char *f = fmt+1;
 			if (f[0] == '-') {
 				ljust = 1;
 				f++;
 			// flags '0' and '+' are recognized but skipped
 			if (f[0] == '0') {
 				f++;
 				if (f[0] == '+')
 					f++;
 			if (f[0] == '+') {
 				f++;
 				if (f[0] == '0')
 					f++;
 			if (isdigit(f[0])) { /* there is a wid */
 				wid = strtol(f, &f, 10);
 			if (f[0] == '.') { /* there is a .prec */
 				prec = strtol(++f, &f, 10);
 			if (prec > 1)           // %c --> only one character
 				prec = 1;
 			pad = wid>prec ? wid - prec : 0;  // has to be >= 0
 			int i;
 			if (ljust) { // print one char from t, then pad blanks
 				for (int i = 0; i < n; i++)
 					*p++ = t[i];
 				for (i = 0; i < pad; i++) {
 					//printf(" ");
 					*p++ = ' ';
 			} else { // print pad blanks, then prec chars from t
 				for (i = 0; i < pad; i++) {
 					//printf(" ");
 					*p++ = ' ';
 				for (int i = 0; i < n; i++)
 					*p++ = t[i];
 			*p = 0;
 			break;
 		default:
 			FATAL("can't happen: bad conversion %c in format()", flag);
+		}
 		tempfree(x);
 		p += strlen(p);
 		s++;
+	}
 	*p = '\0';
 	free(fmt);
 	for ( ; a; a = a->nnext) {		/* evaluate any remaining args */
 		x = execute(a);
 		tempfree(x);
+	}
 	*pbuf = buf;
 	*pbufsize = bufsize;
 	return p - buf;
 @@ -1660,47 +1255,44 @@ Cell *dopa2(Node **a, int n)	/* a[0], a[
+	}
 	return(False);
+}
 Cell *split(Node **a, int nnn)	/* split(a[0], a[1], a[2]); a[3] is type */
+{
 	Cell *x = NULL, *y, *ap;
 	const char *s, *origs, *t;
 	const char *fs = NULL;
 	char *origfs = NULL;
 	int sep;
 	char temp, num[50];
 	int n, tempstat, arg3type;
 	int j;
 	double result;
 	y = execute(a[0]);	/* source string */
 	origs = s = strdup(getsval(y));
 	tempfree(y);
 	arg3type = ptoi(a[3]);
-	if (a[2] == NULL) {		/* BUG: CSV should override implicit fs but not explicit */
+	if (a[2] == NULL)		/* fs string */
 		fs = getsval(fsloc);
-	} else if (arg3type == STRING) {	/* split(str,arr,"string") */
+	else if (arg3type == STRING) {	/* split(str,arr,"string") */
 		x = execute(a[2]);
 		fs = origfs = strdup(getsval(x));
 		tempfree(x);
-	} else if (arg3type == REGEXPR) {
+	} else if (arg3type == REGEXPR)
 		fs = "(regexpr)";	/* split(str,arr,/regexpr/) */
-	} else {
+	else
 		FATAL("illegal type of split");
 	sep = *fs;
 	ap = execute(a[1]);	/* array name */
 /* BUG 7/26/22: this appears not to reset array: see C1/asplit */
 	freesymtab(ap);
 	DPRINTF("split: s=|%s|, a=%s, sep=|%s|\n", s, NN(ap->nval), fs);
 	ap->tval &= ~STR;
 	ap->tval |= ARR;
 	ap->sval = (char *) makesymtab(NSYMTAB);
 	n = 0;
         if (arg3type == REGEXPR && strlen((char*)((fa*)a[2])->restr) == 0) {
 		/* split(s, a, //); have to arrange that it looks like empty sep */
 		arg3type = 0;
 		fs = "";
 		sep = 0;
+	}
 @@ -1734,102 +1326,62 @@ Cell *split(Node **a, int nnn)	/* split(
+				}
 			} while (nematch(pfa,s));
 			pfa->initstat = tempstat; 	/* bwk: has to be here to reset */
 							/* cf gsub and refldbld */
+		}
 		n++;
 		snprintf(num, sizeof(num), "%d", n);
 		if (is_number(s, & result))
 			setsymtab(num, s, result, STR|NUM, (Array *) ap->sval);
 		else
 			setsymtab(num, s, 0.0, STR, (Array *) ap->sval);
   spdone:
 		pfa = NULL;
 	} else if (sep == ' ') {
 	} else if (a[2] == NULL && CSV) {	/* CSV only if no explicit separator */
 		char *newt = (char *) malloc(strlen(s)); /* for building new string; reuse for each field */
 		for (;;) {
 			char *fr = newt;
 			n++;
 			if (*s == '"' ) { /* start of "..." */
 				for (s++ ; *s != '\0'; ) {
 					if (*s == '"' && s[1] != '\0' && s[1] == '"') {
 						s += 2; /* doubled quote */
 						*fr++ = '"';
 					} else if (*s == '"' && (s[1] == '\0' || s[1] == ',')) {
 						s++; /* skip over closing quote */
 						break;
 					} else {
 						*fr++ = *s++;
 				*fr++ = 0;
 			} else {	/* unquoted field */
 				while (*s != ',' && *s != '\0')
 					*fr++ = *s++;
 				*fr++ = 0;
 			snprintf(num, sizeof(num), "%d", n);
 			if (is_number(newt, &result))
 				setsymtab(num, newt, result, STR|NUM, (Array *) ap->sval);
 			else
 				setsymtab(num, newt, 0.0, STR, (Array *) ap->sval);
 			if (*s++ == '\0')
 				break;
 		free(newt);
 	} else if (!CSV && sep == ' ') { /* usual case: split on white space */
 		for (n = 0; ; ) {
 #define ISWS(c)	((c) == ' ' || (c) == '\t' || (c) == '\n')
 			while (ISWS(*s))
 				s++;
 			if (*s == '\0')
 				break;
 			n++;
 			t = s;
 			do
 				s++;
 			while (*s != '\0' && !ISWS(*s));
 			temp = *s;
 			setptr(s, '\0');
 			snprintf(num, sizeof(num), "%d", n);
 			if (is_number(t, & result))
 				setsymtab(num, t, result, STR|NUM, (Array *) ap->sval);
 			else
 				setsymtab(num, t, 0.0, STR, (Array *) ap->sval);
 			setptr(s, temp);
 			if (*s != '\0')
 				s++;
+		}
 	} else if (sep == 0) {	/* new: split(s, a, "") => 1 char/elem */
-		for (n = 0; *s != '\0'; s += u8_nextlen(s)) {
+		for (n = 0; *s != '\0'; s++) {
-			char buf[10];
+			char buf[2];
 			n++;
 			snprintf(num, sizeof(num), "%d", n);
 			buf[0] = *s;
-			for (j = 0; j < u8_nextlen(s); j++) {
+			buf[1] = '\0';
 				buf[j] = s[j];
 			buf[j] = '\0';
 			if (isdigit((uschar)buf[0]))
 				setsymtab(num, buf, atof(buf), STR|NUM, (Array *) ap->sval);
 			else
 				setsymtab(num, buf, 0.0, STR, (Array *) ap->sval);
+		}
 	} else if (*s != '\0') {
 	} else if (*s != '\0') {  /* some random single character */
 		for (;;) {
 			n++;
 			t = s;
 			while (*s != sep && *s != '\n' && *s != '\0')
 				s++;
 			temp = *s;
 			setptr(s, '\0');
 			snprintf(num, sizeof(num), "%d", n);
 			if (is_number(t, & result))
 				setsymtab(num, t, result, STR|NUM, (Array *) ap->sval);
 			else
 				setsymtab(num, t, 0.0, STR, (Array *) ap->sval);
 			setptr(s, temp);
 @@ -1968,47 +1520,46 @@ Cell *instat(Node **a, int n)	/* for (a[
+	}
 	return True;
+}
 static char *nawk_convert(const char *s, int (*fun_c)(int),
     wint_t (*fun_wc)(wint_t))
+{
 	char *buf      = NULL;
 	char *pbuf     = NULL;
 	const char *ps = NULL;
 	size_t n       = 0;
 	wchar_t wc;
 	size_t sz = MB_CUR_MAX;
 	int unused;
 	if (sz == 1) {
 		buf = tostring(s);
 		for (pbuf = buf; *pbuf; pbuf++)
 			*pbuf = fun_c((uschar)*pbuf);
 		return buf;
 	} else {
 		/* upper/lower character may be shorter/longer */
 		buf = tostringN(s, strlen(s) * sz + 1);
 		(void) mbtowc(NULL, NULL, 0);	/* reset internal state */
 		/*
 		 * Reset internal state here too.
 		 * Assign result to avoid a compiler warning. (Casting to void
 		 * doesn't work.)
 		 * Increment said variable to avoid a different warning.
 		 */
-		unused = wctomb(NULL, L'\0');
+		int unused = wctomb(NULL, L'\0');
 		unused++;
 		ps   = s;
 		pbuf = buf;
 		while (n = mbtowc(&wc, ps, sz),
 		       n > 0 && n != (size_t)-1 && n != (size_t)-2)
+		{
 			ps += n;
 			n = wctomb(pbuf, fun_wc(wc));
 			if (n == (size_t)-1)
 				FATAL("illegal wide character %s", s);
 @@ -2042,48 +1593,46 @@ static wint_t towlower(wint_t wc)
+}
 #endif
 static char *nawk_toupper(const char *s)
+{
 	return nawk_convert(s, toupper, towupper);
+}
 static char *nawk_tolower(const char *s)
+{
 	return nawk_convert(s, tolower, towlower);
+}
 Cell *bltin(Node **a, int n)	/* builtin functions. a[0] is type, a[1] is arg list */
+{
 	Cell *x, *y;
 	Awkfloat u;
 	int t;
 	Awkfloat tmp;
 	char *buf;
 	Node *nextarg;
 	FILE *fp;
 	int status = 0;
 	t = ptoi(a[0]);
 	x = execute(a[1]);
 	nextarg = a[1]->nnext;
 	switch (t) {
 	case FLENGTH:
 		if (isarr(x))
 			u = ((Array *) x->sval)->nelem;	/* GROT.  should be function*/
 		else
-			u = u8_strlen(getsval(x));
+			u = strlen(getsval(x));
 		break;
 	case FLOG:
 		errno = 0;
 		u = errcheck(log(getfval(x)), "log");
 		break;
 	case FINT:
 		modf(getfval(x), &u); break;
 	case FEXP:
 		errno = 0;
 		u = errcheck(exp(getfval(x)), "exp");
 		break;
 	case FSQRT:
 		errno = 0;
 @@ -2585,51 +2134,13 @@ void backsub(char **pb_ptr, const char *
 		} else {			/* \\x -> \\x */
 			*pb++ = *sptr++;
 			*pb++ = *sptr++;
+		}
 	} else if (sptr[1] == '&') {	/* literal & */
 		sptr++;
 		*pb++ = *sptr++;
 	} else				/* literal \ */
 		*pb++ = *sptr++;
 	*pb_ptr = pb;
 	*sptr_ptr = sptr;
+}
 static char *wide_char_to_byte_str(int rune, size_t *outlen)
 	static char buf[5];
 	int len;
 	if (rune < 0 || rune > 0x10FFFF)
 		return NULL;
 	memset(buf, 0, sizeof(buf));
 	len = 0;
 	if (rune <= 0x0000007F) {
 		buf[len++] = rune;
 	} else if (rune <= 0x000007FF) {
 		// 110xxxxx 10xxxxxx
 		buf[len++] = 0xC0 | (rune >> 6);
 		buf[len++] = 0x80 | (rune & 0x3F);
 	} else if (rune <= 0x0000FFFF) {
 		// 1110xxxx 10xxxxxx 10xxxxxx
 		buf[len++] = 0xE0 | (rune >> 12);
 		buf[len++] = 0x80 | ((rune >> 6) & 0x3F);
 		buf[len++] = 0x80 | (rune & 0x3F);
 	} else {
 		// 0x00010000 - 0x10FFFF
 		// 11110xxx 10xxxxxx 10xxxxxx 10xxxxxx
 		buf[len++] = 0xF0 | (rune >> 18);
 		buf[len++] = 0x80 | ((rune >> 12) & 0x3F);
 		buf[len++] = 0x80 | ((rune >> 6) & 0x3F);
 		buf[len++] = 0x80 | (rune & 0x3F);
 	*outlen = len;
 	buf[len++] = '\0';
 	return buf;

 @@ -24,28 +24,26 @@ THIS SOFTWARE.
 #define DEBUG
 #include <stdio.h>
 #include <string.h>
 #include <strings.h>
 #include <ctype.h>
 #include <errno.h>
 #include <stdlib.h>
 #include <stdarg.h>
 #include <limits.h>
 #include <math.h>
 #include "awk.h"
 extern int u8_nextlen(const char *s);
 char	EMPTY[] = { '\0' };
 FILE	*infile	= NULL;
 bool	innew;		/* true = infile has not been read by readrec */
 char	*file	= EMPTY;
 char	*record;
 int	recsize	= RECSIZE;
 char	*fields;
 int	fieldssize = RECSIZE;
 Cell	**fldtab;	/* pointers to Cells */
 static size_t	len_inputFS = 0;
 static char	*inputFS = NULL; /* FS at time of input, for field splitting */
 @@ -211,54 +209,48 @@ int getrec(char **pbuf, int *pbufsize, b
 	*pbuf = buf;
 	*pbufsize = savebufsize;
 	return 0;	/* true end of file */
+}
 void nextfile(void)
+{
 	if (infile != NULL && infile != stdin)
 		fclose(infile);
 	infile = NULL;
 	argno++;
+}
 extern int readcsvrec(char **pbuf, int *pbufsize, FILE *inf, bool newflag);
 int readrec(char **pbuf, int *pbufsize, FILE *inf, bool newflag)	/* read one record into buf */
+{
-	int sep, c, isrec; // POTENTIAL BUG? isrec is a macro in awk.h
+	int sep, c, isrec;
-	char *rr = *pbuf, *buf = *pbuf;
+	char *rr, *buf = *pbuf;
 	int bufsize = *pbufsize;
 	char *rs = getsval(rsloc);
-	if (CSV) {
+	if (*rs && rs[1]) {
 		c = readcsvrec(pbuf, pbufsize, inf, newflag);
 		isrec = (c == EOF && rr == buf) ? false : true;
 	} else if (*rs && rs[1]) {
 		bool found;
 		fa *pfa = makedfa(rs, 1);
 		if (newflag)
 			found = fnematch(pfa, inf, &buf, &bufsize, recsize);
 		else {
 			int tempstat = pfa->initstat;
 			pfa->initstat = 2;
 			found = fnematch(pfa, inf, &buf, &bufsize, recsize);
 			pfa->initstat = tempstat;
+		}
 		if (found)
 			setptr(patbeg, '\0');
 		isrec = (found == 0 && *buf == '\0') ? false : true;
 	} else {
 		if ((sep = *rs) == 0) {
 			sep = '\n';
 			while ((c=getc(inf)) == '\n' && c != EOF)	/* skip leading \n's */
+				;
 			if (c != EOF)
 				ungetc(c, inf);
+		}
 		for (rr = buf; ; ) {
 			for (; (c=getc(inf)) != sep && c != EOF; ) {
 				if (rr-buf+1 > bufsize)
 					if (!adjbuf(&buf, &bufsize, 1+rr-buf,
 					    recsize, &rr, "readrec 1"))
 @@ -276,96 +268,47 @@ int readrec(char **pbuf, int *pbufsize,
 			*rr++ = c;
+		}
 		if (!adjbuf(&buf, &bufsize, 1+rr-buf, recsize, &rr, "readrec 3"))
 			FATAL("input record `%.30s...' too long", buf);
 		*rr = 0;
 		isrec = (c == EOF && rr == buf) ? false : true;
+	}
 	*pbuf = buf;
 	*pbufsize = bufsize;
 	DPRINTF("readrec saw <%s>, returns %d\n", buf, isrec);
 	return isrec;
+}
 /*******************
  * loose ends here:
  *   \r\n should become \n
  *   what about bare \r?  Excel uses that for embedded newlines
  *   can't have "" in unquoted fields, according to RFC 4180
 */
 int readcsvrec(char **pbuf, int *pbufsize, FILE *inf, bool newflag) /* csv can have \n's */
 {			/* so read a complete record that might be multiple lines */
 	int sep, c;
 	char *rr = *pbuf, *buf = *pbuf;
 	int bufsize = *pbufsize;
 	bool in_quote = false;
 	sep = '\n'; /* the only separator; have to skip over \n embedded in "..." */
 	rr = buf;
 	while ((c = getc(inf)) != EOF) {
 		if (c == sep) {
 			if (! in_quote)
 				break;
 			if (rr > buf && rr[-1] == '\r')	// remove \r if was \r\n
 				rr--;
 		if (rr-buf+1 > bufsize)
 			if (!adjbuf(&buf, &bufsize, 1+rr-buf,
 			    recsize, &rr, "readcsvrec 1"))
 				FATAL("input record `%.30s...' too long", buf);
 		*rr++ = c;
 		if (c == '"')
 			in_quote = ! in_quote;
 	if (c == '\n' && rr > buf && rr[-1] == '\r') 	// remove \r if was \r\n
 		rr--;
 	if (!adjbuf(&buf, &bufsize, 1+rr-buf, recsize, &rr, "readcsvrec 4"))
 		FATAL("input record `%.30s...' too long", buf);
 	*rr = 0;
 	*pbuf = buf;
 	*pbufsize = bufsize;
 	DPRINTF("readcsvrec saw <%s>, returns %d\n", buf, c);
 	return c;
 char *getargv(int n)	/* get ARGV[n] */
+{
 	Cell *x;
 	char *s, temp[50];
 	extern Array *ARGVtab;
 	snprintf(temp, sizeof(temp), "%d", n);
 	if (lookup(temp, ARGVtab) == NULL)
 		return NULL;
 	x = setsymtab(temp, "", 0.0, STR, ARGVtab);
 	s = getsval(x);
 	DPRINTF("getargv(%d) returns |%s|\n", n, s);
 	return s;
+}
 void setclvar(char *s)	/* set var=value from s */
+{
 	char *e, *p;
 	Cell *q;
 	double result;
 /* commit f3d9187d4e0f02294fb1b0e31152070506314e67 broke T.argv test */
 /* I don't understand why it was changed. */
 	for (p=s; *p != '='; p++)
+		;
 	e = p;
 	*p++ = 0;
 	p = qstring(p, '\0');
 	q = setsymtab(s, p, 0.0, STR, symtab);
 	setsval(q, p);
 	if (is_number(q->sval, & result)) {
 		q->fval = result;
 		q->tval |= NUM;
+	}
 	DPRINTF("command line set %s to |%s|\n", s, p);
 	free(p);
 @@ -390,97 +333,65 @@ void fldbld(void)	/* create fields from
 	n = strlen(r);
 	if (n > fieldssize) {
 		xfree(fields);
 		if ((fields = (char *) malloc(n+2)) == NULL) /* possibly 2 final \0s */
 			FATAL("out of space for fields in fldbld %d", n);
 		fieldssize = n;
+	}
 	fr = fields;
 	i = 0;	/* number of fields accumulated here */
 	if (inputFS == NULL)	/* make sure we have a copy of FS */
 		savefs();
 	if (strlen(inputFS) > 1) {	/* it's a regular expression */
 		i = refldbld(r, inputFS);
-	} else if (!CSV && (sep = *inputFS) == ' ') {	/* default whitespace */
+	} else if ((sep = *inputFS) == ' ') {	/* default whitespace */
 		for (i = 0; ; ) {
 			while (*r == ' ' || *r == '\t' || *r == '\n')
 				r++;
 			if (*r == 0)
 				break;
 			i++;
 			if (i > nfields)
 				growfldtab(i);
 			if (freeable(fldtab[i]))
 				xfree(fldtab[i]->sval);
 			fldtab[i]->sval = fr;
 			fldtab[i]->tval = FLD | STR | DONTFREE;
 			do
 				*fr++ = *r++;
 			while (*r != ' ' && *r != '\t' && *r != '\n' && *r != '\0');
 			*fr++ = 0;
+		}
 		*fr = 0;
-	} else if (CSV) {	/* CSV processing.  no error handling */
+	} else if ((sep = *inputFS) == 0) {		/* new: FS="" => 1 char/field */
-		if (*r != 0) {
+		for (i = 0; *r != '\0'; r += n) {
-			for (;;) {
+			char buf[MB_LEN_MAX + 1];
 				i++;
 				if (i > nfields)
 					growfldtab(i);
 				if (freeable(fldtab[i]))
 					xfree(fldtab[i]->sval);
 				fldtab[i]->sval = fr;
 				fldtab[i]->tval = FLD | STR | DONTFREE;
 				if (*r == '"' ) { /* start of "..." */
 					for (r++ ; *r != '\0'; ) {
 						if (*r == '"' && r[1] != '\0' && r[1] == '"') {
 							r += 2; /* doubled quote */
 							*fr++ = '"';
 						} else if (*r == '"' && (r[1] == '\0' || r[1] == ',')) {
 							r++; /* skip over closing quote */
 							break;
 						} else {
 							*fr++ = *r++;
 					*fr++ = 0;
 				} else {	/* unquoted field */
 					while (*r != ',' && *r != '\0')
 						*fr++ = *r++;
 					*fr++ = 0;
 				if (*r++ == 0)
 					break;
 		*fr = 0;
 	} else if ((sep = *inputFS) == 0) {	/* new: FS="" => 1 char/field */
 		for (i = 0; *r != '\0'; ) {
 			char buf[10];
 			i++;
 			if (i > nfields)
 				growfldtab(i);
 			if (freeable(fldtab[i]))
 				xfree(fldtab[i]->sval);
-			n = u8_nextlen(r);
+			n = mblen(r, MB_LEN_MAX);
-			for (j = 0; j < n; j++)
+			if (n < 0)
-				buf[j] = *r++;
+				n = 1;
-			buf[j] = '\0';
+			memcpy(buf, r, n);
 			buf[n] = '\0';
 			fldtab[i]->sval = tostring(buf);
 			fldtab[i]->tval = FLD | STR;
+		}
 		*fr = 0;
 	} else if (*r != 0) {	/* if 0, it's a null field */
-		/* subtle case: if length(FS) == 1 && length(RS > 0)
+		/* subtlecase : if length(FS) == 1 && length(RS > 0)
 		 * \n is NOT a field separator (cf awk book 61,84).
 		 * this variable is tested in the inner while loop.
 		 */
 		int rtest = '\n';  /* normal case */
 		if (strlen(*RS) > 0)
 			rtest = '\0';
 		for (;;) {
 			i++;
 			if (i > nfields)
 				growfldtab(i);
 			if (freeable(fldtab[i]))
 				xfree(fldtab[i]->sval);
 			fldtab[i]->sval = fr;
 @@ -875,31 +786,31 @@ bool is_valid_number(const char *s, bool
+{
 	double r;
 	char *ep;
 	bool retval = false;
 	bool is_nan = false;
 	bool is_inf = false;
 	if (no_trailing)
 		*no_trailing = false;
 	while (isspace(*s))
 		s++;
-	/* no hex floating point, sorry */
+	// no hex floating point, sorry
 	if (s[0] == '0' && tolower(s[1]) == 'x')
 		return false;
-	/* allow +nan, -nan, +inf, -inf, any other letter, no */
+	// allow +nan, -nan, +inf, -inf, any other letter, no
 	if (s[0] == '+' || s[0] == '-') {
 		is_nan = (strncasecmp(s+1, "nan", 3) == 0);
 		is_inf = (strncasecmp(s+1, "inf", 3) == 0);
 		if ((is_nan || is_inf)
 		    && (isspace(s[4]) || s[4] == '\0'))
 			goto convert;
 		else if (! isdigit(s[1]) && s[1] != '.')
 			return false;
+	}
 	else if (! isdigit(s[0]) && s[0] != '.')
 		return false;
 convert:
 @@ -913,18 +824,18 @@ convert:
 	if (result != NULL)
 		*result = r;
 	/*
 	 * check for trailing stuff
 	 */
 	while (isspace(*ep))
 		ep++;
 	if (no_trailing != NULL)
 		*no_trailing = (*ep == '\0');
-        /* return true if found the end, or trailing stuff is allowed */
+        // return true if found the end, or trailing stuff is allowed
 	retval = *ep == '\0' || trailing_stuff_ok;
 	return retval;
+}

 @@ -33,33 +33,34 @@ extern	int	yylex(void);
 extern	void	startreg(void);
 extern	int	input(void);
 extern	void	unput(int);
 extern	void	unputstr(const char *);
 extern	int	yylook(void);
 extern	int	yyback(int *, int);
 extern	int	yyinput(void);
 extern	fa	*makedfa(const char *, bool);
 extern	fa	*mkdfa(const char *, bool);
 extern	int	makeinit(fa *, bool);
 extern	void	penter(Node *);
 extern	void	freetr(Node *);
 extern	int	hexstr(const uschar **);
 extern	int	quoted(const uschar **);
-extern	int	*cclenter(const char *);
+extern	char	*cclenter(const char *);
 extern	noreturn void	overflo(const char *);
 extern	void	cfoll(fa *, Node *);
 extern	int	first(Node *);
 extern	void	follow(Node *);
-extern	int	member(int, int *);
+extern	int	member(int, const char *);
 extern	int	match(fa *, const char *);
 extern	int	pmatch(fa *, const char *);
 extern	int	nematch(fa *, const char *);
 extern	bool	fnematch(fa *, FILE *, char **, int *, int);
 extern	Node	*reparse(const char *);
 extern	Node	*regexp(void);
 extern	Node	*primary(void);
 extern	Node	*concat(Node *);
 extern	Node	*alt(Node *);
 extern	Node	*unary(Node *);
 extern	int	relex(void);
 extern	int	cgoto(fa *, int, int);
 extern	void	freefa(fa *);

 @@ -298,27 +298,27 @@ Awkfloat setfval(Cell *vp, Awkfloat f)	/
 	f += 0.0;		/* normalise negative zero to positive zero */
 	if ((vp->tval & (NUM | STR)) == 0)
 		funnyvar(vp, "assign to");
 	if (isfld(vp)) {
 		donerec = false;	/* mark $0 invalid */
 		fldno = atoi(vp->nval);
 		if (fldno > *NF)
 			newfld(fldno);
 		DPRINTF("setting field %d to %g\n", fldno, f);
 	} else if (&vp->fval == NF) {
 		donerec = false;	/* mark $0 invalid */
 		setlastfld(f);
-		DPRINTF("setfval: setting NF to %g\n", f);
+		DPRINTF("setting NF to %g\n", f);
 	} else if (isrec(vp)) {
 		donefld = false;	/* mark $1... invalid */
 		donerec = true;
 		savefs();
 	} else if (vp == ofsloc) {
 		if (!donerec)
 			recbld();
+	}
 	if (freeable(vp))
 		xfree(vp->sval); /* free any previous string */
 	vp->tval &= ~(STR|CONVC|CONVO); /* mark string invalid */
 	vp->fmt = NULL;
 	vp->tval |= NUM;	/* mark number ok */
 @@ -338,30 +338,26 @@ void funnyvar(Cell *vp, const char *rw)
 		(void *)vp, vp->nval, vp->sval, vp->fval, vp->tval);
+}
 char *setsval(Cell *vp, const char *s)	/* set string val of a Cell */
+{
 	char *t;
 	int fldno;
 	Awkfloat f;
 	DPRINTF("starting setsval %p: %s = \"%s\", t=%o, r,f=%d,%d\n",
 		(void*)vp, NN(vp->nval), s, vp->tval, donerec, donefld);
 	if ((vp->tval & (NUM | STR)) == 0)
 		funnyvar(vp, "assign to");
 	if (CSV && (vp == rsloc))
 		WARNING("danger: don't set RS when --csv is in effect");
 	if (CSV && (vp == fsloc))
 		WARNING("danger: don't set FS when --csv is in effect");
 	if (isfld(vp)) {
 		donerec = false;	/* mark $0 invalid */
 		fldno = atoi(vp->nval);
 		if (fldno > *NF)
 			newfld(fldno);
 		DPRINTF("setting field %d to %s (%p)\n", fldno, s, (const void*)s);
 	} else if (isrec(vp)) {
 		donefld = false;	/* mark $1... invalid */
 		donerec = true;
 		savefs();
 	} else if (vp == ofsloc) {
 		if (!donerec)
 			recbld();
 @@ -369,27 +365,27 @@ char *setsval(Cell *vp, const char *s)	/
 	t = s ? tostring(s) : tostring("");	/* in case it's self-assign */
 	if (freeable(vp))
 		xfree(vp->sval);
 	vp->tval &= ~(NUM|DONTFREE|CONVC|CONVO);
 	vp->tval |= STR;
 	vp->fmt = NULL;
 	DPRINTF("setsval %p: %s = \"%s (%p) \", t=%o r,f=%d,%d\n",
 		(void*)vp, NN(vp->nval), t, (void*)t, vp->tval, donerec, donefld);
 	vp->sval = t;
 	if (&vp->fval == NF) {
 		donerec = false;	/* mark $0 invalid */
 		f = getfval(vp);
 		setlastfld(f);
-		DPRINTF("setsval: setting NF to %g\n", f);
+		DPRINTF("setting NF to %g\n", f);
+	}
 	return(vp->sval);
+}
 Awkfloat getfval(Cell *vp)	/* get float val of a Cell */
+{
 	if ((vp->tval & (NUM | STR)) == 0)
 		funnyvar(vp, "read value of");
 	if (isfld(vp) && !donefld)
 		fldbld();
 	else if (isrec(vp) && !donerec)
 		recbld();

 @@ -1,46 +1,44 @@
-.\" $NetBSD: nawk.1,v 1.3 2023/09/12 19:16:52 vins Exp $
+.\" $NetBSD: nawk.1,v 1.4 2023/09/17 10:32:06 vins Exp $
 .\"
-.\" This file is copied from awk.1 but with the following modifications
+.\" This file is copied from nawk.1 but with the following modifications
 .\" for pkgsrc:
 .\"
-.\"    * awk is changed to nawk.
+.\"    * nawk is changed to nnawk.
-.\"    * Awk is changed to Nawk.
+.\"    * Nawk is changed to Nnawk.
-.\"    * AWK is changed to NAWK.
+.\"    * NAWK is changed to NNAWK.
 .\"
 .de EX
 .nf
 .ft CW
 ..
 .de EE
 .br
 .fi
 .ft 1
 ..
 .de TF
 .IP "" "\w'\fB\\$1\ \ \fP'u"
 .PD 0
 ..
 .TH NAWK 1
 .CT 1 files prog_other
 .SH NAME
 nawk \- pattern-directed scanning and processing language
 .SH SYNOPSIS
 .B nawk
+[
 .BI \-F
 .I fs
 .B \-\^\-csv
+]
+[
 .BI \-v
 .I var=value
+]
+[
 .I 'prog'
+|
 .BI \-f
 .I progfile
+]
+[
 .I file ...
 @@ -77,32 +75,26 @@ The option
 followed by
 .I var=value
 is an assignment to be done before
 .I prog
 is executed;
 any number of
 .B \-v
 options may be present.
 The
 .B \-F
 .I fs
 option defines the input field separator to be the regular expression
 .IR fs .
 The
 .B \-\^\-csv
 option causes
 .I nawk
 to process records using (more or less) standard comma-separated values
 (CSV) format.
 .PP
 An input line is normally made up of fields separated by white space,
 or by the regular expression
 .BR FS .
 The fields are denoted
 .BR $1 ,
 .BR $2 ,
 \&..., while
 .B $0
 refers to the entire line.
 If
 .BR FS
 is null, the input line is split into one field per character.
 @@ -209,45 +201,45 @@ The built-in function
 flushes any buffered output for the file or pipe
 .IR expr .
 .PP
 The mathematical functions
 .BR atan2 ,
 .BR cos ,
 .BR exp ,
 .BR log ,
 .BR sin ,
 and
 .B sqrt
 are built in.
 Other built-in functions:
-.TF "\fBlength(\fR[\fIv\^\fR]\fB)\fR"
+.TF length
 .TP
-\fBlength(\fR[\fIv\^\fR]\fB)\fR
+.B length
 the length of its argument
 taken as a string,
 number of elements in an array for an array argument,
 or length of
 .B $0
 if no argument.
 .TP
-.B rand()
+.B rand
 random number on [0,1).
 .TP
-\fBsrand(\fR[\fIs\^\fR]\fB)\fR
+.B srand
 sets seed for
 .B rand
 and returns the previous seed.
 .TP
-.BI int( x\^ )
+.B int
 truncates to an integer value.
 .TP
 \fBsubstr(\fIs\fB, \fIm\fR [\fB, \fIn\^\fR]\fB)\fR
 the
 .IR n -character
 substring of
 .I s
 that begins at position
 .I m
 counted from 1.
 If no
 .IR n ,
 use the rest of the string.
 @@ -396,37 +388,37 @@ Regular expressions may also occur in
 relational expressions, using the operators
 .B ~
 and
 .BR !~ .
 .BI / re /
 is a constant regular expression;
 any string (constant or variable) may be used
 as a regular expression, except in the position of an isolated regular expression
 in a pattern.
 .PP
 A pattern may consist of two patterns separated by a comma;
 in this case, the action is performed for all lines
 from an occurrence of the first pattern
-through an occurrence of the second, inclusive.
+though an occurrence of the second.
 .PP
 A relational expression is one of the following:
 .IP
 .I expression matchop regular-expression
 .br
 .I expression relop expression
 .br
 .IB expression " in " array-name
 .br
-.BI ( expr ,\| expr ,\| ... ") in " array-name
+.BI ( expr , expr,... ") in " array-name
 .PP
 where a
 .I relop
 is any of the six relational operators in C,
 and a
 .I matchop
 is either
 .B ~
 (matches)
 or
 .B !~
 (does not match).
 A conditional is an arithmetic expression,
 @@ -506,27 +498,27 @@ is treated as a regular expression, and
 separated by text matching the expression.
 .TP
 .B RSTART
 the start position of a string matched by
 .BR match .
 .TP
 .B SUBSEP
 separates multiple subscripts (default 034).
 .PD
 .PP
 Functions may be defined (at the position of a pattern-action statement) thus:
 .IP
 .B
-function foo(a, b, c) { ... }
+function foo(a, b, c) { ...; return x }
 .PP
 Parameters are passed by value if scalar and by reference if array name;
 functions may be called recursively.
 Parameters are local to the function; all other variables are global.
 Thus local variables may be created by providing excess parameters in
 the function definition.
 .SH ENVIRONMENT VARIABLES
 If
 .B POSIXLY_CORRECT
 is set in the environment, then
 .I nawk
 follows the POSIX rules for
 .B sub
 @@ -572,39 +564,38 @@ Print all lines between start/stop pairs
 .nf
 BEGIN	{	# Simulate echo(1)
 	for (i = 1; i < ARGC; i++) printf "%s ", ARGV[i]
 	printf "\en"
 	exit }
 .fi
 .EE
 .SH SEE ALSO
 .IR grep (1),
 .IR lex (1),
 .IR sed (1)
 .br
 A. V. Aho, B. W. Kernighan, P. J. Weinberger,
-.IR "The NAWK Programming Language, Second Edition" ,
+.IR "The NAWK Programming Language" ,
-Addison-Wesley, 2024.  ISBN 978-0-13-826972-2, 0-13-826972-6.
+Addison-Wesley, 1988.  ISBN 0-201-07981-X.
 .SH BUGS
 There are no explicit conversions between numbers and strings.
 To force an expression to be treated as a number add 0 to it;
 to force it to be treated as a string concatenate
 \&\f(CW""\fP to it.
 .PP
 The scope rules for variables in functions are a botch;
 the syntax is worse.
 .PP
-Input is expected to be UTF-8 encoded. Other multibyte
+Only eight-bit characters sets are handled correctly.
 character sets are not handled.
 .SH UNUSUAL FLOATING-POINT VALUES
 .I Nawk
 was designed before IEEE 754 arithmetic defined Not-A-Number (NaN)
 and Infinity values, which are supported by all modern floating-point
 hardware.
 .PP
 Because
 .I nawk
 uses
 .IR strtod (3)
 and
 .IR atof (3)
 to convert string values to double-precision floating-point values,