Thu Sep 1 16:25:51 2016 UTC ()
Updated miller to 4.5.0.
4.5.0
Customizable output format for redirected output
In a natural follow-on to the 4.4.0 redirected-output feature, the
4.5.0 release allows your tap-files to be in a different output
format from the main program output.
For example, using
mlr --icsv --opprint ... then put --ojson 'tee > "mytap-".$a.".dat",
$*' then ...
the input is CSV, the output is pretty-print tabular, but the
tee-files output is written in JSON format. Likewise --ofs, --ors,
--ops, --jvstack, and all other output-formatting options from the
main help at mlr -h and/or man mlr default to the main command-line
options, and may be overridden with flags supplied to mlr put and
mlr tee.
4.4.0
Redirected output, row-value shift, and other features
The principal feature of Miller 4.4.0 is redirected output. Inspired
by awk, Miller lets you tap/tee your data as it's processed, run
output through subordinate processes such as gzip and jq, split a
single file into multiple files per an account-ID column, and so
on.
Details:
http://johnkerl.org/miller/doc/reference.html#Redirected-output_statements_for_put
Other features:
mlr step -a shift allows you to place the previous record's
values alongside the current record's values:
http://johnkerl.org/miller/doc/reference.html#step
mlr head, when used without the group-by flag (-g), stops after
the specified number of records has been output. For example,
even with a multi-gigabyte data file, mlr head -n 10 hugefile.dat
will complete quickly after producing the first ten records
from the file.
The sec2gmtdate verb, and sec2gmtdate function for filter/put,
is new: please see
http://johnkerl.org/miller/doc/reference.html#sec2gmtdate and
http://johnkerl.org/miller/doc/reference.html#Functions_for_filter_and_put.
sec2gmt and sec2gmtdate both leave non-numbers as-is, rather
than formatting them as (error). This is particularly relevant
for formatting nullable epoch-seconds columns in SQL-table
output: if a column value is NULL then after sec2gmt or
sec2gmtdate it will still be NULL.
The dot operator has been universalized to work with any data
type and produce a string. For example, if the field n has
integers, then instead of typing mlr put '$name = "value:".string($n)'
you can now simply domlr put '$name = "value:".$n'. This is
particularly timely for creating filenames for redirected
print/dump/tee/emit output.
The online documents now have a copy of the Miller manpage:
http://johnkerl.org/miller/doc/manpage.html
Bugfix: inside filter/put, $x=="" was distinct from isempty($x).
This was nonsensical; now both are the same.
(wiz)
diff -r1.7 -r1.8 pkgsrc/textproc/miller/Makefile
diff -r1.8 -r1.9 pkgsrc/textproc/miller/distinfo
--- pkgsrc/textproc/miller/Makefile 2016/07/15 13:10:51 1.7
+++ pkgsrc/textproc/miller/Makefile 2016/09/01 16:25:51 1.8
| @@ -1,16 +1,16 @@ | | | @@ -1,16 +1,16 @@ |
1 | # $NetBSD: Makefile,v 1.7 2016/07/15 13:10:51 mef Exp $ | | 1 | # $NetBSD: Makefile,v 1.8 2016/09/01 16:25:51 wiz Exp $ |
2 | | | 2 | |
3 | DISTNAME= mlr-4.3.0 | | 3 | DISTNAME= mlr-4.5.0 |
4 | PKGNAME= ${DISTNAME:S/mlr/miller/} | | 4 | PKGNAME= ${DISTNAME:S/mlr/miller/} |
5 | CATEGORIES= devel | | 5 | CATEGORIES= devel |
6 | MASTER_SITES= ${MASTER_SITE_GITHUB:=johnkerl/} | | 6 | MASTER_SITES= ${MASTER_SITE_GITHUB:=johnkerl/} |
7 | GITHUB_PROJECT= miller | | 7 | GITHUB_PROJECT= miller |
8 | GITHUB_RELEASE= v${PKGVERSION_NOREV} | | 8 | GITHUB_RELEASE= v${PKGVERSION_NOREV} |
9 | | | 9 | |
10 | MAINTAINER= pkgsrc-users@NetBSD.org | | 10 | MAINTAINER= pkgsrc-users@NetBSD.org |
11 | HOMEPAGE= http://github.com/johnkerl/miller/ | | 11 | HOMEPAGE= http://github.com/johnkerl/miller/ |
12 | COMMENT= Command-line CSV processor | | 12 | COMMENT= Command-line CSV processor |
13 | LICENSE= 2-clause-bsd | | 13 | LICENSE= 2-clause-bsd |
14 | | | 14 | |
15 | BUILD_DEPENDS+= asciidoc-[0-9]*:../../textproc/asciidoc | | 15 | BUILD_DEPENDS+= asciidoc-[0-9]*:../../textproc/asciidoc |
16 | | | 16 | |
--- pkgsrc/textproc/miller/distinfo 2016/07/15 13:10:51 1.8
+++ pkgsrc/textproc/miller/distinfo 2016/09/01 16:25:51 1.9
| @@ -1,6 +1,6 @@ | | | @@ -1,6 +1,6 @@ |
1 | $NetBSD: distinfo,v 1.8 2016/07/15 13:10:51 mef Exp $ | | 1 | $NetBSD: distinfo,v 1.9 2016/09/01 16:25:51 wiz Exp $ |
2 | | | 2 | |
3 | SHA1 (mlr-4.3.0.tar.gz) = 47677dcd106c54ed79c2035931234eec32341d5d | | 3 | SHA1 (mlr-4.5.0.tar.gz) = 8d1cb1c1b32790b92c404e893b2b66659238d0b6 |
4 | RMD160 (mlr-4.3.0.tar.gz) = b2b0435d3a0720accdd1facca6e339c1f05a224b | | 4 | RMD160 (mlr-4.5.0.tar.gz) = c9f05de18c9f9ecb8004ef332ad995efbd5c6793 |
5 | SHA512 (mlr-4.3.0.tar.gz) = d82e11df09f62066b85c2251f80e154bd763fb473f60d608844981d32e6c818b5a7715a60eb5a7c9202857ac4193a6131fd82587145faf4b06d7396fb6248e74 | | 5 | SHA512 (mlr-4.5.0.tar.gz) = 31b1c44b03b36d9ed98986ab6d01afdf5d74e36917d40235bb00ed0294ab83c254081f81e7ed2ef74616549ea54cbd08cb513e91dbf24d22913dba4db43fce55 |
6 | Size (mlr-4.3.0.tar.gz) = 900260 bytes | | 6 | Size (mlr-4.5.0.tar.gz) = 1010180 bytes |