nmh: Update mail/nmh to 1.7

pkgsrc changes:
- Update MASTER_SITES (use https:// and avoid redirects)

- Delete (a bit outdated) comment about locking mechanisms

  Since 02 Feb 2014 (post-1.6) the default locking mechanisms are
  (directly from m4/locking.m4):

    - aix*|cygwin*|linux*: fcntl
    - freebsd*|*netbsd*|openbsd*|darwin*: flock
    - everything else: dot

  The original comment was probably about just NetBSD and maybe Solaris
  (it's dated 1999). Solaris still uses the `dot' mechanisms by default
  but we no longer have any local patches about locking.

- Delete (no more needed) `-O1' hack to CFLAGS

  mh_strcasecmp() was completely replaced by strcasecmp() on 24 Mar 2013, and
  hence present in 1.6. Forcing `-O1' for gcc is no longer needed.

- Adjust --sysconfdir CONFIGURE_ARGS per-upstream change, now the nmh
  directory is created by nmh's configure so pass PKG_SYSCONFBASE instead of
  PKG_SYSCONFDIR.

- Add support for the `test' phase

  Add support for nmh tests. Modify patches/patch-ca accordingly in order to
  adjust TEST_ENVIRONMENT to use the configuration files in $egdir instead of the
  ones in $nmhetcdir.
  Actually all tests are passed except an mhparam test that sposts the
  $egdir/$nmetcdir kludge.

- Do not include bsd.prefs.mk two times (NFC)

- Add `oauth' PKG_OPTIONS (disabled by default) to enable OAuth2 support in
  SMTP and POP auth via curl

Changes:
Release notes for nmh 1.7
=========================
Welcome to nmh, the new version of the classic MH mail handling system.
It's been over three years since the last release of nmh, and there have
been a number of significant changes since the last release.  Long-time
MH and nmh uses should read careful the NOTEABLE CHANGES section, as there
are some significant changes to nmh behavior.  Otherwise, please see the
README and INSTALL files for help on getting started with nmh.

For news of future releases, subscribe to the low-volume
    https://lists.nongnu.org/mailman/listinfo/nmh-announce

---------------
NOTABLE CHANGES
---------------

The largest notable changes in the 1.7 release are:

- Complete unification of network security support.  All network protocols
  (currently, POP and SMTP) have been refactored to use a common set of
  security routines.  This means all protocols support all SASL mechanisms
  (via the Cyrus-SASL library) and TLS.  TLS support has been strengthened
  to perform certificate name validation and to require TLS 1.1 as a
  minimum protocol.  Also, all protocols can make use of the OAuth2/XOAUTH
  SASL mechanism, which is supported by Gmail.
- send(1) now supports adding switches to post(8) based on the address or
  domain of the email address in the From: header; this more easily allows
  users to support multiple identities.
- A generic facility for passing arguments to filter programs in repl(1)
  by use of the -convertargs switch.
- Native support for the manipulation of iCalendar requests; see mhical(1)
  for more details.

------------
NEW FEATURES
------------

The following are new features for the 1.7 release of nmh:

- When building from source, configure will derive ${prefix} from an existing
  nmh installation if it finds one in your $PATH.
- Added welcome message when nmh detects that its version changed.
- The default locations for configuration files and support binaries
  have been changed.  Configuration files now install into ${sysconfdir}/nmh,
  and support binaries are placed in ${libexecdir}/nmh.  If you are upgrading
  an existing installation you should look for old configuration files in
  ${sysconfdir} and merge any local customizations into the new files in
  ${sysconfdir}/nmh, then remove the old files.  ${libdir} will also contain
  obsolete support programs that should be removed.
- All TLS connections now perform certificate validation (including hostname
  matching) by default; can be disabled on a per-application basis.
- post now defaults to port 587 on 'smtp' message submission.
- A value of 0 for the width switch of scan(1), inc(1), ap(1), dp(1),
  fmttest(1), and mhl(1) now means as many characters as the format
  engine can produce [Bug #15274].  That amount is limited by internal
  buffers.
- If a component has trailing whitespace, e.g., body:component="> ",
  mhl now trims that whitespace off when filtering blank text lines.
- An "rtrim" flag has been added to mhl to remove any trailing
  whitespace from filtered text lines.  A corresponding "nortrim" flag
  has also been added.
- Added getmymbox and getmyaddr mh-format(5) function escapes.
- New -[no]changecur, -fixtype, -decodetypes, and -[no]crlflinebreaks switches
  have been added to mhfixmsg(1).
- mhfixmsg now removes an extraneous trailing semicolon from header
  parameter lists.
- Added -convertargs switch to repl(1), to pass arguments to programs
  specified in the user's profile or mhn.defaults to convert message
  content.
- Added mhical(1), to display, reply to, and cancel iCalendar (RFC 5545)
  event requests.
- Added multiply mh-format(5) function.
- "mhparam bindir" prints the path to the directory containing the public
  executables (${bindir}).
- New "-prefer" switch for mhshow (and mhlist and mhshow), to allow specifying
  the preferred content types to show, if present in a multipart alternative.
- mh-format now has %(kilo) and %(kibi) functions, to allow printing
  numbers with SI or IEC quantities, e.g. "10K", "2.3Mi".
- Support for the -sendmail flag to send/post to change the sendmail
  binary when using the sendmail/pipe MTS.
- Added support to send(1) to specify switches to post(1) based on address or
  domain name in From: header line in message draft.
- post(8) -snoop now attempts to decode base64-encoded SMTP traffic.
- folder(1) -nocreate now prints a warning message for a non-existent folder.
- mhfixmsg(1) now allows -decodetext binary, though 8bit is still the default.
- inc(1) and msgchk(1) now support TLS encryption natively.
- All network protocols support the XOAUTH authentication mechanism.
- Support for SMTPUTF8 (RFC 6531) has been added.  mhshow(1) already supported
  RFC 6532, assuming all 8-bit message header field bodies are UTF-8 and use
  of a UTF-8 locale.
- mhfixmsg now replaces RFC 2047 encoding with RFC 2231 encoding of name and
  filename parameters in Content-Type and Content-Disposition headers,
  respectively.
- If a message body contains 8-bit bytes, post(8) uses SMTP 8BITMIME if the
  server supports it.  If not, post fails with a message to the user to
  encode the message for 7-bit transport.
- Fewer lseek(2)s will be used when reading headers in the common case.
- ./configure's --enable-debug has been removed;  it did nothing.
- configure now defaults to enabling each of TLS and Cyrus SASL if the
  necessary headers and libraries are found.
- Moved build_nmh to top-level directory.
- Better error reporting for connections to network services.

---------
BUG FIXES
---------

- The format scanner no longer subtracts 1 from the width.  This has the
  effect of no longer counting the trailing newline in the output of
  scan(1), inc(1), and the other programs that rely on the format scanner.
- The first character of some very short (less than 4 characters) message
  bodies is no longer dropped.
- Single-character headers can be reliably formatted, etc., instead of
  apparently being missing.
- mhfixmsg now adds a Content-Transfer-Encoding header at the message level,
  if needed after decoding text parts.
- mhbuild now checks whether all text parts need a Content-Transfer-Encoding
  header, not just those with a character set not specified.
- mhbuild no longer parses lines that start with # as directives with
  -nodirectives.
- repl now makes sure that any Fcc header in a replied-to message is not
  copied into the outgoing draft by default, and that the -fcc switch
  actually works in the absence of a Fcc header in the replied-to message.
- A Content-ID is generated for message/external-body entities as required
  by RFC 2045, even if -nocontentid is supplied to mhbuild.
- post will now expand aliases on a "From" line when doing a BCC [Bug #51098].
- scan can now handle empty files without violating an assert [Bug #51693].
- An error when writing an error message, e.g. EPIPE, no longer causes
  recursion until the stack is exhausted.

-------------------
DEPRECATED FEATURES
-------------------

- Support for the MHPDEBUG environment variable is deprecated and will be
  removed from a future nmh release.  Instead, use the -debug switch to pick.
- With the move of support binaries from ${libdir} to ${libexecdir}/nmh, the
  mostly undocumented 'libdir' mhparam(1) component has been replaced by a
  new 'libexecdir' component.  'libdir' will be removed in a future release.

-----------------
OBSOLETE FEATURES
-----------------

- The undocumented -queue switch to post was deprecated in nmh 1.6, and was
  removed in this release.
- conflict(8) was deprecated in nmh 1.6, and was removed in this release.
- mhtest(8) was deprecated in nmh 1.6, and was removed in this release.
- msh(1) was deprecated in nmh 1.6, and was removed in this release.
- Support in alias files for the the "*" address-group (everyone) was
  deprecated in nmh 1.6, and was removed in this release.
- Support for multiple hostnames in the "servers" entry of mts.conf has
  been removed.
- Support in alias files for expanding aliases based on group membership
  (=) and primary group (+) has been removed.

As always, feedback is welcome.

(leot)

libgit2: remove python from bl3.mk

cmake says it's only needed for tests

(wiz)

Escape whitespace in PKG_JAVA_HOME when setting JAVA_HOME

(minskim)

doc: Updated devel/py-ipython to 6.2.1nb2

(minskim)

Correct the PYPKGPREFIX value

It should be "py34" rather than "34".

Bump PKGREVISION.

(minskim)

doc: Updated databases/mongo-c-driver to 1.8.2

(fhajny)

Update databases/mongo-c-driver to 1.8.2.

- Remove option to bundle the Snappy compression library, it caused issues
  for programs linking to libmongoc
- Fix pkg-config and CMake config file flags for programs that statically
  link to libmongoc when libmongoc is statically linked to zLib
- The configure flag "--with-zlib=no" was ignored
- Fix crash in authentication when username is NULL

(fhajny)

doc: Updated devel/libbson to 1.8.2

(fhajny)

Update devel/libbson to 1.8.2.

No change since 1.8.1; released to keep pace with libmongoc's version.

(fhajny)

Fails with Python > 2.7 - add PYTHON_VERSIONS_ACCEPTED=27

(abs)

doc: Updated devel/py-argcomplete to 1.9.3

(fhajny)

Update devel/py-argcomplete to 1.9.3.

- Fix handling of COMP_POINT
- Fix crash when writing unicode to debug_stream in Python 2

(fhajny)

doc: Updated sysutils/beats to 6.0.0

(fhajny)

Update sysutils/beats to 6.0.0.

=== Beats version 6.0.0

==== Breaking changes

Affecting all Beats

- The log directory (`path.log`) for Windows services is now set to
  `C:\ProgramData\[beatname]\logs`.
- The _all field is disabled in Elasticsearch 6.0.
- Fail if removed setting output.X.flush_interval is explicitly
  configured.
- Rename the `/usr/bin/beatname.sh` script (e.g. `metricbeat.sh`) to
  `/usr/bin/beatname`.
- Beat does not start if elasticsearch index pattern was modified but
  not the template name and pattern.
- Fail if removed setting output.X.flush_interval is explicitly
  configured.
- Rename `kubernetes` processor to `add_kubernetes_metadata`.
- Rename `.full.yml` config files to `*.reference.yml`.
- The `scripts/import_dashboards` is removed from packages. Use the
  `setup` command instead.
- Change format of the saved kibana dashboards to have a single JSON
  file for each dashboard
- Rename `configtest` command to `test config`.
- Remove setting `queue_size` and `bulk_queue_size`.
- Remove setting `dashboard.snapshot` and `dashboard.snapshot_url`.
- Beats can no longer be launched from Windows Explorer (GUI), command
  line is required.

Filebeat

- Rename `input_type` field to `prospector.type`
- The `@metadata.type` field, added by the Logstash output, is now
  hardcoded to `doc` and will be removed in future versions.

Metricbeat

- Change all `system.cpu..pct` metrics to be scaled by the number of
  CPU cores.
- Remove filters setting from metricbeat modules.
- Added `type` field to filesystem metrics.

Heartbeat

- Renamed the heartbeat RPM/DEB name to `heartbeat-elastic`.

Packetbeat

- Remove not-working `runoptions.uid` and `runoptions.gid` options in
  Packetbeat.
- Remove the already unsupported `pf_ring` sniffer option.

Auditbeat

- Changed file metricset config to make `file.paths` a list instead of
  a dictionary.

==== Bugfixes

Affecting all Beats

- Fix data race accessing watched containers.
- Do not require template if index change and template disabled
- Fix missing ACK in redis output.
- Fix the `/usr/bin/beatname` script to accept `-d ""` as a parameter.
- Combine `fields.yml` properties when they are defined in different
  sources.
- Keep Docker & Kubernetes pod metadata after container dies while
  they are needed by processors.
- Fix `fields.yml` lookup when using `export template` with a custom
  `path.config` param.
- Remove runner creation from every reload check
- Fix add_kubernetes_metadata matcher registry lookup.
- Register kubernetes `field_format` matcher and remove logger in
  `Encode` API
- Fix go plugins not loaded when beat starts
- Add support for `initContainers` in `add_kubernetes_metadata`
  processor.
- Eliminate deprecated _default_ mapping in 6.x
- Fix pod name indexer to use both namespace, pod name to frame index
  key
- Don't stop with error loading the ES template if the ES output is
  not enabled.
- Fix race condition in internal logging rotator.
- Normalize all times to UTC to ensure proper index naming.
- Fix issue with loading dashboards to ES 6.0 when .kibana index did
  not already exist.
- Fix importing the dashboards when the limit for max open files is
  too low.
- Fix configuration documentation for kubernetes processor
- Fix misspelling in `add_locale` configuration option for
  abbreviation.

Filebeat

- Fix machine learning jobs setup for dynamic modules.
- Fix default paths for redis 4.0.1 logs on macOS
- Fix Filebeat not starting if command line and modules configs are
  used together.
- Fix double `@timestamp` field when JSON decoding was used.
- Fix issue where the `fileset.module` could have the wrong value.
- Fix race condition on harvester stopping with reloading enabled.
- Fix recursive glob config parsing and resolution across restarts.
- Allow string characters in user agent patch version (NGINX and
  Apache)
- Fix grok pattern in filebeat module system/auth without hostname.

Winlogbeat

- Removed validation of top-level config keys.

Metricbeat

- Use `beat.name` instead of `beat.hostname` in the Host Overview
  dashboard.
- Fix the loading of 5.x dashboards.
- Fix a memory allocation issue where more memory was allocated than
  needed in the windows-perfmon metricset.
- Don't start metricbeat if external modules config is wrong and
  reload is disabled
- The MongoDB module now connects on each fetch, to avoid stopping the
  whole Metricbeat instance if MongoDB is not up when starting.
- Fix kubernetes events module to be able to index time fields
  properly.
- Fixed `cmd_set` and `cmd_get` being mixed in the Memcache module.
- Added missing mongodb configuration file to the `modules.d` folder.
- Fix wrong MySQL CRUD queries timelion visualization
- Add new metrics to CPU metricsset
- Fix issue affecting Windows services timing out at startup.
- Fix incorrect docker.diskio.total metric calculation.
- Vsphere module: used memory field corrected.
- Set correct format for percent fields in memory module.
- Fix a debug statement that said a module wrapper had stopped when it
  hadn't.
- Use MemAvailable value from /proc/meminfo on Linux 3.14.
- Fix panic when events were dropped by filters.
- Add filtering to system filesystem metricset to remove relative
  mountpoints like those from Linux network namespaces.
- Remove unnecessary print statement in schema apis.
- Fix type of field `haproxy.stat.check.health.last`.

Heartbeat

- Fix monitor.name being empty by default.
- Fix wrong event timestamps.

Packetbeat

- Fix missing length check in the PostgreSQL module.
- Fix panic in ACK handler if event is dropped on blocked queue
- Update flow timestamp on each packet being received.
- Enabled /proc/net/tcp6 scanning and fixed ip v6 parsing.
- Enable memcache filtering only if a port is specified in the config
  file.

Auditbeat

- Fix `file.max_file_size` config option for the audit file metricset.

==== Added

Affecting all Beats

- Enable flush timeout by default.
- Add @metadata.version to events send to Logstash.
- Add setting to enable/disable the slow start in logstash output.
- Update init scripts to use the `test config` subcommand instead of
  the deprecated `-configtest` flag.
- Get by default the credentials for connecting to Kibana from the
  Elasticsearch output configuration.
- Added `cloud.id` and `cloud.auth` settings, for simplifying using
  Beats with the Elastic Cloud.
- Add lz4 compression support to kafka output.
- Add newer kafka versions to kafka output.
- Configure the index name when loading the dashboards and the index
  pattern.
- New cli subcommands interface.
- Allow source path matching in `add_docker_metadata` processor.
- Add support for analyzers and multifields in fields.yml.
- Add support for JSON logging.
- Add `test output` command, to test Elasticsearch and Logstash output
  settings.
- Introduce configurable event queue settings: queue.mem.events,
  queue.mem.flush.min_events and queue.mem.flush.timeout.
- Enable pipelining in Logstash output by default.
- Added 'result' field to Elasticsearch QueryResult struct for
  compatibility with 6.x Index and Delete API responses.
- The sample dashboards are now included in the Beats packages.
- Add `pattern` option to be used in the fields.yml to specify the
  pattern for a number field.
- Upgraded to Golang 1.8.3.
- Added the possibility to set Elasticsearch mapping template settings
  from the Beat configuration file.
- Add a variable to the SysV init scripts to make it easier to change
  the user.
- Add the option to write the generated Elasticsearch mapping template
  into a file.
- Add `instance_name` in GCE add_cloud_metadata processor.
- Add `add_docker_metadata` processor.
- Add `logging.files` `permissions` option.

Filebeat

- Add Kubernetes manifests to deploy Filebeat.
- Changed the number of shards in the default configuration to 3.
- Don't start filebeat if external modules/prospectors config is wrong
  and reload is disabled
- Add `filebeat.registry_flush` setting, to delay the registry
  updates.
- Add experimental Redis module.
- Nginx module: use the first not-private IP address as the remote_ip.
- Load Ingest Node pipelines when the Elasticsearch connection is
  established, instead of only once at startup.
- Add support for loading Xpack Machine Learning configurations from
  the modules, and added sample configurations for the Nginx module.
- Add udp prospector type.
- Enabled Cgo which means libc is dynamically compiled.
- Add Beta module config reloading mechanism
- Remove spooler and publisher components and settings.
- Added ability to sort harvested files.
- Add experimental Redis slow log prospector type.

Winlogbeat

- Changed the number of shards in the default configuration to 3.
- Add the ability to use LevelRaw if Level isn't populated in the
  event XML.

Metricbeat

- Add Kubernetes manifests to deploy Metricbeat.
- Auto-select a hostname (based on the host on which the Beat is
  running) in the Host Overview dashboard.
- Add `filesystem.ignore_types` to system module for ignoring
  filesystem types.
- Add support to exclude labels from kubernetes pod metadata.
- Add random startup delay to each metricset to avoid the thundering
  herd problem.
- Add the ability to configure audit rules to the kernel module.
- Add the ability to configure kernel's audit failure mode.
- Add experimental Aerospike module.
- Vsphere module: collect custom fields from virtual machines.
- Add `test modules` command, to test modules expected output.
- Add `processors` setting to metricbeat modules.
- Support `npipe` protocol (Windows) in Docker module.
- Add macOS implementation of the system diskio metricset.
- Add process_summary metricset that records high level metrics about
  processes.
- Add `kube-state-metrics` based metrics to `kubernetes` module
- Add debug logging to Jolokia JMX metricset.
- Add events metricset for kubernetes metricbeat module
- Change Metricbeat default configuration file to be better optimized
  for most users.
- Add experimental RabbitMQ module.
- Add Kibana dashboard for the Kubernetes modules.

Heartbeat

- Changed the number of shards in the default configuration to 1.
- Enabled Cgo which means libc is dynamically compiled.

Packetbeat

- Changed the number of shards in the default configuration to 3.

Auditbeat

- Changed the number of shards in the default configuration to 3.
- Add support for receiving audit events using a multicast socket.
- Added `file.hash_types` config option for controlling the hash
  types.
- Added the ability to specify byte unit suffixes to
  `file.max_file_size`.
- Add file integrity metricset to the audit module.

==== Deprecated

Affecting all Beats

- The `@metadata.type` field, added by the Logstash output, is
  deprecated, hardcoded to `doc` and will be removed in future
versions.

Filebeat

- The `filebeat.config_dir` option is deprecated. Use
  `filebeat.config.prospector` options instead.
- Deprecate `input_type` prospector config. Use `type` config option
  instead.

=== Beats version 5.6.4

- Fix race condition in internal logging rotator.
- Add support for enabling TLS renegotiation.
- Add setting to enable/disable the slow start in logstash output.
- Packetbeat: Fix missing length check in the PostgreSQL module.

(fhajny)

doc: Updated databases/py-elasticsearch to 6.0.0

(fhajny)

Update databases/py-elasticsearch to 6.0.0.

6.0.0 (2017-11-14)
- compatibility with Elasticsearch 6.0.0

5.5.0 (2017-11-10)
- streaming_bulk helper now supports retries with incremental backoff
- scan helper properly checks for successful shards instead of just
  checking failed
- compatible release with elasticsearch 5.6.4
- fix handling of UTF-8 surrogates

(fhajny)

doc: Updated databases/elasticsearch to 6.0.0

(fhajny)

Update databases/elasticsearch to 6.0.0.

=== Breaking changes

Aggregations::
- Change parsing of numeric `to` and `from` parameters in `date_range`
  aggregation

Aliases::
- Wrong behavior deleting alias

Allocation::
- Remove `cluster.routing.allocation.snapshot.relocation_enabled`
  setting

Analysis::
- Do not allow custom analyzers to have the same names as built-in
  analyzers
- Removing query-string parameters in `_analyze` API

CAT API::
- Write -1 on unbounded queue in cat thread pool

CRUD::
- Disallow `VersionType.FORCE` for GetRequest
- Disallow `VersionType.FORCE` versioning for 6.x indices
- If the index does not exist, delete document will not auto create it

Cluster::
- Disallow : in cluster and index/alias names
- No longer allow cluster name in data path

Core::
- Simplify file store
- Make boolean conversion strict
- Remove the `default` store type.
- Remove store throttling.

Geo::
- Remove deprecated geo search features
- Reduce GeoDistance Insanity

Highlighting::
- Remove the postings highlighter and make unified the default
  highlighter choice

Index APIs::
- Remove (deprecated) support for '+' in index expressions
- Delete index API to work only against concrete indices
- Open/Close index api to allow_no_indices by default
- Remove support for controversial `ignore_unavailable` and
  `allow_no_indices` from indices exists api

Index Templates::
- Allows multiple patterns to be specified for index templates

Indexed Scripts/Templates::
- Scripting: Remove search template actions

Ingest::
- update ingest-user-agent regexes.yml
- remove ingest.new_date_format

Inner Hits::
- Return the _source of inner hit nested as is without wrapping it
  into its full path context

Java API::
- Enforce Content-Type requirement on the rest layer and remove
  deprecated methods

Java REST Client::
- Remove deprecated created and found from index, delete and bulk

Mapping::
- Reject out of range numbers for float, double and half_float
- Enforce at most one type.
- Disallow `include_in_all` for 6.0+ indices
- Disable _all by default, disallow configuring _all on 6.0+ indices
- Throw an exception on unrecognized "match_mapping_type"

Network::
- Remove unused Netty-related settings
- Remove blocking TCP clients and servers
- Remove `modules/transport_netty_3` in favor of `netty_4`
- Remove LocalTransport in favor of MockTcpTransport

Packaging::
- Configure heap dump path out of the box
- Remove support for ES_INCLUDE
- Setup: Change default heap to 1G
- Use config directory to find jvm.options
- Remove implicit 32-bit support
- Remove default path settings
- Remove path.conf setting
- Honor masking of systemd-sysctl.service
- Rename CONF_DIR to ES_PATH_CONF
- Remove customization of ES_USER and ES_GROUP

Percolator::
- Remove deprecated percolate and mpercolate apis

Plugin Analysis ICU::
- Upgrade icu4j for the ICU analysis plugin to 59.1
- Upgrade icu4j to latest version

Plugin Delete By Query::
- Require explicit query in _delete_by_query API

Plugin Discovery Azure Classic::
- Remove `discovery.type` BWC layer from the EC2/Azure/GCE plugins

Plugin Discovery EC2::
- Ec2 Discovery: Cleanup deprecated settings
- Discovery EC2: Remove region setting
- AWS Plugins: Remove signer type setting

Plugin Lang JS::
- Remove lang-python and lang-javascript

Plugin Mapper Attachment::
- Remove mapper attachments plugin

Plugin Repository Azure::
- Remove global `repositories.azure` settings
- Remove auto creation of container for azure repository

Plugin Repository GCS::
- GCS Repository: Remove specifying credential file on disk

Plugin Repository S3::
- S3 Repository: Cleanup deprecated settings
- S3 Repository: Remove region setting
- S3 Repository: Remove bucket auto create
- S3 Repository: Remove env var and sysprop credentials support
- Remove deprecated S3 settings

Plugins::
- Make plugin loading stricter

Query DSL::
- Remove deprecated `type` and `slop` field in `match` query
- Remove several parse field deprecations in query builders
- Remove deprecated parameters from `ids_query`
- Refactor QueryStringQuery for 6.0
- Change `split_on_whitespace` default to false
- Remove deprecated template query
- Throw exception in scroll requests using `from`
- Remove deprecated `minimum_number_should_match` in BoolQueryBuilder
- Remove support for empty queries
- Remove deprecated query names: in, geo_bbox, mlt, fuzzy_match and
  match_fuzzy
- The `terms` query should always map to a Lucene `TermsQuery`.
- Be strict when parsing values searching for booleans
- Remove collect payloads parameter

REST::
- IndexClosedException to return 400 rather than 403
- Remove comma-separated feature parsing for GetIndicesAction
- Improve REST error handling when endpoint does not support HTTP
  verb, add OPTIONS support
- Remove ldjson support and document ndjson for bulk/msearch
- Enable strict duplicate checks for all XContent types
- Enable strict duplicate checks for JSON content
- Remove lenient stats parsing
- Remove allow unquoted JSON
- Remove FORCE version_type

Scripting::
- remove lang url parameter from stored script requests
- Disallow lang to be used with Stored Scripts
- Remove Deprecated Script Settings
- Scripting: Remove native scripts
- Scripting: Remove file scripts
- Make dates be ReadableDateTimes in scripts
- Remove groovy scripting language
- Remove script access to term statistics

Search::
- Make `index` in TermsLookup mandatory
- Removes FieldStats API
- Remove deprecated fielddata_fields from search request
- Removes deprecated fielddata_fields
- ProfileResult and CollectorResult should print machine readable
  timing information
- Remove indices query
- Remove ignored type parameter in search_shards api

Sequence IDs::
- Change certain replica failures not to fail the replica shard

Settings::
- Settings: Remove shared setting property
- Settings: Remove support for yaml and json config files

Shadow Replicas::
- Remove shadow replicas

Similarities::
- Similarity should accept dynamic settings when possible

=== Breaking Java changes

Aggregations::
- Remove the unused SignificantTerms.compareTerm() method
- Make SignificantTerms.Bucket an interface rather than an abstract
  class
- Fix NPE when `values` is omitted on percentile_ranks agg
- Make Terms.Bucket an interface rather than an abstract class
- Compound order for histogram aggregations

Internal::
- Collapses package structure for some bucket aggs

Java API::
- Remove deprecated IdsQueryBuilder ctor
- Removing unneeded getTookInMillis method
- Java api: ActionRequestBuilder#execute to return a PlainActionFuture

Java High Level REST Client::
- Unify the result interfaces from get and search in Java client
- Allow RestHighLevelClient to use plugins

Java REST Client::
- Rename client artifacts

Network::
- Simplify TransportAddress

Plugin Delete By Query::
- Move DeleteByQuery and Reindex requests into core

Plugins::
- Drop name from TokenizerFactory

Query DSL::
- Remove QueryParseContext
- Remove QueryParseContext from parsing QueryBuilders

REST::
- Return index name and empty map for `/{index}/_alias` with no
  aliases

=== Deprecations

Index APIs::
- Deprecated use of + in index expressions

Index Templates::
- Restore deprecation warning for invalid match_mapping_type values

Indexed Scripts/Templates::
- Scripting: Deprecate stored search template apis

Internal::
- Deprecate XContentType auto detection methods in XContentFactory

Percolator::
- Deprecate percolate query's document_type parameter.

Plugins::
- Plugins: Add backcompat for sha1 checksums

Scripting::
- Scripting: Change keys for inline/stored scripts to source/id
- Scripting: Deprecate native scripts
- Scripting: Deprecate index lookup
- Deprecate Fine Grain Settings for Scripts
- Scripting: Deprecate file script settings
- Scripting: Deprecate file scripts

Settings::
- Settings: Update settings deprecation from yml to yaml
- Deprecate settings in .yml and .json

Tribe Node::
- Deprecate tribe service

=== New features

Aggregations::
- SignificantText aggregation - like significant_terms, but for text

Analysis::
- Expose simplepattern and simplepatternsplit tokenizers
- Parse synonyms with the same analysis chain

Core::
- Enable index-time sorting

Internal::
- Automatically adjust search threadpool queue_size

Mapping::
- Add new ip_range field type

Parent/Child::
- Move parent_id query to the parent-join module
- Introduce ParentJoinFieldMapper, a field mapper that creates
  parent/child relation within documents of the same index

Plugin Analysis ICU::
- Add ICUCollationFieldMapper

Search::
- Automatically early terminate search query based on index sorting

Sequence IDs::
- Add a scheduled translog retention check
- Initialize sequence numbers on a shrunken index
- Initialize primary term for shrunk indices
- Introduce translog size and age based retention policies

Stats::
- Adds nodes usage API to monitor usages of actions

Task Manager::
- Task Management

Upgrade API::
- TemplateUpgraders should be called during rolling restart

=== Enhancements

Aggregations::
- Add strict parsing of aggregation ranges
- Adds rewrite phase to aggregations
- Tweak AggregatorBase.addRequestCircuitBreakerBytes
- Add superset size to Significant Term REST response
- Add document count to Matrix Stats aggregation response
- Adds an implementation of LogLogBeta for the cardinality aggregation
- Support distance units in GeoHashGrid aggregation precision
- Reject multiple methods in `percentiles` aggregation
- Use `global_ordinals_hash` execution mode when sorting by sub
  aggregations.
- Add a specialized deferring collector for terms aggregator
- Agg builder accessibility fixes
- Remove support for the include/pattern syntax.
- Promote longs to doubles when a terms agg mixes decimal and
  non-decimal numbers

Allocation::
- Adjust status on bad allocation explain requests
- Promote replica on the highest version node

Analysis::
- [Analysis] Support normalizer in request param
- Enforce validation for PathHierarchy tokenizer
- [analysis-icu] Allow setting unicodeSetFilter
- Match- and MultiMatchQueryBuilder should only allow setting analyzer
  on string values

Bulk::
- Simplify bulk request execution

CAT API::
- expand `/_cat/nodes` to return information about hard drive

CRUD::
- Added validation for upsert request

Circuit Breakers::
- ScriptService: Replace max compilation per minute setting with max
  compilation rate

Cluster::
- Validate a joining node's version with version of existing cluster
  nodes
- Switch indices read-only if a node runs out of disk space
- Add a cluster block that allows to delete indices that are read-only
- Separate publishing from applying cluster states
- Adds cluster state size to /_cluster/state response

Core::
- Allow `InputStreamStreamInput` array size validation where
  applicable
- Refactor bootstrap check results and error messages
- Add BootstrapContext to expose settings and recovered state to
  bootstrap checks
- Unit testable index creation task on MetaDataCreateIndexService
- Ignore .DS_Store files on macOS
- Add max file size bootstrap check
- Add compatibility versions to main action response
- Index ids in binary form.
- Explicitly reject duplicate data paths
- Use SPI in High Level Rest Client to load XContent parsers
- Upgrade to lucene-7.0.0-snapshot-a0aef2f
- Speed up PK lookups at index time.
- Use Java 9 FilePermission model
- Add friendlier message on bad keystore permissions
- Epoch millis and second formats accept float implicitly
- Remove connect SocketPermissions from core
- Add repository-url module and move URLRepository
- Remove accept SocketPermissions from core
- Move IfConfig.logIfNecessary call into bootstrap
- Remove artificial default processors limit
- Simplify write failure handling
- Improve bootstrap checks error messages

Discovery::
- Allow plugins to validate cluster-state on join

Engine::
- Add refresh stats tracking for realtime get
- Introducing a translog deletion policy
- Fill missing sequence IDs up to max sequence ID when recovering from
  store
- Use sequence numbers to identify out of order delivery in replicas &
  recovery
- Add replica ops with version conflict to translog
- Clarify global checkpoint recovery
- Move the IndexDeletionPolicy to be engine internal

Exceptions::
- IllegalStateException: Only duplicated jar instead of classpath

Highlighting::
- Picks offset source for the unified highlighter directly from the es
  mapping

Index APIs::
- Let primary own its replication group
- Create index request should return the index name

Index Templates::
- Fix error message for a put index template request without
  index_patterns

Ingest::
- Add Ingest-Processor specific Rest Endpoints & Add Grok endpoint
- Port support for commercial GeoIP2 databases from Logstash.
- add `exclude_keys` option to KeyValueProcessor
- Allow removing multiple fields in ingest processor
- Add target_field parameter to ingest processors

Inner Hits::
- Reuse inner hit query weight

Internal::
- TemplateUpgradeService should only run on the master
- Cleanup IndexFieldData visibility
- Bump the min compat version to 5.6.0
- "shard started" should show index and shard ID
- Break out clear scroll logic from TransportClearScrollAction
- Add helper methods to TransportActionProxy to identify proxy actions
  and requests
- Add remote cluster infrastructure to fetch discovery nodes.
- Add the ability to set eager_global_ordinals in the new parent-join
  field
- Disallow multiple parent-join fields per mapping
- Remove the need for _UNRELEASED suffix in versions
- Optimize the order of bytes in uuids for better compression.
- Prevent cluster internal `ClusterState.Custom` impls to leak to a
  client
- Use holder pattern for lazy deprecation loggers
- Allow `ClusterState.Custom` to be created on initial cluster states
- Try to convince the JVM not to lose stacktraces
- Make document write requests immutable
- Add assertions enabled helper

Java API::
- Always Accumulate Transport Exceptions

Java High Level REST Client::
- [DOCS] restructure java clients docs pages
- Use SPI in High Level Rest Client to load XContent parsers
- Add support for clear scroll to high level REST client
- Add search scroll method to high level REST client
- Add search method to high level REST client
- Make RestHighLevelClient Closeable and simplify its creation
- Add info method to High Level Rest client
- Add support for named xcontent parsers to high level REST client
- Add BulkRequest support to High Level Rest client
- Add UpdateRequest support to High Level Rest client
- Add delete API to the High Level Rest Client
- Add Index API to High Level Rest Client
- Add get/exists method to RestHighLevelClient
- Add fromxcontent methods to delete response
- Add REST high level client gradle submodule and first simple method
- Add doc_count to ParsedMatrixStats
- Add fromXContent method to ClearScrollResponse
- ClearScrollRequest to implement ToXContentObject
- SearchScrollRequest to implement ToXContentObject
- Add aggs parsers for high level REST Client

Java REST Client::
- Shade external dependencies in the rest client jar
- RestClient uses system properties and system default SSLContext
- Wrap rest httpclient with doPrivileged blocks

Logging::
- Prevent excessive disk consumption by log files
- Use LRU set to reduce repeat deprecation messages

Mapping::
- More efficient encoding of range fields.
- Don't detect source's XContentType in DocumentParser.parseDocument()
- Better validation of `copy_to`.
- Optimize `terms` queries on `ip` addresses to use a
  `PointInSetQuery` whenever possible.
- Loosen the restrictions on disabling _all in 6.x
- Date detection should not rely on a hardcoded set of characters.
- Identify documents by their `_id`.

Network::
- Add additional low-level logging handler
- Unwrap causes when maybe dying
- Move TransportStats accounting into TcpTransport
- Simplify connection closing and cleanups in TcpTransport
- Disable the Netty recycler in the client
- Remove Netty logging hack
- Isolate SocketPermissions to Netty
- Wrap netty accept/connect ops with doPrivileged
- Replace Socket, ServerSocket, and HttpServer usages in tests with
  mocksocket versions

Packaging::
- Remove memlock suggestion from systemd service
- Set address space limit in systemd service file
- Version option should display if snapshot
- Ignore JVM options before checking Java version
- Also skip JAVA_TOOL_OPTIONS on Windows
- Introduce elasticsearch-env for Windows
- Introduce elasticsearch-env
- Stop exporting HOSTNAME from scripts
- Set number of processes in systemd unit file

Parent/Child::
- Remove ParentJoinFieldSubFetchPhase
- Support parent id being specified as number in the _source

Percolator::
- Store the QueryBuilder's Writable representation instead of its
  XContent representation
- Add support for selecting percolator query candidate matches
  containing wildcard / prefix queries

Plugin Discovery EC2::
- Read ec2 discovery address from aws instance tags

Plugin Lang Painless::
- Allow Custom Whitelists in Painless
- Update Painless to Allow Augmentation from Any Class
- Add Needs Methods to Painless Script Context Factories
- Support Script Context Stateful Factory in Painless
- Generate Painless Factory for Creating Script Instances
- Update Painless to Use New Script Contexts
- Optimize instance creation in LambdaBootstrap
- Make Painless Compiler Use an Instance Per Context
- Make PainlessScript An Interface

Plugin Repository GCS::
- GCS Repository: Add secure storage of credentials

Plugin Repository HDFS::
- Add permission checks before reading from HDFS stream
- Add doPrivilege blocks for socket connect ops in repository-hdfs
- Add Kerberos support for Repo HDFS plugin

Plugin Repository S3::
- S3 Repository: Add back repository level credentials

Plugins::
- Adjust SHA-512 supported format on plugin install
- Move tribe to a module
- Plugins can register pre-configured char filters
- Add purge option to remove plugin CLI
- Allow plugins to register pre-configured tokenizers
- Move ReindexAction class to core
- Make PreConfiguredTokenFilter harder to misuse
- Plugins: Remove leniency for missing plugins dir
- Add doPrivilege blocks for socket connect operations in plugins

Query DSL::
- Make slop optional when parsing `span_near` query
- Require a field when a `seed` is provided to the `random_score`
  function.
- Add support for auto_generate_synonyms_phrase_query in match_query,
  multi_match_query, query_string and simple_query_string

REST::
- Cat shards bytes
- Refactor PathTrie and RestController to use a single trie for all
  methods
- Make ObjectParser support string to boolean conversion

Recovery::
- Introduce a History UUID as a requirement for ops based recovery
- Goodbye, Translog Views
- Disallow multiple concurrent recovery attempts for same target shard
- Live primary-replica resync (no rollback)
- Peer Recovery: remove maxUnsafeAutoIdTimestamp hand off
- Introduce sequence-number-based recovery

Scripting::
- Scripting: Rename SearchScript.needsScores to needs_score
- Scripting: Add optional context parameter to put stored script
  requests
- Add New Security Script Settings
- Add StatefulFactoryType as optional intermediate factory in script
  contexts
- Make contexts available to ScriptEngine construction
- Make ScriptEngine.compile generic on the script context
- Add instance and compiled classes to script contexts

Search::
- Add soft limit on allowed number of script fields in request
- Add a soft limit for the number of requested doc-value fields
- Rewrite search requests on the coordinating nodes
- Ensure query resources are fetched asynchronously during rewrite
- Introduce a new Rewriteable interface to streamline rewriting
- Reduce the scope of `QueryRewriteContext`
- Reduce the overhead of timeouts and low-level search cancellation.
- Reduce profiling overhead.
- Prevent `can_match` requests from sending to incompatible nodes
- Add a shard filter search phase to pre-filter shards based on query
  rewriting
- Ensure we rewrite common queries to `match_none` if possible
- Limit the number of concurrent shard requests per search request
- Add cluster name validation to RemoteClusterConnection
- Speed up sorted scroll when the index sort matches the search sort
- Leverage scorerSupplier when applicable.
- Add Cross Cluster Search support for scroll searches
- Track EWMA[1] of task execution time in search threadpool executor
- Query range fields by doc values when they are expected to be more
  efficient than points
- Search: Fairer balancing when routing searches by session ID
-  Add parsing from xContent to Suggest
- Add parsing from xContent to ShardSearchFailure
- Eliminate array access in tight loops when profiling is enabled.
- Support Multiple Inner Hits on a Field Collapse Request
- Expand cross cluster search indices for search requests to the
  concrete index or to it's aliases

Search Templates::
- Add max concurrent searches to multi template search

Sequence IDs::
- Roll translog generation on primary promotion
- Restoring from snapshot should force generation of a new history
  uuid
- Add global checkpoint tracking on the primary
- Introduce global checkpoint background sync
- Move `UNASSIGNED_SEQ_NO` and `NO_OPS_PERFORMED` to SequenceNumbers`
- Move primary term from ReplicationRequest to ConcreteShardRequest
- Add reason to global checkpoint updates on replica
- Introduce primary/replica mode for GlobalCheckPointTracker
- Throw back replica local checkpoint on new primary
- Update global checkpoint when increasing primary term on replica
- Enable a long translog retention policy by default
- Introduce primary context
- Block older operations on primary term transition
- Block global checkpoint advances when recovering
- Add primary term to doc write response
- Preserve multiple translog generations
- Introduce translog generation rolling
- Replicate write failures
- Introduce sequence-number-aware translog
- Introduce translog no-op
- Tighten sequence numbers recovery
- Add BWC layer to seq no infra and enable BWC tests
- Add internal _primary_term doc values field, fix _seq_no indexing
- Add global checkpoint to translog checkpoints
- Sequence numbers commit data for Lucene uses Iterable interface
- Simplify GlobalCheckpointService and properly hook it for cluster
  state updates
- Fill gaps on primary promotion
- Introduce clean transition on primary promotion
- Guarantee that translog generations are seqNo conflict free
- Inline global checkpoints

Settings::
- Add disk threshold settings validation
- Enable cross-setting validation
- Validate `transport.profiles.*` settings
- Cleanup network / transport related settings
- Emit settings deprecation logging at most once
- IndexMetaData: Introduce internal format index setting
- Persist created keystore on startup unless keystore is present
- Settings: Add keystore.seed auto generated secure setting
- Settings: Add keystore creation to add commands

Snapshot/Restore::
- Fixed references to Multi Index Syntax
- Improves snapshot logging and snapshot deletion error handling
- Enhances get snapshots API to allow retrieving repository index only

Stats::
- Update `IndexShard#refreshMetric` via a
  `ReferenceManager.RefreshListener`
- Expose disk usage estimates in nodes stats

Store::
- Remote support for lucene versions without checksums

Suggesters::
- Remove deprecated _suggest endpoint

Task Manager::
- Add descriptions to bulk tasks

Translog::
- Translog file recovery should not rely on lucene commits

=== Bug fixes

Aggregations::
- Do not delegate a null scorer to LeafBucketCollectors
- Create weights lazily in filter and filters aggregation
- Fix IndexOutOfBoundsException in histograms for NaN doubles (#26787)
- Scripted_metric _agg parameter disappears if params are provided
- Fixes array out of bounds for value count agg
- Aggregations bug: Significant_text fails on arrays of text.
- Check bucket metric ages point to a multi bucket agg
- Terms aggregation should remap global ordinal buckets when a
  sub-aggregator is used to sort the terms
- Correctly set doc_count when MovAvg "predicts" values on existing
  buckets
- DateHistogram: Fix `extended_bounds` with `offset`
- Fix ArrayIndexOutOfBoundsException when no ranges are specified in
  the query

Aliases::
- mget with an alias shouldn't ignore alias routing
- GET aliases should 404 if aliases are missing

Allocation::
- Fix DiskThresholdMonitor flood warning
- Allow wildcards for shard IP filtering

Analysis::
- Pre-configured shingle filter should disable graph analysis
- PatternAnalyzer should lowercase wildcard queries when `lowercase`
  is true.

CAT API::
- Fix NPE for /_cat/indices when no primary shard

CRUD::
- Serialize and expose timeout of acknowledged requests in REST layer
- Fix silent loss of last command to _bulk and _msearch due to missing
  newline

Cache::
- Reduce the default number of cached queries.
- fix bug of weight computation

Circuit Breakers::
- Checks the circuit breaker before allocating bytes for a new big
  array

Cluster::
- Register setting `cluster.indices.tombstones.size`

Core::
- Correctly encode warning headers
- Fix cache compute if absent for expired entries
- Timed runnable should delegate to abstract runnable
- Stop invoking non-existent syscall
- MetaData Builder doesn't properly prevent an alias with the same
  name as an index
- Release operation permit on thread-pool rejection
- Node should start up despite of a lingering `.es_temp_file`
- Fix cache expire after access

Dates::
- Fix typo in date format

Discovery::
- MasterNodeChangePredicate should use the node instance to detect
  master change

Engine::
- Die with dignity while merging
- Engine - do not index operations with seq# lower than the local
  checkpoint into lucene

Geo::
- Fix typo in GeoUtils#isValidLongitude

Highlighting::
- Fix percolator highlight sub fetch phase to not highlight query
  twice
- FastVectorHighlighter should not cache the field query globally
- Higlighters: Fix MultiPhrasePrefixQuery rewriting
- Fix nested query highlighting

Index APIs::
- Shrink API should ignore templates
- Rollover max docs should only count primaries
- Validates updated settings on closed indices

Ingest::
- date processor should not fail if timestamp is specified as json
  number
- date_index_name processor should not fail if timestamp is specified
  as json number
- Sort Processor does not have proper behavior with targetField
- fix grok's pattern parsing to validate pattern names in expression
- Remove support for Visio and potm files
- Fix floating-point error when DateProcessor parses UNIX
- add option for _ingest.timestamp to use new ZonedDateTime (5.x
  backport)

Inner Hits::
- Do not allow inner hits that fetch _source and have a non nested
  object field as parent
- When fetching nested inner hits only access stored fields when
  needed
- If size / offset are out of bounds just do a plain count
- Fix Source filtering in new field collapsing feature

Internal::
- Bump version to 6.0.1
- `IndexShard.routingEntry` should only be updated once all internal
  state is ready
- Catch exceptions and inform handler in
  RemoteClusterConnection#collectNodes
- Internal: Add versionless alias for rest client codebase in policy
  files
- Upgrade Lucene to version 7.0.1
- Fix BytesReferenceStreamInput#skip with offset
- Fix race condition in RemoteClusterConnection node supplier
- Initialise empty lists in BaseTaskResponse constructor
- Extract a common base class for scroll executions
- Obey lock order if working with store to get metadata snapshots
- Fix Version based BWC and set correct minCompatVersion
- Fix `_field_caps` serialization in order to support cross cluster
  search
- Avoid race when shutting down controller processes
- Fix handling of document failure exception in InternalEngine
- Ensure remote cluster is connected before fetching `_field_caps`

Java API::
- BulkProcessor flush runnable preserves the thread context from
  creation time

Java High Level REST Client::
- Make RestHighLevelClient's Request class public
- Forbid direct usage of ContentType.create() methods
- Make ShardSearchTarget optional when parsing ShardSearchFailure

Java REST Client::
- Better message text for ResponseException
- rest-client-sniffer: configurable threadfactory

Logging::
- Allow not configure logging without config

Mapping::
- Allow copying from a field to another field that belongs to the same
  nested object.
- Fixed bug that mapper_parsing_exception is thrown for numeric field
  with ignore_malformed=true when inserting "NaN"
- Coerce decimal strings for whole number types by truncating the
  decimal part
- Fix parsing of ip range queries.
- Disable date field mapping changing
- Correctly enable _all for older 5.x indices
- token_count datatype should handle null value
- keep _parent field while updating child type mapping
- ICUCollationKeywordFieldMapper use SortedSetDocValuesField
- Fix serialization of the `_all` field.

More Like This::
- Pass over _routing value with more_like_this items to be retrieved

NOT CLASSIFIED::
- DocumentMissingException during Logstash scripted upsert
- An assertion trips when master opens an index from before 5.x

Nested Docs::
- In case of a single type the _id field should be added to the nested
  document instead of _uid field
- Inner hits source filtering not working

Network::
- Fixed ByteBuf leaking in
  org.elasticsearch.http.netty4.Netty4HttpRequestHandler
- Check for closed connection while opening
- Ensure pending transport handlers are invoked for all channel
  failures
- Notify onConnectionClosed rather than onNodeDisconnect to prune
  transport handlers
- Release pipelined http responses on close
- Fix error message if an incompatible node connects

Packaging::
- Fix handling of Windows paths containing parentheses
- Exit Windows scripts promptly on failure
- Pass config path as a system property
- ES_HOME needs to be made absolute before attempt at traversal
- Fix elasticsearch-keystore handling of path.conf
- Stop disabling explicit GC
- Avoid failing install if system-sysctl is masked
- Get short path name for native controllers
- When stopping via systemd only kill the JVM, not its control group
- remove remaining references to scripts directory
- Handle parentheses in batch file path
- Detect modified keystore on package removal
- Create keystore on RPM and Debian package install
- Add safer empty variable checking for Windows
- Export HOSTNAME environment variable
- Fix daemonization command status test
- Set RuntimeDirectory in systemd service

Parent/Child::
- The default _parent field should not try to load global ordinals

Percolator::
- Also support query extraction for queries wrapped inside a
  ESToParentBlockJoinQuery
- Fix range queries with date range based on current time in
  percolator queries.

Plugin Analysis Kuromoji::
- Fix kuromoji default stoptags

Plugin Analysis Phonetic::
- Fix beidermorse phonetic token filter for unspecified `languageset`

Plugin Discovery File::
- Fix discovery-file plugin to use custom config path

Plugin Ingest Attachment::
- Add missing mime4j library

Plugin Lang Painless::
- Painless: allow doubles to be casted to longs.

Plugin Repository Azure::
- Azure snapshots can not be restored anymore
- Snapshot : azure module - accelerate the listing of files (used in
  delete snapshot)
- Use Azure upload method instead of our own implementation
- Make calls to CloudBlobContainer#exists privileged

Plugin Repository GCS::
- Ensure that gcs client creation is privileged

Plugin Repository HDFS::
- Add Log4j to SLF4J binding for repository-hdfs
- Upgrading HDFS Repository Plugin to use HDFS 2.8.1 Client

Plugin Repository S3::
- Avoid SecurityException in repository-S3 on
  DefaultS3OutputStream.flush()
- Wrap getCredentials() in a doPrivileged() block

Plugins::
- X-Pack plugin download fails on Windows desktop
- Fix plugin installation permissions

Query DSL::
- Fixed incomplete JSON body on count request making
  org.elasticsearch.rest.action.RestActions#parseTopLevelQueryBuilder
go into endless loop
- SpanNearQueryBuilder should return the inner clause when a single
  clause is provided
- Refactor field expansion for match, multi_match and query_string
  query
- WrapperQueryBuilder should also rewrite the parsed query

REST::
- Rest test fixes
- Fix inconsistencies in the rest api specs for cat.snapshots
- Fix inconsistencies in the rest api specs for *_script
- exists template needs a template name
- Fix handling of invalid error trace parameter
- Fix handling of exceptions thrown on HEAD requests
- Fixed NPEs caused by requests without content.
- Fix get mappings HEAD requests

Recovery::
- Close translog view after primary-replica resync

Reindex API::
- Fix update_by_query's default size parameter
- Reindex: don't duplicate _source parameter
- Add qa module that tests reindex-from-remote against pre-5.0
  versions of Elasticsearch

Scroll::
- Fix single shard scroll within a cluster with nodes in version `>=
  5.3` and `<= 5.3`

Search::
- Fail query when a sort is provided in conjunction with rescorers
- Let search phases override max concurrent requests
- Avoid stack overflow on search phases
- Fix search_after with geo distance sorting
- Fix serialization errors when cross cluster search goes to a single
  shard
- Early termination with index sorting should not set terminated_early
  in the response
- Format doc values fields.
- Fix term(s) query for range field
- Caching a MinDocQuery can lead to wrong results.
- Fix random score generation when no seed is provided.
- Merge FunctionScoreQuery and FiltersFunctionScoreQuery
- Respect cluster alias in `_index` aggs and queries
- First increment shard stats before notifying and potentially sending
  response
- Remove assertion about deviation when casting to a float.
- Prevent skipping shards if a suggest builder is present
- Ensure remote cluster alias is preserved in inner hits aggs
- Do not search locally if remote index pattern resolves to no indices
- Adds check for negative search request size
- Make sure range queries are correctly profiled.
- Fix RangeFieldMapper rangeQuery to properly handle relations
- Fix ExpandSearchPhase when response contains no hits
- Refactor simple_query_string to handle text part like multi_match
  and query_string
- Fix `_exists_` in query_string on empty indices.
- Fix script field sort returning Double.MAX_VALUE for all documents
- Compute the took time of the query after the expand phase of field
  collapsing

Sequence IDs::
- Fire global checkpoint sync under system context
- Fix pre-6.0 response to unknown replication actions
- Track local checkpoint on primary immediately
- Initialize max unsafe auto ID timestamp on shrink
- Use correct primary term for replicating NOOPs
- Handle already closed while filling gaps
- TranslogWriter.assertNoSeqNumberConflict failure
- Avoid losing ops in file-based recovery
- Handle primary failure handling replica response

Settings::
- Emit settings deprecation logging on empty update
- Fix filtering for ListSetting
- Fix settings serialization to not serialize secure settings or not
  take the total size into account
- Keystore CLI should use the AddFileKeyStoreCommand for files
- Allow resetting settings that use an IP validator
- Updating an unrecognized setting should error out with that reason
- Settings: Fix setting groups to include secure settings

Similarities::
- Add boolean similarity to built in similarity types

Snapshot/Restore::
- Snapshot/Restore: better handle incorrect chunk_size settings in FS
  repo
- Snapshot/Restore: Ensure that shard failure reasons are correctly
  stored in CS
- Output all empty snapshot info fields if in verbose mode
- Remove redundant and broken MD5 checksum from repository-s3
- Consolidates the logic for cleaning up snapshots on master election
- Removes completed snapshot from cluster state on master change
- Keep snapshot restore state and routing table in sync
- Master failover during snapshotting could leave the snapshot
  incomplete
- Fix inefficient (worst case exponential) loading of snapshot
  repository

Stats::
- Fix RestGetAction name typo
- Keep cumulative elapsed scroll time in microseconds
- _nodes/stats should not fail due to concurrent
  AlreadyClosedException
- Avoid double decrement on current query counter
- Adjust available and free bytes to be non-negative on huge FSes

Suggesters::
- Fix division by zero in phrase suggester that causes assertion to
  fail
- Context suggester should filter doc values field
- Fix context suggester to read values from keyword type field

Templates::
- Tests: Fix FullClusterRestartIT.testSnapshotRestore test failing in
  6.x

Translog::
- Fix Translog.Delete serialization for sequence numbers

Upgrade API::
- Upgrade API: fix excessive logging and unnecessary template updates

=== Regressions

Bulk::
- Only re-parse operation if a mapping update was needed

Highlighting::
- Fix Fast Vector Highlighter NPE on match phrase prefix

Search::
- Always use DisjunctionMaxQuery to build cross fields disjunction

Sequence IDs::
- Indexing performance degradation in 6.0.0-beta1

=== Upgrades

Core::
- Upgrade to Lucene 7.0.0

Logging::
- Upgrade to Log4j 2.9.1

Network::
- Upgrade to Netty 4.1.13.Final

Plugin Ingest Attachment::
- Update to Tika 1.14

Upgrade API::
- Improve stability and logging of TemplateUpgradeServiceIT tests

(fhajny)

doc: Updated archivers/libzip to 1.3.2

(wiz)

libzip: update to 1.3.2.

1.3.2 [2017-11-20]
==================
* Fix bug introduced in last: zip_t was erroneously freed if zip_close() failed.

(wiz)

doc/TODO: make myself responsible for nmh 1.7 update

(Everything is in pkgsrc-wip and probably ready to be imported as-is
but a commit message with pkgsrc and nmh changes need to be filled)

(leot)

xentools48: use pkgsrc iasl, build is broken with iasl 20171110

From Chavdar Ivanov in PR pkg/52732

(maya)

bash: comment broken option, remove dead master sites,
remove conflict with non-existent package.

PR pkg/51519

(maya)

I'm sorry, should call res_ndestroy() instead of res_nclose() to
avoid an fd leak.
Bump PKGREVISION.

(he)

doc: Updated www/contao44 to 4.4.8nb1

(taca)

www/contao44: Update due to repacakge.

Add DIST_SUBDIR due to repacakging.

Bump PKGREVISION.

(taca)

Updated sysutils/pciutils to 3.5.6.
Updated sysutils/intel-microcode-netbsd to 20171117.

(msaitoh)

Update intel-microcode-netbsd to 20171117.

Added:
Family 6, Model 0x55, Stepping 0x4
(Xeon Scalable (Skylake))
00050654-5

Family 6, Model 0x5c, Stepping 0x9
(Atom (Goldmont))
000506c9-0
000506c9-1

Family 6, Model 0x7a, Stepping 0x1
(Atom (Goldmont Plus))
000706a1-0

Family 6, Model 0x9e, Stepping 0x[ab]
(7th gen Core (Kaby Lake))
000906ea-1
000906ea-5
000906eb-1

Updated:
Family 6, Model 0x55, Stepping 0x4
(Xeon Scalable (Skylake))
00050654-0
00050654-1
00050654-2
00050654-4
00050654-7

Family 6, Model 0x8e, Stepping 0xa
(7th gen Core (Kaby Lake))
000806ea-6
000806ea-7

(msaitoh)

Update pciutils to 3.5.6.

2017-11-17  Martin Mares <mj@ucw.cz>

* Released as 3.5.6.

* Improvements of the FreeBSD back-end: read-only access for non-root,
  support DragonFly BSD, support extended config space.

* MN VPD keyword is decoded correctly.

* As usually, updated pci.ids to the current snapshot of the database.

(msaitoh)

doc/TODO: add nmh-1.7.

(wiz)

Updated sysutils/u-boot-pinebook to 2017.09

(jmcneill)

Update to 2017.09

(jmcneill)

Note update of graphics/tiff to 4.0.9.

(he)

Update tiff to version 4.0.9.

Pkgsrc changes:
* Adapt PLIST, remove patches for now-integrated bugfixes.

Upstream changes:
* Many changes related to security & stability.
  See the source's ChangeLog for the details.

(he)

doc: Updated archivers/libzip to 1.3.1

(wiz)

libzip: update to 1.3.1.

1.3.1 [2017-11-19]
==================

* Install zipconf.h into ${PREFIX}/include
* Add zip_libzip_version()
* Fix AES tests on Linux

(wiz)

doc/TODO: add some

+ audacity-2.2.0, binutils-2.29.1, dbus-1.12.2, gtk3-3.22.26,
  gtkmm-3.22.2, haproxy-1.8, harfbuzz-1.7.1, libgsf-1.14.42,
  nickle-2.81, pango-1.40.14, plasma-5.11, poppler-0.61.1,
  py-setuptools-36.7.2, tiff-4.0.9, vala-0.38.3, vim-8.0.1316,
  vim-share-8.0.1316, x264-devel-20171118.

(wiz)

doc: Updated devel/mk-configure to 0.29.2

(cheusov)

Update mk-configure to 0.29.2

  Fix MKC_FEATURES "fgenln"
  mkc_check_common.sh is installed to libexec directory

(cheusov)

doc: Updated editors/nano to 2.9.0

(wiedi)

doc: Updated net/mikutter to 3.5.15

(tsutsui)

nano: updated to 2.9.0

2017.11.18 - GNU nano 2.9.0 "Eta" introduces the ability to record and
replay keystrokes (M-: to start and stop recording, M-;
to play the macro back), makes ^Q and ^S do something
useful by default (^Q starts a backward search, and ^S
saves the current file), changes ^W to start always a
forward search, shows the number of open buffers (when
more than one) in the title bar, no longer asks to press
Enter when there are errors in an rc file, retires the
options '--quiet' and 'set quiet' and 'set backwards',
makes indenting and unindenting undoable, will look in
$XDG_CONFIG_HOME for a nanorc file and in $XDG_DATA_HOME
for the history files, adds a history stack for executed
commands (^R^X), does not overwrite the position-history
file of another nano, and fixes a score of tiny bugs.

(wiedi)

mikutter: update to 3.5.15.

Upstream change:

- use Ruby-GNOME2 3.2.0

(tsutsui)

doc: Updated ruby-gnome2 packages to 3.2.0

(tsutsui)

ruby-gnome2: update to 3.2.0.

Upstream changes (from NEWS):

== Ruby-GNOME2 3.2.0: 2017-11-07

This is a bug fix release for Ruby/Poppler.

=== Changes

==== Ruby/GLib

  * Improvements

    * (({GLib::IOChannel#create_watch})): Supported.
      [GitHub#1106][Reported by kspt-johs]

    * (({GLib::IOChannel#create_watch})): Supported.

==== Ruby/GTK3

  * Improvements

    * Added a sample.
      [GitHub#1109][Patch by cedlemo]

==== Ruby/Poppler

  * Improvements

    * (({Poppler#Document#size})): Added again.

    * (({Poppler#Document#pages})): Added again.

  * Fixes

    * Updated dependencies.
      [GitHub#1107][Reported by OBATA Akio]

    * (({Poppler#Document#initialize(:data => data)})): Added
      workaround. It should be fixed in upstream.

=== Thanks

  * kspt-johs

  * OBATA Akio

  * cedlemo

(tsutsui)

Updated www/mediawiki to 1.29.2

(wen)

Update to 1.29.2

Upstream changes:
MediaWiki 1.29.2

This is a security and maintenance release of the MediaWiki 1.29 branch.
Changes since 1.29.1

    (T166757) Avoid scoped lock errors in Category::refreshCounts() due to nesting.
    (T175439) Unbreak Postgres Updater when setting defaults for a column.
    (T160298) Remove use of implicitGroupBy() in ActiveUsersPager.
    Fixed login button label to accept RawMessage.
    Fixed case of SpecialRecentChanges class usage.
    (T174255) Declare uploadCount property in importDump.php.
    (T163646) Pass a string not an int to mysql_real_escape_string().
    (T180143) Bump justinrainbow/json-schema development dependency to ~5.2.
    Updated dev dependancy phpunit/phpunit from v4.8.35 to v4.8.36.
    (T178451) SECURITY: Potential XSS when $wgShowExceptionDetails = false and browser sends non-standard url escaping.
    (T165846) SECURITY: BotPassword login attempts weren't throttled.
    (T128209) SECURITY: Reflected File Download from api.php.
    (T134100) SECURITY: Do not reveal if user exists during login failure.
    (T176247) SECURITY: Ensure Message::rawParams can't lead to XSS.
    (T125163) SECURITY: Make anchor for headlines escape > and <.
    (T180237) SECURITY: Protect vendor folder with .htaccess.
    (T180231) SECURITY: Remove PHPUnit file with known RCE if exists in update.php.
    (T124404) SECURITY: XSS in langconverter when regex hits pcre.backtrack_limit.
    (T119158) SECURITY: Handle -{}- syntax in attributes safely.
    (T180488) (T125177) "api.log contains passwords in plaintext" wasn't correctly fixed in all branches in the previous security release.

(wen)

Renamed sysutils/u-boot-sinovoip-bpi-m3 to sysutils/u-boot-bananapi-m3

(jmcneill)

Rename u-boot-sinovoip-bpi-m3 to u-boot-bananapi-m3 -- upstream made the
same name change to the .dts files.

(jmcneill)

doc: Updated multimedia/gst-plugins1-libav to 1.12.3nb1

(prlw1)

gst-plugins1-libav: fix mp4 playback

From Jana Saout in https://bugzilla.gnome.org/show_bug.cgi?id=789193

Not meant as "fix" but as temporary workaround for people bothered
by this issue:

This quick & dirty hack replaces the deprecated call to
avcodec_decode_video2 by calls to avcodec_send_packet and
avcodec_receive_frame.

Note that this is hack is really ugly, because the API really isn't
meant to be used this way (the assumption is that each call to
avcodec_send_packet will produce at most one new frame) and also
the error handling and probably other cases are not handled correctly.

However, MP4 videos seem to play fine for me.

(prlw1)

Note update of {py-,}protobuf to version 3.5.0.

(he)

Upgrade protobuf to version 3.5.0.

Pkgsrc changes:
* Adapt PLIST.
* After exchange, set MAINTAINER to pkgsrc-users@

Upstream changes:

2017-11-13 version 3.5.0 (C++/Java/Python/PHP/Objective-C/C#/Ruby/JavaScript)
  Planned Future Changes
  * Make C++ implementation C++11 only: we plan to require C++11 to build
    protobuf code starting from 3.6.0 release. Please join this github issue:
    https://github.com/google/protobuf/issues/2780 to provide your feedback.

  General
  * Unknown fields are now preserved in proto3 for most of the language
    implementations for proto3 by default. See the per-language section for
    details.
  * reserve keyword are now supported in enums

  C++
  * Proto3 messages are now preserving unknown fields by default. If you rely on
    unknowns fields being dropped. Please use DiscardUnknownFields() explicitly.
  * Deprecated the unsafe_arena_release_* and unsafe_arena_add_allocated_*
    methods for string fields.
  * Added move constructor and move assignment to RepeatedField,
    RepeatedPtrField and google::protobuf::Any.
  * Added perfect forwarding in Arena::CreateMessage
  * In-progress experimental support for implicit weak fields with lite protos.
    This feature allows the linker to strip out more unused messages and reduce
    binary size.
  * Various performance optimizations.

  Java
  * Proto3 messages are now preserving unknown fields by default. If you'd like
    to drop unknown fields, please use the DiscardUnknownFieldsParser API. For
    example:
      Parser<Foo> parser = DiscardUnknownFieldsParser.wrap(Foo.parser());
      Foo foo = parser.parseFrom(input);
  * Added a new CodedInputStream decoder for Iterable<ByteBuffer> with direct
    ByteBuffers.
  * TextFormat now prints unknown length-delimited fields as messages if
    possible.
  * FieldMaskUtil.merge() no longer creates unnecessary empty messages when a
    message field is unset in both source message and destination message.
  * Various performance optimizations.

  Python
  * Proto3 messages are now preserving unknown fields by default. Use
    message.DiscardUnknownFields() to drop unknown fields.
  * Add FieldDescriptor.file in generated code.
  * Add descriptor pool FindOneofByName in pure python.
  * Change unknown enum values into unknown field set .
  * Add more Python dict/list compatibility for Struct/ListValue.
  * Add utf-8 support for text_format.Merge()/Parse().
  * Support numeric unknown enum values for proto3 JSON format.
  * Add warning for Unexpected end-group tag in cpp extension.

  PHP
  * Proto3 messages are now preserving unknown fields.
  * Provide well known type messages in runtime.
  * Add prefix "PB" to generated class of reserved names.
  * Fixed all conformance tests for encode/decode json in php runtime. C
    extension needs more work.

  Objective-C
  * Fixed some issues around copying of messages with unknown fields and then
    mutating the unknown fields in the copy.

  C#
  * Added unknown field support in JsonParser.
  * Fixed oneof message field merge.
  * Simplify parsing messages from array slices.

  Ruby
  * Unknown fields are now preserved by default.
  * Fixed several bugs for segment fault.

  Javascript
  * Decoder can handle both paced and unpacked data no matter how the proto is
    defined.
  * Decoder now accept long varint for 32 bit integers.

(he)

Default build without xft - there are problems with switching fonts,
and we end up with a proportional font, which defeats the primary use
of this editor.

(hauke)

Return to using the same system root as editors/xemacs{,-packages}.

(hauke)

The standards.info provided by XEmacs conflicts with the one installed
by devel/autoconf, which is probably newer.

(hauke)

Updated www/libsass, www/sassc

(adam)

libsass/sassc: updated to 3.4.7

3.4.7:
Features
* Implement exponents for numbers
* Implement long file path support for Windows

Fixes
* Error on quoted string in simple selector parsing
* Do not compress colors in selectors
* Fix issue when passing restargs to call
* Fix issue with attribute selector unification
* Improve wrapped pseudo selector handling
* Improve extend of wrapped pseudo selectors
* Fix propagation of named rest arguments
* Do not output invisible support blocks
* Various build makefile and CI related fixes

(adam)

Updated security/libassuan2, time/py-pendulum

(adam)

py-pendulum: updated to 1.3.2

1.3.2
Fixed instance() behaviour with pytz offsets.
Fixed deepcopy / pickle compatibility for TimezoneInfo.

(adam)

libassuan2: updated to 2.4.4

Changes in version 2.4.4:
* New configure option --disable-doc.
* Fix the use of nanosleep.
* Portability fixes for OpenBSD and macOS.
* Portability fix for systems not defining INADDR_LOOPBACK.
* Improve descriptor passing test to fix a Cygwin build problem.

(adam)