--- - branch: MAIN date: Mon Nov 20 15:55:16 UTC 2017 files: - new: '1.20' old: '1.19' path: pkgsrc/sysutils/beats/Makefile pathrev: pkgsrc/sysutils/beats/Makefile@1.20 type: modified - new: '1.2' old: '1.1' path: pkgsrc/sysutils/beats/PLIST pathrev: pkgsrc/sysutils/beats/PLIST@1.2 type: modified - new: '1.17' old: '1.16' path: pkgsrc/sysutils/beats/distinfo pathrev: pkgsrc/sysutils/beats/distinfo@1.17 type: modified id: 20171120T155516Z.926d2f9744e9170837552a827f27e29bc9a3e07a log: | Update sysutils/beats to 6.0.0. === Beats version 6.0.0 ==== Breaking changes Affecting all Beats - The log directory (`path.log`) for Windows services is now set to `C:\ProgramData\[beatname]\logs`. - The _all field is disabled in Elasticsearch 6.0. - Fail if removed setting output.X.flush_interval is explicitly configured. - Rename the `/usr/bin/beatname.sh` script (e.g. `metricbeat.sh`) to `/usr/bin/beatname`. - Beat does not start if elasticsearch index pattern was modified but not the template name and pattern. - Fail if removed setting output.X.flush_interval is explicitly configured. - Rename `kubernetes` processor to `add_kubernetes_metadata`. - Rename `.full.yml` config files to `*.reference.yml`. - The `scripts/import_dashboards` is removed from packages. Use the `setup` command instead. - Change format of the saved kibana dashboards to have a single JSON file for each dashboard - Rename `configtest` command to `test config`. - Remove setting `queue_size` and `bulk_queue_size`. - Remove setting `dashboard.snapshot` and `dashboard.snapshot_url`. - Beats can no longer be launched from Windows Explorer (GUI), command line is required. Filebeat - Rename `input_type` field to `prospector.type` - The `@metadata.type` field, added by the Logstash output, is now hardcoded to `doc` and will be removed in future versions. Metricbeat - Change all `system.cpu..pct` metrics to be scaled by the number of CPU cores. - Remove filters setting from metricbeat modules. - Added `type` field to filesystem metrics. Heartbeat - Renamed the heartbeat RPM/DEB name to `heartbeat-elastic`. Packetbeat - Remove not-working `runoptions.uid` and `runoptions.gid` options in Packetbeat. - Remove the already unsupported `pf_ring` sniffer option. Auditbeat - Changed file metricset config to make `file.paths` a list instead of a dictionary. ==== Bugfixes Affecting all Beats - Fix data race accessing watched containers. - Do not require template if index change and template disabled - Fix missing ACK in redis output. - Fix the `/usr/bin/beatname` script to accept `-d ""` as a parameter. - Combine `fields.yml` properties when they are defined in different sources. - Keep Docker & Kubernetes pod metadata after container dies while they are needed by processors. - Fix `fields.yml` lookup when using `export template` with a custom `path.config` param. - Remove runner creation from every reload check - Fix add_kubernetes_metadata matcher registry lookup. - Register kubernetes `field_format` matcher and remove logger in `Encode` API - Fix go plugins not loaded when beat starts - Add support for `initContainers` in `add_kubernetes_metadata` processor. - Eliminate deprecated _default_ mapping in 6.x - Fix pod name indexer to use both namespace, pod name to frame index key - Don't stop with error loading the ES template if the ES output is not enabled. - Fix race condition in internal logging rotator. - Normalize all times to UTC to ensure proper index naming. - Fix issue with loading dashboards to ES 6.0 when .kibana index did not already exist. - Fix importing the dashboards when the limit for max open files is too low. - Fix configuration documentation for kubernetes processor - Fix misspelling in `add_locale` configuration option for abbreviation. Filebeat - Fix machine learning jobs setup for dynamic modules. - Fix default paths for redis 4.0.1 logs on macOS - Fix Filebeat not starting if command line and modules configs are used together. - Fix double `@timestamp` field when JSON decoding was used. - Fix issue where the `fileset.module` could have the wrong value. - Fix race condition on harvester stopping with reloading enabled. - Fix recursive glob config parsing and resolution across restarts. - Allow string characters in user agent patch version (NGINX and Apache) - Fix grok pattern in filebeat module system/auth without hostname. Winlogbeat - Removed validation of top-level config keys. Metricbeat - Use `beat.name` instead of `beat.hostname` in the Host Overview dashboard. - Fix the loading of 5.x dashboards. - Fix a memory allocation issue where more memory was allocated than needed in the windows-perfmon metricset. - Don't start metricbeat if external modules config is wrong and reload is disabled - The MongoDB module now connects on each fetch, to avoid stopping the whole Metricbeat instance if MongoDB is not up when starting. - Fix kubernetes events module to be able to index time fields properly. - Fixed `cmd_set` and `cmd_get` being mixed in the Memcache module. - Added missing mongodb configuration file to the `modules.d` folder. - Fix wrong MySQL CRUD queries timelion visualization - Add new metrics to CPU metricsset - Fix issue affecting Windows services timing out at startup. - Fix incorrect docker.diskio.total metric calculation. - Vsphere module: used memory field corrected. - Set correct format for percent fields in memory module. - Fix a debug statement that said a module wrapper had stopped when it hadn't. - Use MemAvailable value from /proc/meminfo on Linux 3.14. - Fix panic when events were dropped by filters. - Add filtering to system filesystem metricset to remove relative mountpoints like those from Linux network namespaces. - Remove unnecessary print statement in schema apis. - Fix type of field `haproxy.stat.check.health.last`. Heartbeat - Fix monitor.name being empty by default. - Fix wrong event timestamps. Packetbeat - Fix missing length check in the PostgreSQL module. - Fix panic in ACK handler if event is dropped on blocked queue - Update flow timestamp on each packet being received. - Enabled /proc/net/tcp6 scanning and fixed ip v6 parsing. - Enable memcache filtering only if a port is specified in the config file. Auditbeat - Fix `file.max_file_size` config option for the audit file metricset. ==== Added Affecting all Beats - Enable flush timeout by default. - Add @metadata.version to events send to Logstash. - Add setting to enable/disable the slow start in logstash output. - Update init scripts to use the `test config` subcommand instead of the deprecated `-configtest` flag. - Get by default the credentials for connecting to Kibana from the Elasticsearch output configuration. - Added `cloud.id` and `cloud.auth` settings, for simplifying using Beats with the Elastic Cloud. - Add lz4 compression support to kafka output. - Add newer kafka versions to kafka output. - Configure the index name when loading the dashboards and the index pattern. - New cli subcommands interface. - Allow source path matching in `add_docker_metadata` processor. - Add support for analyzers and multifields in fields.yml. - Add support for JSON logging. - Add `test output` command, to test Elasticsearch and Logstash output settings. - Introduce configurable event queue settings: queue.mem.events, queue.mem.flush.min_events and queue.mem.flush.timeout. - Enable pipelining in Logstash output by default. - Added 'result' field to Elasticsearch QueryResult struct for compatibility with 6.x Index and Delete API responses. - The sample dashboards are now included in the Beats packages. - Add `pattern` option to be used in the fields.yml to specify the pattern for a number field. - Upgraded to Golang 1.8.3. - Added the possibility to set Elasticsearch mapping template settings from the Beat configuration file. - Add a variable to the SysV init scripts to make it easier to change the user. - Add the option to write the generated Elasticsearch mapping template into a file. - Add `instance_name` in GCE add_cloud_metadata processor. - Add `add_docker_metadata` processor. - Add `logging.files` `permissions` option. Filebeat - Add Kubernetes manifests to deploy Filebeat. - Changed the number of shards in the default configuration to 3. - Don't start filebeat if external modules/prospectors config is wrong and reload is disabled - Add `filebeat.registry_flush` setting, to delay the registry updates. - Add experimental Redis module. - Nginx module: use the first not-private IP address as the remote_ip. - Load Ingest Node pipelines when the Elasticsearch connection is established, instead of only once at startup. - Add support for loading Xpack Machine Learning configurations from the modules, and added sample configurations for the Nginx module. - Add udp prospector type. - Enabled Cgo which means libc is dynamically compiled. - Add Beta module config reloading mechanism - Remove spooler and publisher components and settings. - Added ability to sort harvested files. - Add experimental Redis slow log prospector type. Winlogbeat - Changed the number of shards in the default configuration to 3. - Add the ability to use LevelRaw if Level isn't populated in the event XML. Metricbeat - Add Kubernetes manifests to deploy Metricbeat. - Auto-select a hostname (based on the host on which the Beat is running) in the Host Overview dashboard. - Add `filesystem.ignore_types` to system module for ignoring filesystem types. - Add support to exclude labels from kubernetes pod metadata. - Add random startup delay to each metricset to avoid the thundering herd problem. - Add the ability to configure audit rules to the kernel module. - Add the ability to configure kernel's audit failure mode. - Add experimental Aerospike module. - Vsphere module: collect custom fields from virtual machines. - Add `test modules` command, to test modules expected output. - Add `processors` setting to metricbeat modules. - Support `npipe` protocol (Windows) in Docker module. - Add macOS implementation of the system diskio metricset. - Add process_summary metricset that records high level metrics about processes. - Add `kube-state-metrics` based metrics to `kubernetes` module - Add debug logging to Jolokia JMX metricset. - Add events metricset for kubernetes metricbeat module - Change Metricbeat default configuration file to be better optimized for most users. - Add experimental RabbitMQ module. - Add Kibana dashboard for the Kubernetes modules. Heartbeat - Changed the number of shards in the default configuration to 1. - Enabled Cgo which means libc is dynamically compiled. Packetbeat - Changed the number of shards in the default configuration to 3. Auditbeat - Changed the number of shards in the default configuration to 3. - Add support for receiving audit events using a multicast socket. - Added `file.hash_types` config option for controlling the hash types. - Added the ability to specify byte unit suffixes to `file.max_file_size`. - Add file integrity metricset to the audit module. ==== Deprecated Affecting all Beats - The `@metadata.type` field, added by the Logstash output, is deprecated, hardcoded to `doc` and will be removed in future versions. Filebeat - The `filebeat.config_dir` option is deprecated. Use `filebeat.config.prospector` options instead. - Deprecate `input_type` prospector config. Use `type` config option instead. === Beats version 5.6.4 - Fix race condition in internal logging rotator. - Add support for enabling TLS renegotiation. - Add setting to enable/disable the slow start in logstash output. - Packetbeat: Fix missing length check in the PostgreSQL module. module: pkgsrc subject: 'CVS commit: pkgsrc/sysutils/beats' unixtime: '1511193316' user: fhajny