Updated security/p5-IO-Socket-SSL to 2.015

(wiz)

Update to 2.015:

2.014 2015/05/13
- work around problem with IO::Socket::INET6 on windows, by explicitly using
  Domain AF_INET in the tests.
  Fixes RT#104226 reported by CHORNY

(wiz)

Updated x11/qt5-qtbase to 5.4.0nb4

(wiz)

Updated x11/qt4-libs to 4.8.6nb6

(wiz)

Set sysconfdir for qt* packages.
>From Kamil Rytarowski on tech-pkg.
Bump PKGREVISION.

(wiz)

Fix build on NetBSD. From adam@

(wiz)

Updated net/synergy to 1.7.3

(tnn)

Update to synergy-1.7.3.

This contains various bugfixes (hangs, excessive logging) related to SSL
protocol use.

There is no mention in release notes but please be advised that the client
appears to refuse to talk to servers running old versions due to protocol
incompatibility.

(tnn)

Restore PLIST.nis/PLIST.no-nis support, lost at some point during updates.
>From Kamil Rytarowski in PR 49915.

(dholland)

Expose pango in the bl3 file; PR 49925 from Kamil Rytarowski.

(dholland)

Updated databases/postgresql9[0-4]

(adam)

Updated databases/postgresql9[0-4]

(adam)

The PostgreSQL Global Development Group has released an update with multiple functionality and security fixes to all supported versions of the PostgreSQL database system, which includes minor versions 9.4.2, 9.3.7, 9.2.11, 9.1.16, and 9.0.20. The update contains a critical fix for a potential data corruption issue in PostgreSQL 9.3 and 9.4; users of those versions should update their servers at the next possible opportunity.

(adam)

+ asymptote-2.35, frotz-2.44, mupdf-1.7a [pkg/49918], openntpd-5.7p4
  [pkg/49930], php-owncloud-8.0.3 [pkg/49916], py-attic-0.16,
  qlandkartegt-1.8.1 [pkg/49910].

(wiz)

Updated fonts/umefont-ttf to 0.510

(ryoon)

Update to 0.510

Status:
Unadjusted glyphs (characters; 1st pass): Uninode 808e-8f58
Unadjusted glyphs (Bushu; 1st pass): Niku, Kusa-kanmuri, Mushi, Koromo,
Gon-ben, Ashi, Kuruma

(ryoon)

Updated converters/dos2unix to 7.2.2

(ryoon)

Update to 7.2.2

Changelog:
2015-05-22: Version 7.2.2. Check UTF-16 for invalid surrogate pairs.
Print system error and line number when UTF-16 conversion error occurs.
Bug fix: When conversion of an UTF-16 file with binary symbols was forced,
null characters were not written in the output.
Fixed symlink support on FreeBSD.

(ryoon)

Note update of net/exabgp to 3.4.11.

(he)

Update exabgp to version 3.4.11.

Pkgsrc changes: adapt PLIST; the healtcheck script moved

Upstream changes:

This release is a bug fix "mostly" release, back porting fixes
performed in master.

* Change: install healthcheck in bin
* Feature: Allow single line flow route requested by: Pavel Odintsov
* Feature: show route (extensive) can take a neighbor as parameter
  requested by: jtkdpu
* Fix: Do shutdown when waiting for a new connection to a peer
* Fix: Bad ASN enconding when ASN4 is not negotiated reported by:
  Orangefish on github
* Fix: Shutdown when waiting for a new outgoing connection to establish
* Fix: JSON counter reported by:
* Fix: JSON flow printing for source and destination
* Fix: Do not always locate exabgp.env reported by: Florian Obser
* Fix: Correctly drop root privileges reported by: Florian Obser
* Fix: validation of flow routes
* Fix: Python differences between Unix version breaking process forking
* Fix: Allow = with flowspec singleton reported by: Pavel Odintsov
* Fix: selfcheck feature
* Fix: do not refuse to parse multiple MP attributes in an update
* Fix: possible bug with attribute information due to caching

(he)

Remove duplicate entry.

(ryoon)

Updated www/apache-tomcat8 to 8.0.23

(ryoon)

Update to 8.0.23

Changelog:
The Apache Tomcat Project is proud to announce the release of version
8.0.23 of Apache Tomcat. Apache Tomcat 8.0.23 includes a numerous fixes
for issues identified in 8.0.22 as well as a number of other enhancements
and changes. The notable changes since 8.0.22 include:

    Fixed corruption issues with NIO2 and TLS
    Added a workaround for SPNEGO authentication and a JRE regression in Java 8 update 40 onwards
    Added the new HttpHeaderSecurityFilter

(ryoon)

Updated www/apache-tomcat7 to 7.0.62

(ryoon)

Update to 7.0.62

Changelog:
Tomcat 7.0.62 (violetagg)

    Catalina

        add Allow logging of the remote port in the access log using the format pattern %{remote}p. (rjung)
        fix 57765: When checking last modified times as part of the automatic deployment process, account for the fact that File.lastModified() has a resolution of one second to ensure that if a file has been modified within the last second, the latest version of the file is always used. Note that a side-effect of this change is that files with modification times in the future are treated as if they are unmodified. (markt)
        fix Align redeploy resource modification checking with reload modification checking so that now, in both cases, a change in modification time rather than an increase in modification time is used to determine if the resource has changed. (markt)
        fix Cleanup o.a.tomcat.util.digester.Digester from debug messages that do not give any valuable information. Patch provided by Polina Genova. (violetagg)
        fix 57772: When reloading a web application and a directory representing an expanded WAR needs to be deleted, delete the directory after the web application has been stopped rather than before to avoid potential ClassNotFoundExceptions. (markt)
        fix 57801: Improve the error message in the start script in case the PID read from the PID file is already owned by a process. (rjung)
        fix 57824: Correct a regression in the fix for 57252 that broke request listeners for non-async requests that triggered an error that was handled by the ErrorReportingValve. (markt/violetagg)
        fix 57841: Improve error logging during web application start. (markt)
        fix 57856: Ensure that any scheme/port changes implemented by the RemoteIpFilter also affect HttpServletResponse.sendRedirect(). (markt)
        fix 57896: Support defensive copying of "cookie" header so that unescaping double quotes in a cookie value does not corrupt original value of "cookie" header. This is an opt-in feature, enabled by org.apache.tomcat.util.http.ServerCookie.PRESERVE_COOKIE_HEADER system property. (kkolinko)

    Coyote

        fix 57779: When an I/O error occurs on a non-container thread only dispatch to a container thread to handle the error if using Servlet 3+ asynchronous processing. This avoids potential deadlocks if an application is performing I/O on a non-container thread without using the Servlet 3+ asynchronous API. (markt)
        fix 57833: When using JKS based keystores for NIO, ensure that the key alias is always converted to lower caes since that is what JKS key stores expect. Based on a patch by Santosh Giri Govind M. (markt)
        fix 57837: Add text/css to the default list of compressable MIME types. (markt)

    Jasper

        fix 57845: Ensure that, if the same JSP is accessed directly and via a <jsp-file> declaration in web.xml, updates to the JSP are visible (subject to the normal rules on re-compilation) regardless of how the JSP is accessed. (markt)
        fix 57855: Explicitly handle the case where a MethodExpression is invoked with null or the wrong number of parameters. Rather than failing with an ArrayIndexOutOfBoundsException or a NullPointerException throw an IllegalArgumentException with a useful error message. (markt)

    Cluster

        add Add new attribute that send all actions for session across Tomcat cluster nodes. (kfujino)
        fix Remove unused pathname attribute in mbean definition of BackupManager. (kfujino)
        fix 57338: Improve the ability of the ClusterSingleSignOn valve to handle nodes being added and removed from the Cluster at run time. (markt)
        fix Avoid unnecessary call of DeltaRequest.addSessionListener() in non-primary nodes. (kfujino)

    WebSocket

        fix 57762: Ensure that the WebSocket client correctly detects when the connection to the server is dropped. (markt)
        fix 57776: Revert the 8.0.21 fix for the permessage-deflate implementation and incorrect op-codes since the fix was unnecessary (the bug only affected trunk) and the fix broke rather than fixed permessage-deflate if an uncompressed message was converted into more than one compressed message. (markt)
        fix Fix log name typo in WsRemoteEndpointImplServer class, caused by a copy-paste. (markt/kkolinko)
        fix 57788: Avoid NPE when looking up a class hierarchy without finding anything. (remm)

    Web applications

        add 57759: Add information to the keyAlias documentation to make it clear that the order keys are read from the keystore is implementation dependent. (markt)
        fix 57864: Update the documentation web application to make it clearer that hex values are not valid for cluster send options. Based on a patch by Kyohei Nakamura. (markt)

    Tribes

        fix Fix a concurrency issue when a backup message that has all session data and a backup message that has diff data are processing at the same time. This fix ensures that MapOwner is set to ReplicatedMapEntry. (kfujino)
        fix Clarify the handling of Copy message and Copy nodes. (kfujino)
        fix Copy node does not need to send the entry data. It is enough to send only the node information of the entry. (kfujino)
        fix ReplicatedMap should send the Copy message when replicating. (kfujino)
        fix Fix behavior of ReplicatedMap when member has disappeared. If map entrprimary, rebuild the backup members. If primary node of map entry has disappeared, backup node is promoted to primary. (kfujino)
        fix When a map member has been added to ReplicatedMap, make sure to add it to backup nodes list of all other members.

(ryoon)

Updated games/lgogdownloader to 2.23

(wiz)

Update to 2.23:

LGOGDownloader 2.23
- Option to show wishlist is no longer saved to config file
- Added simple login check as a fallback to current complex HTTP login check
* This fallback method should work even if GOG changes the site layout as long as account page location remains the same
- Fixed error with login when cookies didn't exist but API login worked because config.cfg contained valid secret and token

LGOGDownloader 2.22
- Added option to download files by using file id
* --download-file "gamename/fileid"
- Fixed multiple DLCs when using cache (patch by: Ismo Toijala)
- Made progressbar display 10 second average download speed instead of average for the whole download
- Added support for new account page layout
- Added --wishlist option to show wishlist
- Makefile change: "make install" will now install debug build if it has been built

(wiz)

+ afl-1.79b, apr-1.5.2, clutter-1.22.2, convmv-2.0, dialog-1.2.20150513,
  gawk-4.1.3, gbrainy-2.25, gnome-commander-1.4.6, lgogdownloader-2.23,
  libconfig-1.5, libetonyek-0.1.2, nodejs-0.12.4, pinentry-0.9.2,
  pod2mdoc-0.2, postgresql93-9.3.7, py-cffi-1.0.3, py-cparser-2.13,
  py-cryptography-0.9, py-setuptools-16.0, py-test-2.7.1,
  py-twisted-15.2.0, py-virtualenv-13.0.1, webkit-gtk-2.8.3,
  whohas-0.29.1, wireshark-1.12.5.

(wiz)

Updated x11/xkeyboard-config to 2.15

(wiz)

Update to 2.15:

Artur Dryomov (1):
      symbols/typo: rework typographic layout.

Sergey Udaltsov (9):
      Add support for Latinamerican Dvorak layout
      added lk(us)
      fixed the license (removed GPL)
      Removed GPLv3 line
      Fixed Singhala -> Sinhala
      Updated cm layout
      Updated from TP
      Update from TP
      Release 2.15

Stefan Dirsch (1):
      Define <LSGT> key properly in basic layout

(wiz)

Updated x11/libXrandr to 1.5.0

(wiz)

Update to 1.5:

Add support for monitors and fixes requires on libX11.

Keith Packard (2):
      libXrandr: Clean up compiler warnings
      Add monitors, update to version 1.5 (v2)

Michael Joost (1):
      Remove fallback for _XEatDataWords, require libX11 1.6 for it

(wiz)

Updated x11/randrproto to 1.5.0

(wiz)

Update to 1.5.0:

XRandR 1.5 add monitor objects and tiling support.

Dave Airlie (3):
      randrproto: add tile property info for randr 1.5
      randrproto: clarify output XID lifetimes.
      randrproto 1.5.0

Keith Packard (2):
      Fix spelling errors in randrproto.txt
      Add Monitors, update version to 1.5 (v2)

(wiz)

Updated net/samba4 to 4.2.1nb3

(jperkin)

Move rc.d-specific MESSAGE file out of the way and only display if rc.d
is used.  Bump PKGREVISION.

(jperkin)

Various changes to fix the build and install on SunOS.

(jperkin)

Updated devel/userspace-rcu to 0.8.7

(pettai)

2015-04-28 Userspace RCU 0.8.7
        * Fix: deadlock when thread join is issued in read-side C.S.
        * Fix: rename RCU_DEBUG to DEBUG_RCU in urcu-qsbr.h
        * Mark braced-groups within expressions with __extension__
        * Fix: compat_futex_noasync race condition
        * Fix: documentation: urcu-pointer.h: s/rcu_dereference_pointer/rcu_dereference/
        * Fix: call rcu should call internal RCU API

(pettai)

Ensure we can find OpenSSL after rpath changes.

(jperkin)

Fix symbol visibility, package now builds on SunOS.

(jperkin)

Put back missing PLIST entries lost in previous update.

(jperkin)

Update to 31.7.0

* Sync with thunderbird-31.7.0.

(ryoon)

Updated www/firefox31 to 31.7.0

(ryoon)

pdated mail/thunderbird to 31.7.0

(ryoon)

Update to 31.7.0

Changelog:
Fixed in Thunderbird 31.7

    2015-57 Privilege escalation through IPC channel messages
    2015-54 Buffer overflow when parsing compressed XML
    2015-51 Use-after-free during text processing with vertical text enabled
    2015-48 Buffer overflow with SVG content and CSS
    2015-47 Buffer overflow parsing H.264 video with Linux Gstreamer
    2015-46 Miscellaneous memory safety hazards (rv:38.0 / rv:31.7)

(ryoon)

Update to 31.7.0

* Sync with firefox31-31.7.0.

(ryoon)

Put back SunOS support, and update for 4.0.  Build now completes but
install fails due to PLIST errors with the new libmono-2.0 handling.
Need to verify this on other platforms before fixing.

(jperkin)

Added patch to fix compilation error (on -current, at least)

(jaapb)

Updated www/firefox31 to 31.7.0

(ryoon)

Update to 31.7.0

Changelog:
Fixed in Firefox ESR 31.7

    2015-57 Privilege escalation through IPC channel messages
    2015-54 Buffer overflow when parsing compressed XML
    2015-51 Use-after-free during text processing with vertical text enabled
    2015-48 Buffer overflow with SVG content and CSS
    2015-47 Buffer overflow parsing H.264 video with Linux Gstreamer
    2015-46 Miscellaneous memory safety hazards (rv:38.0 / rv:31.7)

(ryoon)

Put back PLIST.monodoc handling.

(jperkin)

MAKE_JOBS_SAFE=yes is a nop, only =no makes sense, remove it.

(jperkin)

Add missing Compat.browser PLIST entries.

(jperkin)

Since distinfo is shared with security/clamav, this file is not used
any where.  I should not add this file, sigh.

(taca)

Added tex-europecv and tex-europecv-doc to SUBDIRs in Makefile

(jaapb)

Added print/tex-europecv-doc version 2014

(jaapb)

Added print/tex-europecv-doc, documentation for print/tex-europecv.

(jaapb)

Added print/tex-europecv version 2014

(jaapb)

Added print/tex-europecv, an unofficial class for European curricula vitae.

(jaapb)

Let's unconditionally depend on useradd on NetBSD. It has been in base
since 1.4 at least.

(joerg)

Updated databases/mysql-cluster to 7.4.6

(jnemeth)

Update to MySQL Cluster 7.4.6:

----

Changes in MySQL Cluster NDB 7.4.6 (5.6.24-ndb-7.4.6)

Bugs Fixed

    During backup, loading data from one SQL node followed by
repeated DELETE statements on the tables just loaded from a different
SQL node could lead to data node failures. (Bug #18949230)

    When an instance of NdbEventBuffer was destroyed, any references
to GCI operations that remained in the event buffer data list were
not freed. Now these are freed, and items from the event bufer data
list are returned to the free list when purging GCI containers.
(Bug #76165, Bug #20651661)

    When a bulk delete operation was committed early to avoid an
additional round trip, while also returning the number of affected
rows, but failed with a timeout error, an SQL node performed no
verification that the transaction was in the Committed state. (Bug
#74494, Bug #20092754)

    References: See also Bug #19873609.

Changes in MySQL Cluster NDB 7.4.5 (5.6.23-ndb-7.4.5)

Bugs Fixed

    In the event of a node failure during an initial node restart
followed by another node start, the restart of the the affected
node could hang with a START_INFOREQ that occurred while invalidation
of local checkpoints was still ongoing. (Bug #20546157, Bug #75916)

    References: See also Bug #34702.

    It was found during testing that problems could arise when the
node registered as the arbitrator disconnected or failed during
the arbitration process.

    In this situation, the node requesting arbitration could never
receive a positive acknowledgement from the registered arbitrator;
this node also lacked a stable set of members and could not initiate
selection of a new arbitrator.

    Now in such cases, when the arbitrator fails or loses contact
during arbitration, the requesting node immediately fails rather
than waiting to time out. (Bug #20538179)

    DROP DATABASE failed to remove the database when the database
directory contained a .ndb file which had no corresponding table
in NDB. Now, when executing DROP DATABASE, NDB performs an check
specifically for leftover .ndb files, and deletes any that it finds.
(Bug #20480035)

    References: See also Bug #44529.

    The maximum failure time calculation used to ensure that normal
node failure handling mechanisms are given time to handle survivable
cluster failures (before global checkpoint watchdog mechanisms
start to kill nodes due to GCP delays) was excessively conservative,
and neglected to consider that there can be at most number_of_data_nodes
/ NoOfReplicas node failures before the cluster can no longer
survive. Now the value of NoOfReplicas is properly taken into
account when performing this calculation.  (Bug #20069617, Bug
#20069624)

    References: See also Bug #19858151, Bug #20128256, Bug #20135976.

    When performing a restart, it was sometimes possible to find
a log end marker which had been written by a previous restart, and
that should have been invalidated. Now when when searching for the
last page to invalidate, the same search algorithm is used as when
searching for the last page of the log to read.  (Bug #76207, Bug
#20665205)

    During a node restart, if there was no global checkpoint
completed between the START_LCP_REQ for a local checkpoint and its
LCP_COMPLETE_REP it was possible for a comparison of the LCP ID
sent in the LCP_COMPLETE_REP signal with the internal value
SYSFILE->latestLCP_ID to fail. (Bug #76113, Bug #20631645)

    When sending LCP_FRAG_ORD signals as part of master takeover,
it is possible that the master is not synchronized with complete
accuracy in real time, so that some signals must be dropped. During
this time, the master can send a LCP_FRAG_ORD signal with its
lastFragmentFlag set even after the local checkpoint has been
completed. This enhancement causes this flag to persist until the
statrt of the next local checkpoint, which causes these signals to
be dropped as well.

    This change affects ndbd only; the issue described did not
occur with ndbmtd. (Bug #75964, Bug #20567730)

    When reading and copying transporter short signal data, it was
possible for the data to be copied back to the same signal with
overlapping memory. (Bug #75930, Bug #20553247)

    NDB node takeover code made the assumption that there would be
only one takeover record when starting a takeover, based on the
further assumption that the master node could never perform copying
of fragments. However, this is not the case in a system restart,
where a master node can have stale data and so need to perform such
copying to bring itself up to date. (Bug #75919, Bug #20546899)

    Cluster API: A scan operation, whether it is a single table
scan or a query scan used by a pushed join, stores the result set
in a buffer. This maximum size of this buffer is calculated and
preallocated before the scan operation is started. This buffer may
consume a considerable amount of memory; in some cases we observed
a 2 GB buffer footprint in tests that executed 100 parallel scans
with 2 single-threaded (ndbd) data nodes.  This memory consumption
was found to scale linearly with additional fragments.

    A number of root causes, listed here, were discovered that led
to this problem:

Result rows were unpacked to full NdbRecord format before
they were stored in the buffer. If only some but not all columns
of a table were selected, the buffer contained empty space (essentially
wasted).

Due to the buffer format being unpacked, VARCHAR and
VARBINARY columns always had to be allocated for the maximum size
defined for such columns.

BatchByteSize and MaxScanBatchSize values were not taken
into consideration as a limiting factor when calculating the maximum
buffer size.

    These issues became more evident in NDB 7.2 and later MySQL
Cluster release series. This was due to the fact buffer size is
scaled by BatchSize, and that the default value for this parameter
was increased fourfold (from 64 to 256) beginning with MySQL Cluster
NDB 7.2.1.

    This fix causes result rows to be buffered using the packed
format instead of the unpacked format; a buffered scan result row
is now not unpacked until it becomes the current row. In addition,
BatchByteSize and MaxScanBatchSize are now used as limiting factors
when calculating the required buffer size.

    Also as part of this fix, refactoring has been done to separate
handling of buffered (packed) from handling of unbuffered result
sets, and to remove code that had been unused since NDB 7.0 or
earlier. The NdbRecord class declaration has also been cleaned up
by removing a number of unused or redundant member variables.  (Bug
#73781, Bug #75599, Bug #19631350, Bug #20408733)

-----

Changes in MySQL Cluster NDB 7.4.4 (5.6.23-ndb-7.4.4)

Bugs Fixed

    When upgrading a MySQL Cluster from NDB 7.3 to NDB 7.4, the
first data node started with the NDB 7.4 data node binary caused
the master node (still running NDB 7.3) to fail with Error 2301,
then itself failed during Start Phase 5. (Bug #20608889)

    A memory leak in NDB event buffer allocation caused an event
to be leaked for each epoch. (Due to the fact that an SQL node uses
3 event buffers, each SQL node leaked 3 events per epoch.) This
meant that a MySQL Cluster mysqld leaked an amount of memory that
was inversely proportional to the size of TimeBetweenEpochs that
is, the smaller the value for this parameter, the greater the amount
of memory leaked per unit of time. (Bug #20539452)

    The values of the Ndb_last_commit_epoch_server and
Ndb_last_commit_epoch_session status variables were incorrectly
reported on some platforms. To correct this problem, these values
are now stored internally as long long, rather than long. (Bug
#20372169)

    When restoring a MySQL Cluster from backup, nodes that failed
and were restarted during restoration of another node became
unresponsive, which subsequently caused ndb_restore to fail and
exit. (Bug #20069066)

    When a data node fails or is being restarted, the remaining
nodes in the same nodegroup resend to subscribers any data which
they determine has not already been sent by the failed node.
Normally, when a data node (actually, the SUMA kernel block) has
sent all data belonging to an epoch for which it is responsible,
it sends a SUB_GCP_COMPLETE_REP signal, together with a count, to
all subscribers, each of which responds with a SUB_GCP_COMPLETE_ACK.
When SUMA receives this acknowledgment from all subscribers, it
reports this to the other nodes in the same nodegroup so that they
know that there is no need to resend this data in case of a subsequent
node failure. If a node failed before all subscribers sent this
acknowledgement but before all the other nodes in the same nodegroup
received it from the failing node, data for some epochs could be
sent (and reported as complete) twice, which could lead to an
unplanned shutdown.

    The fix for this issue adds to the count reported by
SUB_GCP_COMPLETE_ACK a list of identifiers which the receiver can
use to keep track of which buckets are completed and to ignore any
duplicate reported for an already completed bucket.  (Bug #17579998)

    The output format of SHOW CREATE TABLE for an NDB table containing
foreign key constraints did not match that for the equivalent InnoDB
table, which could lead to issues with some third-party applications.
(Bug #75515, Bug #20364309)

    An ALTER TABLE statement containing comments and a partitioning
option against an NDB table caused the SQL node on which it was
executed to fail. (Bug #74022, Bug #19667566)

    Cluster API: When a transaction is started from a cluster
connection, Table and Index schema objects may be passed to this
transaction for use. If these schema objects have been acquired
from a different connection (Ndb_cluster_connection object), they
can be deleted at any point by the deletion or disconnection of
the owning connection. This can leave a connection with invalid
schema objects, which causes an NDB API application to fail when
these are dereferenced.

    To avoid this problem, if your application uses multiple
connections, you can now set a check to detect sharing of schema
objects between connections when passing a schema object to a
transaction, using the NdbTransaction::setSchemaObjectOwnerChecks()
method added in this release. When this check is enabled, the schema
objects having the same names are acquired from the connection and
compared to the schema objects passed to the transaction. Failure
to match causes the application to fail with an error. (Bug #19785977)

    Cluster API: The increase in the default number of hashmap
buckets (DefaultHashMapSize API node configuration parameter) from
240 to 3480 in MySQL Cluster NDB 7.2.11 increased the size of the
internal DictHashMapInfo::HashMap type considerably.  This type
was allocated on the stack in some getTable() calls which could
lead to stack overflow issues for NDB API users.

    To avoid this problem, the hashmap is now dynamically allocated
from the heap. (Bug #19306793)

-----

Changes in MySQL Cluster NDB 7.4.3 (5.6.22-ndb-7.4.3)

Functionality Added or Changed

    Important Change; Cluster API: This release introduces an
epoch-driven Event API for the NDB API that supercedes the earlier
GCI-based model. The new version of this API also simplifies error
detection and handling, and monitoring of event buffer memory usage
has been been improved.

    New event handling methods for Ndb and NdbEventOperation added
by this change include NdbEventOperation::getEventType2(),
pollEvents2(), nextEvent2(), getHighestQueuedEpoch(),
getNextEventOpInEpoch2(), getEpoch(), isEmptyEpoch(), and isErrorEpoch.
The pollEvents(), nextEvent(), getLatestGCI(), getGCIEventOperations(),
isConsistent(), isConsistentGCI(), getEventType(), getGCI(),
getLatestGCI(), isOverrun(), hasError(), and clearError() methods
are deprecated beginning with the same release.

    Some (but not all) of the new methods act as replacements for
deprecated methods; not all of the deprecated methods map to new
ones. The Event Class, provides information as to which old methods
correspond to new ones.

    Error handling using the new API is no longer handled using
dedicated hasError() and clearError() methods, which are now
deprecated as previously noted. To support this change, TableEvent
now supports the values TE_EMPTY (empty epoch), TE_INCONSISTENT
(inconsistent epoch), and TE_OUT_OF_MEMORY (insufficient event
buffer memory).

    Event buffer memory management has also been improved with the
introduction of the get_eventbuffer_free_percent(),
set_eventbuffer_free_percent(), and get_eventbuffer_memory_usage()
methods, as well as a new NDB API error Free percent out of range
(error code 4123). Memory buffer usage can now be represented in
applications using the EventBufferMemoryUsage data structure, and
checked from MySQL client applications by reading the
ndb_eventbuffer_free_percent system variable.

    For more information, see the detailed descriptions for the
Ndb and NdbEventOperation methods listed. See also The Event::TableEvent
Type, as well as The EventBufferMemoryUsage Structure.

    Additional logging is now performed of internal states occurring
during system restarts such as waiting for node ID allocation and
master takeover of global and local checkpoints. (Bug #74316, Bug
#19795029)

    Added the MaxParallelCopyInstances data node configuration
parameter. In cases where the parallelism used during restart copy
phase (normally the number of LDMs up to a maximum of 16) is
excessive and leads to system overload, this parameter can be used
to override the default behavior by reducing the degree of parallelism
employed.

    Added the operations_per_fragment table to the ndbinfo information
database. Using this table, you can now obtain counts of operations
performed on a given fragment (or fragment replica).  Such operations
include reads, writes, updates, and deletes, scan and index operations
performed while executing them, and operations refused, as well as
information relating to rows scanned on and returned from a given
fragment replica. This table also provides information about
interpreted programs used as attribute values, and values returned
by them.

    Cluster API: Two new example programs, demonstrating reads and
writes of CHAR, VARCHAR, and VARBINARY column values, have been
added to storage/ndb/ndbapi-examples in the MySQL Cluster source
tree. For more information about these programs, including source
code listings, see NDB API Simple Array Example, and NDB API Simple
Array Example Using Adapter.

Bugs Fixed

    The global checkpoint commit and save protocols can be delayed
by various causes, including slow disk I/O. The DIH master node
monitors the progress of both of these protocols, and can enforce
a maximum lag time during which the protocols are stalled by killing
the node responsible for the lag when it reaches this maximum. This
DIH master GCP monitor mechanism did not perform its task more than
once per master node; that is, it failed to continue monitoring
after detecting and handling a GCP stop. (Bug #20128256)

    References: See also Bug #19858151, Bug #20069617, Bug #20062754.

    When running mysql_upgrade on a MySQL Cluster SQL node, the
expected drop of the performance_schema database on this node was
instead performed on all SQL nodes connected to the cluster.  (Bug
#20032861)

    The warning shown when an ALTER TABLE ALGORITHM=INPLACE ...
ADD COLUMN statement automatically changes a column's COLUMN_FORMAT
from FIXED to DYNAMIC now includes the name of the column whose
format was changed. (Bug #20009152, Bug #74795)

    The local checkpoint scan fragment watchdog and the global
checkpoint monitor can each exclude a node when it is too slow when
participating in their respective protocols. This exclusion was
implemented by simply asking the failing node to shut down, which
in case this was delayed (for whatever reason) could prolong the
duration of the GCP or LCP stall for other, unaffected nodes.

    To minimize this time, an isolation mechanism has been added
to both protocols whereby any other live nodes forcibly disconnect
the failing node after a predetermined amount of time. This allows
the failing node the opportunity to shut down gracefully (after
logging debugging and other information) if possible, but limits
the time that other nodes must wait for this to occur. Now, once
the remaining live nodes have processed the disconnection of any
failing nodes, they can commence failure handling and restart the
related protocol or protocol, even if the failed node takes an
excessiviely long time to shut down.  (Bug #19858151)

    References: See also Bug #20128256, Bug #20069617, Bug #20062754.

    The matrix of values used for thread configuration when applying
the setting of the MaxNoOfExecutionThreads configuration parameter
has been improved to align with support for greater numbers of LDM
threads. See Multi-Threading Configuration Parameters (ndbmtd),
for more information about the changes.  (Bug #75220, Bug #20215689)

    When a new node failed after connecting to the president but
not to any other live node, then reconnected and started again, a
live node that did not see the original connection retained old
state information. This caused the live node to send redundant
signals to the president, causing it to fail. (Bug #75218, Bug
#20215395)

    In the NDB kernel, it was possible for a TransporterFacade
object to reset a buffer while the data contained by the buffer
was being sent, which could lead to a race condition. (Bug #75041,
Bug #20112981)

    mysql_upgrade failed to drop and recreate the ndbinfo database
and its tables as expected. (Bug #74863, Bug #20031425)

    Due to a lack of memory barriers, MySQL Cluster programs such
as ndbmtd did not compile on POWER platforms. (Bug #74782, Bug
#20007248)

    In spite of the presence of a number of protection mechanisms
against overloading signal buffers, it was still in some cases
possible to do so. This fix adds block-level support in the NDB
kernel (in SimulatedBlock) to make signal buffer overload protection
more reliable than when implementing such protection on a case-by-case
basis. (Bug #74639, Bug #19928269)

    Copying of metadata during local checkpoints caused node restart
times to be highly variable which could make it difficult to diagnose
problems with restarts. The fix for this issue introduces signals
(including PAUSE_LCP_IDLE, PAUSE_LCP_REQUESTED, and
PAUSE_NOT_IN_LCP_COPY_META_DATA) to pause LCP execution and flush
LCP reports, making it possible to block LCP reporting at times
when LCPs during restarts become stalled in this fashion. (Bug
#74594, Bug #19898269)

    When a data node was restarted from its angel process (that
is, following a node failure), it could be allocated a new node ID
before failure handling was actually completed for the failed node.
(Bug #74564, Bug #19891507)

    In NDB version 7.4, node failure handling can require completing
checkpoints on up to 64 fragments. (This checkpointing is performed
by the DBLQH kernel block.) The requirement for master takeover to
wait for completion of all such checkpoints led in such cases to
excessive length of time for completion.

    To address these issues, the DBLQH kernel block can now report
that it is ready for master takeover before it has completed any
ongoing fragment checkpoints, and can continue processing these
while the system completes the master takeover. (Bug #74320, Bug
#19795217)

    Local checkpoints were sometimes started earlier than necessary
during node restarts, while the node was still waiting for copying
of the data distribution and data dictionary to complete.  (Bug
#74319, Bug #19795152)

    The check to determine when a node was restarting and so know
when to accelerate local checkpoints sometimes reported a false
positive. (Bug #74318, Bug #19795108)

    Values in different columns of the ndbinfo tables
disk_write_speed_aggregate and disk_write_speed_aggregate_node were
reported using differing multiples of bytes. Now all of these
columns display values in bytes.

    In addition, this fix corrects an error made when calculating
the standard deviations used in the std_dev_backup_lcp_speed_last_10sec,
std_dev_redo_speed_last_10sec, std_dev_backup_lcp_speed_last_60sec,
and std_dev_redo_speed_last_60sec columns of the
ndbinfo.disk_write_speed_aggregate table. (Bug #74317, Bug #19795072)

    Recursion in the internal method Dblqh::finishScanrec() led to
an attempt to create two list iterators with the same head.  This
regression was introduced during work done to optimize scans for
version 7.4 of the NDB storage engine. (Bug #73667, Bug #19480197)

    Transporter send buffers were not updated properly following
a failed send. (Bug #45043, Bug #20113145)

    Disk Data: An update on many rows of a large Disk Data table
could in some rare cases lead to node failure. In the event that
such problems are observed with very large transactions on Disk
Data tables you can now increase the number of page entries allocated
for disk page buffer memory by raising the value of the
DiskPageBufferEntries data node configuration parameter added in
this release. (Bug #19958804)

    Disk Data: In some cases, during DICT master takeover, the new
master could crash while attempting to roll forward an ongoing
schema transaction. (Bug #19875663, Bug #74510)

    Cluster API: It was possible to delete an Ndb_cluster_connection
object while there remained instances of Ndb using references to
it. Now the Ndb_cluster_connection destructor waits for all related
Ndb objects to be released before completing. (Bug #19999242)

    References: See also Bug #19846392.

-----

Changes in MySQL Cluster NDB 7.4.2 (5.6.21-ndb-7.4.2)

Functionality Added or Changed

    Added the restart_info table to the ndbinfo information database
to provide current status and timing information relating to node
and system restarts. By querying this table, you can observe the
progress of restarts in real time. (Bug #19795152)

    After adding new data nodes to the configuration file of a
MySQL Cluster having many API nodes, but prior to starting any of
the data node processes, API nodes tried to connect to these missing
data nodes several times per second, placing extra loads on management
nodes and the network. To reduce unnecessary traffic caused in this
way, it is now possible to control the amount of time that an API
node waits between attempts to connect to data nodes which fail to
respond; this is implemented in two new API node configuration
parameters StartConnectBackoffMaxTime and ConnectBackoffMaxTime.

    Time elapsed during node connection attempts is not taken into
account when applying these parameters, both of which are given in
milliseconds with approximately 100 ms resolution. As long as the
API node is not connected to any data nodes as described previously,
the value of the StartConnectBackoffMaxTime parameter is applied;
otherwise, ConnectBackoffMaxTime is used.

    In a MySQL Cluster with many unstarted data nodes, the values
of these parameters can be raised to circumvent connection attempts
to data nodes which have not yet begun to function in the cluster,
as well as moderate high traffic to management nodes.

    For more information about the behavior of these parameters,
see Defining SQL and Other API Nodes in a MySQL Cluster. (Bug
#17257842)

Bugs Fixed

    When performing a batched update, where one or more successful
write operations from the start of the batch were followed by write
operations which failed without being aborted (due to the AbortOption
being set to AO_IgnoreError), the failure handling for these by
the transaction coordinator leaked CommitAckMarker resources. (Bug
#19875710)

    References: This bug was introduced by Bug #19451060, Bug #73339.

    Online downgrades to MySQL Cluster NDB 7.3 failed when a MySQL
Cluster NDB 7.4 master attempted to request a local checkpoint with
32 fragments from a data node already running NDB 7.3, which supports
only 2 fragments for LCPs. Now in such cases, the NDB 7.4 master
determines how many fragments the data node can handle before making
the request. (Bug #19600834)

    The fix for a previous issue with the handling of multiple node
failures required determining the number of TC instances the failed
node was running, then taking them over. The mechanism to determine
this number sometimes provided an invalid result which caused the
number of TC instances in the failed node to be set to an excessively
high value. This in turn caused redundant takeover attempts, which
wasted time and had a negative impact on the processing of other
node failures and of global checkpoints. (Bug #19193927)

    References: This bug was introduced by Bug #18069334.

    The server side of an NDB transporter disconnected an incoming
client connection very quickly during the handshake phase if the
node at the server end was not yet ready to receive connections
from the other node. This led to problems when the client immediately
attempted once again to connect to the server socket, only to be
disconnected again, and so on in a repeating loop, until it suceeded.
Since each client connection attempt left behind a socket in
TIME_WAIT, the number of sockets in TIME_WAIT increased rapidly,
leading in turn to problems with the node on the server side of
the transporter.

    Further analysis of the problem and code showed that the root
of the problem lay in the handshake portion of the transporter
connection protocol. To keep the issue described previously from
occurring, the node at the server end now sends back a WAIT message
instead of disconnecting the socket when the node is not yet ready
to accept a handshake. This means that the client end should no
longer need to create a new socket for the next retry, but can
instead begin immediately with a new handshake hello message. (Bug
#17257842)

    Corrupted messages to data nodes sometimes went undetected,
causing a bad signal to be delivered to a block which aborted the
data node. This failure in combination with disconnecting nodes
could in turn cause the entire cluster to shut down.

    To keep this from happening, additional checks are now made
when unpacking signals received over TCP, including checks for byte
order, compression flag (which must not be used), and the length
of the next message in the receive buffer (if there is one).

    Whenever two consecutive unpacked messages fail the checks just
described, the current message is assumed to be corrupted. In this
case, the transporter is marked as having bad data and no more
unpacking of messages occurs until the transporter is reconnected.
In addition, an entry is written to the cluster log containing the
error as well as a hex dump of the corrupted message. (Bug #73843,
Bug #19582925)

    During restore operations, an attribute's maximum length was
used when reading variable-length attributes from the receive buffer
instead of the attribute's actual length. (Bug #73312, Bug #19236945)

-----

Changes in MySQL Cluster NDB 7.4.1 (5.6.20-ndb-7.4.1)

Node Restart Performance and Reporting Enhancements

    Performance: A number of performance and other improvements
have been made with regard to node starts and restarts. The following
list contains a brief description of each of these changes:

Before memory allocated on startup can be used, it must be
touched, causing the operating system to allocate the actual physical
memory needed. The process of touching each page of memory that
was allocated has now been multithreaded, with touch times on the
order of 3 times shorter than with a single thread when performed
by 16 threads.

When performing a node or system restart, it is necessary
to restore local checkpoints for the fragments. This process
previously used delayed signals at a point which was found to be
critical to performance; these have now been replaced with normal
(undelayed) signals, which should shorten significantly the time
required to back up a MySQL Cluster or to restore it from backup.

Previously, there could be at most 2 LDM instances active
with local checkpoints at any given time. Now, up to 16 LDMs can
be used for performing this task, which increases utilization of
available CPU power, and can speed up LCPs by a factor of 10, which
in turn can greatly improve restart times.

Better reporting of disk writes and increased control over
these also make up a large part of this work. New ndbinfo tables
disk_write_speed_base, disk_write_speed_aggregate, and
disk_write_speed_aggregate_node provide information about the speed
of disk writes for each LDM thread that is in use. The DiskCheckpointSpeed
and DiskCheckpointSpeedInRestart configuration parameters have been
deprecated, and are subject to removal in a future MySQL Cluster
release. This release adds the data node configuration parameters
MinDiskWriteSpeed, MaxDiskWriteSpeed, MaxDiskWriteSpeedOtherNodeRestart,
and MaxDiskWriteSpeedOwnRestart to control write speeds for LCPs
and backups when the present node, another node, or no node is
currently restarting.

For more information, see the descriptions of the ndbinfo
tables and MySQL Cluster configuration parameters named previously.

Reporting of MySQL Cluster start phases has been improved,
with more frequent printouts. New and better information about the
start phases and their implementation has also been provided in
the sources and documentation. See Summary of MySQL Cluster Start
Phases.

Improved Scan and SQL Processing

    Performance: Several internal methods relating to the NDB
receive thread have been optimized to make mysqld more efficient
in processing SQL applications with the NDB storage engine. In
particular, this work improves the performance of the
NdbReceiver::execTRANSID_AI() method, which is commonly used
to receive a record from the data nodes as part of a scan
operation. (Since the receiver thread sometimes has to process
millions of received records per second, it is critical that
this method does not perform unnecessary work, or tie up
resources that are not strictly needed.) The associated internal
functions receive_ndb_packed_record() and handleReceivedSignal()
methods have also been improved, and made more efficient.

Per-Fragment Memory Reporting

    Information about memory usage by individual fragments can now
be obtained from the memory_per_fragment view added in this release
to the ndbinfo information database. This information includes
pages having fixed, and variable element size, rows, fixed element
free slots, variable element free bytes, and hash index memory
usage. For information, see The ndbinfo memory_per_fragment Table.

Bugs Fixed

    In some cases, transporter receive buffers were reset by one
thread while being read by another. This happened when a race
condition occurred between a thread receiving data and another
thread initiating disconnect of the transporter (disconnection
clears this buffer). Concurrency logic has now been implemented to
keep this race from taking place. (Bug #19552283, Bug #73790)

    When a new data node started, API nodes were allowed to attempt
to register themselves with the data node for executing transactions
before the data node was ready. This forced the API node to wait
an extra heartbeat interval before trying again.

    To address this issue, a number of HA_ERR_NO_CONNECTION errors
(Error 4009) that could be issued during this time have been changed
to Cluster temporarily unavailable errors (Error 4035), which should
allow API nodes to use new data nodes more quickly than before. As
part of this fix, some errors which were incorrectly categorised
have been moved into the correct categories, and some errors which
are no longer used have been removed. (Bug #19524096, Bug #73758)

    Executing ALTER TABLE ... REORGANIZE PARTITION after increasing
the number of data nodes in the cluster from 4 to 16 led to a crash
of the data nodes. This issue was shown to be a regression caused
by previous fix which added a new dump handler using a dump code
that was already in use (7019), which caused the command to execute
two different handlers with different semantics. The new handler
was assigned a new DUMP code (7024).  (Bug #18550318)

    References: This bug is a regression of Bug #14220269.

    When certain queries generated signals having more than 18 data
words prior to a node failure, such signals were not written
correctly in the trace file. (Bug #18419554)

    Failure of multiple nodes while using ndbmtd with multiple TC
threads was not handled gracefully under a moderate amount of
traffic, which could in some cases lead to an unplanned shutdown
of the cluster. (Bug #18069334)

    For multithreaded data nodes, some threads do communicate often,
with the result that very old signals can remain at the top of the
signal buffers. When performing a thread trace, the signal dumper
calculated the latest signal ID from what it found in the signal
buffers, which meant that these old signals could be erroneously
counted as the newest ones. Now the signal ID counter is kept as
part of the thread state, and it is this value that is used when
dumping signals for trace files. (Bug #73842, Bug #19582807)

    Cluster API: When an NDB API client application received a
signal with an invalid block or signal number, NDB provided only
a very brief error message that did not accurately convey the nature
of the problem. Now in such cases, appropriate printouts are provided
when a bad signal or message is detected.  In addition, the message
length is now checked to make certain that it matches the size of
the embedded signal. (Bug #18426180)

-----

The following improvements to MySQL Cluster have been made in MySQL
Cluster NDB 7.4:

    Conflict detection and resolution enhancements.  A reserved
column name namespace NDB$ is now employed for exceptions table
metacolumns, allowing an arbitrary subset of main table columns to
be recorded, even if they are not part of the original table's
primary key.

    Recording the complete original primary key is no longer
required, due to the fact that matching against exceptions table
columns is now done by name and type only. It is now also possible
for you to record values of columns which not are part of the main
table's primary key in the exceptions table.

    Read conflict detection is now possible. All rows read by the
conflicting transaction are flagged, and logged in the exceptions
table. Rows inserted in the same transaction are not included among
the rows read or logged. This read tracking depends on the slave
having an exclusive read lock which requires setting
ndb_log_exclusive_reads in advance. See Read conflict detection
and resolution, for more information and examples.

    Existing exceptions tables remain supported. For more information,
see Section 18.6.11, "MySQL Cluster Replication Conflict Resolution".

    Circular ("active-active") replication improvements.  When
using a circular or "active-active" MySQL Cluster Replication
topology, you can assign one of the roles of primary of secondary
to a given MySQL Cluster using the ndb_slave_conflict_role server
system variable, which can be employed when failing over from a
MySQL Cluster acting as primary, or when using conflict detection
and resolution with NDB$EPOCH2() and NDB$EPOCH2_TRANS() (MySQL
Cluster NDB 7.4.2 and later), which support delete-delete conflict
handling.

    See the description of the ndb_slave_conflict_role variable,
as well as NDB$EPOCH2(), for more information. See also Section
18.6.11, MySQL Cluster Replication Conflict Resolution.

    Per-fragment memory usage reporting.  You can now obtain data
about memory usage by individual MySQL Cluster fragments from the
memory_per_fragment view, added in MySQL Cluster NDB 7.4.1 to the
ndbinfo information database. For more information, see Section
18.5.10.17, "The ndbinfo memory_per_fragment Table".

    Node restart improvements.  MySQL Cluster NDB 7.4 includes a
number of improvements which decrease the time needed for data
nodes to be restarted. These are described in the following list:

Memory allocated that is allocated on node startup cannot
be used until it has been, which causes the operating system to
set aside the actual physical memory required. In previous versions
of MySQL Cluster, the process of touching each page of memory that
was allocated was singlethreaded, which made it relatively
time-consuming. This process has now been reimplimented with
multithreading. In tests with 16 threads, touch times on the order
of 3 times shorter than with a single thread were observed.

Increased parallelization of local checkpoints; in MySQL
Cluster NDB 7.4, LCPs now support 32 fragments rather than 2 as
before. This greatly increases utilization of CPU power that would
otherwise go unused, and can make LCPs faster by up to a factor of
10; this speedup in turn can greatly improve node restart times.

The degree of parallelization used for the node copy phase
during node and system restarts can be controlled in MySQL Cluster
NDB 7.4.3 and later by setting the MaxParallelCopyInstances data
node configuration parameter to a nonzero value.

Reporting on disk writes is provided by new ndbinfo tables
disk_write_speed_base, disk_write_speed_aggregate, and
disk_write_speed_aggregate_node, which provide information about
the speed of disk writes for each LDM thread that is in use.

This release also adds the data node configuration parameters
MinDiskWriteSpeed, MaxDiskWriteSpeed, MaxDiskWriteSpeedOtherNodeRestart,
and MaxDiskWriteSpeedOwnRestart to control write speeds for LCPs
and backups when the present node, another node, or no node is
currently restarting.

These changes are intended to supersede configuration of
disk writes using the DiskCheckpointSpeed and DiskCheckpointSpeedInRestart
configuration parameters.  These 2 parameters have now been
deprecated, and are subject to removal in a future MySQL Cluster
release.

Faster times for restoring a MySQL Cluster from backup have
been obtained by replacing delayed signals found at a point which
was found to be critical to performance with normal (undelayed)
signals. The elimination or replacement of these unnecessary delayed
signals should noticeably reduce the amount of time required to
back up a MySQL Cluster, or to restore a MySQL Cluster from backup.

Several internal methods relating to the NDB receive thread
have been optimized, to increase the efficiency of SQL processing
by NDB. The receiver thread at time may have to process several
million received records per second, so it is critical that it not
perform unnecessary work or waste resources when retrieving records
from MySQL Cluster data nodes.

    Improved reporting of MySQL Cluster restarts and start phases.
The restart_info table (included in the ndbinfo information database
beginning with MySQL Cluster NDB 7.4.2) provides current status
and timing information about node and system restarts.

    Reporting and logging of MySQL Cluster start phases also provides
more frequent and specific printouts during startup than previously.
See Section 18.5.1, Summary of MySQL Cluster Start Phases, for more
information.

    NDB API: new Event API.  MySQL Cluster NDB 7.4.3 introduces an
epoch-driven Event API that supercedes the earlier GCI-based model.
The new version of the API also simplifies error detection and
handling. These changes are realized in the NDB API by implementing
a number of new methods for Ndb and NdbEventOperation, deprecating
several other methods of both classes, and adding new type values
to Event::TableEvent.

    The event handling methods added to Ndb in MySQL Cluster NDB
7.4.3 are pollEvents2(), nextEvent2(), getHighestQueuedEpoch(),
and getNextEventOpInEpoch2(). The Ndb methods pollEvents(),
nextEvent(), getLatestGCI(), getGCIEventOperations(), isConsistent(),
and isConsistentGCI() are deprecated beginning with the same release.

    MySQL Cluster NDB 7.4.3 adds the NdbEventOperation event handling
methods getEventType2(), getEpoch(), isEmptyEpoch(), and isErrorEpoch;
it obsoletes getEventType(), getGCI(), getLatestGCI(), isOverrun(),
hasError(), and clearError().

    While some (but not all) of the new methods are direct replacements
for deprecated methods, not all of the deprecated methods map to
new ones. The Event Class, provides information as to which old
methods correspond to new ones.

    Error handling using the new API is no longer handled using
dedicated hasError() and clearError() methods, which are now
deprecated (and thus subject to removal in a future release of
MySQL Cluster). To support this change, the list of TableEvent
types now includes the values TE_EMPTY (empty epoch), TE_INCONSISTENT
(inconsistent epoch), and TE_OUT_OF_MEMORY (inconsistent data).

    Improvements in event buffer management have also been made by
implementing new get_eventbuffer_free_percent(),
set_eventbuffer_free_percent(), and get_eventbuffer_memory_usage()
methods. Memory buffer usage can now be represented in application
code using EventBufferMemoryUsage. The ndb_eventbuffer_free_percent
system variable, also implemented in MySQL Cluster NDB 7.4, makes
it possible for event buffer memory usage to be checked from MySQL
client applications.

    For more information, see the detailed descriptions for the
Ndb and NdbEventOperation methods listed. See also The Event::TableEvent
Type, as well as The EventBufferMemoryUsage Structure.

    Per-fragment operations information.  In MySQL Cluster NDB
7.4.3 and later, counts of various types of operations on a given
fragment or fragment replica can obtained easily using the
operations_per_fragment table in the ndbinfo information database.
This includes read, write, update, and delete operations, as well
as scan and index operations performed by these. Information about
operations refused, and about rows scanned and returned from a
given fragment replica, is also shown in operations_per_fragment.
This table also provides information about interpreted programs
used as attribute values, and values returned by them.

MySQL Cluster NDB 7.4 is also supported by MySQL Cluster Manager,
which provides an advanced command-line interface that can simplify
many complex MySQL Cluster management tasks. See MySQL Cluster
Manager 1.3.5 User Manual, for more information.

-----

Changes in MySQL Cluster NDB 7.3.9 (5.6.24-ndb-7.3.9)

Bugs Fixed

    It was found during testing that problems could arise when the
node registered as the arbitrator disconnected or failed during
the arbitration process.

    In this situation, the node requesting arbitration could never
receive a positive acknowledgement from the registered arbitrator;
this node also lacked a stable set of members and could not initiate
selection of a new arbitrator.

    Now in such cases, when the arbitrator fails or loses contact
during arbitration, the requesting node immediately fails rather
than waiting to time out. (Bug #20538179)

    The values of the Ndb_last_commit_epoch_server and
Ndb_last_commit_epoch_session status variables were incorrectly
reported on some platforms. To correct this problem, these values
are now stored internally as long long, rather than long. (Bug
#20372169)

    The maximum failure time calculation used to ensure that normal
node failure handling mechanisms are given time to handle survivable
cluster failures (before global checkpoint watchdog mechanisms
start to kill nodes due to GCP delays) was excessively conservative,
and neglected to consider that there can be at most number_of_data_nodes
/ NoOfReplicas node failures before the cluster can no longer
survive. Now the value of NoOfReplicas is properly taken into
account when performing this calculation.  (Bug #20069617, Bug
#20069624)

    References: See also Bug #19858151, Bug #20128256, Bug #20135976.

    When a data node fails or is being restarted, the remaining
nodes in the same nodegroup resend to subscribers any data which
they determine has not already been sent by the failed node.
Normally, when a data node (actually, the SUMA kernel block) has
sent all data belonging to an epoch for which it is responsible,
it sends a SUB_GCP_COMPLETE_REP signal, together with a count, to
all subscribers, each of which responds with a SUB_GCP_COMPLETE_ACK.
When SUMA receives this acknowledgment from all subscribers, it
reports this to the other nodes in the same nodegroup so that they
know that there is no need to resend this data in case of a subsequent
node failure. If a node failed before all subscribers sent this
acknowledgement but before all the other nodes in the same nodegroup
received it from the failing node, data for some epochs could be
sent (and reported as complete) twice, which could lead to an
unplanned shutdown.

    The fix for this issue adds to the count reported by
SUB_GCP_COMPLETE_ACK a list of identifiers which the receiver can
use to keep track of which buckets are completed and to ignore any
duplicate reported for an already completed bucket.  (Bug #17579998)

    When performing a restart, it was sometimes possible to find
a log end marker which had been written by a previous restart, and
that should have been invalidated. Now when when searching for the
last page to invalidate, the same search algorithm is used as when
searching for the last page of the log to read.  (Bug #76207, Bug
#20665205)

    When reading and copying transporter short signal data, it was
possible for the data to be copied back to the same signal with
overlapping memory. (Bug #75930, Bug #20553247)

    When a bulk delete operation was committed early to avoid an
additional round trip, while also returning the number of affected
rows, but failed with a timeout error, an SQL node performed no
verification that the transaction was in the Committed state. (Bug
#74494, Bug #20092754)

    References: See also Bug #19873609.

    An ALTER TABLE statement containing comments and a partitioning
option against an NDB table caused the SQL node on which it was
executed to fail. (Bug #74022, Bug #19667566)

    Cluster API: When a transaction is started from a cluster
connection, Table and Index schema objects may be passed to this
transaction for use. If these schema objects have been acquired
from a different connection (Ndb_cluster_connection object), they
can be deleted at any point by the deletion or disconnection of
the owning connection. This can leave a connection with invalid
schema objects, which causes an NDB API application to fail when
these are dereferenced.

    To avoid this problem, if your application uses multiple
connections, you can now set a check to detect sharing of schema
objects between connections when passing a schema object to a
transaction, using the NdbTransaction::setSchemaObjectOwnerChecks()
method added in this release. When this check is enabled, the schema
objects having the same names are acquired from the connection and
compared to the schema objects passed to the transaction. Failure
to match causes the application to fail with an error. (Bug #19785977)

    Cluster API: The increase in the default number of hashmap
buckets (DefaultHashMapSize API node configuration parameter) from
240 to 3480 in MySQL Cluster NDB 7.2.11 increased the size of the
internal DictHashMapInfo::HashMap type considerably.  This type
was allocated on the stack in some getTable() calls which could
lead to stack overflow issues for NDB API users.

    To avoid this problem, the hashmap is now dynamically allocated
from the heap. (Bug #19306793)

    Cluster API: A scan operation, whether it is a single table
scan or a query scan used by a pushed join, stores the result set
in a buffer. The maximum size of this buffer is calculated and
preallocated before the scan operation is started. This buffer may
consume a considerable amount of memory; in some cases we observed
a 2 GB buffer footprint in tests that executed 100 parallel scans
with 2 single-threaded (ndbd) data nodes.  This memory consumption
was found to scale linearly with additional fragments.

    A number of root causes, listed here, were discovered that led
to this problem:

Result rows were unpacked to full NdbRecord format before
they were stored in the buffer. If only some but not all columns
of a table were selected, the buffer contained empty space (essentially
wasted).

Due to the buffer format being unpacked, VARCHAR and
VARBINARY columns always had to be allocated for the maximum size
defined for such columns.

BatchByteSize and MaxScanBatchSize values were not taken
into consideration as a limiting factor when calculating the maximum
buffer size.

    These issues became more evident in NDB 7.2 and later MySQL
Cluster release series. This was due to the fact buffer size is
scaled by BatchSize, and that the default value for this parameter
was increased fourfold (from 64 to 256) beginning with MySQL Cluster
NDB 7.2.1.

    This fix causes result rows to be buffered using the packed
format instead of the unpacked format; a buffered scan result row
is now not unpacked until it becomes the current row. In addition,
BatchByteSize and MaxScanBatchSize are now used as limiting factors
when calculating the required buffer size.

    Also as part of this fix, refactoring has been done to separate
handling of buffered (packed) from handling of unbuffered result
sets, and to remove code that had been unused since NDB 7.0 or
earlier. The NdbRecord class declaration has also been cleaned up
by removing a number of unused or redundant member variables.  (Bug
#73781, Bug #75599, Bug #19631350, Bug #20408733)

(jnemeth)

Add patches to prevent a segfault when playing certain WAV files. From
Onno van der Linden in PR pkg/49929. Also submitted upstream.
Bump PKGREVISION.

(bsiegert)

Updated x11/deforaos-libdesktop to 0.2.0

(khorben)

Package DeforaOS libDesktop 0.2.0

This release brings:
- license update as 2-clause BSD
- new DesktopWidget class
- updated interaction with helper(1) (from www/deforaos-surfer)
- portability fixes

(khorben)

Updated devel/deforaos-configure to 0.2.1

(khorben)

Package DeforaOS configure 0.2.1

This release is a bugfix release for configure 0.2.0, which brought:
- support for building out-of-tree (OBJDIR)
- new "distcheck" target generation
- support for phony targets
- updated helper scripts (OBJDIR support, portability fixes...)
- new helper script for Python conformance (PEP 8)
- initial support for Objective-C/C++ programs
- documentation update
- test suite

(khorben)

pullup #4720

(hiramatsu)

Pullup ticket #4720 - requested by taca
lang/ruby193-base: build fix

Revisions pulled up:
- pkgsrc/lang/ruby193-base/Makefile                            1.50
- pkgsrc/lang/ruby193-base/distinfo                            1.42-1.44
- pkgsrc/lang/ruby193-base/patches/patch-configure              1.12
- pkgsrc/lang/ruby193-base/patches/patch-lib_mkmf.rb            1.3

---
  Module Name:    pkgsrc
  Committed By:  jperkin
  Date:          Fri Apr  3 08:59:08 UTC 2015

  Modified Files:
          pkgsrc/lang/ruby193-base: distinfo
          pkgsrc/lang/ruby193-base/patches: patch-configure

  Log Message:
  Disable CPU detection on Darwin, the result for 32-bit (i486) is incompatible
  with pkgsrc MACHINE_ARCH (i386).  Fixes 32-bit build, no change for 64-bit.

---
  Module Name:    pkgsrc
  Committed By:  taca
  Date:          Thu Apr 30 03:25:56 UTC 2015

  Modified Files:
          pkgsrc/lang/ruby193-base: Makefile distinfo
  Added Files:
          pkgsrc/lang/ruby193-base/patches: patch-lib_mkmf.rb

  Log Message:
  Avoid to generate empty command line on some platforms:

  $(DLLIB): $(OBJS) Makefile
          ...
          $(Q) $(POSTLINK)

  And POSTLINK is empty macro.  In such case, GNU make ignore empty command
  line but BSD make tries to execute it and causes error.

  Bump PKGREVISION.

(hiramatsu)

Updated mail/roundcube to 1.1.1nb1

(jym)

- install SQL update scripts used by installer for upgrading DB schemas;
- change rights for the spool, log and tmp directories from 0755 to 0750,
  they contain sensitive information depending on configuration;
- fix the default paths of potential mime.types files;
- change config.inc.php to respect pkgsrc paths especially VARBASE;

No regression expected. Bump rev.

ok taca@.

(jym)

Manually set locale to en_us.UTF-8 on Mac OS to fix rst2man invocations
in the build of multimedia/mpv. Patch originally from Homebrew, via
Leonardo Taccari in PR pkg/49921.

(bsiegert)

Updated security/p5-Crypt-Rijndael to 1.13

(mef)

Update to 1.13
--------------
1.13
    * Correctly null-terminate encoded/decoded strings (RT 769/#5)

(mef)

Updated security/p5-Crypt-CipherSaber to 1.01

(mef)

Update 1.00 to 1.01
-------------------
  Changes are not explicitly given. Seems wording in help,
  adustment on URL and Mail address etc.

(mef)

Pullup tickets #4731, #4732 and #4733.

(tron)

Pullup ticket #4733 - requested by sborrill
www/apache22: security patch

Revisions pulled up:
- www/apache22/Makefile                                        1.103
- www/apache22/distinfo                                        1.61
- www/apache22/patches/patch-modules_ssl_ssl__engine__dh.c      1.1

---
  Module Name: pkgsrc
  Committed By: sborrill
  Date: Fri May 22 09:20:20 UTC 2015

  Modified Files:
  pkgsrc/www/apache22: Makefile distinfo
  Added Files:
  pkgsrc/www/apache22/patches: patch-modules_ssl_ssl__engine__dh.c

  Log Message:
  Add patch to mitigate Logjam TLS vulnerabilities (CVE-2015-4000).
  Based on FreeBSD ports.

(tron)

Used github infrastructure. As suggested by mef@.

(jaapb)

Pullup ticket #4732 - requested by bouyer
security/clamav: security update

Revisions pulled up:
- security/clamav/Makefile                                      1.24
- security/clamav/Makefile.common                              1.2
- security/clamav/distinfo                                      1.19

---
  Module Name: pkgsrc
  Committed By: bouyer
  Date: Wed May 20 21:15:26 UTC 2015

  Modified Files:
  pkgsrc/security/clamav: Makefile Makefile.common distinfo

  Log Message:
  Update clamav to 0.98.7.
  This release contains new scanning features and bug fixes.
      - Improvements to PDF processing: decryption, escape sequence
        handling, and file property collection.
      - Scanning/analysis of additional Microsoft Office 2003 XML format.
      - Fix infinite loop condition on crafted y0da cryptor file. Identified
        and patch suggested by Sebastian Andrzej Siewior. CVE-2015-2221.
      - Fix crash on crafted petite packed file. Reported and patch
        supplied by Sebastian Andrzej Siewior. CVE-2015-2222.
      - Fix false negatives on files within iso9660 containers. This issue
        was reported by Minzhuan Gong.
      - Fix a couple crashes on crafted upack packed file. Identified and
        patches supplied by Sebastian Andrzej Siewior.
      - Fix a crash during algorithmic detection on crafted PE file.
        Identified and patch supplied by Sebastian Andrzej Siewior.
      - Fix an infinite loop condition on a crafted "xz" archive file.
        This was reported by Dimitri Kirchner and Goulven Guiheux.
        CVE-2015-2668.
      - Fix compilation error after ./configure --disable-pthreads.
        Reported and fix suggested by John E. Krokes.
      - Apply upstream patch for possible heap overflow in Henry Spencer's
        regex library. CVE-2015-2305.
      - Fix crash in upx decoder with crafted file. Discovered and patch
        supplied by Sebastian Andrzej Siewior. CVE-2015-2170.
      - Fix segfault scanning certain HTML files. Reported with sample by
        Kai Risku.
      - Improve detections within xar/pkg files.

(tron)