--- - branch: netbsd-8 date: Sat Oct 21 19:43:56 UTC 2017 files: - new: 1.18.4.1 old: '1.18' path: src/crypto/dist/ipsec-tools/src/setkey/parse.y pathrev: src/crypto/dist/ipsec-tools/src/setkey/parse.y@1.18.4.1 type: modified - new: 1.19.8.1 old: '1.19' path: src/crypto/dist/ipsec-tools/src/setkey/token.l pathrev: src/crypto/dist/ipsec-tools/src/setkey/token.l@1.19.8.1 type: modified - new: 1.752.2.4 old: 1.752.2.3 path: src/distrib/sets/lists/tests/mi pathrev: src/distrib/sets/lists/tests/mi@1.752.2.4 type: modified - new: 1.11.6.1 old: '1.11' path: src/doc/TODO.smpnet pathrev: src/doc/TODO.smpnet@1.11.6.1 type: modified - new: 1.31.4.1 old: '1.31' path: src/sys/net/pfkeyv2.h pathrev: src/sys/net/pfkeyv2.h@1.31.4.1 type: modified - new: 1.22.20.1 old: '1.22' path: src/sys/net/raw_cb.c pathrev: src/sys/net/raw_cb.c@1.22.20.1 type: modified - new: 1.27.4.1 old: '1.27' path: src/sys/net/raw_cb.h pathrev: src/sys/net/raw_cb.h@1.27.4.1 type: modified - new: 1.56.4.1 old: '1.56' path: src/sys/net/raw_usrreq.c pathrev: src/sys/net/raw_usrreq.c@1.56.4.1 type: modified - new: 1.213.2.3 old: 1.213.2.2 path: src/sys/net/rtsock.c pathrev: src/sys/net/rtsock.c@1.213.2.3 type: modified - new: 1.123.4.1 old: '1.123' path: src/sys/netinet/in_proto.c pathrev: src/sys/netinet/in_proto.c@1.123.4.1 type: modified - new: 1.355.2.1 old: '1.355' path: src/sys/netinet/ip_input.c pathrev: src/sys/netinet/ip_input.c@1.355.2.1 type: modified - new: 1.357.4.2 old: 1.357.4.1 path: src/sys/netinet/tcp_input.c pathrev: src/sys/netinet/tcp_input.c@1.357.4.2 type: modified - new: 1.196.2.1 old: '1.196' path: src/sys/netinet/tcp_output.c pathrev: src/sys/netinet/tcp_output.c@1.196.2.1 type: modified - new: 1.177.10.1 old: '1.177' path: src/sys/netinet/tcp_var.h pathrev: src/sys/netinet/tcp_var.h@1.177.10.1 type: modified - new: 1.211.6.2 old: 1.211.6.1 path: src/sys/netinet6/icmp6.c pathrev: src/sys/netinet6/icmp6.c@1.211.6.2 type: modified - new: 1.117.4.1 old: '1.117' path: src/sys/netinet6/in6_proto.c pathrev: src/sys/netinet6/in6_proto.c@1.117.4.1 type: modified - new: 1.87.2.1 old: '1.87' path: src/sys/netinet6/ip6_forward.c pathrev: src/sys/netinet6/ip6_forward.c@1.87.2.1 type: modified - new: 1.178.2.1 old: '1.178' path: src/sys/netinet6/ip6_input.c pathrev: src/sys/netinet6/ip6_input.c@1.178.2.1 type: modified - new: 1.191.6.2 old: 1.191.6.1 path: src/sys/netinet6/ip6_output.c pathrev: src/sys/netinet6/ip6_output.c@1.191.6.2 type: modified - new: 1.25.10.1 old: '1.25' path: src/sys/netinet6/ip6protosw.h pathrev: src/sys/netinet6/ip6protosw.h@1.25.10.1 type: modified - new: 1.99.2.1 old: '1.99' path: src/sys/netipsec/ipsec.c pathrev: src/sys/netipsec/ipsec.c@1.99.2.1 type: modified - new: 1.50.2.1 old: '1.50' path: src/sys/netipsec/ipsec.h pathrev: src/sys/netipsec/ipsec.h@1.50.2.1 type: modified - new: 1.17.4.1 old: '1.17' path: src/sys/netipsec/ipsec6.h pathrev: src/sys/netipsec/ipsec6.h@1.17.4.1 type: modified - new: 1.17.4.1 old: '1.17' path: src/sys/netipsec/key_debug.c pathrev: src/sys/netipsec/key_debug.c@1.17.4.1 type: modified - new: 1.43.2.1 old: '1.43' path: src/sys/netipsec/ipsec_input.c pathrev: src/sys/netipsec/ipsec_input.c@1.43.2.1 type: modified - new: 1.40.6.1 old: '1.40' path: src/sys/netipsec/ipsec_netbsd.c pathrev: src/sys/netipsec/ipsec_netbsd.c@1.40.6.1 type: modified - new: 1.48.2.1 old: '1.48' path: src/sys/netipsec/ipsec_output.c pathrev: src/sys/netipsec/ipsec_output.c@1.48.2.1 type: modified - new: 1.4.4.1 old: '1.4' path: src/sys/netipsec/ipsec_private.h pathrev: src/sys/netipsec/ipsec_private.h@1.4.4.1 type: modified - new: 1.163.2.1 old: '1.163' path: src/sys/netipsec/key.c pathrev: src/sys/netipsec/key.c@1.163.2.1 type: modified - new: 1.19.2.1 old: '1.19' path: src/sys/netipsec/key.h pathrev: src/sys/netipsec/key.h@1.19.2.1 type: modified - new: 1.8.4.1 old: '1.8' path: src/sys/netipsec/key_debug.h pathrev: src/sys/netipsec/key_debug.h@1.8.4.1 type: modified - new: 1.15.2.1 old: '1.15' path: src/sys/netipsec/keydb.h pathrev: src/sys/netipsec/keydb.h@1.15.2.1 type: modified - new: 1.58.2.1 old: '1.58' path: src/sys/netipsec/keysock.c pathrev: src/sys/netipsec/keysock.c@1.58.2.1 type: modified - new: 1.9.2.1 old: '1.9' path: src/sys/netipsec/keysock.h pathrev: src/sys/netipsec/keysock.h@1.9.2.1 type: modified - new: 1.8.10.1 old: '1.8' path: src/sys/netipsec/xform.h pathrev: src/sys/netipsec/xform.h@1.8.10.1 type: modified - new: 1.54.2.1 old: '1.54' path: src/sys/netipsec/xform_ah.c pathrev: src/sys/netipsec/xform_ah.c@1.54.2.1 type: modified - new: 1.55.2.1 old: '1.55' path: src/sys/netipsec/xform_esp.c pathrev: src/sys/netipsec/xform_esp.c@1.55.2.1 type: modified - new: 1.38.2.1 old: '1.38' path: src/sys/netipsec/xform_ipcomp.c pathrev: src/sys/netipsec/xform_ipcomp.c@1.38.2.1 type: modified - new: 1.49.2.1 old: '1.49' path: src/sys/netipsec/xform_ipip.c pathrev: src/sys/netipsec/xform_ipip.c@1.49.2.1 type: modified - new: 1.11.4.1 old: '1.11' path: src/sys/netipsec/xform_tcp.c pathrev: src/sys/netipsec/xform_tcp.c@1.11.4.1 type: modified - new: 1.169.6.1 old: '1.169' path: src/sys/rump/librump/rumpkern/Makefile.rumpkern pathrev: src/sys/rump/librump/rumpkern/Makefile.rumpkern@1.169.6.1 type: modified - new: 1.26.4.1 old: '1.26' path: src/sys/rump/librump/rumpnet/net_stub.c pathrev: src/sys/rump/librump/rumpnet/net_stub.c@1.26.4.1 type: modified - new: 1.66.10.1 old: '1.66' path: src/sys/sys/protosw.h pathrev: src/sys/sys/protosw.h@1.66.10.1 type: modified - new: 1.18.2.2 old: 1.18.2.1 path: src/tests/net/net_common.sh pathrev: src/tests/net/net_common.sh@1.18.2.2 type: modified - new: 1.6.2.1 old: '1.6' path: src/tests/net/carp/t_basic.sh pathrev: src/tests/net/carp/t_basic.sh@1.6.2.1 type: modified - new: 1.10.2.1 old: '1.10' path: src/tests/net/if_gif/t_gif.sh pathrev: src/tests/net/if_gif/t_gif.sh@1.10.2.1 type: modified - new: 1.2.8.1 old: '1.2' path: src/tests/net/if_l2tp/t_l2tp.sh pathrev: src/tests/net/if_l2tp/t_l2tp.sh@1.2.8.1 type: modified - new: 1.6.2.1 old: '1.6' path: src/tests/net/ipsec/Makefile pathrev: src/tests/net/ipsec/Makefile@1.6.2.1 type: modified - new: 1.4.2.1 old: '1.4' path: src/tests/net/ipsec/algorithms.sh pathrev: src/tests/net/ipsec/algorithms.sh@1.4.2.1 type: modified - new: 1.4.2.1 old: '1.4' path: src/tests/net/ipsec/t_ipsec_transport.sh pathrev: src/tests/net/ipsec/t_ipsec_transport.sh@1.4.2.1 type: modified - new: 1.3.2.1 old: '1.3' path: src/tests/net/ipsec/common.sh pathrev: src/tests/net/ipsec/common.sh@1.3.2.1 type: modified - new: 1.1.8.1 old: '1.1' path: src/tests/net/ipsec/t_ipsec_ah_keys.sh pathrev: src/tests/net/ipsec/t_ipsec_ah_keys.sh@1.1.8.1 type: modified - new: 1.1.8.1 old: '1.1' path: src/tests/net/ipsec/t_ipsec_esp_keys.sh pathrev: src/tests/net/ipsec/t_ipsec_esp_keys.sh@1.1.8.1 type: modified - new: 1.5.2.1 old: '1.5' path: src/tests/net/ipsec/t_ipsec_gif.sh pathrev: src/tests/net/ipsec/t_ipsec_gif.sh@1.5.2.1 type: modified - new: 1.5.2.1 old: '1.5' path: src/tests/net/ipsec/t_ipsec_l2tp.sh pathrev: src/tests/net/ipsec/t_ipsec_l2tp.sh@1.5.2.1 type: modified - new: 1.6.2.2 old: 1.6.2.1 path: src/tests/net/ipsec/t_ipsec_misc.sh pathrev: src/tests/net/ipsec/t_ipsec_misc.sh@1.6.2.2 type: modified - new: 1.2.2.2 old: '0' path: src/tests/net/ipsec/t_ipsec_sockopt.sh pathrev: src/tests/net/ipsec/t_ipsec_sockopt.sh@1.2.2.2 type: added - new: 1.2.2.2 old: '0' path: src/tests/net/ipsec/t_ipsec_tcp.sh pathrev: src/tests/net/ipsec/t_ipsec_tcp.sh@1.2.2.2 type: added - new: 1.2.2.2 old: '0' path: src/tests/net/ipsec/t_ipsec_tunnel_ipcomp.sh pathrev: src/tests/net/ipsec/t_ipsec_tunnel_ipcomp.sh@1.2.2.2 type: added - new: 1.8.2.1 old: '1.8' path: src/tests/net/ipsec/t_ipsec_tunnel.sh pathrev: src/tests/net/ipsec/t_ipsec_tunnel.sh@1.8.2.1 type: modified - new: 1.2.2.1 old: '1.2' path: src/tests/net/ipsec/t_ipsec_tunnel_odd.sh pathrev: src/tests/net/ipsec/t_ipsec_tunnel_odd.sh@1.2.2.1 type: modified - new: 1.5.4.1 old: '1.5' path: src/tests/net/mcast/t_mcast.sh pathrev: src/tests/net/mcast/t_mcast.sh@1.5.4.1 type: modified - new: 1.10.4.1 old: '1.10' path: src/tests/net/net/t_ipaddress.sh pathrev: src/tests/net/net/t_ipaddress.sh@1.10.4.1 type: modified - new: 1.2.28.1 old: '1.2' path: src/tests/net/npf/t_npf.sh pathrev: src/tests/net/npf/t_npf.sh@1.2.28.1 type: modified - new: 1.15.6.2 old: 1.15.6.1 path: src/tests/net/route/t_flags.sh pathrev: src/tests/net/route/t_flags.sh@1.15.6.2 type: modified - new: 1.12.6.2 old: 1.12.6.1 path: src/tests/net/route/t_flags6.sh pathrev: src/tests/net/route/t_flags6.sh@1.12.6.2 type: modified - new: 1.21.4.1 old: '1.21' path: src/usr.bin/netstat/fast_ipsec.c pathrev: src/usr.bin/netstat/fast_ipsec.c@1.21.4.1 type: modified id: 20171021T194356Z.47809ba93384dc43445e6376b9ed144252b755cb log: "Pull up following revision(s) (requested by ozaki-r in ticket #300):\n\tcrypto/dist/ipsec-tools/src/setkey/parse.y: 1.19\n\tcrypto/dist/ipsec-tools/src/setkey/token.l: 1.20\n\tdistrib/sets/lists/tests/mi: 1.754, 1.757, 1.759\n\tdoc/TODO.smpnet: 1.12-1.13\n\tsys/net/pfkeyv2.h: 1.32\n\tsys/net/raw_cb.c: 1.23-1.24, 1.28\n\tsys/net/raw_cb.h: 1.28\n\tsys/net/raw_usrreq.c: 1.57-1.58\n\tsys/net/rtsock.c: 1.228-1.229\n\tsys/netinet/in_proto.c: 1.125\n\tsys/netinet/ip_input.c: 1.359-1.361\n\tsys/netinet/tcp_input.c: 1.359-1.360\n\tsys/netinet/tcp_output.c: 1.197\n\tsys/netinet/tcp_var.h: 1.178\n\tsys/netinet6/icmp6.c: 1.213\n\tsys/netinet6/in6_proto.c: 1.119\n\tsys/netinet6/ip6_forward.c: 1.88\n\tsys/netinet6/ip6_input.c: 1.181-1.182\n\tsys/netinet6/ip6_output.c: 1.193\n\tsys/netinet6/ip6protosw.h: 1.26\n\tsys/netipsec/ipsec.c: 1.100-1.122\n\tsys/netipsec/ipsec.h: 1.51-1.61\n\tsys/netipsec/ipsec6.h: 1.18-1.20\n\tsys/netipsec/ipsec_input.c: 1.44-1.51\n\tsys/netipsec/ipsec_netbsd.c: 1.41-1.45\n\tsys/netipsec/ipsec_output.c: 1.49-1.64\n\tsys/netipsec/ipsec_private.h: 1.5\n\tsys/netipsec/key.c: 1.164-1.234\n\tsys/netipsec/key.h: 1.20-1.32\n\tsys/netipsec/key_debug.c: 1.18-1.21\n\tsys/netipsec/key_debug.h: 1.9\n\tsys/netipsec/keydb.h: 1.16-1.20\n\tsys/netipsec/keysock.c: 1.59-1.62\n\tsys/netipsec/keysock.h: 1.10\n\tsys/netipsec/xform.h: 1.9-1.12\n\tsys/netipsec/xform_ah.c: 1.55-1.74\n\tsys/netipsec/xform_esp.c: 1.56-1.72\n\tsys/netipsec/xform_ipcomp.c: 1.39-1.53\n\tsys/netipsec/xform_ipip.c: 1.50-1.54\n\tsys/netipsec/xform_tcp.c: 1.12-1.16\n\tsys/rump/librump/rumpkern/Makefile.rumpkern: 1.170\n\tsys/rump/librump/rumpnet/net_stub.c: 1.27\n\tsys/sys/protosw.h: 1.67-1.68\n\ttests/net/carp/t_basic.sh: 1.7\n\ttests/net/if_gif/t_gif.sh: 1.11\n\ttests/net/if_l2tp/t_l2tp.sh: 1.3\n\ttests/net/ipsec/Makefile: 1.7-1.9\n\ttests/net/ipsec/algorithms.sh: 1.5\n\ttests/net/ipsec/common.sh: 1.4-1.6\n\ttests/net/ipsec/t_ipsec_ah_keys.sh: 1.2\n\ttests/net/ipsec/t_ipsec_esp_keys.sh: 1.2\n\ttests/net/ipsec/t_ipsec_gif.sh: 1.6-1.7\n\ttests/net/ipsec/t_ipsec_l2tp.sh: 1.6-1.7\n\ttests/net/ipsec/t_ipsec_misc.sh: 1.8-1.18\n\ttests/net/ipsec/t_ipsec_sockopt.sh: 1.1-1.2\n\ttests/net/ipsec/t_ipsec_tcp.sh: 1.1-1.2\n\ttests/net/ipsec/t_ipsec_transport.sh: 1.5-1.6\n\ttests/net/ipsec/t_ipsec_tunnel.sh: 1.9\n\ttests/net/ipsec/t_ipsec_tunnel_ipcomp.sh: 1.1-1.2\n\ttests/net/ipsec/t_ipsec_tunnel_odd.sh: 1.3\n\ttests/net/mcast/t_mcast.sh: 1.6\n\ttests/net/net/t_ipaddress.sh: 1.11\n\ttests/net/net_common.sh: 1.20\n\ttests/net/npf/t_npf.sh: 1.3\n\ttests/net/route/t_flags.sh: 1.20\n\ttests/net/route/t_flags6.sh: 1.16\n\tusr.bin/netstat/fast_ipsec.c: 1.22\nDo m_pullup before mtod\n\nIt may fix panicks of some tests on anita/sparc and anita/GuruPlug.\n---\nKNF\n---\nEnable DEBUG for babylon5\n---\nApply C99-style struct initialization to xformsw\n---\nTweak outputs of netstat -s for IPsec\n\n- Get rid of \"Fast\"\n- Use ipsec and ipsec6 for titles to clarify protocol\n- Indent outputs of sub protocols\n\nOriginal outputs were organized like this:\n\n(Fast) IPsec:\nIPsec ah:\nIPsec esp:\nIPsec ipip:\nIPsec ipcomp:\n(Fast) IPsec:\nIPsec ah:\nIPsec esp:\nIPsec ipip:\nIPsec ipcomp:\n\nNew outputs are organized like this:\n\nipsec:\n\tah:\n\tesp:\n\tipip:\n\tipcomp:\nipsec6:\n\tah:\n\tesp:\n\tipip:\n\tipcomp:\n---\nAdd test cases for IPComp\n---\nSimplify IPSEC_OSTAT macro (NFC)\n---\nKNF; replace leading whitespaces with hard tabs\n---\nIntroduce and use SADB_SASTATE_USABLE_P\n---\nKNF\n---\nAdd update command for testing\n\nUpdating an SA (SADB_UPDATE) requires that a process issuing\nSADB_UPDATE is the same as a process issued SADB_ADD (or SADB_GETSPI).\nThis means that update command must be used with add command in a\nconfiguration of setkey. This usage is normally meaningless but\nuseful for testing (and debugging) purposes.\n---\nAdd test cases for updating SA/SP\n\nThe tests require newly-added udpate command of setkey.\n---\nPR/52346: Frank Kardel: Fix checksumming for NAT-T\nSee XXX for improvements.\n---\nRemove codes for PACKET_TAG_IPSEC_IN_CRYPTO_DONE\n\nIt seems that PACKET_TAG_IPSEC_IN_CRYPTO_DONE is for network adapters\nthat have IPsec accelerators; a driver sets the mtag to a packet\nwhen its device has already encrypted the packet.\n\nUnfortunately no driver implements such offload features for long\nyears and seems unlikely to implement them soon. (Note that neither\nFreeBSD nor Linux doesn't have such drivers.) Let's remove related\n(unused) codes and simplify the IPsec code.\n---\nFix usages of sadb_msg_errno\n---\nAvoid updating sav directly\n\nOn SADB_UPDATE a target sav was updated directly, which was unsafe.\nInstead allocate another sav, copy variables of the old sav to\nthe new one and replace the old one with the new one.\n---\nSimplify; we can assume sav->tdb_xform cannot be NULL while it's valid\n---\nRename key_alloc* functions (NFC)\n\nWe shouldn't use the term \"alloc\" for functions that just look up\ndata and actually don't allocate memory.\n---\nUse explicit_memset to surely zero-clear key_auth and key_enc\n---\nMake sure to clear keys on error paths of key_setsaval\n---\nAdd missing KEY_FREESAV\n---\nMake sure a sav is inserted to a sah list after its initialization completes\n---\nRemove unnecessary zero-clearing codes from key_setsaval\n\nkey_setsaval is now used only for a newly-allocated sav. (It was\nused to reset variables of an existing sav.)\n---\nCorrect wrong assumption of sav->refcnt in key_delsah\n\nA sav in a list is basically not to be sav->refcnt == 0. And also\nKEY_FREESAV assumes sav->refcnt > 0.\n---\nLet key_getsavbyspi take a reference of a returning sav\n---\nUse time_mono_to_wall (NFC)\n---\nSeparate sending message routine (NFC)\n---\nSimplify; remove unnecessary zero-clears\n\nkey_freesaval is used only when a target sav is being destroyed.\n---\nOmit NULL checks for sav->lft_c\n\nsav->lft_c can be NULL only when initializing or destroying sav.\n---\nOmit unnecessary NULL checks for sav->sah\n---\nOmit unnecessary check of sav->state\n\nkey_allocsa_policy picks a sav of either MATURE or DYING so we\ndon't need to check its state again.\n---\nSimplify; omit unnecessary saidx passing\n\n- ipsec_nextisr returns a saidx but no caller uses it\n- key_checkrequest is passed a saidx but it can be gotton by\n another argument (isr)\n---\nFix splx isn't called on some error paths\n---\nFix header size calculation of esp where sav is NULL\n---\nFix header size calculation of ah in the case sav is NULL\n\nThis fix was also needed for esp.\n---\nPass sav directly to opencrypto callback\n\nIn a callback, use a passed sav as-is by default and look up a sav\nonly if the passed sav is dead.\n---\nAvoid examining freshness of sav on packet processing\n\nIf a sav list is sorted (by lft_c->sadb_lifetime_addtime) in advance,\nwe don't need to examine each sav and also don't need to delete one\non the fly and send up a message. Fortunately every sav lists are sorted\nas we need.\n\nAdded key_validate_savlist validates that each sav list is surely sorted\n(run only if DEBUG because it's not cheap).\n---\nAdd test cases for SAs with different SPIs\n---\nPrepare to stop using isr->sav\n\nisr is a shared resource and using isr->sav as a temporal storage\nfor each packet processing is racy. And also having a reference from\nisr to sav makes the lifetime of sav non-deterministic; such a reference\nis removed when a packet is processed and isr->sav is overwritten by\nnew one. Let's have a sav locally for each packet processing instead of\nusing shared isr->sav.\n\nHowever this change doesn't stop using isr->sav yet because there are\nsome users of isr->sav. isr->sav will be removed after the users find\na way to not use isr->sav.\n---\nFix wrong argument handling\n---\nfix printf format.\n---\nDon't validate sav lists of LARVAL or DEAD states\n\nWe don't sort the lists so the validation will always fail.\n\nFix PR kern/52405\n---\nMake sure to sort the list when changing the state by key_sa_chgstate\n---\nRename key_allocsa_policy to key_lookup_sa_bysaidx\n---\nSeparate test files\n---\nCalculate ah_max_authsize on initialization as well as esp_max_ivlen\n---\nRemove m_tag_find(PACKET_TAG_IPSEC_PENDING_TDB) because nobody sets the tag\n---\nRestore a comment removed in previous\n\nThe comment is valid for the below code.\n---\nMake tests more stable\n\nsleep command seems to wait longer than expected on anita so\nuse polling to wait for a state change.\n---\nAdd tests that explicitly delete SAs instead of waiting for expirations\n---\nRemove invalid M_AUTHIPDGM check on ESP isr->sav\n\nM_AUTHIPDGM flag is set to a mbuf in ah_input_cb. An sav of ESP can\nhave AH authentication as sav->tdb_authalgxform. However, in that\ncase esp_input and esp_input_cb are used to do ESP decryption and\nAH authentication and M_AUTHIPDGM never be set to a mbuf. So\nchecking M_AUTHIPDGM of a mbuf on isr->sav of ESP is meaningless.\n---\nLook up sav instead of relying on unstable sp->req->sav\n\nThis code is executed only in an error path so an additional lookup\ndoesn't matter.\n---\nCorrect a comment\n---\nDon't release sav if calling crypto_dispatch again\n---\nRemove extra KEY_FREESAV from ipsec_process_done\n\nIt should be done by the caller.\n---\nDon't bother the case of crp->crp_buf == NULL in callbacks\n---\nHold a reference to an SP during opencrypto processing\n\nAn SP has a list of isr (ipsecrequest) that represents a sequence\nof IPsec encryption/authentication processing. One isr corresponds\nto one opencrypto processing. The lifetime of an isr follows its SP.\n\nWe pass an isr to a callback function of opencrypto to continue\nto a next encryption/authentication processing. However nobody\nguaranteed that the isr wasn't freed, i.e., its SP wasn't destroyed.\n\nIn order to avoid such unexpected destruction of isr, hold a reference\nto its SP during opencrypto processing.\n---\nDon't make SAs expired on tests that delete SAs explicitly\n---\nFix a debug message\n---\nDedup error paths (NFC)\n---\nUse pool to allocate tdb_crypto\n\nFor ESP and AH, we need to allocate an extra variable space in addition\nto struct tdb_crypto. The fixed size of pool items may be larger than\nan actual requisite size of a buffer, but still the performance\nimprovement by replacing malloc with pool wins.\n---\nDon't use unstable isr->sav for header size calculations\n\nWe may need to optimize to not look up sav here for users that\ndon't need to know an exact size of headers (e.g., TCP segmemt size\ncaclulation).\n---\nDon't use sp->req->sav when handling NAT-T ESP fragmentation\n\nIn order to do this we need to look up a sav however an additional\nlook-up degrades performance. A sav is later looked up in\nipsec4_process_packet so delay the fragmentation check until then\nto avoid an extra look-up.\n---\nDon't use key_lookup_sp that depends on unstable sp->req->sav\n\nIt provided a fast look-up of SP. We will provide an alternative\nmethod in the future (after basic MP-ification finishes).\n---\nStop setting isr->sav on looking up sav in key_checkrequest\n---\nRemove ipsecrequest#sav\n---\nStop setting mtag of PACKET_TAG_IPSEC_IN_DONE because there is no users anymore\n---\nSkip ipsec_spi_*_*_preferred_new_timeout when running on qemu\n\nProbably due to PR 43997\n---\nAdd localcount to rump kernels\n---\nRemove unused macro\n---\nFix key_getcomb_setlifetime\n\nThe fix adjusts a soft limit to be 80% of a corresponding hard limit.\n\nI'm not sure the fix is really correct though, at least the original\ncode is wrong. A passed comb is zero-cleared before calling\nkey_getcomb_setlifetime, so\n comb->sadb_comb_soft_addtime = comb->sadb_comb_soft_addtime * 80 / 100;\nis meaningless.\n---\nProvide and apply key_sp_refcnt (NFC)\n\nIt simplifies further changes.\n---\nFix indentation\n\nPointed out by knakahara@\n---\nUse pslist(9) for sptree\n---\nDon't acquire global locks for IPsec if NET_MPSAFE\n\nNote that the change is just to make testing easy and IPsec isn't MP-safe yet.\n---\nLet PF_KEY socks hold their own lock instead of softnet_lock\n\nOperations on SAD and SPD are executed via PF_KEY socks. The operations\ninclude deletions of SAs and SPs that will use synchronization mechanisms\nsuch as pserialize_perform to wait for references to SAs and SPs to be\nreleased. It is known that using such mechanisms with holding softnet_lock\ncauses a dead lock. We should avoid the situation.\n---\nMake IPsec SPD MP-safe\n\nWe use localcount(9), not psref(9), to make the sptree and secpolicy (SP)\nentries MP-safe because SPs need to be referenced over opencrypto\nprocessing that executes a callback in a different context.\n\nSPs on sockets aren't managed by the sptree and can be destroyed in softint.\nlocalcount_drain cannot be used in softint so we delay the destruction of\nsuch SPs to a thread context. To do so, a list to manage such SPs is added\n(key_socksplist) and key_timehandler_spd deletes dead SPs in the list.\n\nFor more details please read the locking notes in key.c.\n\nProposed on tech-kern@ and tech-net@\n---\nFix updating ipsec_used\n\n- key_update_used wasn't called in key_api_spddelete2 and key_api_spdflush\n- key_update_used wasn't called if an SP had been added/deleted but\n a reply to userland failed\n---\nFix updating ipsec_used; turn on when SPs on sockets are added\n---\nAdd missing IPsec policy checks to icmp6_rip6_input\n\nicmp6_rip6_input is quite similar to rip6_input and the same checks exist\nin rip6_input.\n---\nAdd test cases for setsockopt(IP_IPSEC_POLICY)\n---\nDon't use KEY_NEWSP for dummy SP entries\n\nBy the change KEY_NEWSP is now not called from softint anymore\nand we can use kmem_zalloc with KM_SLEEP for KEY_NEWSP.\n---\nComment out unused functions\n---\nAdd test cases that there are SPs but no relevant SAs\n---\nDon't allow sav->lft_c to be NULL\n\nlft_c of an sav that was created by SADB_GETSPI could be NULL.\n---\nClean up clunky eval strings\n\n- Remove unnecessary \\ at EOL\n - This allows to omit ; too\n- Remove unnecessary quotes for arguments of atf_set\n- Don't expand $DEBUG in eval\n - We expect it's expanded on execution\n\nSuggested by kre@\n---\nRemove unnecessary KEY_FREESAV in an error path\n\nsav should be freed (unreferenced) by the caller.\n---\nUse pslist(9) for sahtree\n---\nUse pslist(9) for sah->savtree\n---\nRename local variable newsah to sah\n\nIt may not be new.\n---\nMP-ify SAD slightly\n\n- Introduce key_sa_mtx and use it for some list operations\n- Use pserialize for some list iterations\n---\nIntroduce KEY_SA_UNREF and replace KEY_FREESAV with it where sav will never be actually freed in the future\n\nKEY_SA_UNREF is still key_freesav so no functional change for now.\n\nThis change reduces diff of further changes.\n---\nRemove out-of-date log output\n\nPointed out by riastradh@\n---\nUse KDASSERT instead of KASSERT for mutex_ownable\n\nBecause mutex_ownable is too heavy to run in a fast path\neven for DIAGNOSTIC + LOCKDEBUG.\n\nSuggested by riastradh@\n---\nAssemble global lists and related locks into cache lines (NFCI)\n\nAlso rename variable names from *tree to *list because they are\njust lists, not trees.\n\nSuggested by riastradh@\n---\nMove locking notes\n---\nUpdate the locking notes\n\n- Add locking order\n- Add locking notes for misc lists such as reglist\n- Mention pserialize, key_sp_ref and key_sp_unref on SP operations\n\nRequested by riastradh@\n---\nDescribe constraints of key_sp_ref and key_sp_unref\n\nRequested by riastradh@\n---\nHold key_sad.lock on SAVLIST_WRITER_INSERT_TAIL\n---\nAdd __read_mostly to key_psz\n\nSuggested by riastradh@\n---\nTweak wording (pserialize critical section => pserialize read section)\n\nSuggested by riastradh@\n---\nAdd missing mutex_exit\n---\nFix setkey -D -P outputs\n\nThe outputs were tweaked (by me), but I forgot updating libipsec\nin my local ATF environment...\n---\nMP-ify SAD (key_sad.sahlist and sah entries)\n\nlocalcount(9) is used to protect key_sad.sahlist and sah entries\nas well as SPD (and will be used for SAD sav).\n\nPlease read the locking notes of SAD for more details.\n---\nIntroduce key_sa_refcnt and replace sav->refcnt with it (NFC)\n---\nDestroy sav only in the loop for DEAD sav\n---\nFix KASSERT(solocked(sb->sb_so)) failure in sbappendaddr that is called eventually from key_sendup_mbuf\n\nIf key_sendup_mbuf isn't passed a socket, the assertion fails.\nOriginally in this case sb->sb_so was softnet_lock and callers\nheld softnet_lock so the assertion was magically satisfied.\nNow sb->sb_so is key_so_mtx and also softnet_lock isn't always\nheld by callers so the assertion can fail.\n\nFix it by holding key_so_mtx if key_sendup_mbuf isn't passed a socket.\n\nReported by knakahara@\nTested by knakahara@ and ozaki-r@\n---\nFix locking notes of SAD\n---\nFix deadlock between key_sendup_mbuf called from key_acquire and localcount_drain\n\nIf we call key_sendup_mbuf from key_acquire that is called on packet\nprocessing, a deadlock can happen like this:\n- At key_acquire, a reference to an SP (and an SA) is held\n- key_sendup_mbuf will try to take key_so_mtx\n- Some other thread may try to localcount_drain to the SP with\n holding key_so_mtx in say key_api_spdflush\n- In this case localcount_drain never return because key_sendup_mbuf\n that has stuck on key_so_mtx never release a reference to the SP\n\nFix the deadlock by deferring key_sendup_mbuf to the timer\n(key_timehandler).\n---\nFix that prev isn't cleared on retry\n---\nLimit the number of mbufs queued for deferred key_sendup_mbuf\n\nIt's easy to be queued hundreds of mbufs on the list under heavy\nnetwork load.\n---\nMP-ify SAD (savlist)\n\nlocalcount(9) is used to protect savlist of sah. The basic design is\nsimilar to MP-ifications of SPD and SAD sahlist. Please read the\nlocking notes of SAD for more details.\n---\nSimplify ipsec_reinject_ipstack (NFC)\n---\nAdd per-CPU rtcache to ipsec_reinject_ipstack\n\nIt reduces route lookups and also reduces rtcache lock contentions\nwhen NET_MPSAFE is enabled.\n---\nUse pool_cache(9) instead of pool(9) for tdb_crypto objects\n\nThe change improves network throughput especially on multi-core systems.\n---\nUpdate\n\nipsec(4), opencrypto(9) and vlan(4) are now MP-safe.\n---\nWrite known issues on scalability\n---\nShare a global dummy SP between PCBs\n\nIt's never be changed so it can be pre-allocated and shared safely between PCBs.\n---\nFix race condition on the rawcb list shared by rtsock and keysock\n\nkeysock now protects itself by its own mutex, which means that\nthe rawcb list is protected by two different mutexes (keysock's one\nand softnet_lock for rtsock), of course it's useless.\n\nFix the situation by having a discrete rawcb list for each.\n---\nUse a dedicated mutex for rt_rawcb instead of softnet_lock if NET_MPSAFE\n---\nfix localcount leak in sav. fixed by ozaki-r@n.o.\n\nI commit on behalf of him.\n---\nremove unnecessary comment.\n---\nFix deadlock between pserialize_perform and localcount_drain\n\nA typical ussage of localcount_drain looks like this:\n\n mutex_enter(&mtx);\n item = remove_from_list();\n pserialize_perform(psz);\n \ localcount_drain(&item->localcount, &cv, &mtx);\n mutex_exit(&mtx);\n\nThis sequence can cause a deadlock which happens for example on the following\nsituation:\n\n- Thread A calls localcount_drain which calls xc_broadcast after releasing\n a specified mutex\n- Thread B enters the sequence and calls pserialize_perform with holding\n the mutex while pserialize_perform also calls xc_broadcast\n- Thread C (xc_thread) that calls an xcall callback of localcount_drain tries\n to hold the mutex\n\nxc_broadcast of thread B doesn't start until xc_broadcast of thread A\nfinishes, which is a feature of xcall(9). This means that pserialize_perform\nnever complete until xc_broadcast of thread A finishes. On the other hand,\nthread C that is a callee of xc_broadcast of thread A sticks on the mutex.\nFinally the threads block each other (A blocks B, B blocks C and C blocks A).\n\nA possible fix is to serialize executions of the above sequence by another\nmutex, but adding another mutex makes the code complex, so fix the deadlock\nby another way; the fix is to release the mutex before pserialize_perform\nand instead use a condvar to prevent pserialize_perform from being called\nsimultaneously.\n\nNote that the deadlock has happened only if NET_MPSAFE is enabled.\n---\nAdd missing ifdef NET_MPSAFE\n---\nTake softnet_lock on pr_input properly if NET_MPSAFE\n\nCurrently softnet_lock is taken unnecessarily in some cases, e.g.,\nicmp_input and encap4_input from ip_input, or not taken even if needed,\ne.g., udp_input and tcp_input from ipsec4_common_input_cb. Fix them.\n\nNFC if NET_MPSAFE is disabled (default).\n---\n- sanitize key debugging so that we don't print extra newlines or unassociated\n \ debugging messages.\n- remove unused functions and make internal ones static\n- print information in one line per message\n---\nhumanize printing of ip addresses\n---\ncast reduction, NFC.\n---\nFix typo in comment\n---\nPull out ipsec_fill_saidx_bymbuf (NFC)\n---\nDon't abuse key_checkrequest just for looking up sav\n\nIt does more than expected for example key_acquire.\n---\nFix SP is broken on transport mode\n\nisr->saidx was modified accidentally in ipsec_nextisr.\n\nReported by christos@\nHelped investigations by christos@ and knakahara@\n---\nConstify isr at many places (NFC)\n---\nInclude socketvar.h for softnet_lock\n---\nFix buffer length for ipsec_logsastr\n" module: src subject: 'CVS commit: [netbsd-8] src' unixtime: '1508615036' user: snj