Fri Jul 15 14:51:16 2016 UTC ()
Revise ABI discussion

Substantially revise the ABI 32/64 discussion, separating the
close-in-time changes about the default ABI vs how it is encoded.
Thanks to jperkin for off-list clarifications.


(gdt)
diff -r1.30 -r1.31 pkgsrc/bootstrap/README.MacOSX

cvs diff -r1.30 -r1.31 pkgsrc/bootstrap/Attic/README.MacOSX (expand / switch to unified diff)

--- pkgsrc/bootstrap/Attic/README.MacOSX 2016/07/14 23:35:05 1.30
+++ pkgsrc/bootstrap/Attic/README.MacOSX 2016/07/15 14:51:16 1.31
@@ -1,14 +1,14 @@ @@ -1,14 +1,14 @@
1$NetBSD: README.MacOSX,v 1.30 2016/07/14 23:35:05 gdt Exp $ 1$NetBSD: README.MacOSX,v 1.31 2016/07/15 14:51:16 gdt Exp $
2 2
3This file describes the use of current versions of pkgsrc with 3This file describes the use of current versions of pkgsrc with
4multiple versions of Darwin and OS X, omitting information about previous pkgsrc 4multiple versions of Darwin and OS X, omitting information about previous pkgsrc
5versions. 5versions.
6 6
7* Darwin vs OS X 7* Darwin vs OS X
8 8
9(Apple has renamed Mac OS X to OS X, but the filename 9(Apple has renamed Mac OS X to OS X, but the filename
10remains.) 10remains.)
11 11
12OS X consists of Darwin (kernel/userland) plus Mac stuff on top. 12OS X consists of Darwin (kernel/userland) plus Mac stuff on top.
13pkgsrc used to target Darwin, but given the tools issued discussed 13pkgsrc used to target Darwin, but given the tools issued discussed
14below it is not clear that it works on Darwin without OS X. Darwin 14below it is not clear that it works on Darwin without OS X. Darwin
@@ -36,61 +36,86 @@ could treat it like a relatively normal  @@ -36,61 +36,86 @@ could treat it like a relatively normal
36approximately 10.9, headers were no longer available at the standard 36approximately 10.9, headers were no longer available at the standard
37location, and one has to use an SDK that puts headers someplace else. 37location, and one has to use an SDK that puts headers someplace else.
38pkgsrc supports this, but there has been some confusion where a 10.9 38pkgsrc supports this, but there has been some confusion where a 10.9
39system produced binaries for 10.10, which only mostly works. The 39system produced binaries for 10.10, which only mostly works. The
40confusion is believed to be resolved. 40confusion is believed to be resolved.
41 41
42** gcc vs clang 42** gcc vs clang
43 43
44Older versions of OS X (when XCode is installed) provided gcc, and 44Older versions of OS X (when XCode is installed) provided gcc, and
45pkgsrc defaulted to using gcc. With 10.9, gcc is no longer present. 45pkgsrc defaulted to using gcc. With 10.9, gcc is no longer present.
46 46
47** i386 vs x86_64 ABI issue 47** i386 vs x86_64 ABI issue
48 48
 49This entire section is only about Intel Macs.
 50
49OS X 10.6 and higher supports x86-64 binaries on Intel Macs with 51OS X 10.6 and higher supports x86-64 binaries on Intel Macs with
50x86-64 processors, which is now most of them. 52x86-64 processors, which is now most of them. i386 binaries are also
 53supported on most (all?) Intel machines.
 54
 55*** issues related to ABI 32 vs 64
51 56
52This has caused problems with packages which get confused because 57Note that a pkgsrc package built in x86_64 mode will not run on an
53"MACHINE_ARCH" is in some OS versions set to "i386" (on a 64-bit 58Intel Mac that is i386 only. For a longer discussion, see:
54system!). 59http://mail-index.NetBSD.org/pkgsrc-users/2009/09/24/msg010817.html
 60
 61Somewhat separately from pkgsrc's ABI choice, there have been issues
 62with packages which get confused because "MACHINE_ARCH" is in some OS
 63versions set to "i386" (on a 64-bit system!). As of 2016 this should
 64be mostly resolved.
55 version: uname -m : uname -p 65 version: uname -m : uname -p
56 10.6: i386 : i386 66 10.6: i386 : i386
57 10.9: x86_64 : i386 67 10.9: x86_64 : i386
58 68
59On Intel machines, pkgsrc currently defaults to i386 mode (--abi=32) 69*** default ABI
60on OS X, and can be set to x86_64 mode (--abi=64). 
61Note that a pkgsrc build in x86_64 mode will not run on an Intel Mac 
62that is i386 only. For a longer discussion, see: 
63 http://mail-index.NetBSD.org/pkgsrc-users/2009/09/24/msg010817.html 
64 70
65As of 2015-11-09, the default ABI is x86_64 on machines where "uname 71The ABI is chosen at bootstrap time and encoded into mk.conf. So a
 72change in the default is about what a new bootstrap will do;
 73already-bootstrapped systems should remain unchanged. They should be
 74able to build and run new packages using the old ABI value.
 75
 76pkgsrc used to set the default ABI as i386, both on systems with i386
 77processors and on systems with x86_64 processors. On 2015-11-09 the
 78default was changed so that ABI=64 is chosen on machines where "uname
66-m" reports x86_64. (It remains i386 on others, which are not capable 79-m" reports x86_64. (It remains i386 on others, which are not capable
67of running x86_64 binaries.) 80of running x86_64 binaries.)
68 81
69*** resolving issues from a change in default ABI 82Generally, users will not need to deal with the default ABI change,
 83except that packages are mostly only portable across machines with the
 84same bootstrapping parameters.
 85
 86If one unpacks a new binary bootstrap kit over an existing
 87installation, one can end up with a mix. The standard advice is not to
 88do this, and to rrebuild/reinstall all packages from scratch or a
 89compatible binary package set. But, one could also mark packages with
 90the wrong ABI as rebuild=YES and use pkg_rolling-replace.
 91
 92*** change in storage of ABI information
 93
 94On 2016-01-24, the way ABI information was stored in pkgsrc was
 95rationalized and simplified. The new code could compute the wrong ABI
 96for some previously-bootstrapped installations. The problem can be
 97resolved by building bmake with MACHINE_ARCH=x86_64 and updating that
 98package, as described in mail archives:
70 99
71When the ABI changes from 32 to 64, you can recover without a 
72rebootstrap by replacing bmake and pkg_install. See 
73 https://mail-index.netbsd.org/pkgsrc-users/2016/01/25/msg022870.html 100 https://mail-index.netbsd.org/pkgsrc-users/2016/01/25/msg022870.html
74 101
75In /usr/pkgsrc/devel/bmake, do: 102(One would expect to be able to use make replace to do this. One
76# bmake MACHINE_ARCH=x86_64 replace 103minor issue is that it requires pkg_tarup, although that will be
77and then rerun the failed pkg_add -U with an additional -f. 104present on systems of those who use make replace. There also may be
78Then, in /usr/pkgsrc/pkgtools/pkg_install, do: 105an error with architecture mismatch from pkg_install requiring a "-f"
79# bmake replace 106option. Repeatable data about recovery is somewhat hard to obtain, as
80 107most are past this issue already and no longer interested in
81Then, rebuild all packages; ABI=32 and ABI=64 packages both work 108experimenting.)
82individually, but mixing them in a single program via dynamic linking 
83will not work. 
84 109
85** sed in 10.9 110** sed in 10.9
86 111
87The sed that comes with 10.9 appears to be broken; it exits when 112The sed that comes with 10.9 appears to be broken; it exits when
88called on files with UTF-8 or other apparently-binary content. 113called on files with UTF-8 or other apparently-binary content.
89Therefore, pkgsrc uses nbsed on 10.9. 114Therefore, pkgsrc uses nbsed on 10.9.
90 115
91* Developer tools and prerequisites 116* Developer tools and prerequisites
92 117
93** XCode 118** XCode
94 119
95This section applies to 10.6 through 10.10. 120This section applies to 10.6 through 10.10.
96 121