fergofrog’s blog

Compiling XNU 4903.221.2 for ARM64

2018-12-23T00:00:00+00:00

This is a follow up to the previous post on compling XNU 4570.41.2 for ARM64, but covering the newly released XNU 4903.221.2 (macOS 14.1). This guide is, again, mostly based on the kernelshamin guide.

The future of compilation for arm64 looked bad from Apple’s release of xnu-4570.71.2 which contained several broken components for the architecture. Thankfully, this release required only two minor modifications (and nothing to code!):

We don’t have access to the device map database, nor the embedded_device_map tool, thankfully these are only needed to actually build for the device
As before, Availability.h is needed but not included in the installhdrs stage - simply fixed by modifying EXTERNAL_HEADERS/Makefile

This release included Apple’s intended use for the MACHINE_CONFIG build variable. This allows the targetted AP to be specified as part of the build (or for multiple device builds to occur). In the original modifications, I hardcoded the T8011 board, but later included a great suggestion from @zhouwei (see issue #1) which did exactly what Apple has now released. An additional machine is also included, the BCM2837, the AP for the Raspberry Pi 3 - very cool! The big challenges that someone would need to resolve for the rpi3 is that a device map is needed, ideally they would create an embedded_device_map tool, and a custom iBoot to take over from the Pi’s bootloader and pass off to XNU. @zhouwei has done this for qemu and of course Corellium have done it better than anyone - it can be done!

tl;dr

The complication process is as simple as following the kernelshaman blog, with the following modifications:

Use my repository for xnu 4903.221.2 - it contains modifications to the xnu source necessary for building
- Note: if you would rather build from Apple’s source, check the diff from my repo for changes I made
a copy of the TargetConditionals.h header must be made from the iPhoneOS SDK’s /usr/include directory to the iPhoneOS SDK’s /System/Library/Frameworks/Kernel.framework/Versions/A/Headers directory in order to build libfirehose
When building: the SDK is “iphoneos”, the ARCH is “arm64”, and XNU’s build also needs ARCH_STRING_FOR_CURRENT_MACHINE_CONFIG=arm64

The Details

This process was tested on a mostly-clean macOS 10.14.2 with Xcode 10.1.

CTF Tools

The CTF tools (ctfconvert, ctfdump and ctfmerge) are executed as part of XNU’s build and are expected to be in your hosts’s SDK path.

For macOS 10.14.1 dtrace has had a version bump to 284.200.15. Compilation requires some additional header search paths to resolve locating elf.h and a few others.

tar zxvf dtrace-284.200.15.tar.gz
cd dtrace-284.200.15
mkdir obj sym dst
xcodebuild install -sdk macosx \
    -target ctfconvert -target ctfdump -target ctfmerge \
    ARCHS=x86_64 SRCROOT=$PWD OBJROOT=$PWD/obj \
    SYMROOT=$PWD/sym DSTROOT=$PWD/dst \
    HEADER_SEARCH_PATHS="$PWD/compat/opensolaris/** $PWD/lib/**"
sudo ditto \
    $PWD/dst/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain \
    /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain
cd ..

AvailabilityVersions

AvailabilityVersions provides a perl script used as part of the XNU build. Build and installation is per the blog, except the install target is the iphoneos SDK.

AvailabilityVersions had a version bump to 33.200.4 in macOS 10.14.1, however the installation is as before.

tar zxvf AvailabilityVersions-33.200.4.tar.gz
cd AvailabilityVersions-33.200.4
mkdir dst
make install SRCROOT=$PWD DSTROOT=$PWD/dst
sudo ditto \
    $PWD/dst/usr/local/libexec \
    $(xcrun -sdk iphoneos -show-sdk-path)/usr/local/libexec
cd ..

libplatform Headers

Just three header files are required from this project. These are now at version 177.200.16, but install is the same.

tar zxvf libplatform-177.200.16.tar.gz
cd libplatform-177.200.16
sudo mkdir -p \
    $(xcrun -sdk iphoneos -show-sdk-path)/usr/local/include/os/internal
sudo ditto $PWD/private/os/internal \
    $(xcrun -sdk iphoneos -show-sdk-path)/usr/local/include/os/internal
cd ..

XNU Headers

The headers are ready to be built now. I’d recommend using my repo and check the diff to see what I did.

After the headers are installed, a copy of the TargetConditionals.h header must be made from the SDK’s /usr/include directory to the SDK’s /System/Library/Frameworks/Kernel.framework/Versions/A/Headers directory in order to build libfirehose.

git clone -b xnu-4903.221.2 https://github.com/fergofrog/xnu.git
cd xnu
make LOGCOLORS=y SDKROOT=iphoneos ARCH_CONFIGS=ARM64 MACHINE_CONFIGS=T8011 \
     KERNEL_CONFIGS=RELEASE ARCH_STRING_FOR_CURRENT_MACHINE_CONFIG=arm64 \
     installhdrs
sudo ditto $PWD/BUILD/dst $(xcrun -sdk iphoneos -show-sdk-path)
sudo cp $(xcrun -sdk iphoneos -show-sdk-path)/usr/include/TargetConditionals.h \
    $(xcrun -sdk iphoneos -show-sdk-path)/System/Library/Frameworks/Kernel.framework/Versions/A/Headers/TargetConditionals.h
cd ..

libfirehose_kernel From libdispatch

The kernel libfirehose header and objects are required to build XNU. As before, the XNU headers and TargetConditionals.h need to be in place, however the new libdispatch 1008.220.2 requires several additional modifications, documented by the kernelsharman blog.

tar zxvf libdispatch-1008.220.2.tar.gz
cd libdispatch-1008.220.2
mkdir obj sym dst
awk '/include "<DEVELOPER/ {next;} /SDKROOT =/ {print "SDKROOT = macosx"; next;} {print $0}' \
    xcodeconfig/libdispatch.xcconfig > .__tmp__ && \
    mv -f .__tmp__ xcodeconfig/libdispatch.xcconfig
awk '/#include / { next; } { print $0 }' \
    xcodeconfig/libfirehose_kernel.xcconfig > .__tmp__ && \
    mv -f .__tmp__ xcodeconfig/libfirehose_kernel.xcconfig
xcodebuild install -sdk iphoneos -target libfirehose_kernel \
    SRCROOT=$PWD OBJROOT=$PWD/obj SYMROOT=$PWD/sym DSTROOT=$PWD/dst \
    ENABLE_BITCODE=no
sudo ditto $PWD/dst/usr/local \
    $(xcrun -sdk iphoneos -show-sdk-path)/usr/local
cd ..

XNU

Now it’s time to build XNU. With all the setup from before, this process is as simple as one command.

cd xnu
make LOGCOLORS=y SDKROOT=iphoneos ARCH_CONFIGS=ARM64 MACHINE_CONFIGS=T8011 \
     KERNEL_CONFIGS=RELEASE ARCH_STRING_FOR_CURRENT_MACHINE_CONFIG=arm64

Compiling XNU 4570.41.2 for ARM64

2018-07-07T00:00:00+00:00

Compiling XNU for x86_64 is quite easy, particularly when armed with a good guide. Aside from an extremely helpful gist and a Twitter thread from Proteas, there didn’t seem to be much on compiling XNU for arm64.

Why would you want to compile XNU for arm64? Well, you definitely can’t run it on your i-device. But there could be many other reasons. For me, a clean compile means I can generate a clean compile-commands.json file for use with Woboq.

tl;dr

The long and short of this process is, follow the steps from this blog, with the following ammendments:

Use my repository for xnu - it contains modifications to the xnu source necessary for building
- Note: if you would rather build from Apple’s source, check the diff from my repo for changes I made
a copy of the TargetConditionals.h header must be made from the iPhoneOS SDK’s /usr/include directory to the iPhoneOS SDK’s /System/Library/Frameworks/Kernel.framework/Versions/A/Headers directory in order to build libfirehose
When building: the SDK is “iphoneos”, the ARCH is “arm64”, and XNU’s build also needs BUILD_WERROR=0

The Details

This process worked for me compiling XNU 4570.41.2 with a clean install of macOS High Sierra (10.13.5) and XCode 9.4, however, your mileage may vary.

CTF Tools

The CTF tools (ctfconvert, ctfdump and ctfmerge) are executed as part of XNU’s build and are expected to be in your hosts’s SDK path. Build is per the blog.

tar zxvf dtrace-262.tar.gz
cd dtrace-262
mkdir obj sym dst
xcodebuild install \
    -target ctfconvert -target ctfdump -target ctfmerge \
    ARCHS=x86_64 SRCROOT=$PWD OBJROOT=$PWD/obj \
    SYMROOT=$PWD/sym DSTROOT=$PWD/dst
sudo ditto \
    $PWD/dst/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain \
    /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain
cd ..

AvailabilityVersions

AvailabilityVersions provides a perl script used as part of the XNU build. Build and installation is per the blog, except the install target is the iphoneos SDK.

tar zxvf AvailabilityVersions-32.30.1.tar.gz
cd AvailabilityVersions-32.30.1
mkdir dst
make install SRCROOT=$PWD DSTROOT=$PWD/dst
sudo ditto \
    $PWD/dst/usr/local/libexec \
    $(xcrun -sdk iphoneos -show-sdk-path)/usr/local/libexec
cd ..

libplatform Headers

Just three header files are required from this project.

tar zxvf libplatform-161.20.1.tar.gz
cd libplatform-161.20.1
sudo mkdir -p \
    $(xcrun -sdk iphoneos -show-sdk-path)/usr/local/include/os/internal
sudo ditto $PWD/private/os/internal \
    $(xcrun -sdk iphoneos -show-sdk-path)/usr/local/include/os/internal
cd ..

XNU Headers

The headers are ready to be built now. This is where the first modifications are required. Proteas’ suggestion of copying the compile and link flags for the ARM64 architecture at this point just works. I’d recommend using my repo and check the diff to see what I did.

git clone -b xnu-4570.41.2 https://github.com/fergofrog/xnu.git
cd xnu
make LOGCOLORS=y SDKROOT=iphoneos ARCH_CONFIGS=ARM64 KERNEL_CONFIGS=RELEASE installhdrs
sudo ditto $PWD/BUILD/dst $(xcrun -sdk iphoneos -show-sdk-path)
sudo cp $(xcrun -sdk iphoneos -show-sdk-path)/usr/include/TargetConditionals.h \
    $(xcrun -sdk iphoneos -show-sdk-path)/System/Library/Frameworks/Kernel.framework/Versions/A/Headers/TargetConditionals.h
cd ..

libfirehose_kernel From libdispatch

The kernel libfirehose header and objects are required to build XNU. With the XNU headers and TargetConditionals.h header in the right place, this step is the same as before, however ENABLE_BITCODE must be set to “no” in the build.

tar zxvf libdispatch-913.30.4.tar.gz
cd libdispatch-913.30.4
mkdir obj sym dst
xcodebuild install -sdk iphoneos -target libfirehose_kernel \
    SRCROOT=$PWD OBJROOT=$PWD/obj SYMROOT=$PWD/sym DSTROOT=$PWD/dst \
    ENABLE_BITCODE=no
sudo ditto $PWD/dst/usr/local \
    $(xcrun -sdk iphoneos -show-sdk-path)/usr/local
cd ..

XNU

Now it’s time to build XNU. With several code modifications (one fix and some code stubbing) and help to the make scripts, this is as simple as running one make command.

cd xnu
make LOGCOLORS=y SDKROOT=iphoneos ARCH_CONFIGS=ARM64 KERNEL_CONFIGS=RELEASE BUILD_WERROR=0

From there, you’ll have your very own XNU kernel targetting ARM64 located at BUILD/obj/RELEASE_ARM64/mach (and an unstripped version at BUILD/obj/RELEASE_ARM64/mach.unstripped)