Core ML Compute Placement

The question

MLComputeUnits reads like a switch but behaves like an allow-list. You can permit the Neural Engine, the GPU or both, and the runtime then places each operation as it sees fit. No public API reports the outcome, so the usual advice, leave it on the default and let Core ML decide, cannot be checked from inside an app. This work checks it: the default (ALL) against the two explicit choices, Neural Engine with CPU and GPU with CPU, on five models and two Apple silicon configurations.

Method

Five models converted with coremltools 9.0 to fp16 ML programs at batch 16: SigLIP base, ResNet-50, MobileNetV2, DistilBERT, and the Whisper tiny encoder. The hardware is three M4 Pro Mac minis and one M5 Max MacBook Pro. Throughput is the median of repeated burst runs; sustained behaviour comes from 120- and 600-second soaks measured in windows. Placement is read from Core ML's own MLComputePlan, weighted by cost. That is the runtime's plan, not a trace of execution, and the write-up says so wherever it matters.

The repository is the instrument as much as the result: one script reproduces the benchmarks on any Apple silicon Mac and prints a table ready to paste into an issue, a placement check can fail a CI job when a model stops landing on the Neural Engine, and the paper's figures are recomputed from the committed results files in CI, so a claim that drifts from its data fails the build.

What it found

The faster engine flips between the two configurations on three of the five models. On SigLIP the Neural Engine is 1.14× faster than the GPU on the M4 Pro; on the M5 Max the GPU runs 1,077.7 images a second against the Neural Engine's 232.9, about 4.6×. Between the two, the Neural Engine gains 1.09 to 1.18× and the GPU 2.89 to 6.31×. MobileNet, usually held up as the Neural Engine's best case, is faster on the GPU on both.

The default is not a safe middle. On the M4 Pro it splits work across engines on four of the five models, and on three of them it is slower than either explicit choice:

M4 Pro, images/s	Default	Neural Engine	GPU
SigLIP	172.2	204.4	178.8
ResNet-50	386.1	929.6	659.5
Whisper tiny encoder	28.0	56.3	145.1

On the M5 Max the default puts everything on the GPU and costs only 1.01 to 1.09×. Over time the picture moves again: the Neural Engine gives back at most 0.14% under sustained load in 41 of 42 soaks, while the GPU gives back 1.1 to 13.2% on the M4 Pro and 4.8 to 16.3% on the M5 Max. On SigLIP the M5 Max GPU's lead falls from 4.38× in the first ten seconds to 3.67× by the end of a soak.

What it does not show

Two configurations are not a population, and each pairs a chip with a different macOS build, so chip and operating system cannot be separated. That is why the paper keeps a hardware explanation for the inversion but withdraws it for the cost of the default, and why the conclusion is stated per configuration: placement has to be measured on the machine it will run on. There is no decoder-only language model in the set, which is the largest gap, and Python call overhead is inside every figure.

Release

Version 2.0.0, released 30 August 2026 under the MIT licence, with the paper, tools and every results file in the repository. The DOI below always resolves to the newest release.