Apple revealed that its new Apple Intelligence features were developed using Google’s Tensor Processing Units (TPUs) instead of the more commonly used hardware accelerators from NVIDIA, such as the H100. This surprising decision was explained in an official Apple research paper, which provided insights into their AI development process. The paper detailed how Google’s TPUv4 and TPUv5 chips were essential in developing the Apple Foundation Models (AFMs).
These models, including AFM-server and AFM-on-device, are designed to support both online and offline Apple Intelligence features introduced at WWDC 2024. For training the AFM-server, Apple’s largest language model with 6.4 billion parameters, the company used an impressive setup of 8,192 TPUv4 chips, organized into 8×1024 chip slices. This training process was done in three stages, processing a total of 7.4 trillion tokens. On the other hand, the smaller AFM-on-device model, with 3 billion parameters and optimized for on-device processing, was trained using 2,048 TPUv5p chips.
Apple’s training data was sourced from the Applebot web crawler and licensed high-quality datasets. The company also used selected code, math, and public datasets to improve the models’ abilities. According to the benchmark results in the paper, both AFM-server and AFM-on-device perform exceptionally well in areas like Instruction Following, Tool Use, and Writing. This positions Apple as a formidable player in the AI field, despite entering the market later than others. However, Apple’s strategy for breaking into the AI market is more intricate than that of any other competitor.
With Apple’s vast user base and millions of devices compatible with Apple Intelligence, the AFM could significantly transform how users interact with their devices, especially for everyday tasks. Therefore, it’s crucial to refine these AI models before a large-scale rollout. Interestingly, Apple, a company usually known for its secrecy, has shown an unusual level of transparency. The AI boom is prompting some changes in Apple’s approach, and it’s intriguing to see the company reveal these inner workings.
Via Tom’s Hardware
Images: Apple
