What does a layered AI strategy look like?

This week, I've been thinking a bit more about how a business should develop and implement a strategy for maximizing the impact of AI. The landscape is complex, with dozens of choices and various tradeoffs. 

I spend a fair amount of time meeting with customers, acting as an executive sponsor, and offering my expertise to support their technology goals. 

Lately, conversations have been dominated by the rapid pace of change, the glut of available options, and competing priorities as technology leaders struggle to solidify an approach to AI transformation. This week, my interest in this topic has been rekindled as a result of experimentation and announcements from an industry titan.

Lessons From the Lab

In a previous post, I talked about Tascii, my platform for task management. In short, Tascii is a Python library, REST web service, and responsive web app that turns a directory structure of Markdown notes into a data store for tasks. (Note: I am still intending to open source Tascii, but it needs more time to bake first). 

Last week, I started building an MCP server for Tascii, which has given me the ability to use MCP clients like Amazon Q Developer and Anthropic Claude Desktop to interact with my tasks using natural language, powered by the latest and greatest foundation models from Amazon and Anthropic. 

I have also made it possible to leverage local models via Ollama, but, the infrastructure in my home lab is constrained compared to what AWS or Anthropic have at their disposal. As a result, the end-user experience is significantly better using hosted models. For example, if I want to use Deepseek-R1, I can pick from a number of different versions, including smaller "distilled" models with as few as 1.1 billion parameters, all the way up to the full 671 billion parameter beast of a model. The smaller models range from 1 GB to 9 GB in size, but once you pass the 14 billion parameter threshold, that balloons to 20 GB for the 32 billion parameter distilled model, and a staggering 404 GB for the full 671 billion parameter model. 

I'm a big advocate for personal data privacy and security, and while the appeal of running local models on endpoints is strong, there is clearly a benefit to using much more capable models in the cloud or on hosted platforms. 

A simple, one-size-fits-all approach is unlikely to give the best results. Instead, we will need to deploy a more layered strategy. What does that look like? Let's take a look at an example.

Apple's "Onion" AI Strategy

This week is Apple's annual Worldwide Developer Conference (WWDC), where the undisputed king of consumer devices announces new tools available to developers and new features for customers. 

WWDC in 2024 leaned heavily into Apple's take on AI, which they call "Apple Intelligence." It's fair to say that the past 12 months have been a bit rocky for Apple and its AI initiatives as they've struggled mightily to deliver on their promises. 

This week, Apple's announcements reinforced aspects of their 2024 strategy but brought some clear refinements relevant to our exploration. Apple has chosen to build its own foundation models. This task is truly out of reach for all but the world's largest, richest companies. Like Deepseek-R1's collection of models, Apple's foundation models range from datacenter-scale monsters to tiny, on-device models for specific features and use cases. 

Apple Intelligence has struggled since its launch in 2024, and I believe that is in part due to Apple's predilection for having complete, full-stack control over every aspect of its products and services. This week's WWDC announcements indicate that Apple is beginning to see the light. While Apple still makes a big deal about their on-device models, they've also crafted a privacy-respecting cloud service for developers to access their larger, more resource-hungry models. 

Critically, they've also partnered with OpenAI to make the models powering ChatGPT available to their developers, promising more flexibility and choice coming soon. This is a noticeable shift from one of the most "NIH syndrome" companies in the world. With small on-device models, powerful Apple models in the cloud, and hosted partner models from major AI players, Apple has created a multi-layered "onion" strategy that may prove to be a critical pivot point for AI at Apple.

AAI Model Tug of War

Last week, I had the pleasure of presenting at a CDW Leadership Technology Conference in Austin, Texas. The event brought together seasoned, experienced technology leaders from the CDW customer base to explore a variety of timely and relevant topics. 

I certainly enjoyed presenting, but another presenter managed to tickle my brain.

The presentation in question was from Roger Haney, one of CDW’s Chief Architects, and an all-around smart dude. Roger gave an in-depth overview of “platform engineering,” which is a discipline focused on enabling software teams to move quickly, with self-service workflows and tooling, all while maintaining adherence to a consistent, opinionated set of constraints. 

What The Zen has done for Python, platform engineering can do for software teams, providing a shared set of technologies, design patterns, and constraints that enable rapid innovation in a well-considered sandbox.

If you’d like to learn more about platform engineering, let me know! 

I’m also interested in hearing your reaction to The Zen. What is your favorite line? What are your thoughts on what we can learn from it?