When you are sitting in the Apple Theatre during the Apple Worldwide Developers Conference, wave after wave of information comes crashing upon you. It is not easy to comprehend it all at the moment. Plus, it runs all week so there is even more to digest.
After re-watching the Monday program, and reviewing my notes for the week, some thoughts.
And, at the end of this post, the real big stuff yet to come.
17 June 2024 – – Almost no one outside Apple has really used Apple Intelligence yet. I know of only 2 people but having been on Apple beta teams, I suspect there are more.
It won’t launch until this autumn and even then it won’t work on 80-90% of the iPhone installed base. And we all know nothing is ever as good as the demos (well, except the original iPhone was), and Apple might mess up the developer incentives (as has happened quite a few times), and it all might just not be as good as it looks. We’ll find out in due course.
But meanwhile …
Apple is, I think, signalling a view that generative AI (and ChatGPT) is really just a commodity technology that is most useful when it is:
1: embedded in a system that gives it broader context about the user (which might be search, social, a device OS, or a vertical application) and
2: unbundled into individual features (ditto), which are inherently much easier to run as small power-efficient models on small power-efficient devices on the edge (paid for by users, not your capex budget) which is just as well, because
3: this stuff will never work for the mass-market if we have marginal cost every time the user presses “OK” and we need a fleet of new nuclear power-stations to power it all.
First, Apple has built an LLM with no chatbot. There is no where that you can plug a raw prompt directly into the model and get a raw output back – there is always set of buttons and options shaping what you ask. Apple’s model won’t suggest putting glue on your pizza, as Gemini famously did, because you can’t ask it open-ended questions at all. Rather, Apple is treating this as a technology to enable new classes of features and capabilities, not as an oracle that you ask for things. Of course, incumbents always try to make the new thing a feature, but it is very unclear to me that the raw chatbot itself is a product.
Second, Apple is drawing a clear split between an on-device “context model” and a “world model” in the cloud. If you ask it questions about your data, Apple’s models have access to that context about you (all private and on the device). If you ask it for ideas for what to make with a photo of your grocery shopping, it will offer to send that to a world model – today, ChatGPT.
It is hard for anyone else to match that package. OpenAI of course has none of your context, and Google has the “world model”, and the context if you use Android (a distinct minority in the U.S.), but Android has only a fraction of the high-end install base that the iPhone has, and without anything like the same level of integrated AI silicon (except perhaps in the Pixel, but where sales remain a rounding error) so it would struggle to do this locally and hence for free. Microsoft’s AI PCs have some of this, but the smartphone is the primary device with all the real context for most people now, not the PC. Does Meta have that context? Maybe. They are keeping their cards close to their chest.
Third, this past May, as we wrote, an internal Google memo leaked that claimed “there are no moats in LLMs”, because everyone has essentially the same access to training data and there would be good open source models. And that’s pretty much what has happened: the only moat is capital, and access to Nvidia chips for now, and depending on how you count it there are anything from half-a-dozen to a dozen top-tier models, with OpenAI still a little ahead, but not by enough. And Apple wisely does not claim its new foundation model is the best for everything- but it does appear to be good enough for the features it wants to provide. This isn’t going to play out like search, or operating systems. There is no sign, yet, of clear winner-takes-all effects. Apple built its own model. A lot of people fell out of their chairs when they saw that.
Fourth, everyone in tech is trying to commoditize someone else, give their product away for free, or both. Meta is giving Llama away for free (both the model itself and, for now, even embedding free queries in its apps) where the hyper-scalers want to charge for models, because Meta wants this to be cheap commodity infrastructure where it will differentiate with services and features on top.
Apple is doing something very similar. A lot of the compute to run Apple Intelligence is in end-user devices paid for by the users, not Apple’s capex budget, and Apple Intelligence is free. We still have no indication on how much the Apple Private Cloud will cost, not the likely mix of local versus cloud queries.
But this is also an integration story. There was a time when “spell check” was separate product that you had to buy (anybody on here old enough to remember that?) for several hundred dollars, and then it was integrated first into the word processor and then the OS. Today “summarise this document” is a paid service from a cloud LLM – but tomorrow the OS will do that.
Hence, fifth, OpenAI is being given (apparently) “free” distribution to a few hundred million Apple users – for which it bears all the inference costs and has (apparently) no obvious upsell path.
But it’s also being treated as an interchangeable plug-in. There’s a parallel here with Goldman Sachs powering the Apple credit card, and of course with Google paying Apple $20bn a year to be the default search engine. Apple’s AI chief John Giannandrea made this comparison explicity after the event this week: “I think of it a bit like the way Safari deals with search engines“.
But Apple decided that it makes no sense to try to build a search engine as good as Google – and, incidentally, no one else has really managed it either. On the other hand, it did build Maps, and though it messed-up badly at the beginning, Apple Maps are now at least “good enough” because again, there was no real moat except capital.
And, surely, the foundation model that Apple built itself to run in the cloud IS a “world model”, and you could ask it for pizza recipes – it’s just that so far, Apple has decided not to offer that to the users. It’s letting OpenAI take the brand risk of creating pizza glue recipes, and making error rates and abuse someone else’s problem. The next step, probably, is to take bids from Bing and Google for the default slot, but meanwhile, more and more use-cases will be quietly shifted from the third party to Apple’s own models.
Sixth, what does this mean for Nvidia? Chips and data centre analysis talk a lot about Nvidia’s moats, both in the silicon itself and in the software moats for developers that it has built on top. Apple, of course, is big enough to take its own path, just as it did moving the Mac to Apple silicon: it controls the software and the APIs on top that are the basis of those developer network effects, and it has a world class chip team and privileged access to TSMC.
Who else compares to that today? Google TPUs? It seems unlikely that many other tech companies, even huge ones, will build their own completely custom silicon-to-GUI AI stack. The place for change is in where the models get run: the full foundation model doesn’t fit on a phone, yet, but the more that the real use cases come from unbuilding that model and the “oracle” into features, the more that the inference could shift to the edge a lot faster.
Finally, all of these are still just theses. None of this stuff worked just 18 months ago, and we don’t have product-market fit yet. Anywhere up to half of the population of many developed world countries have already tried generative AI, but half of those never tried it twice. That’s great awareness for something so new, but it’s not traction.
So we don’t know what the product will be, nor the market, nor the science, and everything is still changing really fast. There may be new breakthroughs in agents or error rates that change the use cases entirely. But meanwhile, I keep coming back to the old observation that AI is whatever doesn’t work yet, because once it works it’s just software. Apple is trying to bottle utopian AI maximalism into software.
And the big stuff still to come?
Someone (no names; I signed an NDA) is working with a LLM platform on an AI that would create a graph of people who are connected to other people through far more criteria than networks like LinkedIn or Facebook. It would take into account all of those connections and others. Kind of like a current-day geneology of relationships.
I find I look for new stuff in the social web the way I used to look for new stuff on news websites before we got RSS and feed readers. It made a little sense when it was only Twitter. But now I have to check Bluesky, Mastodon and Threads, too. Plus the usual suspects. It’s not just the writing that has to be distributed manually, but also our time as readers. And then there’s the question of where you reply if you have something to add.
Such an AI as described above solves that issue. The reply/comment is shared with the entire network at one time.
If you and I both have accounts on ChatGPT, it would be nice if I could include you in one of my conversations with the bot, and we could explore an idea together with access to all the information we might want to call up. And actually, this is what all the chat companies are trying, but I think the one in the best position to do this is OpenAI, because they have the top rung in the positioning ladder for AI apps. It’s where most of us go for our AI.
Like Visicalc which had the top rung of the spreadsheet ladder until Lotus 1-2-3 took over, which was then knocked off the top rung by Excel. This is why these developers need to move forward aggressively because that’s the way tech works. Before too long all the chat apps with have their AI bots in the loop and it will be too late for OpenAI to dominate. But right now they are the obvious choice for this.
