Asking Different Questions
Don't miss the forest
Disclaimer: The views and opinions expressed in this blog are entirely my own and do not necessarily reflect the views of my current or any previous employer. This blog may also contain links to other websites or resources. I am not responsible for the content on those external sites or any changes that may occur after the publication of my posts.
End Disclaimer
I had a friend of mine, who happens to be the CEO at a large company, ask me the other day regarding AI,
“Do you find that you are beginning to ask different questions?”
I thought that was a great question.
So let me respond, no pressure (close up on a single bead of sweat running down my temple).
Here goes:
LLMs are current AI.
They are amazing, albeit wrapped up in a clunky chat interface.
They are the embryonic phase of what AI will eventually become.
They are a patch of trees in the forest.
Don't miss the forest for the trees.
It’s early days.
Chat GPT was released to the public as a “research preview” November 30th, 2022.
Open AI says 400mm people are using GPT weekly. That sounds like a big number in a relatively short amount of time (roughly ~5% of global population). By comparison, Facebook launched in 2004, has ~3 billion average monthly users (~38% of global population).
The future of AI will not be chat, will not be summary and extraction, and comparison- all the things people are focused on now. All the things companies with terrible data are throwing at gen AI and expecting a good result.
The near future of AI will be doing things- taking steps, actions, getting stuff done for you- this currently obtuse, formless, and edgeless promise being called Agentic AI.
The Agentic Curse of Complexity
Agentic AI is nowhere close to being ready for prime time, despite what your 3rd party vendors may try to sell you.
As Chip Huyen points out, among others- say you have 5 agentic AIs each kicking ass at their respective agentic “jobs”- each scoring 98% on whatever metric.
Isn’t that awesome?
Well not when those jobs are combinatory .98*.98*.98*.98*.98 = ~%90
90% accuracy/efficacy is a non-starter for many uses cases.
But, put a placeholder in that.
2025 will be a year of companies trying to build Agentic Agents, and sell “Agentic Agent Adjacent”, aka bullshit agents, cause agents aren’t ready yet.
But they will be.
Innovation equals Ass in Chair
Seeking differentiated ways of doing things and asking more abstracted questions about the future requires something commonly referred to as innovation, an often romanticized “pay no attention to the man behind the curtain” sorcery.
It may look like magic, but that man behind the curtain needs to keep grinding.
Practical or applied innovation requires a similar process to how Stephen King and Steven Pressfield describe the art of writing. For King, “Writing equals ass in chair”. Pressfield calls the process, “Put your ass where your heart wants to be”.
All this to say, the process or act of innovation should be, to some extent, unsexy and tedious.
What psychologist Anders Ericsson termed “deliberate practice”.
There are seldom any running down the street naked, Archimedic “Eureka” moments.
To call innovation anything other than work is a trap.
Hey What’s the Problem? Just innovate.
Hey wait, isn’t there a job for that?
Yes yes I know.
It must be tough being a Chief Innovation Officer, where one must innovate from 9-5 each weekday, allowed only brief bathrooms breaks and a 40 minute lunch innovation respite.
That’s an impossible ask.
Eureka on demand.
So then how to think about the future and start asking different questions?
Take whatever exists currently and assume its solved for. Then assume the next thing is solved for.
Repeat and iterate until you get a nosebleed from all the abstractive thinking.
Of course, never assume anything.
One of my old trading bosses used to say that makes an “ass out of u and me”.
But this is a thought experiment, so just go with it.
What will you do after the thing is solved for?
The thing in this case can be all the current projects (extraction, comparison, summary) everyone is doing with llms that are going to be table stakes in 2 years time. Right now people are concerned about things like hallucinations and how to go from something like 88% accuracy to 92% accuracy.
They are asking the wrong questions.
Forget about that…sort of. That stuffs all going to be completely solved for.
Build the infrastructure that can support improvements to the underlying models.
Buy vs. Build
What do you build?
The pricing reduction and commoditization of llms has happened so quickly that, were you to start building things as a company with llms right now, you could probably wait it out a year or 18 months for someone else to build what you need in such a way that the both the pricing and efficacy make sense.
Right now your team runs the risk of working on something for 6-12 months that one of the big tech companies solves for in a random inconsequential model upgrade or software update.
Take the time and get your data in order first.
Use that data, your actual proprietary data, for the secret sauce projects.
And now I’m going to tell you exactly what those secret sauce projects are.
Wait…Whoops.
Secret sauce is reserved for the one who brought me to the dance.
I hope we can still be friends.
What AI vendors do you choose to buy?
It’s going to be really hard for smaller vendors to differentiate and keep up long enough not to get run over.
There are very few real llm/foundational model moats right now between the big tech companies.
Meta, Google, Apple, and couple others can throw infinity money at compute, pre, and post training. They all have their pre-existing ecosystems.
Companies like Open AI and Anthropic need to raise money.
They only sell LLMs.
Those two are in a race to achieve AGI and the wish/hope/expectation that Artificial General Intelligence, whatever that entails, immediately makes money.
There are gigantic, seemingly insurmountable moats between the big AI companies and the small ones.
There are AI vendors right now aiming to have a niche or a silo in a particular industry, often times one that hasn’t historically embraced tech.
The one-eyed king vendors in the land of the blind, non tech-aware companies.
Generally speaking, I would be weary of placing my (company’s) money or tying my company’s fate and future technical debt to their platforms.
Will there be disruptive startups that achieve great success? Always.
But I’m not a VC and that’s not the bet I’m tasked with making.
So for things we can’t or I don’t want my team to build for the foreseeable future, I need to choose those companies with the widest, deepest moats.
Preferably stocked with alligators.
Ask different questions.
Think on top of, inside of, and outside the box.
Don’t miss the forest.
Don’t slow down.