Transcript
Alex Palcuie: I'm Alex. I'm on the AI reliability team at Anthropic, which means my job is keeping Claude up. I've been using LLMs as part of actual incident response, and I'd like to share a candid and humorous discussion about what works and what doesn't. I'm one of the two people that joined the reliability team in London. I did on-call for Claude's serving stack, Solo, for my first three months, which is a very effective and fast way, but really stressful, to learn about the system. Then I onboarded everyone else so I could stop being on-call forever. Before that, I was at Google, SRE on Google Cloud's compute product, GCE. I was on the SRE for SRE team, which is the escalation layer for when an outage is bad enough that even normal SREs want to back up. I've been carrying pages for a while. I do have opinions of what makes an incident response process good and what makes one bad. While naturally skeptical in the beginning, since about January this year, I've started doing something that feels slightly transgressive to admit, which is, I start reaching out for Claude before I reach out to my monitoring dashboards.
Can Claude Fix Your Incidents?
Then you're asking yourselves the one question, and I get asked all the time at dinner parties, someone finds me at this conference hallway, my friends at my old company where their VPs are telling them to do something with AI, does Claude really fix your incidents? Is the on-call rotation just Claude right now? Have you basically automated yourself out of the job, just like the original SRE book wrote more than 10 years ago? I understand why people ask this. If anyone has this working, it's the company that makes the model. We have unlimited tokens. Researchers sit a desk away from me. I get involved in training the models. Surely, if it's anyone, it's us. Let me get the answer out of the way. It's a no. I want to sit with the no for a second. It would be genuinely hypocritical for me to stand up here and tell you that Claude fixes everything.
My team just shortly will have its one-year anniversary. If an LLM could carry a pager, we might not need to hire so much. The fact that my team exists, the fact that we're hiring for many positions, in London, in Dublin, and the U.S., and we have staff positions, this should show to you that, no, it doesn't work. However, there's an asterisk over there. The asterisk is related to the timelines. I think many of us would not be surprised if somewhere in the future we would be able to do such a thing. Today, I also point out the useful ways in which Claude helps me during my on-call. Going back, Claude is down more often than any of us would like. Earlier, I was involved in an incident, even if I'm at a conference. You may have noticed, and some of you might have tweeted, and I have the same sentiment when Claude goes down. Full of opinions, people are tagging me. Please continue doing these. These are the positive tweets. I also got the mean ones, but that's social media for you nowadays.
AI SRE
There's even more. I know at least 10 companies right now whose entire pitch is some version of AI SRE. Since I'm also pretty open about angel investing, and somehow people think I would be less skeptical about them, and I'm more skeptical about them. There are now benchmarks. There are curated datasets. There are historical incidents. There's the game of how much percentage can your app solve of these incidents. There are papers. There's like the whole on-call industry. The cynical take that we could have in a room full of senior engineers who've seen hype cycles before, is, of course, this will be it. There's VC money sloshing around. Someone was always going to slap AI on ops, say that the TAM is billions, that it will swallow the observability industry, and our above-average compensation. They go, they raise a Series A from investors. Those investors are building a diversified portfolio.
There's AI law, there's AI healthcare, and there's AIOps. Yes, it's a lot. This is not an exhaustive list. By the time I was writing the slides, at least two other companies appeared. The cynicism is warranted. Some of the benchmarks are like, can the model solve these pre-packaged problems into a clean prompt? This is not what something at 3 a.m. when you get paged looks at. Actual incidents are not well-formatted problems. I'm not cynical about the goal. Genuinely, unironically, I'm cheering for these people. I want them to practice. It's not for the reasons that people typically assume. The reason is that on-call is a tax we levy on humans because our systems are not good enough to look after themselves. How many people here have been on-call? You know the physical reality of it. Your phone buzzes, there's a half a second where you go from asleep to probably incident commander mode.
It's not that any individual page is bad. I like a good incident. Every other day, it's the cumulative weight of this problem. It's never quite relaxing. By on-call standards, I'm one of the lucky ones. I have followed the sun rotation. London hands off to the West Coast at 6 p.m., so I can go to the pub. The worst case for me is I'm up at 6 a.m. Like some of you, you're up at 3 a.m. or 1 a.m. You're on a 24/7 on-call rotation. There's probably four people. You get paged at 3 a.m. You go to sleep, get paged at 4.30 a.m. because the other database is broken. Then at 9 a.m., you show up at work and you have to be at your stand-up and look professional and presentable. I've been there in my career. Also, I want to be clear, this should not be a badge of honor. Our industry sometimes treats it like this, like we call them war stories. In panic rooms, we use combat metaphors. It is a cost in sleep, in our attention, in our relationships with the people. When someone tells me, we're building AI SRE, my first reaction could be, that'll never work. I'm like, I hope you get it right.
There's a second reason, which is more cold-blooded. Some of the best early career advice that I got from someone who was more senior than me was that you don't get promoted for fixing incidents. I remember thinking that was wrong the first time I heard it, because it feels like it should. You're the hero. You got paged. You figured it out. You fixed it. The service came back up and people said thank you on the Slack channel. It does count at first when you're a junior. Like, can I fix a production system on fire? That's a real signal that you're on your way to be a good engineer. It stops counting quite quickly. By the time you're a senior, of course you can fix this. That's table stakes. It's in your job description. What actually moves the needle in big tech parlance, what gets you promoted, what makes you the tech lead, is when people are like, they're operating at a different level.
It's building the thing that makes the whole class of incidents just not happen. Not fixing the one fire, but making it structurally so that those fires will never happen again. Here's where the AI on-call dream happens for me, if, and big if, an LLM can handle the generic mitigations, the obvious rollbacks, the have you tried, the standard remediation stuff. Then, us, the humans, can go and work on the prevention part, the platform part, the stuff that scales. That's a dream, and I want it to work.
The OODA Loop (Observe, Orient, Decide, Act)
Here's the framework that I use to explain incident management. I'll be using it across, for a good chunk of the presentation. It's from a U.S. Air Force colonel that used to train fighter pilots. He was trying to explain why some pilots consistently win against other pilots, even though they're flying less better aircraft. The answer was, it wasn't the aircraft. The winner was the person who would cycle the most through this loop of observe, what's happening? Orient, what does it mean? What's my mental model? Decide, what am I going to do about it? Act, taking the decision. Then, again, taking the feedback, because the world changed, and running this loop again. I feel this maps, not perfectly, but very well on incident response. You get paged. You look at your graphs. You look at your logs. You build a mental model of what might have been broken.
You decide on a mitigation. Then you go to a terminal and write the command and submit it. The reason I like this frame is that LLMs and incident response, they're not uniformly good or uniformly bad. They're just wildly uneven, we use the term jagged in the industry, across the stages of this loop. It's generally superhuman at one of them. It's pretty dangerous in another part. It's really murky at two of the parts.
Observe Loop - Superhuman
Let me walk you through the observe loop, where I think LLMs are superhuman, by gathering signal from a system, reading the dashboards, querying the metrics, pulling the logs, finding the needle in the haystack. This is something that, while we train ourselves, and we get good at it, it doesn't come naturally. It's also, our limited attention is not parallelizable, not scalable. Claude, I'm just going to say it, and you can replace Claude with any LLM that you use, is just better. It's not smarter, necessarily, but it's so fast and it can clone itself. It can hit every metrics endpoint you've got in parallel with PromQL, and it can write the syntax with almost no mistakes. It doesn't get tired about changing the variables. It reads the logs at the speed of I/O, just as they come. It doesn't get bored anymore by 2,000, 4,000 lines. This at scale is something that no one human can match.
What we don't like the most, then we'll have stories. It's December 31st, New Year's Eve, skeleton crew. I don't remember if I was on call. I don't think I was on call, but 500 HTTP internal errors are picking up on Claude Opus 4.5. I open my Claude Code, and I ask it to have a look with that particular prompt. It's not just that prompt, I have written my own SKILL.md that teaches the model the life of a request, how a request hits the edge servers, and then hits the API frontends, where we do admission control, and quota checking, and all the boring business logic. Then it goes to the inference backends, where we have accelerators like GPUs, TPUs, and Trainiums across clouds, and the matrix multiplication happens, and the tokens go out to the user. Once I've thought that, there's another file that's common across the company, where it teaches Claude where's our data warehouse, what are the most important tables, what are the fields, and importantly, how to introspect and self-serve by itself if it needs more information. The next part of the conversation, it has been abridged, and I have removed proprietary information. I have posted internally a big chunk of it at Anthropic.
Claude goes and pulls up the last hour of server errors. It just writes a SQL query, and within seconds, it has the answer, an unhandled exception in the image processing path. Weird. 31st of December. It then goes in the codebase, because it's in the monorepo, it finds this error type where it is raised, and figures out a possible explanation for what happened. It actually posted the entire stack trace, but I won't bore you with Python. However, it doesn't stop there. This code path has been live for weeks. It does get logged. It looks if someone changed it recently, and nobody's hit it until tonight. It pulls three failing requests, reads the raw JSON of the payload, and one has an image array with exactly 22 images, and a PDF is attached. The second, the third, it grabs even more, and the bug happens to trigger on 22.
Why would that happen? It asks, who's sending these? It runs another query, where it reps the request logs, and it sees roughly 200 accounts, all sending exactly 22 images, starting tonight, roughly at the same time. Now this, as the younger generation says, nowadays, is pretty sus. It runs another query. It doesn't stop, it's relentless, to find out how many accounts were created in that timeline. Roughly 4,000 of them. Same window. Same email template. Same provider. To quote the famous singer-songwriter, I knew you were trouble when you walked in, the other 3,800 accounts haven't sent a single request. They're dormant, sitting there since December 28th, waiting. It runs another query, and then it looks at how often these accounts were created, and it saw that they were using 9 signups a minute. That looks roughly what I would expect someone in account abuse, maybe setting a limit.
It then says, stop looking at the 500s, this is fraud. Not might be suspicious, not worth flagging. I have literally left the Claude slop, where it gets really excited, where it finds something, because I was shocked. I would have only looked at the 500, internal errors. I would have marked this as a bug for my API team. I would not have paged account abuse on the 31st of December, to join in with me, and start looking on their side of things, because I do not have access on PII, for what was happening there. You would think this would be my keyboard from now on. I went to get a computer science degree, and I get three keys. In AI, there's this famous move 37, where AlphaGo, the DeepMind algorithm, was playing Lee Sedol, over 10 years ago. All the Go players were shocked, of like, why was the AI doing that? Many people have personal move 37 moments. This was my move 37 moment. I knew on that day, that this year would be special.
Story number two. This is also a positive one. We had Rust panics in production. While I was going through the logs manually, because this is something that I still do, I don't know why, one of my newer teammates, not even on-call trained, two to three months in the team, just pointed Claude Code at the logs. It instantly found the root cause, like a Rust panic in checkpoint.rs, with some segment ID validation, the file, the line number, the assertion message, even before I finished reading my two pages of logs. Then he prompts again to get the full volume breakdown, like a gazillion panics in five minutes, then drops, then spikes again. Claude is very helpful, like across two independent hardware platforms, so I don't need to go to my CSPs and ask them. It's not reasoning about the cause yet. It's just gathering signals way faster than even the subject matter experts can do it.
The panics are all in the prefill servers. It involves duplicate segment ID. Let me check when the binary was deployed, whether there's a rollout change right before the panics started or not. It just finds it, before me. The new person in the team is way more effective than the people with experience, because you're unencumbered by the fact that they should go manually and look at these things.
Orient Loop
Now we talk about the Orient part. This is where the OODA loop gets interesting for LLMs. This is a failure story, finally. The first thing I need to explain to you is something that happens with LLM inference, so you actually understand the business logic and what went wrong. When you prompt Claude, or any transformer, the naive way to generate tokens is to take the entire sequence, never going to give, it's the first one, and then put it entirely through the transformer, and then you would get the first token called up. Then you take that entire sequence again, put it again through that transformer, and then you would get the next token, which is never. You're appending the whole thing, but this is wildly inefficient, and no one does the naive one. This is just used in academia to explain. Because by the time you're producing token 1,000, you've reprocessed token 1 a thousand times for the same matrix multiplications.
The trick is called a KV cache, during what we call the attention step, where each token produces a key and a value vector, and crucially, these don't change. We can save them somewhere. That's the diagram at the bottom. Now inference has two phases. There's prefill, where you process the entire prompt in one big parallel pass. You save all the key-value pairs into this cache, and this is what we call compute bound, because you're doing a lot of matrix multiplications. Then on the decode, or generate side, you take this KV cache, and then you feed one token at a time in the machine. Instead of reprocessing the whole prefix, you just have one pass. This is why, if you've ever wondered, tokens appear like a stream in Claude, in ChatGPT, in Gemini, because they are really generated one by one. If your inference provider is using a big batch size, you can see it even really slow.
If one machine goes broken, it's the same. Typically, we maintain it so you have a good user experience. This KV cache can be gigabytes in size, and it's really easy to break it. It's very finicky. It's fragile. When the KV cache blows up, you suddenly have to re-prefill a lot of prompts. That's a lot of compute you didn't have prepared. This is a class of incidents that happens quite often with Claude. When this happens, this is how my graphs look like. The gray line is filter requests. The red line is the number of errors. Orange is the incident window. Look at the shape. Like, requests roughly double at exactly the moment errors appear. Then they both drop together. Every single time, I would ask Claude, what happened here with this graph? Claude will say, request volume increased. This is a capacity problem. You just need to add more servers.
I have corrected it on this six, seven times. You add it to CLAUDE.md, and it understands about this situation. It will get wrong correlation versus causation in the other 99 situations. It's not helpful. If you have a new joiner in your team, they will immediately be swayed in this direction. They will immediately think, it's a capacity problem, where actually what you did is you lost your cache. How about you go and fix your cache and understand what was happening. Maybe you can salvage it. This is why I think we can't trust LLMs for incident response. If it could take a step back and try discerning between causation and correlation. I know for us humans, it is hard as well. We fight every day with this. We see two lines, and we're like, it was a rollout, obviously. Or it was something else. We have those scars. We have experience. I have seen this outage too many times in the past year. I cannot ignore this avenue. I can go and explore the other avenues.
Automation
Let's talk about automation from a higher-level perspective. We in software engineering are not the first ones to have to deal with automated systems. The Society for Automotive Engineers, the people who make cars, published a framework in 2014, over 10 years ago, about self-driving cars. Because that's when we started talking about self-driving cars. Still not in London. They tried mapping how would self-driving cars look like so they could have mental models. They wrote about several parts with several stages. One thing self-driving cars need to do is the execution, like steering and acceleration. The other thing is monitoring the driving environment. The other part is fallback performance, which is code name for like something unexpected happened, like an intersection is blocked, or you need to overtake a parked car. Finally, their biggest challenge, and they call it the scope and the system capability, which actually in my mind reads, would this car just self-drive itself when it's sunny in California?
Or, can it actually self-drive itself when it's raining in London? Because you can't just throw one into the other. As you go up in the level, more and more of those columns get filled by systems instead of humans. Sometimes it's both at a particular level. Most modern cars you can buy now are actually level 2. We get automated braking because we trust that technology. We have line following, there is cruise control. Level 3 is where it gets interesting. I think Tesla has full self-driving in the U.S., and in Europe we've got Mercedes and BMW, but only in Germany. That was level 3. Level 4 is the exciting one. That's Waymo. You don't have a driver, it's just a taxi. You go inside of the car, and you look at a steering wheel moving itself. You watch on the street how cars with no one in are moving from A to B.
Waymos are in the streets in London as well, in trial. I have complained to a friend of mine working there, why are they taking so long? In my mind, you take the model weights from San Francisco, flip the signs, and the car can drive on the other side of the road. He tells me that's not how research is done.
I wanted to map this to my job, to production engineering. Some of you have seen this before, maybe in a previous presentation. In my mind, the columns for me are detection, who identifies that something is wrong. Either spotting anomalies or correlating signals, which are the fancy ones, or actually, do I have good alerts when I have internal errors. Then there's the initial response, who executes the mitigations, like restoring the servers, doing the kubectl scale commands, the failover, the rollbacks, decreasing the user quota for the errant user, stop the bleeding that we typically do. Then there is prevention, which someone will do a deeper walk to improve the system, like looking at the contributing factors, what are the architecture changes. The, what makes sure this doesn't happen again work. Then, as the people from self-driving cars have, the operational domain. Does this system handle only the carefully instrumented flagship service that I have, or can I have this AI agent just parachute it in a different team?
It will learn by itself, and figure out what's going on. I've removed level 0. We were lucky. We got automation, because computers are pretty good. Level 1 is just some assistance for detection. Level 2 is where most mature systems get to. Observability is fully automated. You have alerts firing reliably. Even if it's not AI, that's still automation. You've correlated some signals, and it's where we arrive. The humans still execute the mitigations, and the humans still write all the fixes. Can we get from level 3, where right now we are, human and AI in the response, can we get to level 4? Which is what I've been talking about. Like, remove the humans from the incident response. Level 5, that's just AGI in my mind. If you can throw an agent everywhere, and you can have it as a teammate, that would be really cool.
Recipes of What to Do
That was the broad picture. That was the generic. That was the setting the scene. I now have five things that I have seen working in my team, in my company, and for myself. They've emerged from about one year of trial and error across everyone. Pattern 1, you've been paged, and there's an alert. You're a sensible person, so your alert is somewhere committed in your infrastructure as a code, and it has a PromQL or whatever query language is behind it. Don't open a blank chat. Don't just copy and paste things. Just give your AI agent that expression, and hand Claude some instructions that roughly read as, be curious. Pivot this for me. Here's how my system is set up, find things that I wouldn't normally look at. Your Claude is parallel, or mine is. It will look at your endpoints. It will look if it's a per-region or per-cloud thing.
It'll do things by status code. It will look at last week, how the traffic compared. It will issue those queries, I think, faster than you would be able to drill down into your dashboards. Or, if you start this when you get paged, and then you arrive 5 to 10 minutes at your computer, you already have a scene set up for you. We've found this extremely helpful. As in my previous example with the fraud, Claude is able to catch a lot more things, and it is very helpful. This is not a big change, just putting the alert in a prompt asynchronously for an agent.
Number two, get an exemplar, trace it through your system, but actually just ask Claude to do it for you. I run disparate systems with different logging patterns and with different log sources in different clouds. The one thing that unites them is a trace ID. Whenever I need to look at what happened at the ingress or at the API or at the inference level, previously, I would just open each of those windows. I would have the bookmarks, or I would have a bookmark of bookmarks. If you tell Claude to just do it itself, it actually is very useful because it builds a timeline in its context, and you can start talking with the timeline. You can see, that request bounced around three servers, went back to the API, where the 10-second timeout expired, and this is why I returned the error. That shows to you that, yes, it wasn't at the API, the timeout, it actually went to the backend.
It does this, and it's able to, with coherence, keep a life of a request. It's also able to tell you, it was this server. You can start talking about, what about that server? Did it have a CPU high, or was the condition on the memory? It does open a lot of avenues, even if sometimes you do have to re-verify the information that you got from the model.
Pattern number three is, what changed in this window? I opened a discussion in the unconference for this track. I joined a company where there were not many deploys, and there were not many config pushes, and while rolling out monoliths, it's great, it also doesn't scale. Now I live in a microservices world, where services go out all the time, in config pushes, in feature flags, in cron jobs. It's really hard for me to correlate what changed with the moment my incident started. We've built this handmade tracker of all the big changes that happen across the company. It is curated, in the sense that we're not going to DDoS it with 100 QPS of events. If you give Claude the time window of when your incident happened, it will let you know really fast about the deployments. Not only that, it will go in and check, maybe there's a commit there that affects the image preprocessing.
Which is very useful for getting avenues of debugging and discovering. It's a bit like what I would do by hand previously. Now, obviously you need to stop it to be really excited by, I found the root cause. It's like, no. You found three possible issues that could have happened, and we really need to dive through. If it says, maybe roll this back, and big if your rollbacks are easy, then, yes, just roll back. Check if your errors are now down. It's much faster, and it saves a lot of effort.
Number four, postmortems or retrospectives. Claude is good at the tedious parts. Like, I used to hate these. I'm really sorry for some of the engineers that I worked with, but I sometimes felt it was teaching through fire to get your more juniors or medium engineers that joined the team. I was like, yes, you were part of this incident. You did not fight the big fight, but as a prize that you are part of it, you will now write the two- to three-page postmortem, so you understand the moving parts, so you're able to learn what could go wrong. It was always a chore. Like, no one wants to collate timelines or Slack threads. Now, I take the whole Slack channel, I take my Google Meet transcript, I put it in text format. I give Claude a prompt with the postmortem template that's like 80% the one from the SRE book, and it does produce something.
Two issues, though. One, it gets really bad at root causes. It's very jumpy. It's like, ok, this was the thing, and we all know it is not one thing. We all know it's not one root cause. There are many contributing factors. There are many issues. They're like the Swiss cheese model that you have to go through. It was never the rollout. It was never the code change. It was all the processes in the company that allow you to go in the incident. Claude doesn't know the history of your system, especially if your system has been there for 10 years. It doesn't know the reason you didn't test your secondary database fallback. It doesn't know tacit knowledge that you know. While you get an 80% story that's readable and convincible, I ask you strongly to not share it until you have reviewed it. If documents like these proliferate across the company and they're not vetted by humans, there's a loss of truth that happens, and I've seen it.
One trick that we're now doing for postmortems is we tell Claude, if you don't feel certain about something, insert a todo for a human. It has a capacity of self-reflection. Not perfect, but it is there. We know that a postmortem is not for official consumption until all the todos have been solved, until we've fixed all the issues where we're not sure on that text. Because if you feed these postmortems again in a loop, garbage in, garbage out, but you can also have a good source of truth of what were your incidents. When Claude summarizes that vetted collection of documents, it is much better to write your roadmaps.
Number five, shift handoffs. Just as postmortems, you can point Claude to write your handoff. I have to stress, you need to build in public. All your on-call actions are in a channel where the on-caller thinks loudly by writing the debug sequence in everything that they've done during the handoff. It's good. You can also prompt Claude to just stick to the summarizing part. The person that's coming in doesn't necessarily need to read the 200 or 400 messages that happened during the London daytime. They just can read the one paragraph with links to go to the threads. This is probably the single most uncomplicated win you can get into.
The Learning Problem
I want to talk about something that I'm generally not resolved on. It's the learning problem. If Claude found the stack trace and suggested the rollback, and you approved it, and it worked, what did you learn? Senior incident responders aren't smarter. They don't know the system beforehand. They have been burnt before. They have seen other people debug the systems. They have the scar tissue. If AI starts doing this, will we have our skills atrophied? Since there's no more feedback loop on the actions that you need to take to know what happens to the system. Like in the SRE book, the reason we had a small team of humans be on call for a mission-critical system, and we didn't have a Rota of 50 or 100 senior engineers who would do it once a year, was because if we would put those engineers on call, and it was their job, when they were not on call, they would treat fixing the issues behind the pages as their priority.
Because you don't want to get paged again. This is your whole mission. This is what you do in your entire team. If the AI starts fixing this, you don't get the pain anymore. Your incentives might not be aligned. That's on the senior side. On the junior and the mid-side, you won't get trained in the easier incidents. You won't have to be put on the spot to type the rollback command without thinking, or looking at the history in your playbook. When the big thing happens that the model can't fix, you might be miscalibrated on how to respond to such an incident. I'm genuinely worried about this. A researcher for OpenAI, with their name roon, tweeted in 2023, so this is before Claude Code, and before even people knew about Anthropic, that Jevons paradox will allow software complexity to drastically increase until it's very hard to do software engineering.
Jevons paradox is the favorite paradox in the AI industry. It's when technological improvements increase the efficiency of a resource's use, but the resulting lower cost actually causes consumption to rise up rather than fall. In our case specifically, it's easier to write software, so we write much more of it, so the complexity goes up and not down, which means things break in more interesting ways, which means more incidents, which means more on-call. The follow-up reply is that all improvements in dev tooling that we can do will be canceled by this ever-growing complexity. However, and like pulls the magic wand, we could have agents break this cycle. We could spend arbitrary amounts of compute to simplify and manage the complex systems. We here have made recipes. We know how to grow teams. We know how to grow organizations. We have microservices. Maybe the AI agents can do what we've collectively learned in our industry. That's a big if. Unsurprisingly, roon is a prolific poster, and this has aged very well, and it keeps me up at night.
AI Models Trajectory, with Time
I'll let you with the favorite chart in AI. METR is an organization that evaluates AI capabilities, and one of their studies is how long can a model run autonomously. On the x-axis, you can see the release date of models. On the y-axis, you can see aggregate duration task. Each model is evaluated on a set of tasks where humans were previously asked to do it and measured for their success and timed. If a model manages to complete at least 50% of the tasks, we mark the duration on the chart. You can see the exponential trajectory of like models are getting better, and it's not that they're just getting better as we would expect linearly. It seems we're accelerating. Very many tweets have been spent debating this. Very many people have been drawing sigmoids that the growth will stop, just like growth has stopped for some of these lines. People have been claiming since 2022 that, yes, it will. We have the AI scaling laws. We know that models will improve once we have more data and more compute. They will become more intelligent. The models are the worst today that they'll ever be, and they'll be getting better from tomorrow onwards as we're working on this.
See more presentations with transcripts
