It’s a really good model. Over the past few days, I give Opus some general directives to basically speed up our CI, and telling it I care both about billing minutes and wall clock time. I told it to create a plan after analyzing everything in our CI, run the plan by a Fable subagent, and then focus on low-risk, high-reward changes.
9 hours later, I had 12 PRs ready to be merged, and the net result is CI time has dropped from ~10 minutes to ~4 minutes, and billing minutes have dropped around 60%. Less than an hour of my attention.
Agree Opus 5.5 is incredible and efficient with Claude Max Usage.What a jump after the writing slop you got from Opus 5. I had moved to using Fable for my orchestration workflow mainly due to the communication issue (I think Opus 5 was capable enough but spoke in riddles so you lost confidence quickly)..With 5.5 it needs less steering now and communicates well, and I have had the same CC session running for the last week (obviously compacting with durable plans etc as the post), with it PM'ing my home built agent orchestration of the other coding agents(antigravity, codex, pi etc) and making decent decisions and all the recommendations are normally usually good.
It’s great, but you still need to know what you are doing, not just the goal/s. I built a platform years ago from scratch, and now I am remaking it with more features and more polished design, I know exactly what needs to be done to tiniest details. The first prompt was very well detailed about the architecture and how everything should work, after an hour work at Xhigh, it did create the blueprint artifacts that I asked for, then I spent 3 days reading every single thing and writing notes, turned out it made the system overly complicated without adding extra value, plus I can see how some of the architecture design will be potentially a security risk. So after few days I fed my notes, this time took 4hours and 1M tokens! Later I spent more few days reviewing and writing notes, it was closer to what I want but still made architecture errors, the third run took around an hour and finally made it how it supposed to be, although there are still more notes on non critical stuff. So I don’t think we are yet at the stage where sitting goals and some high level is enough to produce quality results.
Yes. It spawned multiple subagents to run different experiments to benchmark a lot of different things, reviewed CI logs from past runs, etc. In the end, there were changes to what/how we cached, various code quality checks, speeding up test runners, and many other things.
Trying to make us feel old? Very common term among people around my age and higher (40+). Please don't turn "I've never heard that expression before" into "must be AI saying it".
It's a standard term and has been for ages. It distinguishes end to end time vs eg the amount of CPU time an individual process uses (which excludes time spent waiting for the system or time when the process was otherwise not scheduled on the CPU).
“Wall time” is a pretty common systems term (you see it when comparing total runtime to kernel time, for example). I wouldn’t have indexed on “wall clock time” or other variants as an LLMism.
In the meantime, I have cancelled my Anthropic subscription...
I have a simple test that I have been running iteratively across the SOTA models from several vendors, including one Chinese vendor.
I start with some code produced by an Anthropic SOTA model...let’s call that Code A.
Then I get Code B and Code C for the same task from models by two other vendors.
Then I ask each model to review and critique the other proposals.
By the end, both the Anthropic model and I usually run out of arguments...
against them and agree that proposals B and C are better.
Claude then always asks whether it can incorporate the code or ideas
from B and C into its own solution...
Been very impressed with my most recent project. I wanted to simulate some older electronics circuits. I handed it a folder with scans of old service manuals which contained circuit diagrams. It managed to correctly interpret the circuits, including figuring out some were the same topology despite the diagrams being quite different, or some that had some subtle but very important differences despite looking almost identical at a glance.
In a few cases it asked me to check some subcircuits and some component values because it couldn't read it right. So instead of just making things up it deferred to me.
It also ran tons of small simulation experiments while doing this to verify claims from the service manual, like that the RC filter it had read off the schematics actually had a cutoff frequency that was sensible in relation to some bandwidth number in the manual.
I had uploaded datasheet PDFs for many of the ICs and it used those to cross-reference and validate.
It kept on working for over an hour. When it asked for the manual verification, I described circuit connections in words, like "from pin 3 on IC 2 there's a series resistor of 3k in parallel with a 10 pF capacitor, it then connects to a 18k resistor to ground, a reverse-biased diode to ground, and then finally into pin 6 of IC 4", and it correctly understood the topology in all the cases. Sometimes it asked me to check again because it though something was off, and indeed I had mis-read the schematics.
I also provided reference articles on the underlying theory. Scannded stuff from the 40s and 50s. It correctly read the equations and cross-validated them across papers, and even caught several typos along the way.
I barely had to do anything apart from providing the PDFs and some occasional manual schematic interpretation.
Claude 5.5 on High. Burned through about 50% of my weekly $20 subscription usage, but I didn't try to optimize much.
I did use Sonnet 5.5 Medium on some datasheets and it also did very well on the extraction, but did have to correct itself more often on the conclusions.
It's much better than 5, but I've had a couple of situations this week where it was too interested in being independent, making calls that went directly against my recommendations. It can also do fun things like convince auto-mode to go way past what I have autorized. For instance, specific permission to run process X in region abz-1 suddenly became running X in 5 other regions, with no warning, and doing modifications that it never mentioned in the summaries. And a few of the times it got the calls very wrong, by assuming it understood systems it didn't. It'd even argue with me when corrected, as it assumed similar names were referring to the same thing, when they weren't.
So asking it to do things on its own for a long time? Given last week, absolutely not.
I've given it some big tasks and asked it to parallelize as much as possible etc.
It did burn through my weekly tokens in about a day (20x max), but the output was completely on point.
(I knew there was a "reset token usage - opus 5.5" button in my account.)
I've now come to a point where I even delegate my discovery for new features to it.
You still need to give it methodologies though to get the proper output, but the outcome is way beyond what I would be able to realize with a team of 5 in a month.
Is it still necessary to ask Claude to spin up sub-agents? If Opus 5.5 decides how carefully it needs to think after each question, surely it can also decide whether it needs to spin up sub-agents? GPT 6 series models at least seem to do this agent management automatically.
I really hate long tasks. Claude never gets things right, at least for me, and wastes tons of time when a simple question would have gotten me to the right result rather than several turns of correcting bad decisions in addition to the long amounts of wasted thinking time.
What sort of workloads do well with these long tasks? The big labs are optimizing for long run time on their own, but it seems like a terrible thing to optimize on unless you're trying to do something like prove a hard math theorem, which success is clearly defined and the route doesn't matter a ton.
Plan mode has been made increasingly useless. I need to discuss to iterate to get the desired design, explore options, because Claude never gets it right first try and I don't have enough knowledge of options to specify everything up front.
Ah well, the Chinese models will still work well, I guess.
It’s amazing at debugging too. I had it running in Powershell controlling an lldb session in MSYS2. The way it can read addresses and so root cause analysis is amazing! It takes a lot of mind power to do those things.
I like to watch it work though because it honestly teaches me some tricks.
Opus 5.5 is great and cheaper if you compare to fable with close quality in coding (tested in refactoring java to nodejs), but I do not understand why the week before the release Opus 5 started hallucinating (long loop and waste of token for single tasks)
I think what makes it great is that they trained it to write harnesses for the code it writes, so it can test stuff even if the supplied code is not complete.
It’s a really good model. Over the past few days, I give Opus some general directives to basically speed up our CI, and telling it I care both about billing minutes and wall clock time. I told it to create a plan after analyzing everything in our CI, run the plan by a Fable subagent, and then focus on low-risk, high-reward changes.
9 hours later, I had 12 PRs ready to be merged, and the net result is CI time has dropped from ~10 minutes to ~4 minutes, and billing minutes have dropped around 60%. Less than an hour of my attention.
I would also say: I think this works great because the goal is well-defined and measurable.
I’ve also used Opus 5.5 on some hill-climbing, and a lot more steering is required here, because … eval is hard.
Agree Opus 5.5 is incredible and efficient with Claude Max Usage.What a jump after the writing slop you got from Opus 5. I had moved to using Fable for my orchestration workflow mainly due to the communication issue (I think Opus 5 was capable enough but spoke in riddles so you lost confidence quickly)..With 5.5 it needs less steering now and communicates well, and I have had the same CC session running for the last week (obviously compacting with durable plans etc as the post), with it PM'ing my home built agent orchestration of the other coding agents(antigravity, codex, pi etc) and making decent decisions and all the recommendations are normally usually good.
It’s great, but you still need to know what you are doing, not just the goal/s. I built a platform years ago from scratch, and now I am remaking it with more features and more polished design, I know exactly what needs to be done to tiniest details. The first prompt was very well detailed about the architecture and how everything should work, after an hour work at Xhigh, it did create the blueprint artifacts that I asked for, then I spent 3 days reading every single thing and writing notes, turned out it made the system overly complicated without adding extra value, plus I can see how some of the architecture design will be potentially a security risk. So after few days I fed my notes, this time took 4hours and 1M tokens! Later I spent more few days reviewing and writing notes, it was closer to what I want but still made architecture errors, the third run took around an hour and finally made it how it supposed to be, although there are still more notes on non critical stuff. So I don’t think we are yet at the stage where sitting goals and some high level is enough to produce quality results.
9 hours?!
Yes. It spawned multiple subagents to run different experiments to benchmark a lot of different things, reviewed CI logs from past runs, etc. In the end, there were changes to what/how we cached, various code quality checks, speeding up test runners, and many other things.
I dare not ask about the cost, having burned $60 on a task running for 1h 16min once.
I’m on the $100/month subscription; this session took about $500 in token-equivalent costs.
(Note that it wasn’t all Opus 5.5; I have a setup that uses Fable 5.1 as an advisor, Sonnet 5.5 for mechanical changes, etc.)
Meh... There's a reason Opus 4.6 is still an option.
Pros know these are lower cost models.
Is "wall-clock" an actual term you used before Claude? I had never heard it before the model used it and I can't stand it.
It's a pretty old term, to distinguish from e.g. CPU time. This was in common usage even 30 years ago.
Example from 15 years ago: https://stackoverflow.com/questions/7335920/what-specificall...
Trying to make us feel old? Very common term among people around my age and higher (40+). Please don't turn "I've never heard that expression before" into "must be AI saying it".
Wow...it's super common. For example, timing a program you can get user CPU time and then wall clock time, which can be two very different numbers.
As old as time.
https://ss64.com/bash/time.html
It's a standard term and has been for ages. It distinguishes end to end time vs eg the amount of CPU time an individual process uses (which excludes time spent waiting for the system or time when the process was otherwise not scheduled on the CPU).
“Wall time” is a pretty common systems term (you see it when comparing total runtime to kernel time, for example). I wouldn’t have indexed on “wall clock time” or other variants as an LLMism.
I've used wall clock for many years, normally when compared to CPU time when talking about parallelizing some process.
CPU time might go up while wall clock time goes down
Lolwut? You seriously have never heard anyone say this before?
>> It’s a really good model.
In the meantime, I have cancelled my Anthropic subscription...
I have a simple test that I have been running iteratively across the SOTA models from several vendors, including one Chinese vendor.
I start with some code produced by an Anthropic SOTA model...let’s call that Code A. Then I get Code B and Code C for the same task from models by two other vendors.
Then I ask each model to review and critique the other proposals.
By the end, both the Anthropic model and I usually run out of arguments... against them and agree that proposals B and C are better.
Claude then always asks whether it can incorporate the code or ideas from B and C into its own solution...
Been very impressed with my most recent project. I wanted to simulate some older electronics circuits. I handed it a folder with scans of old service manuals which contained circuit diagrams. It managed to correctly interpret the circuits, including figuring out some were the same topology despite the diagrams being quite different, or some that had some subtle but very important differences despite looking almost identical at a glance.
In a few cases it asked me to check some subcircuits and some component values because it couldn't read it right. So instead of just making things up it deferred to me.
It also ran tons of small simulation experiments while doing this to verify claims from the service manual, like that the RC filter it had read off the schematics actually had a cutoff frequency that was sensible in relation to some bandwidth number in the manual.
I had uploaded datasheet PDFs for many of the ICs and it used those to cross-reference and validate.
It kept on working for over an hour. When it asked for the manual verification, I described circuit connections in words, like "from pin 3 on IC 2 there's a series resistor of 3k in parallel with a 10 pF capacitor, it then connects to a 18k resistor to ground, a reverse-biased diode to ground, and then finally into pin 6 of IC 4", and it correctly understood the topology in all the cases. Sometimes it asked me to check again because it though something was off, and indeed I had mis-read the schematics.
I also provided reference articles on the underlying theory. Scannded stuff from the 40s and 50s. It correctly read the equations and cross-validated them across papers, and even caught several typos along the way.
I barely had to do anything apart from providing the PDFs and some occasional manual schematic interpretation.
Claude 5.5 on High. Burned through about 50% of my weekly $20 subscription usage, but I didn't try to optimize much.
I did use Sonnet 5.5 Medium on some datasheets and it also did very well on the extraction, but did have to correct itself more often on the conclusions.
It's much better than 5, but I've had a couple of situations this week where it was too interested in being independent, making calls that went directly against my recommendations. It can also do fun things like convince auto-mode to go way past what I have autorized. For instance, specific permission to run process X in region abz-1 suddenly became running X in 5 other regions, with no warning, and doing modifications that it never mentioned in the summaries. And a few of the times it got the calls very wrong, by assuming it understood systems it didn't. It'd even argue with me when corrected, as it assumed similar names were referring to the same thing, when they weren't.
So asking it to do things on its own for a long time? Given last week, absolutely not.
Superb model indeed.
I've given it some big tasks and asked it to parallelize as much as possible etc.
It did burn through my weekly tokens in about a day (20x max), but the output was completely on point. (I knew there was a "reset token usage - opus 5.5" button in my account.)
I've now come to a point where I even delegate my discovery for new features to it.
You still need to give it methodologies though to get the proper output, but the outcome is way beyond what I would be able to realize with a team of 5 in a month.
Is it still necessary to ask Claude to spin up sub-agents? If Opus 5.5 decides how carefully it needs to think after each question, surely it can also decide whether it needs to spin up sub-agents? GPT 6 series models at least seem to do this agent management automatically.
I really hate long tasks. Claude never gets things right, at least for me, and wastes tons of time when a simple question would have gotten me to the right result rather than several turns of correcting bad decisions in addition to the long amounts of wasted thinking time.
What sort of workloads do well with these long tasks? The big labs are optimizing for long run time on their own, but it seems like a terrible thing to optimize on unless you're trying to do something like prove a hard math theorem, which success is clearly defined and the route doesn't matter a ton.
Plan mode has been made increasingly useless. I need to discuss to iterate to get the desired design, explore options, because Claude never gets it right first try and I don't have enough knowledge of options to specify everything up front.
Ah well, the Chinese models will still work well, I guess.
"Don’t ask it to show its reasoning in the reply" “Explain why you chose this approach in three sentences” says it all really
Agreed on Opus 5.5 being a great model. It's the first one that I trust for long running (>1 hour) tasks.
It’s amazing at debugging too. I had it running in Powershell controlling an lldb session in MSYS2. The way it can read addresses and so root cause analysis is amazing! It takes a lot of mind power to do those things.
I like to watch it work though because it honestly teaches me some tricks.
I'm convinced gpt 4.5 was the best model humanity has ever made.
Now we are getting downgraded models that do 100x COT because it's cheaper.
Opus 5.5 is great and cheaper if you compare to fable with close quality in coding (tested in refactoring java to nodejs), but I do not understand why the week before the release Opus 5 started hallucinating (long loop and waste of token for single tasks)
Phenomenal model, not sure what they did, but I have been able to do so much with my $20 plan!
I think what makes it great is that they trained it to write harnesses for the code it writes, so it can test stuff even if the supplied code is not complete.
A significant part of it is that with Opus 5.5, cache reads are priced at 5% of inputs, instead of 10% for previous models.
If enough people keep saying it I'm sure they will nerf it
The best model I've ever used easily. It's incredible. I've done so much in the past week. About three months of work I reckon.