I just had Claude try to process an RSS feed and it was about to ZALGΌ IS TOƝȳ THË PO NY itself and I pointed that out and it immediately said "Wordpress has a json interface, I'll use that".
If you draw a sloppy circle and fill it in, it'll "escape" and try to paint the whole canvas (and back in the day would get my slow computer stuck until I spam "esc").
You have to be able to draw a good circle to use it.
Human writting will come at a premium. I don't think LLM-generated prose goes away at all; the scary part is that I'm starting to catch people adopt the same tropes and language patterns in their speech/writing, manufactured contrarianism for example seems rampant in social media and blog posts ... but I guess nobody is talking about that :D.
Excellent article, with impeccable analysis, that will fundamentally change how I work with Claude myself. I have already learned to give Claude both a "do" and a "don't" in order to limit unpleasant surprises.
The "Form data" section is doing two completely different things, the large one is two different implementations of a React component mixed together while the short one doesn't store "data" anywhere for use later like the React one does.
Edit: Similar with the "UI components" section, the long one is missing the UI while the short one is UI without the trigger to activate it. You'd probably combine the two, using state from the first to control the UI in the second (replacing the contents of the useEffect with the dialog API to get the modal effect).
Reviewing my experience using LLMs, to improve results, reduce churn and token usage. Discovering the gap between what they produce and what I'd normally do is a significant source of output cost, regressions and surfacing a bit of why and how to fix it. Notably Claude is remarkably bad at/about this, producing errors even when directed toward modern Web solutions—that cut token use a lot, like toward 90% occasionally, which together with the frustrating churn led me to review how I'm working, what is happening and generate this article.
This matches my experience. I've been using OpenRouter with
GPT-4o for an image verification service, and the prompt
engineering choices have a measurable impact on cost.
One thing I found: asking the model to respond in structured
JSON (with a strict schema) vs free-form text cuts token output
by ~40% on average. The model stops "explaining itself" and just
gives you the answer.
Also noticed that including a reference image in vision calls
roughly doubles the input cost but improves accuracy enough that
you save on retries. Net cost ended up lower for my use case.
Curious if you've measured the difference between asking for
"concise" output vs actually constraining the response format.
That's an excellent idea I plan to try, thanks—re using structured JSON with schema. The most success I've had is saying "be brief" or an explicit size, like one line, or do not explain, etc. I haven't measured other instructions so extensively. They do work but the more specific the better. Other strategies around outputs that are more natural language seem to be hands-down the direction to take, and get away from the machine language habits we've used in the past. It's super interesting seeing this new practice emerging and more or less inventing parts of it along the way. Right now I'm at the place where my brute force and elaborate explanations were reaching their limit and in the frustration just realized I need to take a few days and try to figure out the tool. Across all these the pattern seems entirely that the constraints bound the probability space, whether it's the format like you suggested, or the instruction we give, including the space we point it toward (Web APIs, runtime, schema, etc). In all instances where it's not working the solution seems to be what does the pattern reduce to, and what specifics are the do/don't to go with that, and most of the time the results improve immediately. Your tip seems excellent for this. An easy-button.
I just had Claude try to process an RSS feed and it was about to ZALGΌ IS TOƝȳ THË PO NY itself and I pointed that out and it immediately said "Wordpress has a json interface, I'll use that".
You need to know the shape of the solution ...
Context about tony the pony
https://stackoverflow.com/questions/1732348/regex-match-open...
Thank you for helpfully de-darmoking that for the less pony-attuned of us.
If feels like the photoshop paint bucket tool.
If you draw a sloppy circle and fill it in, it'll "escape" and try to paint the whole canvas (and back in the day would get my slow computer stuck until I spam "esc").
You have to be able to draw a good circle to use it.
The code comments are an especially brutal thing to add cruft and bloat and confuse the coding agents.
And it feels like claude code has gotten more verbose with the multiline comments lately
It's really hard for me to parse LLM-generated prose in blog posts - the reasoning is disjointed, logic is split everywhere.
Is it already too late to have humans just write down what they are thinking instead of passing it through a lossy expander?
Human writting will come at a premium. I don't think LLM-generated prose goes away at all; the scary part is that I'm starting to catch people adopt the same tropes and language patterns in their speech/writing, manufactured contrarianism for example seems rampant in social media and blog posts ... but I guess nobody is talking about that :D.
Excellent article, with impeccable analysis, that will fundamentally change how I work with Claude myself. I have already learned to give Claude both a "do" and a "don't" in order to limit unpleasant surprises.
The "Form data" section is doing two completely different things, the large one is two different implementations of a React component mixed together while the short one doesn't store "data" anywhere for use later like the React one does.
Edit: Similar with the "UI components" section, the long one is missing the UI while the short one is UI without the trigger to activate it. You'd probably combine the two, using state from the first to control the UI in the second (replacing the contents of the useEffect with the dialog API to get the modal effect).
Reviewing my experience using LLMs, to improve results, reduce churn and token usage. Discovering the gap between what they produce and what I'd normally do is a significant source of output cost, regressions and surfacing a bit of why and how to fix it. Notably Claude is remarkably bad at/about this, producing errors even when directed toward modern Web solutions—that cut token use a lot, like toward 90% occasionally, which together with the frustrating churn led me to review how I'm working, what is happening and generate this article.
This matches my experience. I've been using OpenRouter with GPT-4o for an image verification service, and the prompt engineering choices have a measurable impact on cost.
One thing I found: asking the model to respond in structured JSON (with a strict schema) vs free-form text cuts token output by ~40% on average. The model stops "explaining itself" and just gives you the answer.
Also noticed that including a reference image in vision calls roughly doubles the input cost but improves accuracy enough that you save on retries. Net cost ended up lower for my use case.
Curious if you've measured the difference between asking for "concise" output vs actually constraining the response format.
That's an excellent idea I plan to try, thanks—re using structured JSON with schema. The most success I've had is saying "be brief" or an explicit size, like one line, or do not explain, etc. I haven't measured other instructions so extensively. They do work but the more specific the better. Other strategies around outputs that are more natural language seem to be hands-down the direction to take, and get away from the machine language habits we've used in the past. It's super interesting seeing this new practice emerging and more or less inventing parts of it along the way. Right now I'm at the place where my brute force and elaborate explanations were reaching their limit and in the frustration just realized I need to take a few days and try to figure out the tool. Across all these the pattern seems entirely that the constraints bound the probability space, whether it's the format like you suggested, or the instruction we give, including the space we point it toward (Web APIs, runtime, schema, etc). In all instances where it's not working the solution seems to be what does the pattern reduce to, and what specifics are the do/don't to go with that, and most of the time the results improve immediately. Your tip seems excellent for this. An easy-button.
why still use gpt-4o?