51 points | by ankitvgupta 2 hours ago
13 comments
I did something like this with Opus 4.5: https://github.com/cpldcpu/smollm.c
It seems implementing transformers is the new "hello world" of AI assisted coding.
Does not say what type of model or size they were serving, but one of the traces mentions Maple, which is a 20b model. Still, getting ~70+ tok/s is quite good.
Fpga is software-driven, model can verify its output easily.
Zero surprise it did it
it only cost them $10,000 in Claude API usage. :)
You get that much usage in a monthly $200 subscription.
So who's paying the cost?
API pricing vs. individual plan pricing is just price discrimination. They're both positive gross margin.
Corporate customers
And a kitten.
This is cool
Next step is tuning your post training pipeline to optimize your model for your chosen FPGA architecture...
Good chance I'm not the only one thinking this: share the code
Now do silicon. ;)
i wonder if we can emulate an FPGA on a rtx 3090? then test it on an AWS EC2 F2 few hours a day.
I did something like this with Opus 4.5: https://github.com/cpldcpu/smollm.c
It seems implementing transformers is the new "hello world" of AI assisted coding.
Does not say what type of model or size they were serving, but one of the traces mentions Maple, which is a 20b model. Still, getting ~70+ tok/s is quite good.
Fpga is software-driven, model can verify its output easily.
Zero surprise it did it
it only cost them $10,000 in Claude API usage. :)
You get that much usage in a monthly $200 subscription.
So who's paying the cost?
API pricing vs. individual plan pricing is just price discrimination. They're both positive gross margin.
Corporate customers
And a kitten.
This is cool
Next step is tuning your post training pipeline to optimize your model for your chosen FPGA architecture...
Good chance I'm not the only one thinking this: share the code
Now do silicon. ;)
i wonder if we can emulate an FPGA on a rtx 3090? then test it on an AWS EC2 F2 few hours a day.