I think my favorite of the bunch is the "Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model" paper. Easy to read, gets the point across very intuitively and quickly, and the point is very interesting and relevant to a lot of people.
About the Superposition paper - this is close to what I've been thinking about over the past week. I'm thinking that concepts or choices in a "superposition" are harder for a fully-differentiable neural net to reason about. For example, if there's a "green" vs "purple" choice to be made, it can't fully commit to either (especially if they're 50-50), and will have to reason about both simultaneously (difficult due to nonlinear manifold space). Discretizing to tokens (non-differentiable argmax) forces a choice, and that allows it to reason about a single concept separately and easier.
Are there any talks about these papers on youtube or somewhere? I think I find it easier to listen and watch then read or maybe I'm just lazy, not sure.
Interesting that 3 names I recognized as physicists from stat mech adjacent fields. They continue to punch above their expectations (as sampled by general dismissal of physicists in AI/ML on HN and reddit).
Some of the best software engineers I know are ex-physics PhDs… it’s one of those “can’t fake it” skillsets that also happens to have high transferability to ML/AI fields. On the other hand, I snuck through the CS major without ever multiplying a matrix.
I think my favorite of the bunch is the "Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model" paper. Easy to read, gets the point across very intuitively and quickly, and the point is very interesting and relevant to a lot of people.
About the Superposition paper - this is close to what I've been thinking about over the past week. I'm thinking that concepts or choices in a "superposition" are harder for a fully-differentiable neural net to reason about. For example, if there's a "green" vs "purple" choice to be made, it can't fully commit to either (especially if they're 50-50), and will have to reason about both simultaneously (difficult due to nonlinear manifold space). Discretizing to tokens (non-differentiable argmax) forces a choice, and that allows it to reason about a single concept separately and easier.
Does some have a similar award for papers that are innovative? Like new, relatively unproven architectures?
Are there any talks about these papers on youtube or somewhere? I think I find it easier to listen and watch then read or maybe I'm just lazy, not sure.
There conference had interesting lectures. Will they be posted online?
Interesting that 3 names I recognized as physicists from stat mech adjacent fields. They continue to punch above their expectations (as sampled by general dismissal of physicists in AI/ML on HN and reddit).
Some of the best software engineers I know are ex-physics PhDs… it’s one of those “can’t fake it” skillsets that also happens to have high transferability to ML/AI fields. On the other hand, I snuck through the CS major without ever multiplying a matrix.
Haha, nice bio. Seeing that font on HN is quite a shock.