Wow! This kid is only 15 and built some pretty cool shit™ here! Can anyone explain how this works beyond trying to read/understand the code (which might take a while). I think I get the idea of 3-hidden neurons in the hidden layer, but not how this results in how it bypasses integer quantization bottlenecks.
By using integers instead of floats, you lose precision — weights/activations can only take on coarse, discrete values. With just 2 neurons, that severely limits the space of functions the network can represent. The 3rd neuron gives the network more degrees of freedom, so it can better approximate the target function despite the coarse resolution — not by 'stepping in' when another one fails, but by giving the network more combinatorial capacity overall.
Wow! This kid is only 15 and built some pretty cool shit™ here! Can anyone explain how this works beyond trying to read/understand the code (which might take a while). I think I get the idea of 3-hidden neurons in the hidden layer, but not how this results in how it bypasses integer quantization bottlenecks.
By using integers instead of floats, you lose precision — weights/activations can only take on coarse, discrete values. With just 2 neurons, that severely limits the space of functions the network can represent. The 3rd neuron gives the network more degrees of freedom, so it can better approximate the target function despite the coarse resolution — not by 'stepping in' when another one fails, but by giving the network more combinatorial capacity overall.