This looks great, but I'm wondering how effective this would be for full model weights rather than just the KV cache. Their paper only gives results for the KV cache use case, which strikes me as strange since the algos are claimed to be near optimal.
This looks great, but I'm wondering how effective this would be for full model weights rather than just the KV cache. Their paper only gives results for the KV cache use case, which strikes me as strange since the algos are claimed to be near optimal.