Earlier this month, Twitter was aflame with rumors that OpenAI’s new Strawberry project was about to go live. This could be it! Artificial general intelligence! Maybe! Possibly! (The gossip is that…
I remember one time in a research project I switched out the tokeniser to see what impact it might have on my output. Spent about a day re-running and the difference was minimal. I imagine it’s wholly the same thing.
*Disclaimer: I don’t actually imagine it is wholly the same thing.
I remember one time in a research project I switched out the tokeniser to see what impact it might have on my output. Spent about a day re-running and the difference was minimal. I imagine it’s wholly the same thing.
*Disclaimer: I don’t actually imagine it is wholly the same thing.
there’s a research result that the precise tokeniser makes bugger all difference, it’s almost entirely the data you put in
because LLMs are lossy compression for text
latent space go brrrr