Scorpil@discuss.tchncs.de to ChatGPT@lemmy.worldEnglish · 1 year agoUnderstanding Generative AI: Part One - Tokenizerscorpil.comexternal-linkmessage-square2fedilinkarrow-up115arrow-down13
arrow-up112arrow-down1external-linkUnderstanding Generative AI: Part One - Tokenizerscorpil.comScorpil@discuss.tchncs.de to ChatGPT@lemmy.worldEnglish · 1 year agomessage-square2fedilink
What are tokens and how to count them?