You must log in or # to comment.
Is running using a GPU a bad thing? I’m new to this…
No, a GPU would be ideal but not everyone has one, especially one with enough VRAM. I have an AMD card with 12gb of VRAM and I can run 7B - 13B models but even the 7B models (which seems to be the lowest that is still good) use a little more than 8gb of VRAM and most people probably have an Nvidia card with 8gb or less. 13B models get very close to using the full 12gb.
There’s a long listing of models to choose from. How to pick one? What are differences, benefits/drawbacks?
