I have an rx 6600 and 16gb of ram and an i5 10400f

I am using oobabooga web-ui and I happened to have a gguf file of LLama2-13B-Tiefighter.Q4_K_S .

But it always says that the connection errored out when I load the model.

Anyway, please suggest any good model that I can get started with.

  • Magical Thinker@lemmy.dbzer0.comEnglish
    41·
    5 months ago

    I’d suggest checking out WolframRavenwolf on raddit, he does regular LLM tests.

    I’m looking at Beyonder 4x7B, Mistral Instruct 2x7B, Laser Dolphin 2x7B, and previously used Una Cybertron.