Those are pretty good results! Unfortunately, I don’t have a GPU at all yet. Any recommendations on something capable yet affordable? What would you buy next?
I stick to what I have as long as it continues to work for me, therefore I haven’t looked into GPUs the last years. New Radeon RX 580 is still at 200€, I paid 300€ 8 years ago. Looks like RX 580 2048SP 8GB is actually one of the cheapest one can get currently, but not entirely sure. (My GPU has 2304 shading units.) It’s so old that ROCm support was dropped and you really need software that can do AI via Vulkan
edit: looks like Intel Arc A750 is 5 years younger, has better performance and more memory bandwidth
Thanks! I’ll make sure to include Vulcan in criteria for a new machine and I’ll look at intel options. Only been looking at NVIDIA. Even 4GB VRAM will be a huge upgrade for me.
All modern GPU drivers support Vulkan, at least on Linux. I don’t know how bad Vulkan does in comparison to CUDA, since I’ve never used that stuff. The bigger issue is software-side support. Ollama, llama-cpp and KoboldCPP all support Vulkan by now. ComfyUI doesn’t seem to support it.
Thanks! I’ve been using llama.cpp but not married to it. What’s your opinion on used machines? Presumably risky to get a used GPU or they don’t really just fail?
Those are pretty good results! Unfortunately, I don’t have a GPU at all yet. Any recommendations on something capable yet affordable? What would you buy next?
I stick to what I have as long as it continues to work for me, therefore I haven’t looked into GPUs the last years. New Radeon RX 580 is still at 200€, I paid 300€ 8 years ago. Looks like RX 580 2048SP 8GB is actually one of the cheapest one can get currently, but not entirely sure. (My GPU has 2304 shading units.) It’s so old that ROCm support was dropped and you really need software that can do AI via Vulkan
edit: looks like Intel Arc A750 is 5 years younger, has better performance and more memory bandwidth
Thanks! I’ll make sure to include Vulcan in criteria for a new machine and I’ll look at intel options. Only been looking at NVIDIA. Even 4GB VRAM will be a huge upgrade for me.
All modern GPU drivers support Vulkan, at least on Linux. I don’t know how bad Vulkan does in comparison to CUDA, since I’ve never used that stuff. The bigger issue is software-side support. Ollama, llama-cpp and KoboldCPP all support Vulkan by now. ComfyUI doesn’t seem to support it.
Thanks! I’ve been using llama.cpp but not married to it. What’s your opinion on used machines? Presumably risky to get a used GPU or they don’t really just fail?