leo@sh.itjust.works to LocalLLaMA@sh.itjust.worksEnglish · 2 months agoLocal LLM Inference Optimization: The Complete Guidecarteakey.devexternal-linkmessage-square2linkfedilinkarrow-up148arrow-down14file-text
arrow-up144arrow-down1external-linkLocal LLM Inference Optimization: The Complete Guidecarteakey.devleo@sh.itjust.works to LocalLLaMA@sh.itjust.worksEnglish · 2 months agomessage-square2linkfedilinkfile-text
minus-squaretheunknownmuncher@lemmy.worldlinkfedilinkEnglisharrow-up3·2 months ago Enabling XMP took my machine from roughly one-third speed back to normal. Huge red flag. XMP is not designed for use with memory-intense workloads like running LLMs.
minus-squareComradePenguin@lemmy.mllinkfedilinkEnglisharrow-up2·2 months agoI have a PC that has been crashing a lot recently, but only when running long RAM+CPU and RAM+GPU intensive tasks. It uses about 60%+ of ny RAM. I ran a memory test for 9 passes, no errors. No errors with the CPU only, and no errors with GPU only. So it can be the XMP that crashes it?
Huge red flag. XMP is not designed for use with memory-intense workloads like running LLMs.
I have a PC that has been crashing a lot recently, but only when running long RAM+CPU and RAM+GPU intensive tasks. It uses about 60%+ of ny RAM.
I ran a memory test for 9 passes, no errors. No errors with the CPU only, and no errors with GPU only.
So it can be the XMP that crashes it?