RandAlThor@lemmy.ca to Technology@lemmy.worldEnglish · 15 hours agoElon Musk’s Grok Goes Haywire, Boasts About Billionaire’s Pee-Drinking Skills and ‘Blowjob Prowess’www.mediaite.comexternal-linkmessage-square100fedilinkarrow-up1716arrow-down114
arrow-up1702arrow-down1external-linkElon Musk’s Grok Goes Haywire, Boasts About Billionaire’s Pee-Drinking Skills and ‘Blowjob Prowess’www.mediaite.comRandAlThor@lemmy.ca to Technology@lemmy.worldEnglish · 15 hours agomessage-square100fedilink
minus-squarebrucethemoose@lemmy.worldlinkfedilinkEnglisharrow-up1·edit-22 hours agoYou should be running hybrid inference of GLM Air with a setup like that. Qwen 8B is kinda obsolete. I dunno what kind of speeds you absolutely need, but I bet you could get at least 12 tokens/s.
You should be running hybrid inference of GLM Air with a setup like that. Qwen 8B is kinda obsolete.
I dunno what kind of speeds you absolutely need, but I bet you could get at least 12 tokens/s.