Avatar

turkishdelight

turkishdelight@lemmy.ml
Joined
1 posts • 39 comments

Turkish from İstanbul

Open source enthusiast

Direct message

Indonesian women are based.

permalink
report
reply

llama.cpp quantizes the heck out of language models, which allows consumer cpus to run them. my laptop can run most 7b or 13b LLMs with 4bit quantization (and they are trying to push the level of quantization even further to 2 or 1.5 bits!)

The same will happen with stable diffusion. Most SD models are still around fp16 levels of quantization, and will soon be going lower. I expect we’ll all be running SDXL or larger models on our laptop CPUs without breaking a sweat at 4bit level.

permalink
report
reply

Crisis is not the right word. It’s Palestinian Genocide, perpetuated by Israel, and supported by the Western world.

permalink
report
reply

The most exciting part is that they plan to release multiple models, so I will probably be able to run it on my laptop

permalink
report
reply

I have been a loyal Lenovo customer for years. Their laptops are not cool or sexy, but they are reliable.

permalink
report
reply

In the mean time Waydroid works like a charm on Linux!

permalink
report
reply

You can’t shrink a model to 1/8 the size and expect it to run at the same quality. Quantization allows me to move from a cloud gpu to my laptops crappy cpu/igpu, so I’m ok with that tradeoff.

permalink
report
parent
reply

Having seen the NYC Subway as a tourist, I worry about the safety of those guardsmen.

You can probably get tethanus or dysentery just by looking at the NY Subway.

permalink
report
reply

Westerns imperialists dream of a world where all non-Westerners are murdered.

permalink
report
reply

I’ve done it at times. It works!

permalink
report
parent
reply