Helps me fall asleep by making me stupid, but otherwise not really great for regular use

Unsloth did a test and their dynamic quants were competitive even at 1 bit in aider benchmark https://docs.unsloth.ai/new/unsloth-dynamic-ggufs-on-aider-polyglot

Sometimes speaking to an older model feels way more human and natural, newer ones seems to be trained too much on “helpful assistant” stuff and especially on the previous AI dialogues, to the point where some of them from time to time claim to be chatgpt because that’s what they have in their training data.

Datasets should be cleared and everything newer than the release of chatgpt should be carefully vetted to make sure the models are not just regurgitating generated output to the point where they all blend into the same style of speech.

Also, it seems like models should be rewarded more for saying “I’m not sure” or “I don’t know” for things that are not in their training data and context, because every one of them still has a huge tendency to be confidently wrong.

I personally get a headache trying to read this, so it would be hard to practice… Words seem to flow faster though, need someone record their time reading with and without this

Yes 😭 And also delayed circadian rhythms, so the body wants to go to sleep later and stay in bed longer

I’m not sure if I’m doing something wrong here, but openwebui has been weird for me. I’ve tried running nanonets-ocr, but it only read the last lines visible on a photo. And other models would start reprocessing the whole chat and ignoring the last image I post, answering with the context of the previous reply instead… Using the websearch is easy with it though, so I think I’ll keep an eye on it and maybe will try again later

Managed to run it with llama.cpp. It was a great suggestion, thank you! MiniCPM-o-2_6 iq4 managed to read text from a picture of a shirt that gemma could not get right

Ok, turned out to be as simple to run as downloading llama.cpp binaries, gguf of gemma3 and an mmproj file and running it all like this

./llama-server -m ~/LLM-models/gemma-3-4b-it-qat-IQ4_NL.gguf --mmproj ~/LLM-models/gemma-3-4b-it-qat-mmproj-F16.gguf --port 5002

(Could be even easier if I’d let it download weights itself, and just used -hf option instead of -m and —mmproj).

And now I can use it from my browser at localhost:5002, llama.cpp already provides an interface there that supports images!

Tested high resolution images and it seems to either downscale or cut them into chunks or both, but the main thing is that 20 megapixels photos work fine, even on my laptop with no gpu, they just take a couple of minutes to get processed. And while 4b model is not very smart (especially quantized), it could still read and translate text for me.

Need to test more with other models but just wanted to leave this here already in case someone stumbles upon this question and wants to do it themselves. It turned out to be much more accessible than expected.

Sounds like what I’m looking for! What do you use for inference?

Looks like what I’m looking for, and llama.cpp has added support this year, so should be easy to try, thank you!

Thank you, haven’t heard of it before and it looks really interesting! I need to test how it works with llama.cpp, I wonder how it works with resolutions higher than supported, will it get downscaled

also legal drugs, other psychiatric drugs, if they want to, to get off of SSRIs, to get off of benzos, to get off of Adderall, and to spend time as much time as they need — three or four years if they need it — to learn to get reparented, to reconnect with communities, to understand how to talk to people. There’ll be job training, particularly in the trades

Seems more like slavery.

And also no devices there, it will be amish paradise

Why can’t Biden do his own supreme court appointments?

Earth Defence Force: nothing better than blowing up 5 million bugs a minute after a tough working day