...and quantized ones from the usual suspect: https://huggingface.co/TheBloke/Or...

		T-A on Nov 21, 2023 \| parent \| context \| favorite \| on: Orca 2: Teaching Small Language Models How to Reas... ...and quantized ones from the usual suspect: https://huggingface.co/TheBloke/Orca-2-7B-GGUF https://huggingface.co/TheBloke/Orca-2-13B-GGUF The 7B Q5_K_M one is small enough to run on an 8GB consumer GPU.

All the 13B files seems to be quantized.