Is there any improvement to GPU inference at all? CPU inference is not cost effe... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

		fareesh on Dec 7, 2022 \| parent \| context \| favorite \| on: Show HN: Port of OpenAI's Whisper model in C/C++ Is there any improvement to GPU inference at all? CPU inference is not cost effective for my use-case

ggerganov on Dec 7, 2022 [–]

If you have a good GPU then you don't need `whisper.cpp`. Best use case currently is if you are running on Apple Silicon.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact