I’d like to generate Minecraft fan fic. GPT4 does a terrible job because it’s been lobotomized. Fine tuning a model with a corpus of fan fic generates better fan fic. If I could fine tune GPT4 then ok I agree. But “general purpose lobotomized oracle” is only one use case for a language model, albeit a useful one, is the least creative one.
They tend to not have the right base of training materials - the models are constrained not just by the lobotomization but by some specialized writing not being part of the corpus. Training a lora on a specific corpus - fan fic, alt.sex.stories, or a persons specific writing, can help induce behavior or language choices specific to that corpus. You can also use context injection but I’ve found combining a model weight adjustment with priming the context to be the most effective.
I think over time it’ll become standard practice to train special purpose models on top of general purpose models. If I have a very specific task domain a model tuned to that domain will less often wander off and will be more likely to respond in the desired way.
