nsorros .com
online
← back to writing

Is the finetuning era over?

Is the finetuning era over?

If so, it's a big shift, though one so gradual it won't shock anyone.

Here is the transition: • Pre-deep-learning: All of AI meant training from scratch. • Deep learning arrived. The pretrain → finetune era began. Vision came first, NLP soon after. • LLMs landed. Suddenly models worked out of the box. Prompting became the first adjustment and finetuning the last, one most teams never did.

Now we're past that point too. The right instructions, well-scoped skills, and tools delivers most, if not all of the value fine tuning did. That does not mean fine tuning is not needed, smaller models still have benefits. Some verticals still need it. But as frontier models get cheaper and faster, that need is evaporating quickly.

This begs the question: are we heading toward general models for everything, steered rather than tuned? Where does finetuning still earn its place?

From scratch → pretrain + finetune → prompt first → ?