Could this be about other labs getting better at instruction tuning their models?
@simonw
@Yuchenj_UW That's an interesting possibility: maybe GPT-4's strength was in its instruction and fine-tuning, not so much its size - and the other labs have caught up in terms of that training by data now
A year ago nobody outside OpenAI had trained a model as good as GPT-4
Today there are dozens - and if you trust the benchmarks that includes some that you can run on a laptop (Qwen2.5-32B perhaps?)
What changed? What techniques are used now that weren't known a year ago?Here's what I wrote about this in December last yearCould this be about other labs getting better at instruction tuning their models?
yes
A year ago nobody outside OpenAI had trained a model as good as GPT-4
Today there are dozens - and if you trust the benchmarks that includes some that you can run on a laptop (Qwen2.5-32B perhaps?)
What changed? What techniques are used now that weren't known a year ago? ... Here's what I wrote about this in December last year ... Could this be about other labs getting better at instruction tuning their models?
Missing some Tweet in this thread? You can try to
Update
