![](/static/253f0d9b/assets/icons/icon-96x96.png)
![](https://lemmy.dbzer0.com/pictrs/image/a18b0c69-23c9-4b2a-b8e0-3aca0172390d.png)
Humans learn a lot through repetition, no reason to believe that LLMs wouldn’t benefit from reinforcement of higher quality information. Especially because seeing the same information in different contexts helps mapping the links between the different contexts and helps dispel incorrect assumptions. But like I said, the only viable method they have for this kind of emphasis at scale is incidental replication of more popular works in its samples. And when something is duplicated too much it overfits instead.
They need to fundamentally change big parts of how learning happens and how the algorithm learns to fix this conflict. In particular it will need a lot more “introspective” training stages to refine what it has learned, and pretty much nobody does anything even slightly similar on large models because they don’t know how, and it would be insanely expensive anyway.
And while they tried to DRM it, the DVD standard still ended up having to maintain compatibility across all readers and discs, but for bluray they regularly deprecate older readers who no longer can play newer movies because new releases use new encryption keys which the old readers don’t have access to (and for this reason the PS consoles are the best bluray movie players because Sony keeps them updated)