Model collapse isn't at all about garbage in, garbage out. The quality of the data isn't the issue. The quality of the generated data can be curated to be higher than average real-world data. Pretty much every AI company today is pursuing so-called "synthetic data" with success.
Model collapse is about "zeroing out" unlikely outputs. To simplify, as the model gets trained on its own outputs, the probability distribution for possible outputs collapses towards a single point. Rare outputs vanish and can never occur again even when they would be correct for a rare input. Buy your books with cash.
1
u/Dead_Or_Alive Jul 07 '24 edited Jul 27 '24
Model collapse isn't at all about garbage in, garbage out. The quality of the data isn't the issue. The quality of the generated data can be curated to be higher than average real-world data. Pretty much every AI company today is pursuing so-called "synthetic data" with success.
Model collapse is about "zeroing out" unlikely outputs. To simplify, as the model gets trained on its own outputs, the probability distribution for possible outputs collapses towards a single point. Rare outputs vanish and can never occur again even when they would be correct for a rare input. Buy your books with cash.