Bingo - this tree is non-existent outside my homeland, so people barely speak about it in English - and odds are that the model was trained with almost no pictures of it. However one of the names you see for it in English is Paraná pine, so it’s modelling it after images of European pines - because odds are those are plenty in its training set.
What I mean is that if people keep making it produce garbage tied to some keyword or phrase and people publish said garbage, that’ll only strengthen AIs’ neural network between the bad data and that keyword, so AI results for such trees will drift even further away from the truth.
Publishing fake data that outweighs the data on the real plant is a way, but that doesn’t require a plant, you can publish bad images today on any subject
Ah. I think I get you, but unfortunately it would probably be a lot easier to unlearn an obscure topic, not the other way around. Poisoning is done at a pixel level if that makes sense
Bingo - this tree is non-existent outside my homeland, so people barely speak about it in English - and odds are that the model was trained with almost no pictures of it. However one of the names you see for it in English is Paraná pine, so it’s modelling it after images of European pines - because odds are those are plenty in its training set.
So we could keep having it generate these and poison its own training data!
No. What are you on about?
What I mean is that if people keep making it produce garbage tied to some keyword or phrase and people publish said garbage, that’ll only strengthen AIs’ neural network between the bad data and that keyword, so AI results for such trees will drift even further away from the truth.
Publishing fake data that outweighs the data on the real plant is a way, but that doesn’t require a plant, you can publish bad images today on any subject
Right, but I think it’d be harder to get it to unlearn the wrong data if the topic itself is obscure.
Ah. I think I get you, but unfortunately it would probably be a lot easier to unlearn an obscure topic, not the other way around. Poisoning is done at a pixel level if that makes sense
Thanks, I have more to learn about this topic!