DALLE-2が独自言語を獲得！？

Giannis Daras @giannis_daras

Our gibberish tokens have varying degrees of robustness in combinations with contexts. E.g. if xx produces birds, ‘xx flying’ is an easy prompt ‘xx on a table’ is a neutral prompt, and ‘xx in space’ is a hard prompt. (8/N)

Giannis Daras @giannis_daras

Our hidden vocabulary seems robust in easy and sometimes neutral prompts but not in hard ones. These tokens may produce low confidence in the generator and small perturbations move it in random directions. "vicootes" means vegetables in some contexts and not in others. (9/N) pic.twitter.com/r2s5I6Bnp0

拡大

Giannis Daras @giannis_daras

We want to emphasize that this is an adversarial attack and hence does not need to work all the time. If a system behaves in an unpredictable way, even if that happens 1/10 times, that is still a massive security and interpretability issue, worth understanding. (10/N, N=10).

Giannis Daras @giannis_daras

@benjamin_hilton, @realmeatyhuman, @BarneyFlames, @mattgroh, @rctatman, @Plinz, @Thomas_Woodside hopefully some of your concerns are addressed! Let us know what you think. We will update the pre-print with this discussion: arxiv.org/abs/2206.00169

Alex Dimakis @AlexGDimakis

My student Giannis discovered that DALLE2 has a secret language. This can be used to crate absurd prompts that generate images. E.g. ''Apoploe vesrreaitais eating Contarra ccetnxniams luryca tanniounons'' generates Birds eating Bugs! We wrote a short paper on our experiments. twitter.com/giannis_daras/…

Giannis Daras @giannis_daras

DALLE-2 has a secret language. "Apoploe vesrreaitais" means birds. "Contarra ccetnxniams luryca tanniounons" means bugs or pests. The prompt: "Apoploe vesrreaitais eating Contarra ccetnxniams luryca tanniounons" gives images of birds eating bugs. A thread (1/n)🧵 pic.twitter.com/VzWfsCFnZo

2022-06-01 02:44:25

Greg Brockman @gdb

Turns out DALL-E can read the seemingly gibberish writing it produces. Built its own mini-language that is consistent between its text input space and image output space: twitter.com/giannis_daras/…

Greg Brockman @gdb

A pretty good hypothesis on how this arises (as an artifact of how the text input space is tokenized): twitter.com/BarneyFlames/s…

Aperiodic Tile @BarneyFlames

I took a look at the BPE encoding of the name DALL-E uses for birds. Its "apo, plo, e</w>, ,ve, sr, re, ait, ais</w>". Apo-didae & Plo-ceidae are families of birds, each with 100+ species. Apo-diformes is the biggest order of birds with 400+ species of birds. twitter.com/giannis_daras/…

2022-06-01 05:37:44