Thx to Scott Aaronson, GPT outputs will soon be watermarked w/ a random seed, making it much harder to submit your GPT-written homework without getting caught He doesn’t give too many details about how it works, but I suspect its possible to bypass using a clever decoding strat

Src:

@mezaoptimizer Anything that can be subverted by the use of AI, should not qualify as education.

@mezaoptimizer This feels like making people still work in a post-scarcity world.

In near term, I think it could very beneficial in some applications, e.g. detecting a propaganda campaign

@mezaoptimizer Some other company will make a product whose sole purpose will be to strategically modify the generated text to corrupt or otherwise defeat the watermark, either by changing words to synonyms when it won’t change meaning, or recasting sentence structure, or reordering things, etc

Yeah I could probably build this..

@mezaoptimizer More explained here. It seems, as he describes, that it will be possible to circumvent by using a different LLM to paraphrase the output of GPT. I expect this will be a possibly useful signal but not a reliable one.

Thanks for the thread!

@mezaoptimizer Naive question: Couldnt this be circumvented by automated post-processing? Like swapping out synonyms, post-processing with another LLM, adding spelling errors?

Yes, depends on how the watermark is designed, but all those listed should work to dilute the signal (although perhaps not entirely)