Reiner Pope

And the one you call out is a pretty strong difference.

The way it shows up, what makes neural nets... If you just randomly initialize a neural network, actually, maybe it's a reasonable cryptographic cipher as well, because the random initialization is going to jumble stuff in a complicated way.

It may even do what you want.

Who knows?

The thing that makes it interpretable is the gradient descent.

So you can differentiate a neural network and get a meaningful derivative.

And we do a lot of work to...

like not over complicate the derivative so the residual connection keeps it like contained and simple um and the uh and so it's like the layer norm uh stuff that we do um

One of the biggest attacks against cryptographic ciphers is also to differentiate the cipher.

Ciphers run in a different number field.

They run in the field of two elements, so just binary, whereas neural nets run in theory in the field of real numbers.

And so you have to differentiate with respect to binary numbers.

But you can absolutely differentiate a cipher, and this is called differential cryptanalysis.

And basically what it says is that if you take a small difference of the input, it's quite difficult to make the difference of the output be small.

The whole job of a well-designed cipher is to make the difference of the output very large.

So I guess the distinction is that the optimization goals at that point are about complexifying.

They don't have the same residual connections or layer norms.

Yeah.

So, yeah, I mean, in fact, this is actually a place where you get exactly the sort of avalanche property that ciphers have as well.

like adversarial attacks on typically like image classification models, right, are can I find a perturbation of the image that, a very, very small perturbation of the image that totally changes the classification, totally changes the output.