It's very interesting that cryptographic protocols and neural networks have the same high-level architecture (where they jumble information as it moves sequentially across many layers). This is the result of a convergent evolution - cryptographic protocols need every output bit to depend on every input bit in complicated ways,... and similarly, NNs need output to make connections between inputs. But they're in some sense doing opposite things. While cryptographic protocols take something which has a lot of structure and make it seem indistinguishable from random, NNs take something which may look random and extract structure from it. Much more on this idea in the full episode with Reiner Popeshow more

It's XOR all the way down.

my undergrad crypto professor was onto some line of suspicion linking quantum crypto and machine learning, should catch up with him… last thing we talked about was random walks in lollipop networks (which you get when evolve certain systems over time…) baller GOAT of a guy, too!

This was a great session overall but the premise that a randomly initialized NN is a good cipher is false. Cryptographic security comes from carefully engineered nonlinearity, provable diffusion, and many rounds chosen specifically to drive differential and linear biases below 2⁻¹²⁸. Random init gives you the opposite: a smooth, locally-linear, gradient-friendly function with weak statistical properties: exactly what an adversary wants. Many recent results on learning ReLU networks show polynomial-time algorithms for recovering small random ReLU networks under various assumptions. Anything polynomially learnable is, by definition, not cryptographically secure.

feels like we are circling around some natural laws of the universe when things like this converge

That's the point - if consciousness arises via convergent evolutionary means, the total differential equation in either case is of dense architecture (topology) and energetics (thermodynamics). Recent research (Wissner-Gross et al.) asserts the discharge of current alone in uploaded Fruit Fly connectomes produces the effect of species level consciousness. For example, species level grooming behaviour, curiosity, flight, full motor movement etc. From the humble myocyte to interconnected motor neuron. Why then wouldn't consciousness be purely electromagnetic, and explanatory from a fully classical physics? There is no extra layer between energy flow (this case discharged electrochemical work) and topology, only that emergence is the density layer upon layer giving the 'illusion of consciousness'. With benefit of billions of years iteration and Natural Selection. The simulation has no attempt regulating cascade resting/action potentials specific to mV, nor programmed gradient recovery (mimicking ATP hydrolysis coupled sodium-potassium pump via autocatalysis). There is no added complexity in terms Δpμ + Δψ across gap junctions nor the product of Donnan equilibrium, no bespoke capacitance, no novel node connections between simulated synapses (the potential for learning, memory). Each node in the simulation has a voltage Δψ, no microtubules or associated oscillation (nor any quantum superposition effects). What there is, crudely for now, is permanent steady current discharge. Pattern recognition, intelligence essentially, is downstream thermodynamics, electrical signalling and topology. Whether said emergence may arise from a wet steady flow device by means chemomechanical transductions (organisms), or abiotic circuit geometry AI systems - the input/output function is the same. So either extend consciousness to every dissipative structure with inbuilt pattern recognition, or make clear even humans lack this in principle.

@LilysAI_ summarize

makes me want to work on that jane street puzzle again

I was also struck by this when reading about normalizing flows and realizing how much their construction reminded me of symmetric cipher architectures, especially the Feistel network. (Both create very complex but invertible transformations out of small non-invertible parts).

GANs embody both protocols. Generator encrypts to noise, discriminator extracts structure. Information theory makes this convergence inevitable.

Dwarkesh, I really do love your takes man you've got a convoluted way of thinking that's somehow unraveling? It's always a pleasure to pluck from the tree of your deep insights. (pause)

SP-networks and Transformers share the same iterative nonlinear mapping DNA. Diffusion isn't just for crypto; it's the core of feature disentanglement. Convergence is peaking as FHE becomes the preferred runtime for ZK-inference.

Which one is the human brain closer to would be a fantastic folllow-up question for a neuroscientist (a computational one if possible).

Honestly this is freshman undergrad level “woah dude…have you ever noticed how…” level thinking. That people take you seriously is a real indictment of the AI scene.

Well, there are probably quite a few similar situations e.g decoding a Turbocode or an LDPC(Gallager)-code. What they have in common is algorithms based on message passing and Bayesian belief propagation. It is an efficient structure that evolution discovered too.

Dwarkesh should do a podcast on how he’s investing in this Ai age. Looks like he is doing a lot of angel investments.

This is exactly why I love following cybersecurity trends. The same layered architecture securing e-commerce payments underpins the neural networks analyzing them. Chaos creating structure. Structure creating chaos. Elegant symmetry!

would love to hear what @keoneHD thinks of this

Makes sense, you’re just applying something in the direction of chaos

Isn't entropy the root of all: one increases it (cryptography), the other removes it (prediction).