I vibecoded this neural network visualization for my students and open sourced it. It shows a simple MLP trained on MNIST handwritten digits at several training steps. The visualization is using Three.js and it comes with training code in PyTorch . Link + repo 👇

Before I say a couple more words here is a link to play with it: Everything runs in a browser and the weights are stored in a json. Might take a bit to load on slower connection. Also its intended for desktop/larger screens and menu overlaps on mobile.

Here is the github repo with all the code for training and visualization all under Apache 2.0 licence.

So first of all sorry for all the text being in German. I wanted to translate it to English before sharing it, but didn't have the time. Also in the current form the educational information is limited since it was used in class with me explaining it.

When I have a bit more time I go over it and will translate it and make it more standalone educational. Also I am currently talking with an exhibition to use it and I might add an option to connect a tablet to it via WebRTC to paint the numbers there.

This was 100% vibecoded using Codex and similar to other things this worked flawless, because of how amazing @threejs is. The @pytorch code for an MLP is obviously trival and also was no challenge. See this thread for something similar that I vibecoded.

The visualisation is inspired by @3blue1brown cover image for this video. If you don't know his content and want to learn more about neural networks I recommend watching his videos.

If you enjoy posts consider retweeting it. And if you like tweets about vibecoding, neural networks, AI, 3D, philsophy and shitposting consider following me.

Since a couple of people are posting fail cases. Yes this is known and basically intended. I showed the fail cases in class. It’s a simple neural network architecture and small. We known better methods since the 90s for this (conv nets) and training should do data augmentation.

Here is a video of @ylecun showing convolution neural networks in 1989 that work better than this MLP here (but are a bit more complicated to explain and visualize). Fun fact: the MNIST dataset used is also from Yann LeCun.

Honestly.. this is a brilliant way to bridge the gap between theory and application for students. Visualizing something like an MLP in real-time makes the learning process so much more intuitive. Am thinking if tools like this could also help researchers experiment with architectures more interactively.. or is it primarily built with teaching in mind?

@threejs @PyTorch I think its more useful for teaching core principles. The architecture is super small. Its just 110.000 parameters. LLMs have billions of parameter and more complicated connections.

@threejs @PyTorch Seems almost like a copy of

@threejs @PyTorch Yeah I saw that one when I researched whats already out there to show my students, but I didn't really like it flattened out neurons and that it doesn't show (some) weights while updating. It's actually much more inspired by (or copying) this from @3blue1brown

@threejs @PyTorch @3blue1brown I appreciate your efforts to demonstrate your students in the best possible way.

@threejs @PyTorch

@threejs @PyTorch Awesome. Showing the training is cool!

@threejs @PyTorch Also I made some experiment simulating combinational logic system and a secuential RISC-V architecture (flip-flops, etc...) and fusing it with the NN system too 😂

@threejs @PyTorch

@threejs @PyTorch Awesome! The most complex thing I designed with flip flops was a number counter with a single digit seven-segment LCD display. NNs on RISC-V is great!

@threejs @PyTorch Did the same thing not too long ago, super interesting to dig through

@threejs @PyTorch This looks awesome. I like the glow effect.

@threejs @PyTorch

@threejs @PyTorch ??

@threejs @PyTorch Yeah this is a common issues of MLPs. If something is larger or smaller, shifted left/right/up/down or rotated its already "out of training distribution". This is where "Conv nets" and/or data augmentation (flipping, rotatating and scaling training images) helps.

@threejs @PyTorch beeindruckend 👍

@threejs @PyTorch Please make it reliable via conformal prediction because what it produces now is absolute bolloks

- no dropout - just relu - no data augmentation - no regularization - no residual connections - no convolutions - just 110k parameters This is build for an AI introduction class of students that don’t even do CS or AI (but VR/AR). It’s deliberately kept simple to specifically show the core principles as well as general limitations. In the lecture I described all those fail cases (lack of invariances etc) and gave a basic introduction how that got solved over the years. This was already fairly advanced for a non science class and we mostly showed AI tooling. I know this is a way too simple network, but probably should have made that more clear in the post. This blew up way more than I expected.

@threejs @PyTorch Understood. It should come with disclosure, top viz though

@threejs @PyTorch Yeah I agree. Should have made that much more clear. That’s on me. In class I explained that. I didn’t have time to make this more standalone educational app, but still wanted to share it.

@threejs @PyTorch When it blows up to 500K views there is a chance some people would start testing 😉

@threejs @PyTorch Yeah. This blew up way more than I expected. Which it would explain more in the app itself. Maybe I make it a more standalone experience when I have some time.

@threejs @PyTorch Vibe coding is the future.

@threejs @PyTorch Very nice! Could you please share a bit about the prompting process? I see that you used three.js for it. Was specifying this part of the prompting? Also, did you ask the LLM to use 3blue1brown video (or screenshot) as well?

I didn’t use any screenshots but described everything. It was very specific: make an array for the input layer with cubes. Make an array for neurons with spheres. Make the final output layer vertically arranged. Add numbers. Make the material unshaded. Make the connections red for negative values and green for blue. Etc… I didn’t do much for PyTorch code and export though. Just told it to make small MLP and have weight export and codex used json for it and did the training export. That was basically one shotted. I use MacWhisper for transcription so it’s fairly fast to describe what i want. It’s usually more robust than providing screenshots. I found screenshots tricky for 3D stuff so far.

@threejs @PyTorch Great job ! I like how it shows it's predictions in real-time while drawing, makes it a great tool for gamifying the core idea behind such networks.

@threejs @PyTorch Noooo this is bad! Code generation is bad! You should have done nothing!

@threejs @PyTorch 😂 I could have done it without vibe coding, but I wouldn’t have had the time to do so. The guest lecture payment isn’t great and I am only paid for the time of the course and not preparation time, so doing this is really only a hobby.

@threejs @PyTorch defo one of the most sick things i have come across recently

@threejs @PyTorch so dope!!

@threejs @PyTorch 8 is a 7 ?! excuse me, what is your training data?

@threejs @PyTorch Its good old MNIST and this result is no surprise. Please note that I am not doing data augmentation (scaling, rotation) and I am not using cnns (translation). This is for educational purposes including the failure cases of this simple architecture.

@threejs @PyTorch I should have made this more clear though earlier in the thread and the web app. I explained all this in the lecture and didn't expect this thread to blow up like this.

@threejs @PyTorch very interesting software! just a little bit of a question, isnt the formula with the sigmoid function being applied across bias as well?

In general yes - activation function is applied to everything including the bias. But here it’s just the sum of all weights and input + bias and then ReLu is only applied after that and isn’t shown in the formula. It’s just a sum over all elements symbol ∑. With activation function it would be max(∑(w_i * in_i) + b_i),0) in that case.

@threejs @PyTorch @alightinastorm looks like your vibe and perhaps interesting to @BrianRoemmele too.

@threejs @PyTorch 👏👏👏

@threejs @PyTorch This is wonderful!

@threejs @PyTorch No wonder my friends wanted to move to TU Darmstadt so badly. Great work sir

@threejs @PyTorch I was teaching the course at Hochschule Darmstadt (h_da) not the TU Darmstadt. It’s for the Extended Reality course (AR/VR design) there. But TU Darmstadt is also great.

@threejs @PyTorch Really useful for learning I have never understood how CNNs worked until I coded one up from scratch This might help comprehension without the need to devote an entire semester to the process

@threejs @PyTorch Its missing some good old overfitting

@threejs @PyTorch It’s a fairly small MLP (110.000 params) and a fairly large dataset (60.000 images -> didn’t do a validation split).

@threejs @PyTorch damnn bro 🔥🔥🔥🔥🔥

@threejs @PyTorch Toll gemacht. Den Follow hast Du Dir verdient!

@threejs @PyTorch Great work !

@threejs @PyTorch

@threejs @PyTorch

@threejs @PyTorch Yes it’s a simple architecture. I added more information about the fail cases to the thread.

@threejs @PyTorch Fickin’ cool

I just want to thank you and say this is an amazing job! Exactly what I was looking for! Bibecode shibecode, this is actually a useful learning tool!

@threejs @PyTorch wow Great Job!.

@threejs @PyTorch Such a cool way to teach the intuition behind neural networks. Visualizing how an MLP evolves during training makes the concepts click instantly. Students are going to remember this far more than a slide deck.

@threejs @PyTorch Can you walk us through how did you vibe code it. What tools did you use for vibe coding

@threejs @PyTorch This is so cool. Amazing visualization!

@threejs @PyTorch great work!