Rudy Gilman's banner

Rudy Gilman

@rgilman33 • 3,109 subscribers

substrate agnostic

Shorts

The attention layers in the VAEs for FLUX, Stable Diffusion 3.5, and SDXL don't do anything. You can ablate them with almost no effect. At first I thought they might be involved in some clever circuitry—maybe moving global information—but no they're just flailing around doing nothing.

The attention layers in the VAEs for FLUX, Stable Diffusion 3.5, and SDXL don't do anything. You can ablate them with almost no effect. At first I thought they might be involved in some clever circuitry—maybe moving global information—but no they're just flailing around doing nothing.

88,587 views

Half the channels in the last layer of the sdxl vae are disabled intentionally by the model itself. I've never seen this type of dead neuron before—it's like a form of apoptosis where the model kills off a part of itself so the rest may thrive. This is the final conv layer where we produce the output RGB. Notice half the neurons never contribute. The weights on the ignored channels are tiny.

Half the channels in the last layer of the sdxl vae are disabled intentionally by the model itself. I've never seen this type of dead neuron before—it's like a form of apoptosis where the model kills off a part of itself so the rest may thrive. This is the final conv layer where we produce the output RGB. Notice half the neurons never contribute. The weights on the ignored channels are tiny.

59,882 views

The sdxl-VAE models a substantial amount of noise. Things we can't even see. It meticulously encodes the noise, uses precious bottleneck capacity to store it, then faithfully reconstructs it in the decoder. I grabbed what I thought was a simple black vector circle on a white background but the VAE latched on to a veritable Navajo quilt of background noise. I was absolutely flummoxed until I realized what was going on! A lot of capacity being used on things we don't need to be modelling at all. Suspected culprit is group norm.

The sdxl-VAE models a substantial amount of noise. Things we can't even see. It meticulously encodes the noise, uses precious bottleneck capacity to store it, then faithfully reconstructs it in the decoder. I grabbed what I thought was a simple black vector circle on a white background but the VAE latched on to a veritable Navajo quilt of background noise. I was absolutely flummoxed until I realized what was going on! A lot of capacity being used on things we don't need to be modelling at all. Suspected culprit is group norm.

52,110 views

The later features in DINO-v2 are more abstract and semantically meaningful than I'd expected from the training objectives. This neuron responds only to hugs. Nothing else, just hugs.

The later features in DINO-v2 are more abstract and semantically meaningful than I'd expected from the training objectives. This neuron responds only to hugs. Nothing else, just hugs.

34,372 views

This is Siglip-2's dedicated DEI neuron. It fires for LGBTQ and indigenous flags, BLM imagery, and especially mixed-race groups doing happy things together (e.g. business meetings, jumping triumphantly, reaching summits etc)

This is Siglip-2's dedicated DEI neuron. It fires for LGBTQ and indigenous flags, BLM imagery, and especially mixed-race groups doing happy things together (e.g. business meetings, jumping triumphantly, reaching summits etc)

22,825 views

A challenge for ML diagnosticians: The patient, Segment-Anything-Model-2 (SAM-2), is presenting with two prominent symptoms: Symptom 1) Extremely high-magnitude tokens spread evenly across spatial dimensions.

A challenge for ML diagnosticians: The patient, Segment-Anything-Model-2 (SAM-2), is presenting with two prominent symptoms: Symptom 1) Extremely high-magnitude tokens spread evenly across spatial dimensions.

25,931 views

Videos

Anya Rossi

sweetdream.ai

SweetDream.ai•Sponsored•Livecam

Watch Anya Live

Anya is streaming live right now! Join her private show and enjoy exclusive content.

Exclusive private shows

1.2k viewers online

Private Show

Join now for exclusive access

Free preview available • Premium content

DINO-v3 has a single high-magnitude channel on its residual pathway, channel 416. Turning off this single channel affects DINO's entire output by 50-80%. For context, turning off a random channel has an effect of less than one percent. The model builds up channel 416 in its last two layer-scale operations, using a single high-magnitude weight in each op to drastically ramp up channel 416's magnitude. This channel doesn't depend on the input, every image fires with a constant overlay. After bringing channel 416 up to a value of about ten thousand, DINO-v3 then scales it down in the final layer-norm to almost nothing, removing it without a trace.

DINO-v3 has a single high-magnitude channel on its residual pathway, channel 416. Turning off this single channel affects DINO's entire output by 50-80%. For context, turning off a random channel has an effect of less than one percent. The model builds up channel 416 in its last two layer-scale operations, using a single high-magnitude weight in each op to drastically ramp up channel 416's magnitude. This channel doesn't depend on the input, every image fires with a constant overlay. After bringing channel 416 up to a value of about ten thousand, DINO-v3 then scales it down in the final layer-norm to almost nothing, removing it without a trace.

85,749 views • 9 months ago

The VAE used in SDXL has extremely high-magnitude "splotches" in its latents. The individual neurons in these blobs fire with magnitudes of close to a million. These aren't some accident of training or initialization—the model creates these high-magnitude splotches for a specific reason: to circumvent the group-norm operations.

The VAE used in SDXL has extremely high-magnitude "splotches" in its latents. The individual neurons in these blobs fire with magnitudes of close to a million. These aren't some accident of training or initialization—the model creates these high-magnitude splotches for a specific reason: to circumvent the group-norm operations.

104,780 views • 1 year ago

This layer in DINO-v2 dedicates about half its attention mass to a single operation. Each of the sixteen heads independently learns the same circuit to perform this task. What is this all-important operation? The “no-op”. That’s right, we’re spending half our computation to do… absolutely nothing.

This layer in DINO-v2 dedicates about half its attention mass to a single operation. Each of the sixteen heads independently learns the same circuit to perform this task. What is this all-important operation? The “no-op”. That’s right, we’re spending half our computation to do… absolutely nothing.

97,122 views • 1 year ago

SDXL-turbo isn't given positional information—so it makes its own. You can see the positional grid forming in the first few blocks, starting from the borders and rippling inwards, carried by successive layers of 3x3 convs.

SDXL-turbo isn't given positional information—so it makes its own. You can see the positional grid forming in the first few blocks, starting from the borders and rippling inwards, carried by successive layers of 3x3 convs.

66,379 views • 1 year ago

Do diffusion models want Perlin-like noise? SDXL-turbo spends the first blocks simply creating Perlin-like noise of different frequencies, amplitudes and orientations. The model ignores the prompt conditioning during these layers.

Do diffusion models want Perlin-like noise? SDXL-turbo spends the first blocks simply creating Perlin-like noise of different frequencies, amplitudes and orientations. The model ignores the prompt conditioning during these layers.

59,764 views • 1 year ago

The last stage of Segment-Anything-Model-2 (SAM-2) is filled with what look to be dead GELUs. It's a graveyard out there.

The last stage of Segment-Anything-Model-2 (SAM-2) is filled with what look to be dead GELUs. It's a graveyard out there.

26,564 views • 1 year ago

The majority of features in this layer of Siglip-2 are multimodal. I'd expected some multimodality but was surprised that two-thirds of the neurons I tested bind together their visual and linguistic features. This neuron fires for images of mustaches and for the word "mustache"

The majority of features in this layer of Siglip-2 are multimodal. I'd expected some multimodality but was surprised that two-thirds of the neurons I tested bind together their visual and linguistic features. This neuron fires for images of mustaches and for the word "mustache"

18,518 views • 1 year ago

Introducing darkspark, a gui for your neural network. It traces your pytorch code and brings up a visual representation for you to interact with. We have a hosted gallery of popular model architectures pre-traced and ready to explore. Here’s stable-diffusion-v1.5

Introducing darkspark, a gui for your neural network. It traces your pytorch code and brings up a visual representation for you to interact with. We have a hosted gallery of popular model architectures pre-traced and ready to explore. Here’s stable-diffusion-v1.5

16,657 views • 1 year ago

No more content to load