Video wird geladen...

Video konnte nicht geladen werden

Beim Laden dieses Videos ist ein Problem aufgetreten. Dies könnte an einem vorübergehenden Netzwerkproblem liegen oder das Video ist möglicherweise nicht verfügbar.

low quant weights make the embedding model lose all discriminative power. I plotted the cosine correlation matrix of jina-v5, and one can see that low quant makes the model really blind. The off-diagonal similarities are pretty high on Q1/2/3, meaning everything looks similar in the semantic space. Q4 is... show more

Han Xiao

21,083 subscribers

63,134 Aufrufe • vor 6 Monaten •via X (Twitter)

Bildung Wissenschaft & Technologie

Anya Rossi• Live Now

Private livecam show

19 Kommentare

Profilbild von deep Manifold

deep Manifoldvor 6 Monaten

once high order nonlinearity and discontinuous is truly understood, quantization is dead end.

Profilbild von dazzafact

dazzafactvor 6 Monaten

Why not use a form of "WebP" Like compression instead of "JPEG"-like?

Profilbild von Han Xiao

Han Xiaovor 6 Monaten

damn u r right!

Profilbild von Yug Shende

Yug Shendevor 6 Monaten

Q8 is gonna become the new FLAC. It’s not gonna benefit most and yet some will swear by it.

Profilbild von Pabli! 🌿 💥💫

Pabli! 🌿 💥💫vor 6 Monaten

the best visualization of quantization that i've seen

Profilbild von Ping He

Ping Hevor 6 Monaten

My guess is the results you are seeing has more to do with the specific quantization method. The type of quant you can run on GGUF are "toy" quantization, they are not sota, and do not reflect non-linear quant frameworks, there is an ocean out there

Profilbild von Doxy

Doxyvor 6 Monaten

exactly why i stopped trusting q2 for anything that isn't a toy, just switched to llama.cpp with -q4_0 and the retrieval latency dropped while accuracy stayed solid honestly if your vectors look like soup you're either quanting too hard or using a model that was never meant to

Profilbild von High Jack

High Jackvor 6 Monaten

Nice viz. Q4's the floor for embeddings. Below that, you're just burning compute on semantic noise.

Profilbild von kodee

kodeevor 6 Monaten

I’m writing a paper about this right now. The effects of floating point non determinism on quantization

Profilbild von Logan Bolton

Logan Boltonvor 6 Monaten

Love this visualization!

Profilbild von Chahid Chirchi

Chahid Chirchivor 6 Monaten

yeah low quant kills the separation, those matrices look rough!!

Profilbild von Jason McGhee

Jason McGheevor 6 Monaten

Would love to see this coloration on a logarithmic scale. In my personal experience the Q8_0 is a significantly more stable and generally higher quality experience.

Profilbild von Sense

Sensevor 6 Monaten

Interesting. For me I see it like topographic map. 4-bit gives you 16 elevations.. that's where you can really start to contrast outliers from the near zeros etc.

Profilbild von ethantsliu

ethantsliuvor 6 Monaten

very cool!

Profilbild von Martin Skorunka

Martin Skorunkavor 6 Monaten

For me it would be very informative how the situation looks between q3_k_xl and something like iq4_k_xs.

Profilbild von Tom Dörr

Tom Dörrvor 6 Monaten

Correlations between what?

Profilbild von Han Xiao

Han Xiaovor 6 Monaten

Same 1k sentences -> diff quants weights -> 1k embeddings -> Cos(1k,1k)

Profilbild von KeenZ😶‍🌫️

KeenZ😶‍🌫️vor 6 Monaten

A very intuitive demonstration, fantastic!

Profilbild von gabor

gaborvor 6 Monaten

why would higher weight similarity imply 'blindness'? obviously at Q1 weights are either fully or zero correlated, and quant necessarily shifts toward higher correlation. Doesn't mean multiple non-linear stacked layers with high correlation don't retain useful patterns.

Ähnliche Videos

Tesla is recalling 20,000 Model 3 and Model Y vehicles with low-beam headlights that violate federal brightness standards—the latest in a string of safety concerns for the automaker.

Tesla is recalling 20,000 Model 3 and Model Y vehicles with low-beam headlights that violate federal brightness standards—the latest in a string of safety concerns for the automaker.

Forbes

58,172 Aufrufe • vor 2 Monaten

You need to know 4 things to run local AI on your own machine: 1) The model is the brain file. Gemma, Llama, Mistral, Qwen, Phi. 2) The warehouse is where you find the model. Hugging Face. 3) The software is what runs the model. LM Studio for the first try, Ollama when you build apps. 4) The workflow is the product you make from the other three. The size of the model decides the hardware: - 2B to 4B for phones and small tasks - 12B for the middle ground - 26B to 31B for a workstation Quantization compresses the model until it fits on a normal machine: - Q4 is easier to run - Q8 keeps more quality, but it needs more memory Q4 costs you a little quality but it saves you a $5,000 workstation. Start on the spare laptop from 2021. Open Hugging Face and read one model card slowly.

You need to know 4 things to run local AI on your own machine: 1) The model is the brain file. Gemma, Llama, Mistral, Qwen, Phi. 2) The warehouse is where you find the model. Hugging Face. 3) The software is what runs the model. LM Studio for the first try, Ollama when you build apps. 4) The workflow is the product you make from the other three. The size of the model decides the hardware: - 2B to 4B for phones and small tasks - 12B for the middle ground - 26B to 31B for a workstation Quantization compresses the model until it fits on a normal machine: - Q4 is easier to run - Q8 keeps more quality, but it needs more memory Q4 costs you a little quality but it saves you a $5,000 workstation. Start on the spare laptop from 2021. Open Hugging Face and read one model card slowly.

The Startup Ideas Podcast (SIP) 🧃

10,947 Aufrufe • vor 1 Monat

Kimi K2.7 Code is the first open-weight model you can select in the GitHub Copilot model picker. What does that mean for you? Burke Holland explains how this low-cost, high-performance model gives you more choice and flexibility in your workflow. ▶️

Kimi K2.7 Code is the first open-weight model you can select in the GitHub Copilot model picker. What does that mean for you? Burke Holland explains how this low-cost, high-performance model gives you more choice and flexibility in your workflow. ▶️

GitHub

125,671 Aufrufe • vor 4 Monaten

Steven MacKinnon: "There is very little or no correlation between the operating model of an airport, public, private, and so on, and those fees that are embedded in the cost of an airfare now, facilities improvement fees and other fees. There's no correlation between the operating model of an airport and its ownership model."

Steven MacKinnon: "There is very little or no correlation between the operating model of an airport, public, private, and so on, and those fees that are embedded in the cost of an airfare now, facilities improvement fees and other fees. There's no correlation between the operating model of an airport and its ownership model."

Scott Robertson

57,741 Aufrufe • vor 1 Monat

Same model psd for both Same rigger Left is 2022 and the one on the Right is 2025 a few days ago It's crazy how much a riggers can really change how a model looks LIKE THAT DOESN'T EVEN LOOK LIKE THE SAME MODEL, BUT IT IS!!

Same model psd for both Same rigger Left is 2022 and the one on the Right is 2025 a few days ago It's crazy how much a riggers can really change how a model looks LIKE THAT DOESN'T EVEN LOOK LIKE THE SAME MODEL, BUT IT IS!!

💙💉 Khione || Vtuber

86,688 Aufrufe • vor 1 Jahr

NEW: Google releases EmbeddingGemma, a state-of-the-art multilingual embedding model perfect for on-device use cases! At only 308M params, the model can run 100% locally in your browser! 🤯 Explore your documents in an interactive 3D universe with our demo: "The Semantic Galaxy"

NEW: Google releases EmbeddingGemma, a state-of-the-art multilingual embedding model perfect for on-device use cases! At only 308M params, the model can run 100% locally in your browser! 🤯 Explore your documents in an interactive 3D universe with our demo: "The Semantic Galaxy"

Xenova

23,338 Aufrufe • vor 1 Jahr

JD Vance: "There are things at some of the AI companies you see happening, where you feel like there is some really, really weird spiritual dark energy. Like one of the companies that develops these models, when they get rid of one model and announce a new model, they actually hold a burial ceremony for the old model, and the employees apparently will cry."

JD Vance: "There are things at some of the AI companies you see happening, where you feel like there is some really, really weird spiritual dark energy. Like one of the companies that develops these models, when they get rid of one model and announce a new model, they actually hold a burial ceremony for the old model, and the employees apparently will cry."

Harlan Stewart

361,627 Aufrufe • vor 2 Monaten

GLM 5.2 @ 19.8 tok/sec on 2 DGX Sparks ⚡️ I switched to a dspark draft model with K=2 (model from Red Hat AI) Acceptance is ~68% because the drafter was trained with the full FP8 model... Next step is to fine tune the drafter so acceptance on this super quant goes up

GLM 5.2 @ 19.8 tok/sec on 2 DGX Sparks ⚡️ I switched to a dspark draft model with K=2 (model from Red Hat AI) Acceptance is ~68% because the drafter was trained with the full FP8 model... Next step is to fine tune the drafter so acceptance on this super quant goes up

Joe Muller

73,297 Aufrufe • vor 3 Monaten

i have been testing with my models a lot customization and sounds, and i figure out how to put low hp themes! jump sounds and put more than 3 clothes/skin color in one single model, here all the videos even with the themes of the 3 models that have the low hp theme for now

i have been testing with my models a lot customization and sounds, and i figure out how to put low hp themes! jump sounds and put more than 3 clothes/skin color in one single model, here all the videos even with the themes of the 3 models that have the low hp theme for now

samurott lover #1

15,685 Aufrufe • vor 8 Monaten

Ever wondered what neural networks are and how they work? Systems like ChatGPT use neural networks to work as well as they do. Neural networks are composed of "layers" of neurons, layers with different functions; connections between layers called "weights"; and mathematical functions called "activation functions". If you’re interested in learning about these systems, check the comments. Ultimately, the neural network structure of the model serves to visually demonstrate that it is, in fact, a complex mathematical equation. When companies release the model's weights, they are releasing a key component needed to run the model's complete equation. Without the weights, the equation is incomplete. For the math-minded: the weights of a model are the learned numbers (they are variables during training) that are then used as constants in the mathematical functions that make up the model. Neural networks are ultimately just one big, hyper-complex mathematical function, and when a model is trained, it learns the constants associated with the high-dimensional input.

Ever wondered what neural networks are and how they work? Systems like ChatGPT use neural networks to work as well as they do. Neural networks are composed of "layers" of neurons, layers with different functions; connections between layers called "weights"; and mathematical functions called "activation functions". If you’re interested in learning about these systems, check the comments. Ultimately, the neural network structure of the model serves to visually demonstrate that it is, in fact, a complex mathematical equation. When companies release the model's weights, they are releasing a key component needed to run the model's complete equation. Without the weights, the equation is incomplete. For the math-minded: the weights of a model are the learned numbers (they are variables during training) that are then used as constants in the mathematical functions that make up the model. Neural networks are ultimately just one big, hyper-complex mathematical function, and when a model is trained, it learns the constants associated with the high-dimensional input.

Harper Carroll

50,905 Aufrufe • vor 1 Jahr

I turned Google Gemma EmbeddingGemma 2, an embedding model, into a speech-to-text model, and it turned out to be one of the fastest STT models you can run on device. On a base M3 MacBook Air it transcribes an hour of audio in about 13 seconds: 288× real time. The idea came in March with Gemini Embedding 2. It puts text, images, audio and video into one shared space, so a spoken word lands right next to the written word. If that's true, you should be able to read a transcript straight out of the embedding model. I couldnt tried it then, but last week EmbeddingGemma 2 came out as an open model, so I tried it.

I turned Google Gemma EmbeddingGemma 2, an embedding model, into a speech-to-text model, and it turned out to be one of the fastest STT models you can run on device. On a base M3 MacBook Air it transcribes an hour of audio in about 13 seconds: 288× real time. The idea came in March with Gemini Embedding 2. It puts text, images, audio and video into one shared space, so a spoken word lands right next to the written word. If that's true, you should be able to read a transcript straight out of the embedding model. I couldnt tried it then, but last week EmbeddingGemma 2 came out as an open model, so I tried it.

Shahir

297,477 Aufrufe • vor 24 Tagen

Hong Kong has officially become "Tesla-land" 🇭🇰 Standing on a street corner in Central, you can witness the brand’s entire evolution in a single traffic light cycle. It is one of the few places on earth where you see total saturation across the board—from the high-end luxury of the Model S and the iconic falcon-wing doors of the Model X to the massive waves of Model 3s and Model Ys that have become the city’s new standard. It’s no longer just a trend; it’s the city's new automotive DNA. It’s the Octopus card of the road.

Ming

17,538 Aufrufe • vor 6 Monaten

NEWS: The Tesla Model 3 has won Edmunds Top Rated Electric Car award for 2026. "The Model 3 is quite simply the best small affordable EV on the market. This is the best Model 3 yet. It is impressively well rounded thanks to improved build quality, ride comfort, and a compelling combination of efficiency, performance and value." "The Model 3 offers just about the perfect combination of everything — speed, range, comfort, space, tech, accessibility and convenience. It's a no-brainer if you want a sensible EV." - Jonathan Elfalan, director of vehicle testing.

NEWS: The Tesla Model 3 has won Edmunds Top Rated Electric Car award for 2026. "The Model 3 is quite simply the best small affordable EV on the market. This is the best Model 3 yet. It is impressively well rounded thanks to improved build quality, ride comfort, and a compelling combination of efficiency, performance and value." "The Model 3 offers just about the perfect combination of everything — speed, range, comfort, space, tech, accessibility and convenience. It's a no-brainer if you want a sensible EV." - Jonathan Elfalan, director of vehicle testing.

Sawyer Merritt

94,745 Aufrufe • vor 8 Monaten

this is what one prompt looks like now a detective, five locations, fog you can taste, and a low cello under all of it made on Qwen Create with the wan3.0 model prompt below, go break it:

this is what one prompt looks like now a detective, five locations, fog you can taste, and a low cello under all of it made on Qwen Create with the wan3.0 model prompt below, go break it:

Muhammad Ayan

56,500 Aufrufe • vor 2 Monaten

I just added 3D low-poly model generation to Retro Diffusion! Create super high quality low poly models with pixel art textures in minutes. Then animate them however you want with just a prompt! Check it out on the site and the API/MCP!

I just added 3D low-poly model generation to Retro Diffusion! Create super high quality low poly models with pixel art textures in minutes. Then animate them however you want with just a prompt! Check it out on the site and the API/MCP!

Astropulse

40,749 Aufrufe • vor 1 Monat

i'm running a 27b uncensored🚨model at 50 tok/s on a 12gb rtx 3060 and it does not say no to anything ⚠️ model just finished full harmbench red-team suite against it, 200 behaviors, and it refused zero. not a few. zero. aligned models sit in the 90s. the model is ternary bonsai 2 with the heretic uncensoring pass, the weights are ternary, just -1, 0 and +1, so the whole 27b is a file small enough that a gaming card from 2020 holds it with room to spare, and it's a reasoning model. it thinks before it answers, and it still keeps the pace the speed is the head i grafted on, a draft model that predicts the next token before the big model checks it. speculative decoding. the card fires a guess, the 27b confirms, and you get 50 tok/s instead of 39 the whole thing, weights, head and a 128k context window, sits in 10.7 of the 12 gb. kv cache at q4. nothing exotic, one card, one power plug why uncensored. because it's mine. no api keys, no terms of service, no model deciding what an adult is allowed to ask. the refusals left in it are ones it chose, and there are almost none that's the whole point of running local. the policy lives in your house, not someone else's

i'm running a 27b uncensored🚨model at 50 tok/s on a 12gb rtx 3060 and it does not say no to anything ⚠️ model just finished full harmbench red-team suite against it, 200 behaviors, and it refused zero. not a few. zero. aligned models sit in the 90s. the model is ternary bonsai 2 with the heretic uncensoring pass, the weights are ternary, just -1, 0 and +1, so the whole 27b is a file small enough that a gaming card from 2020 holds it with room to spare, and it's a reasoning model. it thinks before it answers, and it still keeps the pace the speed is the head i grafted on, a draft model that predicts the next token before the big model checks it. speculative decoding. the card fires a guess, the 27b confirms, and you get 50 tok/s instead of 39 the whole thing, weights, head and a 128k context window, sits in 10.7 of the 12 gb. kv cache at q4. nothing exotic, one card, one power plug why uncensored. because it's mine. no api keys, no terms of service, no model deciding what an adult is allowed to ask. the refusals left in it are ones it chose, and there are almost none that's the whole point of running local. the policy lives in your house, not someone else's

Sudo su

59,951 Aufrufe • vor 22 Tagen

80% of Most traders are asleep when the real setup forms. The 5AM window is where CRT quietly builds the day’s opportunity. In this 10-minute lecture, you’ll see: 5AM CRT Model: High Probability CRT Model (with ICT concepts) You’ll learn: • Why the 5AM candle is critical to the day’s range • How CRT defines the true high/low from this window • Where ICT concepts align with the setup • The confirmation that signals expansion • The mistake traders make before New York opens This isn’t a random time. It’s a repeatable window where structure is formed. If you trade CRT and ICT this model will change how you frame the morning. 🎥 10-minute in-depth breakdown attached. More CRT MODEL to be posted so be patient and wait for the remaining and chose the best that suits you.

80% of Most traders are asleep when the real setup forms. The 5AM window is where CRT quietly builds the day’s opportunity. In this 10-minute lecture, you’ll see: 5AM CRT Model: High Probability CRT Model (with ICT concepts) You’ll learn: • Why the 5AM candle is critical to the day’s range • How CRT defines the true high/low from this window • Where ICT concepts align with the setup • The confirmation that signals expansion • The mistake traders make before New York opens This isn’t a random time. It’s a repeatable window where structure is formed. If you trade CRT and ICT this model will change how you frame the morning. 🎥 10-minute in-depth breakdown attached. More CRT MODEL to be posted so be patient and wait for the remaining and chose the best that suits you.

R1X TRADES (GER40)

27,966 Aufrufe • vor 6 Monaten

I tested the AI model Sora 2 on classic anime, the result is hardly believable... I can already see the hundreds of fanmades and parodies that are going to come out! Sora 2 is definitely a new step in AI anime..

I tested the AI model Sora 2 on classic anime, the result is hardly believable... I can already see the hundreds of fanmades and parodies that are going to come out! Sora 2 is definitely a new step in AI anime..

Naegiko

25,464,193 Aufrufe • vor 1 Jahr