Video yükleniyor...

Video Yüklenemedi

Bu video yüklenirken bir sorun oluştu. Bu geçici bir ağ sorunundan kaynaklanıyor olabilir veya video kullanılamıyor olabilir.

Ana Sayfaya Dön

Given an embedding vector, you can tell which model produced it. I trained a 0.8M transformer that fingerprints embedding models by reading raw float digits (vocab size: 15). Full end-to-end, zero feature engineering.

Han Xiao

18,820 subscribers

19,695 görüntüleme • 4 ay önce •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

0 Yorum

Yorum bulunmuyor

Orijinal gönderinin yorumları burada görünecek

Benzer Videolar

RTFM can be seen as a learned renderer: it is an autoregressive diffusion transformer trained end-to-end on large-scale video data, and it learns to model 3D geometry, reflections, shadows and more just by observing them in its training set.

RTFM can be seen as a learned renderer: it is an autoregressive diffusion transformer trained end-to-end on large-scale video data, and it learns to model 3D geometry, reflections, shadows and more just by observing them in its training set.

World Labs

15,870 görüntüleme • 9 ay önce

Here we go. The full video with translations for $LMND COO Adina Eckstein’s interview. BTW, this was done end-to-end in less than 5 minutes by AI: download -->transcription --> subtitle embedding --> X posting. If I can do something like this with zero programming background, imagine how much Lemonade can automate across its business with AI.

Here we go. The full video with translations for $LMND COO Adina Eckstein’s interview. BTW, this was done end-to-end in less than 5 minutes by AI: download -->transcription --> subtitle embedding --> X posting. If I can do something like this with zero programming background, imagine how much Lemonade can automate across its business with AI.

CyberCat

11,210 görüntüleme • 1 ay önce

Holy sh*t this is gold THE FULL GRAPH ENGINEERING PIPELINE, GROUNDED IN REAL, OPEN SOURCE WORK. Representation. Ontology. Entity extraction. Relations. Events. Quality gate. Fusion. Embeddings. Memory your agent can actually traverse. Real repos already cover this end to end. Strwythura walks through the full ontology pipeline with entity resolution and embedding-enhanced GraphRAG. llm2kg covers entity and relationship extraction against an ontology, Neo4j storage with vector embeddings, and a ReAct agent that queries the graph directly. No PhD required. Real, working code you can run against your own data today.

Holy sh*t this is gold THE FULL GRAPH ENGINEERING PIPELINE, GROUNDED IN REAL, OPEN SOURCE WORK. Representation. Ontology. Entity extraction. Relations. Events. Quality gate. Fusion. Embeddings. Memory your agent can actually traverse. Real repos already cover this end to end. Strwythura walks through the full ontology pipeline with entity resolution and embedding-enhanced GraphRAG. llm2kg covers entity and relationship extraction against an ontology, Neo4j storage with vector embeddings, and a ReAct agent that queries the graph directly. No PhD required. Real, working code you can run against your own data today.

CyrilXBT

48,495 görüntüleme • 6 gün önce

New short course on advanced retrieval for RAG (retrieval augmented generation)! RAG fetches relevant documents to give context to an LLM. In Advanced Retrieval for AI with Chroma, taught by Chroma founder anton 🇺🇸, you’ll learn: (i) Query expansion using an LLM to rewrite and improve a query, by either generating either additional relevant queries or a hypothetical answer to the query. (ii) Reranking using a cross-encoder - a model trained to measure similarity between two inputs presented simultaneously. Reranking reorders retrieved documents based on the cross-encoder similarity measure. (iii) Constructing and training an Embedding Adaptor, which is a model that adapts the embedding values to be more relevant to your use case. Each of these techniques can help you build much better RAG systems. Please sign up for the course here:

New short course on advanced retrieval for RAG (retrieval augmented generation)! RAG fetches relevant documents to give context to an LLM. In Advanced Retrieval for AI with Chroma, taught by Chroma founder anton 🇺🇸, you’ll learn: (i) Query expansion using an LLM to rewrite and improve a query, by either generating either additional relevant queries or a hypothetical answer to the query. (ii) Reranking using a cross-encoder - a model trained to measure similarity between two inputs presented simultaneously. Reranking reorders retrieved documents based on the cross-encoder similarity measure. (iii) Constructing and training an Embedding Adaptor, which is a model that adapts the embedding values to be more relevant to your use case. Each of these techniques can help you build much better RAG systems. Please sign up for the course here:

Andrew Ng

191,219 görüntüleme • 2 yıl önce

1. Project Digits: NVIDIA's Latest AI Supercomputer Introduces an exciting breakthrough with the new NVIDIA GB10 Superchip, offering a compact AI supercomputer that efficiently handles 200B-parameter models. NVIDIA’s Project DIGITS is a powerhouse you can hold in the palm of your hand. With two Project DIGITS units, developers can easily work with AI models up to 405B parameters in size. Available by May.

1. Project Digits: NVIDIA's Latest AI Supercomputer Introduces an exciting breakthrough with the new NVIDIA GB10 Superchip, offering a compact AI supercomputer that efficiently handles 200B-parameter models. NVIDIA’s Project DIGITS is a powerhouse you can hold in the palm of your hand. With two Project DIGITS units, developers can easily work with AI models up to 405B parameters in size. Available by May.

Shruti

236,550 görüntüleme • 1 yıl önce

New short course: Building Multimodal Search and RAG", by Weaviate AI Database's Sebastia(N_) Witalec ✊🏽✊🏾✊🏿. Contrastive learning is used to train models to map vectors into an embedding space by pulling similar concepts closer together and pushing dissimilar concepts away from each other. This technique is also used to train multimodal embedding models that capture semantic similarity across different modalities like text, images, and audio. These multimodal embeddings can be used to build multimodal search and RAG systems. In this course, you'll learn how contrastive learning works, and how to add multimodality to RAG – so your models can draw on diverse, relevant context to answer questions. For example, a query about a financial report might synthesize information from text snippets, graphs, tables, and slides. You will also learn how visual instruction tuning lets you integrate image understanding into language models, and build a multi-vector recommender system using Weaviate’s open-source vector database. Please sign up here:

New short course: Building Multimodal Search and RAG", by Weaviate AI Database's Sebastia(N_) Witalec ✊🏽✊🏾✊🏿. Contrastive learning is used to train models to map vectors into an embedding space by pulling similar concepts closer together and pushing dissimilar concepts away from each other. This technique is also used to train multimodal embedding models that capture semantic similarity across different modalities like text, images, and audio. These multimodal embeddings can be used to build multimodal search and RAG systems. In this course, you'll learn how contrastive learning works, and how to add multimodality to RAG – so your models can draw on diverse, relevant context to answer questions. For example, a query about a financial report might synthesize information from text snippets, graphs, tables, and slides. You will also learn how visual instruction tuning lets you integrate image understanding into language models, and build a multi-vector recommender system using Weaviate’s open-source vector database. Please sign up here:

Andrew Ng

104,371 görüntüleme • 2 yıl önce

Vector Database by hand ✍️ ~ 10 steps walkthrough below Vector databases are the backbone of Retrieval Augmented Generation (RAG). How do they actually work? Goal: index three sentences, then answer a query by finding the nearest one, filling in every cell yourself. = 1. Given = A dataset of three sentences, three words each. In practice it is millions of them. = 2. Word embeddings = Let us look up each word in an embedding table. Here the vocabulary is 22 words; in practice it is tens of thousands, and the vectors have thousands of dimensions rather than four. = 3. Encoding = We feed the sequence to an encoder, one linear layer and a ReLU, and get one feature vector per word. In practice the encoder is a transformer. = 4. Mean pooling = Let us average across the columns. Three word vectors collapse into one, which is what people mean by a text embedding or a sentence embedding. = 5. Indexing = We multiply by a projection matrix and the four dimensions become two. It is doing the job of a hash: a short representation that is faster to compare, and it is what gets saved in the vector storage. = 6. Process "who are you" = Let us repeat steps 2 to 5 on the second sentence. = 7. Process "who am I" = We do it a third time. The database is now indexed. = 8. Query "am I you" = Let us push the query through the very same pipeline: lookup, encoder, mean pooling, projection, and it lands as a 2D vector in the same space. = 9. Dot products = We transpose the query and multiply, which takes the dot product against every stored vector at once. The dot product is the estimate of similarity. = 10. Nearest neighbour = Let us scan for the largest: 60/9 beats 44/9 and 40/9, so the answer is "who am I". Scanning billions of vectors one at a time is what makes this the slow step in practice, which is why real databases use an approximate nearest neighbour index like HNSW. The outputs: Stored index vectors = [5/3, 2/3], [5/3, 0], [7/3, 2/3] Query vector = [8/3, 2/3] Dot products = 44/9, 40/9, 60/9 Nearest neighbour = "who am I" The takeaway: a vector database is an embedding pipeline, a projection, and a dot product. Every step here is arithmetic you can do in pen, which is worth remembering when the word "database" makes it sound like something else. 💾 Save this post!

Vector Database by hand ✍️ ~ 10 steps walkthrough below Vector databases are the backbone of Retrieval Augmented Generation (RAG). How do they actually work? Goal: index three sentences, then answer a query by finding the nearest one, filling in every cell yourself. = 1. Given = A dataset of three sentences, three words each. In practice it is millions of them. = 2. Word embeddings = Let us look up each word in an embedding table. Here the vocabulary is 22 words; in practice it is tens of thousands, and the vectors have thousands of dimensions rather than four. = 3. Encoding = We feed the sequence to an encoder, one linear layer and a ReLU, and get one feature vector per word. In practice the encoder is a transformer. = 4. Mean pooling = Let us average across the columns. Three word vectors collapse into one, which is what people mean by a text embedding or a sentence embedding. = 5. Indexing = We multiply by a projection matrix and the four dimensions become two. It is doing the job of a hash: a short representation that is faster to compare, and it is what gets saved in the vector storage. = 6. Process "who are you" = Let us repeat steps 2 to 5 on the second sentence. = 7. Process "who am I" = We do it a third time. The database is now indexed. = 8. Query "am I you" = Let us push the query through the very same pipeline: lookup, encoder, mean pooling, projection, and it lands as a 2D vector in the same space. = 9. Dot products = We transpose the query and multiply, which takes the dot product against every stored vector at once. The dot product is the estimate of similarity. = 10. Nearest neighbour = Let us scan for the largest: 60/9 beats 44/9 and 40/9, so the answer is "who am I". Scanning billions of vectors one at a time is what makes this the slow step in practice, which is why real databases use an approximate nearest neighbour index like HNSW. The outputs: Stored index vectors = [5/3, 2/3], [5/3, 0], [7/3, 2/3] Query vector = [8/3, 2/3] Dot products = 44/9, 40/9, 60/9 Nearest neighbour = "who am I" The takeaway: a vector database is an embedding pipeline, a projection, and a dot product. Every step here is arithmetic you can do in pen, which is worth remembering when the word "database" makes it sound like something else. 💾 Save this post!

Tom Yeh

35,070 görüntüleme • 7 gün önce

Learn a development pattern to systematically improve the accuracy and reliability of LLM applications in our new short course, Improving Accuracy of LLM Applications, built in partnership with Lamini and Meta, and taught by Lamini’s CEO Sharon Zhou, and Meta’s Senior Director of Partner Engineering, Amit Sangani. (Disclosure: I am an investor in Lamini.) The path to tuning an LLM application can be complex. In this course, you'll learn a systematic sequence of steps for improving accuracy by reducing hallucinations: - Create an evaluation dataset to measure model accuracy - Add prompt engineering and self-reflection - Fine-tune your model including "memory-tuning" which is a new method of embedding facts in an LLM Using the Llama 3-8B parameter model, you will: - Build a text-to-SQL agent with a custom schema and simulate situations where it hallucinates - Understand the difference between instruction fine-tuning, which gives pre-trained LLMs instructions to follow, and memory fine-tuning - See how Performance-Efficient Fine-tuning (PEFT) techniques like Low-Rank Adaptation (LoRA) reduce training time by 100x and Mixture of Memory Experts (MoME) reduces it even further I appreciate Meta releasing the Llama's family of open models -- this course gives an example of the unique type of work that developers can do with such models. Please sign up here:

Learn a development pattern to systematically improve the accuracy and reliability of LLM applications in our new short course, Improving Accuracy of LLM Applications, built in partnership with Lamini and Meta, and taught by Lamini’s CEO Sharon Zhou, and Meta’s Senior Director of Partner Engineering, Amit Sangani. (Disclosure: I am an investor in Lamini.) The path to tuning an LLM application can be complex. In this course, you'll learn a systematic sequence of steps for improving accuracy by reducing hallucinations: - Create an evaluation dataset to measure model accuracy - Add prompt engineering and self-reflection - Fine-tune your model including "memory-tuning" which is a new method of embedding facts in an LLM Using the Llama 3-8B parameter model, you will: - Build a text-to-SQL agent with a custom schema and simulate situations where it hallucinates - Understand the difference between instruction fine-tuning, which gives pre-trained LLMs instructions to follow, and memory fine-tuning - See how Performance-Efficient Fine-tuning (PEFT) techniques like Low-Rank Adaptation (LoRA) reduce training time by 100x and Mixture of Memory Experts (MoME) reduces it even further I appreciate Meta releasing the Llama's family of open models -- this course gives an example of the unique type of work that developers can do with such models. Please sign up here:

Andrew Ng

66,435 görüntüleme • 1 yıl önce

Google just proved that bigger isn't always better. Their 308M parameter model is outperforming models 2x its size. Google just released 𝗘𝗺𝗯𝗲𝗱𝗱𝗶𝗻𝗴𝗚𝗲𝗺𝗺𝗮, and it's proving that lightweight embedding models can punch way above their weight class. At just 308M parameters (578MB), it's the new state-of-the-art for models under 500M parameters across MTEB multilingual, English, and code benchmarks. But the really impressive part is that it ranks 8th overall on MTEB(Multilingual, v2) - that's 𝟭𝟳 𝗽𝗹𝗮𝗰𝗲𝘀 above the second-best sub-500M model, and it's delivering performance 𝗰𝗼𝗺𝗽𝗮𝗿𝗮𝗯𝗹𝗲 𝘁𝗼 𝗺𝗼𝗱𝗲𝗹𝘀 𝗻𝗲𝗮𝗿𝗹𝘆 𝗱𝗼𝘂𝗯𝗹𝗲 𝗶𝘁𝘀 𝘀𝗶𝘇𝗲. There are three key parts of their training recipe that sets it apart: 𝟭. 𝗘𝗻𝗰𝗼𝗱𝗲𝗿-𝗗𝗲𝗰𝗼𝗱𝗲𝗿 𝗜𝗻𝗶𝘁𝗶𝗮𝗹𝗶𝘇𝗮𝘁𝗶𝗼𝗻 Instead of starting from a decoder-only Gemma 3 model, they first adapted it to encoder-decoder, then used just the encoder. By basing EmbeddingGemma off an LLM that already has world and language understanding, it gives it a stronger starting point. 𝟮. 𝗧𝗵𝗿𝗲𝗲-𝗟𝗼𝘀𝘀 𝗧𝗿𝗮𝗶𝗻𝗶𝗻𝗴 They combine three different loss functions, instead of just having one: • Contrastive loss (NCE) with in-batch negatives and hardness weighting • Spread-out regularization to ensure embeddings utilize the full space (for quantization and ANN retrieval) • Embedding matching distillation from Gemini Embedding - not just learning from relevance scores, but directly aligning the embedding space with the teacher model 𝟯. 𝗠𝗼𝗱𝗲𝗹 𝗦𝗼𝘂𝗽𝗶𝗻𝗴 Rather than just averaging checkpoints from the same training run, they use optimization techniques to find multiple specialized training mixtures. Each mixture creates an "expert" model in different domains, and averaging all their parameters creates a final model that's actually better than individual models. Extras: • Matryoshka embeddings supporting 768, 512, 256, and 128 dimensions • Quantization-aware training - maintains quality even at int4 precision • 100+ languages from Gemma 3 pretraining • Exceptional performance on low-resource languages (check their XTREME-UP results) Is it the absolute best embedding model? No - Gemini Embedding still leads overall. But that's not really the point. EmbeddingGemma proves you can achieve state-of-the-art performance in a small package that's actually deployable on-device, in low-latency applications, and in resource-constrained environments. This makes good embeddings accessible for use cases that I'm seeing more and more: offline applications, privacy-sensitive deployments, and high-throughput scenarios where inference cost actually matters. Full paper: Shoutout to the EmbeddingGemma team at Google DeepMind for this awesome open source work 💙 and to Daniel Williams for helping me with this video! 🫶

Google just proved that bigger isn't always better. Their 308M parameter model is outperforming models 2x its size. Google just released 𝗘𝗺𝗯𝗲𝗱𝗱𝗶𝗻𝗴𝗚𝗲𝗺𝗺𝗮, and it's proving that lightweight embedding models can punch way above their weight class. At just 308M parameters (578MB), it's the new state-of-the-art for models under 500M parameters across MTEB multilingual, English, and code benchmarks. But the really impressive part is that it ranks 8th overall on MTEB(Multilingual, v2) - that's 𝟭𝟳 𝗽𝗹𝗮𝗰𝗲𝘀 above the second-best sub-500M model, and it's delivering performance 𝗰𝗼𝗺𝗽𝗮𝗿𝗮𝗯𝗹𝗲 𝘁𝗼 𝗺𝗼𝗱𝗲𝗹𝘀 𝗻𝗲𝗮𝗿𝗹𝘆 𝗱𝗼𝘂𝗯𝗹𝗲 𝗶𝘁𝘀 𝘀𝗶𝘇𝗲. There are three key parts of their training recipe that sets it apart: 𝟭. 𝗘𝗻𝗰𝗼𝗱𝗲𝗿-𝗗𝗲𝗰𝗼𝗱𝗲𝗿 𝗜𝗻𝗶𝘁𝗶𝗮𝗹𝗶𝘇𝗮𝘁𝗶𝗼𝗻 Instead of starting from a decoder-only Gemma 3 model, they first adapted it to encoder-decoder, then used just the encoder. By basing EmbeddingGemma off an LLM that already has world and language understanding, it gives it a stronger starting point. 𝟮. 𝗧𝗵𝗿𝗲𝗲-𝗟𝗼𝘀𝘀 𝗧𝗿𝗮𝗶𝗻𝗶𝗻𝗴 They combine three different loss functions, instead of just having one: • Contrastive loss (NCE) with in-batch negatives and hardness weighting • Spread-out regularization to ensure embeddings utilize the full space (for quantization and ANN retrieval) • Embedding matching distillation from Gemini Embedding - not just learning from relevance scores, but directly aligning the embedding space with the teacher model 𝟯. 𝗠𝗼𝗱𝗲𝗹 𝗦𝗼𝘂𝗽𝗶𝗻𝗴 Rather than just averaging checkpoints from the same training run, they use optimization techniques to find multiple specialized training mixtures. Each mixture creates an "expert" model in different domains, and averaging all their parameters creates a final model that's actually better than individual models. Extras: • Matryoshka embeddings supporting 768, 512, 256, and 128 dimensions • Quantization-aware training - maintains quality even at int4 precision • 100+ languages from Gemma 3 pretraining • Exceptional performance on low-resource languages (check their XTREME-UP results) Is it the absolute best embedding model? No - Gemini Embedding still leads overall. But that's not really the point. EmbeddingGemma proves you can achieve state-of-the-art performance in a small package that's actually deployable on-device, in low-latency applications, and in resource-constrained environments. This makes good embeddings accessible for use cases that I'm seeing more and more: offline applications, privacy-sensitive deployments, and high-throughput scenarios where inference cost actually matters. Full paper: Shoutout to the EmbeddingGemma team at Google DeepMind for this awesome open source work 💙 and to Daniel Williams for helping me with this video! 🫶

Victoria Slocum

21,592 görüntüleme • 8 ay önce

An AI mental health therapist sounds like a terrible idea. It will literally just tell you what you want to hear. I can already see how bad this is going to end.

An AI mental health therapist sounds like a terrible idea. It will literally just tell you what you want to hear. I can already see how bad this is going to end.

Michael

19,857 görüntüleme • 3 ay önce

Keir Starmer is a Globalist. This means he’s fully committed to embedding their regime change, which by 2030 will amount to technocratic communism. This includes:- - Digital ID’s & CBDC - Social Credit scores & Co2 Trackers - Open Borders - End of private motor vehicles - 15 Minute Cities - Mandatory vaccines & passports - Fake Meat & insect consumption - Limits on flying & travel - Universal Basic State approved income “You will own nothing & be happy”

Keir Starmer is a Globalist. This means he’s fully committed to embedding their regime change, which by 2030 will amount to technocratic communism. This includes:- - Digital ID’s & CBDC - Social Credit scores & Co2 Trackers - Open Borders - End of private motor vehicles - 15 Minute Cities - Mandatory vaccines & passports - Fake Meat & insect consumption - Limits on flying & travel - Universal Basic State approved income “You will own nothing & be happy”

Concerned Citizen

941,127 görüntüleme • 2 yıl önce

Introducing VL-JEPA: Vision-Language Joint Embedding Predictive Architecture for streaming, live action recognition, retrieval, VQA, and classification tasks with better performance and higher efficiency than large VLMs. • VL-JEPA is the first non-generative model that can perform general-domain vision-language tasks in real-time, built on a joint embedding predictive architecture. • We demonstrate in controlled experiments that VL-JEPA, trained with latent space embedding prediction, outperforms VLMs that rely on data space token prediction. • We show that VL-JEPA delivers significant efficiency gains over VLMs for online video streaming applications, thanks to its non-autoregressive design and native support for selective decoding. • We highlight that our VL-JEPA model, with an unified model architecture, can effectively handle a wide range of classification, retrieval, and VQA tasks at the same time. by Delong Chen (陈德龙) Mustafa Shukor Théo Moutakanni Willy Jade Lei Yu Tejaswi Kasarla Allen Bolourchi Yann LeCun Pascale Fung

Introducing VL-JEPA: Vision-Language Joint Embedding Predictive Architecture for streaming, live action recognition, retrieval, VQA, and classification tasks with better performance and higher efficiency than large VLMs. • VL-JEPA is the first non-generative model that can perform general-domain vision-language tasks in real-time, built on a joint embedding predictive architecture. • We demonstrate in controlled experiments that VL-JEPA, trained with latent space embedding prediction, outperforms VLMs that rely on data space token prediction. • We show that VL-JEPA delivers significant efficiency gains over VLMs for online video streaming applications, thanks to its non-autoregressive design and native support for selective decoding. • We highlight that our VL-JEPA model, with an unified model architecture, can effectively handle a wide range of classification, retrieval, and VQA tasks at the same time. by Delong Chen (陈德龙) Mustafa Shukor Théo Moutakanni Willy Jade Lei Yu Tejaswi Kasarla Allen Bolourchi Yann LeCun Pascale Fung

Pascale Fung

90,144 görüntüleme • 7 ay önce

MILEI: I'LL DO A SINGING SHOW UPON ZERO-INFLATION Milei: Inflation will cease to exist by mid 2026. Federico Sturzenegger: No, it will end sooner than that. Milei: Let's bet on it... "I bet that if it ended sooner, I was going to do a signing show. I believe I will lose the bet. So, I can tell you that I am already starting to rehearse six songs." -President Javier Milei Javier Milei

MILEI: I'LL DO A SINGING SHOW UPON ZERO-INFLATION Milei: Inflation will cease to exist by mid 2026. Federico Sturzenegger: No, it will end sooner than that. Milei: Let's bet on it... "I bet that if it ended sooner, I was going to do a signing show. I believe I will lose the bet. So, I can tell you that I am already starting to rehearse six songs." -President Javier Milei Javier Milei

Argentina's Milei News 🇦🇷🤝🌎

64,178 görüntüleme • 1 yıl önce

Trump tonight is asked how he would end the war in Ukraine: “It can be negotiated within 24 hours. You have to get them both in a room. And there are things you can say to them, which I won’t reveal now, which will guarantee that this war will end immediately.”

Trump tonight is asked how he would end the war in Ukraine: “It can be negotiated within 24 hours. You have to get them both in a room. And there are things you can say to them, which I won’t reveal now, which will guarantee that this war will end immediately.”

Ron Filipkowski

4,598,165 görüntüleme • 3 yıl önce

Wild. Kimi K2 Thinking just released and it's insane. It's an AI model that can run by itself for hours on end and make HUNDREDS of tool calls It's the 1st model I think that can replace humans In this video I show why it's so special and how to use it to build your first app

Wild. Kimi K2 Thinking just released and it's insane. It's an AI model that can run by itself for hours on end and make HUNDREDS of tool calls It's the 1st model I think that can replace humans In this video I show why it's so special and how to use it to build your first app

Alex Finn

70,013 görüntüleme • 8 ay önce

🎥 Videos DiTs are painfully slow, HunyuanVideo takes 16 min to generate a 5s 720P video on H100. 🤯 Announcing Sliding Tile Attention (STA): * Accelerate 3D full attention (FA3) by up to 10x * Slash the end-to-end time from 16 --> 5 mins * NO extra training. NO quality loss! 🚀 Can you tell which videos are generated by the original HunyuanVideo, and which by STA? 👀 Blog:

🎥 Videos DiTs are painfully slow, HunyuanVideo takes 16 min to generate a 5s 720P video on H100. 🤯 Announcing Sliding Tile Attention (STA): * Accelerate 3D full attention (FA3) by up to 10x * Slash the end-to-end time from 16 --> 5 mins * NO extra training. NO quality loss! 🚀 Can you tell which videos are generated by the original HunyuanVideo, and which by STA? 👀 Blog:

Hao AI Lab

58,034 görüntüleme • 1 yıl önce

Announcing How Transformer LLMs Work, created with Jay Alammar and Maarten Grootendorst, co-authors of the beautifully illustrated book, “Hands-On Large Language Models.” This course offers a deep dive into the inner workings of the transformer architecture that powers large language models (LLMs). The transformer architecture revolutionized generative AI; in fact, the "GPT" in ChatGPT stands for "Generative Pre-Trained Transformer." Originally introduced in the Google Brain team's groundbreaking 2017 paper "Attention Is All You Need," by Vaswani and others, transformers were a highly scalable model for machine translation tasks. Variants of this architecture now power today’s LLMs such as those from OpenAI, Google, Meta, Cohere, Anthropic and DeepSeek. In this course, you’ll learn in detail how LLMs process text. You'll also work through code examples that illustrate that transformer's individual components. In details, you’ll learn: - How the representation of language has evolved, from Bag-of-Words to Word2Vec embeddings to the transformer architecture that captures a word's meanings taking into account the context of other words in the input. - How inputs are broken down into tokens before they are sent to the language model. - The details of a transformer's main stages: Tokenization and embedding, the stack of transformer blocks, and the language model head. - The inner workings of the transformer block, including attention, which calculates relevance scores, and the feedforward layer, which incorporates stored information learned in training. - How cached calculations make transformers faster. - Some of the most recent ideas in the latest models such as Mixture-of-Experts (MoE) which uses multiple sub-models and a router on each layer to improve the quality of LLMs. By the end of this course, you’ll have a deep understanding of how LLMs actually process text and be able to read through papers describing the latest models and understand the details. Gaining this intuition will improve your approach to building LLM applications. Please sign up here:

Announcing How Transformer LLMs Work, created with Jay Alammar and Maarten Grootendorst, co-authors of the beautifully illustrated book, “Hands-On Large Language Models.” This course offers a deep dive into the inner workings of the transformer architecture that powers large language models (LLMs). The transformer architecture revolutionized generative AI; in fact, the "GPT" in ChatGPT stands for "Generative Pre-Trained Transformer." Originally introduced in the Google Brain team's groundbreaking 2017 paper "Attention Is All You Need," by Vaswani and others, transformers were a highly scalable model for machine translation tasks. Variants of this architecture now power today’s LLMs such as those from OpenAI, Google, Meta, Cohere, Anthropic and DeepSeek. In this course, you’ll learn in detail how LLMs process text. You'll also work through code examples that illustrate that transformer's individual components. In details, you’ll learn: - How the representation of language has evolved, from Bag-of-Words to Word2Vec embeddings to the transformer architecture that captures a word's meanings taking into account the context of other words in the input. - How inputs are broken down into tokens before they are sent to the language model. - The details of a transformer's main stages: Tokenization and embedding, the stack of transformer blocks, and the language model head. - The inner workings of the transformer block, including attention, which calculates relevance scores, and the feedforward layer, which incorporates stored information learned in training. - How cached calculations make transformers faster. - Some of the most recent ideas in the latest models such as Mixture-of-Experts (MoE) which uses multiple sub-models and a router on each layer to improve the quality of LLMs. By the end of this course, you’ll have a deep understanding of how LLMs actually process text and be able to read through papers describing the latest models and understand the details. Gaining this intuition will improve your approach to building LLM applications. Please sign up here:

Andrew Ng

259,920 görüntüleme • 1 yıl önce

NEW: Bill Maher leaves journalist stunned when he tells her the “End Racism” message in the NFL end zone does nothing to end racism. MAHER: “I noticed that at the Super Bowl, they’re, for the first time in, I think, four years now, the Trump administration is making them take away ‘End Racism,’ which they had written in the end zone.” TARA PALMERI: “But why? It just seems silly.” MAHER: “To do it or not to do it?” TARA PALMERI: “Why get rid of it?” MAHER: “Oh, I could tell you why. Because it was stupid to begin with. But let me ask you, who is it for? And if you’re a racist and you see ‘End Racism’ in the end zone, you’re gonna stop being a racist?” TARA PALMERI: (Stunned) “But the sentiment is basically like, don’t be an asshole.” MAHER: “But I think it’s an asshole to nag us during a football game about something that doesn’t change anything. If I’m not a racist and I see it, it doesn’t matter. And if I am a racist, it’s gonna make me more of a racist.” BYRON DONALDS: “Look, I think if you write ‘Don’t be an asshole’ in the end zone, everybody will agree with that.” (Round of Applause)

NEW: Bill Maher leaves journalist stunned when he tells her the “End Racism” message in the NFL end zone does nothing to end racism. MAHER: “I noticed that at the Super Bowl, they’re, for the first time in, I think, four years now, the Trump administration is making them take away ‘End Racism,’ which they had written in the end zone.” TARA PALMERI: “But why? It just seems silly.” MAHER: “To do it or not to do it?” TARA PALMERI: “Why get rid of it?” MAHER: “Oh, I could tell you why. Because it was stupid to begin with. But let me ask you, who is it for? And if you’re a racist and you see ‘End Racism’ in the end zone, you’re gonna stop being a racist?” TARA PALMERI: (Stunned) “But the sentiment is basically like, don’t be an asshole.” MAHER: “But I think it’s an asshole to nag us during a football game about something that doesn’t change anything. If I’m not a racist and I see it, it doesn’t matter. And if I am a racist, it’s gonna make me more of a racist.” BYRON DONALDS: “Look, I think if you write ‘Don’t be an asshole’ in the end zone, everybody will agree with that.” (Round of Applause)

The Vigilant Fox 🦊

2,610,848 görüntüleme • 1 yıl önce

I know that people can wake up to reality because I did. I made this video in one take (I was out of breath by the end) when I discovered how wrong I had been. You can tell how bad I was at video editing then and can hear the traffic in the background, but it is a raw and real video that I had put together out of sheer frustration of the ridiculousness I was seeing.

I know that people can wake up to reality because I did. I made this video in one take (I was out of breath by the end) when I discovered how wrong I had been. You can tell how bad I was at video editing then and can hear the traffic in the background, but it is a raw and real video that I had put together out of sheer frustration of the ridiculousness I was seeing.

Defender of the Republic 🇺🇸

405,937 görüntüleme • 1 yıl önce

Only a Russian propagandist like Solovyov can start a monologue explaining to the Europeans that Russia has zero interest in capturing Europe and end it with a threat to capture Europe. By the way, did you know that Solovyov’s beloved Italy belongs to Russia? 😂

Only a Russian propagandist like Solovyov can start a monologue explaining to the Europeans that Russia has zero interest in capturing Europe and end it with a threat to capture Europe. By the way, did you know that Solovyov’s beloved Italy belongs to Russia? 😂

Natalka

96,991 görüntüleme • 1 yıl önce