Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

๐—”๐—ด๐—ป๐—ฒ๐˜€ ๐Ÿฎ.๐Ÿฌ ๐—ถ๐˜€ ๐—ณ๐—ฟ๐—ฒ๐—ฒ. ๐—œ๐—ป๐—ฑ๐—ฒ๐—ณ๐—ถ๐—ป๐—ถ๐˜๐—ฒ๐—น๐˜†. ๐—ก๐—ผ ๐˜„๐—ฎ๐—ถ๐˜๐—น๐—ถ๐˜€๐˜. Text. Image. Video. One model series. We built Agnes-2.0 for the developers who got priced out โ€” high token costs, API limits, geography. That ends today. ๐—ช๐—ต๐—ฎ๐˜'๐˜€ ๐—น๐—ถ๐˜ƒ๐—ฒ: โ†’ ๐—”๐—ด๐—ป๐—ฒ๐˜€-๐Ÿฎ.๐Ÿฌ-๐—™๐—น๐—ฎ๐˜€๐—ต โ€” text and agentic, top 10 on Claw-Eval, ahead of Gemini and...

agnesai_sapiens's profile picture

Agnes AI

3,696 subscribers

3,526,144 Aufrufe โ€ข vor 3 Monaten โ€ขvia X (Twitter)

Wissenschaft & Technologie
LIVE

Anya Rossiโ€ข Live Now

Private livecam show

0 Kommentare

Keine Kommentare verfรผgbar

Kommentare vom Original-Post werden hier angezeigt

ร„hnliche Videos

๐—”๐—ด๐—ป๐—ฒ๐˜€ ๐Ÿฎ.๐Ÿฌ: AI models ranked among the world's best. One suite for text, image, and video generation. Top 10 across leading AI benchmarks. Trained in Singapore and available now with no waitlist.
2:19

๐—”๐—ด๐—ป๐—ฒ๐˜€ ๐Ÿฎ.๐Ÿฌ: AI models ranked among the world's best. One suite for text, image, and video generation. Top 10 across leading AI benchmarks. Trained in Singapore and available now with no waitlist.

Tech in Asia

4,765,968 Aufrufe โ€ข vor 3 Monaten

SITUATION EXPLAINED: MiniMax released H3 with open weights, and it's already #1 in video editing. โ€ข Priced at $7.80 per minute of 2K video, against Seedance 2.0 at $22.45 and Kling 3.0 at $20.16 โ€ข It ranks #1 on Artificial Analysis for video editing, #2 for text-to-video behind Gemini Omni Flash, and #3 for image-to-video โ€ข H3 takes text, images, video, and audio in a single context and generates 5-15 second clips at native 2K, 24fps, with native stereo audio โ€ข MiniMax is releasing the weights under its Community License within days, which would make it the strongest open-weights video model by a wide margin โ€ข It launched the same week ByteDance rolled out Seedance 2.5, which now generates 30-second 4K video from a single prompt โ€ข Seedance 2.5's face consistency across a full 30 seconds is the visible jump, earlier models drifted into a different face partway through โ€ข Voices are still the obvious tell, the video reads as real long before the audio does โ€ข Smaller labs building purpose-built video models are now beating frontier labs at this specific task Theo Jaffee: "This is gonna be one of the biggest examples of things that people are gonna do with these models, historical simulations, basically. Nostalgia videos for zoomers."
7:17

SITUATION EXPLAINED: MiniMax released H3 with open weights, and it's already #1 in video editing. โ€ข Priced at $7.80 per minute of 2K video, against Seedance 2.0 at $22.45 and Kling 3.0 at $20.16 โ€ข It ranks #1 on Artificial Analysis for video editing, #2 for text-to-video behind Gemini Omni Flash, and #3 for image-to-video โ€ข H3 takes text, images, video, and audio in a single context and generates 5-15 second clips at native 2K, 24fps, with native stereo audio โ€ข MiniMax is releasing the weights under its Community License within days, which would make it the strongest open-weights video model by a wide margin โ€ข It launched the same week ByteDance rolled out Seedance 2.5, which now generates 30-second 4K video from a single prompt โ€ข Seedance 2.5's face consistency across a full 30 seconds is the visible jump, earlier models drifted into a different face partway through โ€ข Voices are still the obvious tell, the video reads as real long before the audio does โ€ข Smaller labs building purpose-built video models are now beating frontier labs at this specific task Theo Jaffee: "This is gonna be one of the biggest examples of things that people are gonna do with these models, historical simulations, basically. Nostalgia videos for zoomers."

MTS

55,187 Aufrufe โ€ข vor 1 Monat

TokenFlow: Consistent Diffusion Features for Consistent Video Editing paper page: The generative AI revolution has recently expanded to videos. Nevertheless, current state-of-the-art video models are still lagging behind image models in terms of visual quality and user control over the generated content. In this work, we present a framework that harnesses the power of a text-to-image diffusion model for the task of text-driven video editing. Specifically, given a source video and a target text-prompt, our method generates a high-quality video that adheres to the target text, while preserving the spatial layout and motion of the input video. Our method is based on a key observation that consistency in the edited video can be obtained by enforcing consistency in the diffusion feature space. We achieve this by explicitly propagating diffusion features based on inter-frame correspondences, readily available in the model. Thus, our framework does not require any training or fine-tuning, and can work in conjunction with any off-the-shelf text-to-image editing method. We demonstrate state-of-the-art editing results on a variety of real-world videos.
0:36

TokenFlow: Consistent Diffusion Features for Consistent Video Editing paper page: The generative AI revolution has recently expanded to videos. Nevertheless, current state-of-the-art video models are still lagging behind image models in terms of visual quality and user control over the generated content. In this work, we present a framework that harnesses the power of a text-to-image diffusion model for the task of text-driven video editing. Specifically, given a source video and a target text-prompt, our method generates a high-quality video that adheres to the target text, while preserving the spatial layout and motion of the input video. Our method is based on a key observation that consistency in the edited video can be obtained by enforcing consistency in the diffusion feature space. We achieve this by explicitly propagating diffusion features based on inter-frame correspondences, readily available in the model. Thus, our framework does not require any training or fine-tuning, and can work in conjunction with any off-the-shelf text-to-image editing method. We demonstrate state-of-the-art editing results on a variety of real-world videos.

AK

143,469 Aufrufe โ€ข vor 3 Jahren

[CLIP] by Hand โœ๏ธ The CLIP (Contrastive Languageโ€“Image Pre-training) model, a groundbreaking work by OpenAI, redefines the intersection of computer vision and natural language processing. It is the basis of all the multi-modal foundation models we see today. How does CLIP work? Goal: ๐ŸŸจ Learn a shared embedding space for text and image [1] Given โ†ณ A mini batch of 3 text-image pairs โ†ณ OpenAI used 400 million text-image pairs to train its original CLIP model. Process 1st pair: "big table" [2] ๐ŸŸช Text โ†’ 2 Vectors (3D) โ†ณ Look up word embedding vectors using word2vec. [3] ๐ŸŸฉ Image โ†’ 2 Vectors (4D) โ†ณ Divide the image into two patches. โ†ณ Flatten each patch [4] Process other pairs โ†ณ Repeat [2]-[3] [5] ๐ŸŸช Text Encoder & ๐ŸŸฉ Image Encoder โ†ณ Encode input vectors into feature vectors โ†ณ Here, both encoders are simple one layer perceptron (linear + ReLU) โ†ณ In practice, the encoders are usually transformer models. [6] ๐ŸŸช ๐ŸŸฉ Mean Pooling: 2 โ†’ 1 vector โ†ณ Average 2 feature vectors into a single vector by averaging across the columns โ†ณ The goal is to have one vector to represent each image or text [7] ๐ŸŸช ๐ŸŸฉ -> ๐ŸŸจ Projection โ†ณ Note that the text and image feature vectors from the encoders have different dimensions (3D vs. 4D). โ†ณ Use a linear layer to project image and text vectors to a 2D shared embedding space. ๐Ÿ‹๏ธ Contrastive Pre-training ๐Ÿ‹๏ธ [8] Prepare for MatMul โ†ณ Copy text vectors (T1,T2,T3) โ†ณ Copy the transpose of image vectors (I1,I2,I3) โ†ณ They are all in the 2D shared embedding space. [9] ๐ŸŸฆ MatMul โ†ณ Multiply T and I matrices. โ†ณ This is equivalent to taking dot product between every pair of image and text vectors. โ†ณ The purpose is to use dot product to estimate the similarity between a pair of image-text. [10] ๐ŸŸฆ Softmax: e^x โ†ณ Raise e to the power of the number in each cell โ†ณ To simplify hand calculation, we approximate e^โ–ก with 3^โ–ก. [11] ๐ŸŸฆ Softmax: โˆ‘ โ†ณ Sum each row for ๐ŸŸฉ imageโ†’๐ŸŸช text โ†ณ Sum each column for ๐ŸŸช textโ†’ ๐ŸŸฉ image [12] ๐ŸŸฆ Softmax: 1 / sum โ†ณ Divide each element by the column sum to obtain a similarity matrix for ๐ŸŸช textโ†’๐ŸŸฉ image โ†ณ Divide each element by the row sum to obtain a similarity matrix for ๐ŸŸฉ imageโ†’๐ŸŸช text [13] ๐ŸŸฅ Loss Gradients โ†ณ The "Targets" for the similarity matrices are Identity Matrices. โ†ณ Why? If I and T come from the same pair (i=j), we want the highest value, which is 1, and 0 otherwise. โ†ณ Apply the simple equation of [Similarity - Target] to compute gradients of for both directions. โ†ณ Why so simple? Because when Softmax and Cross-Entropy Loss are used together, the math magically works out that way. โ†ณ These gradients kick off the backpropagation process to update weights and biases of the encoders and projection layers (red borders).
1:20

[CLIP] by Hand โœ๏ธ The CLIP (Contrastive Languageโ€“Image Pre-training) model, a groundbreaking work by OpenAI, redefines the intersection of computer vision and natural language processing. It is the basis of all the multi-modal foundation models we see today. How does CLIP work? Goal: ๐ŸŸจ Learn a shared embedding space for text and image [1] Given โ†ณ A mini batch of 3 text-image pairs โ†ณ OpenAI used 400 million text-image pairs to train its original CLIP model. Process 1st pair: "big table" [2] ๐ŸŸช Text โ†’ 2 Vectors (3D) โ†ณ Look up word embedding vectors using word2vec. [3] ๐ŸŸฉ Image โ†’ 2 Vectors (4D) โ†ณ Divide the image into two patches. โ†ณ Flatten each patch [4] Process other pairs โ†ณ Repeat [2]-[3] [5] ๐ŸŸช Text Encoder & ๐ŸŸฉ Image Encoder โ†ณ Encode input vectors into feature vectors โ†ณ Here, both encoders are simple one layer perceptron (linear + ReLU) โ†ณ In practice, the encoders are usually transformer models. [6] ๐ŸŸช ๐ŸŸฉ Mean Pooling: 2 โ†’ 1 vector โ†ณ Average 2 feature vectors into a single vector by averaging across the columns โ†ณ The goal is to have one vector to represent each image or text [7] ๐ŸŸช ๐ŸŸฉ -> ๐ŸŸจ Projection โ†ณ Note that the text and image feature vectors from the encoders have different dimensions (3D vs. 4D). โ†ณ Use a linear layer to project image and text vectors to a 2D shared embedding space. ๐Ÿ‹๏ธ Contrastive Pre-training ๐Ÿ‹๏ธ [8] Prepare for MatMul โ†ณ Copy text vectors (T1,T2,T3) โ†ณ Copy the transpose of image vectors (I1,I2,I3) โ†ณ They are all in the 2D shared embedding space. [9] ๐ŸŸฆ MatMul โ†ณ Multiply T and I matrices. โ†ณ This is equivalent to taking dot product between every pair of image and text vectors. โ†ณ The purpose is to use dot product to estimate the similarity between a pair of image-text. [10] ๐ŸŸฆ Softmax: e^x โ†ณ Raise e to the power of the number in each cell โ†ณ To simplify hand calculation, we approximate e^โ–ก with 3^โ–ก. [11] ๐ŸŸฆ Softmax: โˆ‘ โ†ณ Sum each row for ๐ŸŸฉ imageโ†’๐ŸŸช text โ†ณ Sum each column for ๐ŸŸช textโ†’ ๐ŸŸฉ image [12] ๐ŸŸฆ Softmax: 1 / sum โ†ณ Divide each element by the column sum to obtain a similarity matrix for ๐ŸŸช textโ†’๐ŸŸฉ image โ†ณ Divide each element by the row sum to obtain a similarity matrix for ๐ŸŸฉ imageโ†’๐ŸŸช text [13] ๐ŸŸฅ Loss Gradients โ†ณ The "Targets" for the similarity matrices are Identity Matrices. โ†ณ Why? If I and T come from the same pair (i=j), we want the highest value, which is 1, and 0 otherwise. โ†ณ Apply the simple equation of [Similarity - Target] to compute gradients of for both directions. โ†ณ Why so simple? Because when Softmax and Cross-Entropy Loss are used together, the math magically works out that way. โ†ณ These gradients kick off the backpropagation process to update weights and biases of the encoders and projection layers (red borders).

Tom Yeh

67,892 Aufrufe โ€ข vor 2 Jahren

Weโ€™re shipping two major updates to streamline your creative workflow, allowing you to generate high-speed images with one model and then instantly animate them with the otherโ€”all at a fraction of the cost ๐ŸŒโšก๏ธ 1๏ธโƒฃ Introducing Nano Banana 2 Lite: Our fastest and most cost-efficient Gemini Image model yet delivers text-to-image outputs in under 4 seconds. Now available via the Gemini API and Google AI Studio, and rolling out soon across @NotebookLM, Google Flow, Google Gemini, Stitch by Google, Google Search and Google Photos. 2๏ธโƒฃ Gemini Omni Flash in Public Preview: Our natively multimodal model for cost-efficient video generation and conversational editing. Now available via the Gemini API, Google AI Studio, and Gemini Enterprise Agent Platform so you can integrate the model into your workflow. While exciting on their own, the real magic happens when you build using these models together. Watch how our interior design demo integrates Nano Banana 2 Lite and Omni to instantly reimagine any space. Upload a photo, swipe through tailored design concepts, and see Omni bring the details to life in cinematic motion. Try out the demo app in AI Studio:
0:27

Weโ€™re shipping two major updates to streamline your creative workflow, allowing you to generate high-speed images with one model and then instantly animate them with the otherโ€”all at a fraction of the cost ๐ŸŒโšก๏ธ 1๏ธโƒฃ Introducing Nano Banana 2 Lite: Our fastest and most cost-efficient Gemini Image model yet delivers text-to-image outputs in under 4 seconds. Now available via the Gemini API and Google AI Studio, and rolling out soon across @NotebookLM, Google Flow, Google Gemini, Stitch by Google, Google Search and Google Photos. 2๏ธโƒฃ Gemini Omni Flash in Public Preview: Our natively multimodal model for cost-efficient video generation and conversational editing. Now available via the Gemini API, Google AI Studio, and Gemini Enterprise Agent Platform so you can integrate the model into your workflow. While exciting on their own, the real magic happens when you build using these models together. Watch how our interior design demo integrates Nano Banana 2 Lite and Omni to instantly reimagine any space. Upload a photo, swipe through tailored design concepts, and see Omni bring the details to life in cinematic motion. Try out the demo app in AI Studio:

Google AI

129,340 Aufrufe โ€ข vor 2 Monaten

Today we're launching FilmArena [dot] ai. Biggest release we've ever shipped. The problem: a new SOTA AI image & video model drops every week. OpenAI shipped GPT Image 2 literally yesterday. 1000 endpoints, no honest answer to "which one is best right now?" Arena AI and Artificial Analysis cracked LLM benchmarking. But for creative AI, they're solving the wrong problem. There is no "best image-to-video model." Best at what? World building? Action Sequences? Expressions? Every category has a different winner, and it flips weekly. So we benchmarked 40+ SOTA models with 2M+ votes across 190 countries. By subcategory. Re-ranked every week as new models ship. With thousands of real humans. But that's just half of it. Then there's the SANDBOX. Higgsfield and other aggregators put models in one place. But creators still juggle twelve subscriptions to do one job. FilmArena collapses it into one tab. Every frontier model. One interface. Same price. Call the best, or call 3 at once and compare. The AI film stack has been fragmented since day one. Today it isn't. Go try it free: filmarena [dot] ai Already wild results from this week: 1. OpenAI GPT Image 2 (released yesterday) dominates image editing, but Google DeepMind Nano Banana 2 still wins animated 2. C-1 beats Dreamina AI Seedance 2 on animated 3. Veo 3.1 Fast outperforms Veo 3.1 on realistic video And a lot more. Go check it out.
1:13

Today we're launching FilmArena [dot] ai. Biggest release we've ever shipped. The problem: a new SOTA AI image & video model drops every week. OpenAI shipped GPT Image 2 literally yesterday. 1000 endpoints, no honest answer to "which one is best right now?" Arena AI and Artificial Analysis cracked LLM benchmarking. But for creative AI, they're solving the wrong problem. There is no "best image-to-video model." Best at what? World building? Action Sequences? Expressions? Every category has a different winner, and it flips weekly. So we benchmarked 40+ SOTA models with 2M+ votes across 190 countries. By subcategory. Re-ranked every week as new models ship. With thousands of real humans. But that's just half of it. Then there's the SANDBOX. Higgsfield and other aggregators put models in one place. But creators still juggle twelve subscriptions to do one job. FilmArena collapses it into one tab. Every frontier model. One interface. Same price. Call the best, or call 3 at once and compare. The AI film stack has been fragmented since day one. Today it isn't. Go try it free: filmarena [dot] ai Already wild results from this week: 1. OpenAI GPT Image 2 (released yesterday) dominates image editing, but Google DeepMind Nano Banana 2 still wins animated 2. C-1 beats Dreamina AI Seedance 2 on animated 3. Veo 3.1 Fast outperforms Veo 3.1 on realistic video And a lot more. Go check it out.

Mehul Agarwal

51,701 Aufrufe โ€ข vor 4 Monaten

CLIP by hand โœ๏ธ ~ 13 steps walkthrough below CLIP, Contrastive Language-Image Pre-training, is OpenAI's answer to a question that sounds impossible: how do you put a sentence and a picture in the same space? CLIP shipped when OpenAI was still open, and those embeddings were shared far and wide. Almost every multimodal model you use today descends from them. How does it work? Goal: learn one shared embedding space for text and images. = 1. Given = A mini batch of three text-image pairs. OpenAI trained the original on 400 million. = 2. Text to vectors = Let us look up each word with word2vec. = 3. Image to vectors = We cut each image into two patches and flatten them. Now text and pixels are both just numbers. = 4. The other pairs = Repeat steps 2 and 3 for the rest of the batch. = 5. Encode = Let us push both sides through their encoders, a linear layer and a ReLU. In practice these are transformers, but the shape of the operation is the same. = 6. Mean pooling = We average across the columns, so each image and each sentence collapses to a single vector. = 7. Projection = The text vectors are 3D and the image vectors are 4D, so they cannot be compared at all. A linear layer projects both to 2D. That 2D space is the shared embedding space, and getting here is the whole point of the model. = 8. Prepare for matmul = Let us copy the text vectors down and the transposed image vectors across. = 9. MatMul = We multiply, which takes the dot product of every text vector with every image vector. Each cell is one estimate of how well a sentence matches a picture. = 10. Softmax, e to the power = Raise e to each cell. To keep it hand sized we approximate e with 3. = 11. Softmax, sum = Sum each row for image to text, each column for text to image. = 12. Softmax, normalize = Divide, and out come two similarity matrices, one per direction. = 13. Loss gradients = The targets are identity matrices: a pair that belongs together should score 1, every other cell 0. Subtract the target from the similarity and you have the gradients, in both directions. The takeaway: pairing a picture with a sentence comes down to a single dot product. Everything before step 9 is the work of getting them into one shared space, so that the dot product finally means something. ๐Ÿ’พ Save this post!
0:36

CLIP by hand โœ๏ธ ~ 13 steps walkthrough below CLIP, Contrastive Language-Image Pre-training, is OpenAI's answer to a question that sounds impossible: how do you put a sentence and a picture in the same space? CLIP shipped when OpenAI was still open, and those embeddings were shared far and wide. Almost every multimodal model you use today descends from them. How does it work? Goal: learn one shared embedding space for text and images. = 1. Given = A mini batch of three text-image pairs. OpenAI trained the original on 400 million. = 2. Text to vectors = Let us look up each word with word2vec. = 3. Image to vectors = We cut each image into two patches and flatten them. Now text and pixels are both just numbers. = 4. The other pairs = Repeat steps 2 and 3 for the rest of the batch. = 5. Encode = Let us push both sides through their encoders, a linear layer and a ReLU. In practice these are transformers, but the shape of the operation is the same. = 6. Mean pooling = We average across the columns, so each image and each sentence collapses to a single vector. = 7. Projection = The text vectors are 3D and the image vectors are 4D, so they cannot be compared at all. A linear layer projects both to 2D. That 2D space is the shared embedding space, and getting here is the whole point of the model. = 8. Prepare for matmul = Let us copy the text vectors down and the transposed image vectors across. = 9. MatMul = We multiply, which takes the dot product of every text vector with every image vector. Each cell is one estimate of how well a sentence matches a picture. = 10. Softmax, e to the power = Raise e to each cell. To keep it hand sized we approximate e with 3. = 11. Softmax, sum = Sum each row for image to text, each column for text to image. = 12. Softmax, normalize = Divide, and out come two similarity matrices, one per direction. = 13. Loss gradients = The targets are identity matrices: a pair that belongs together should score 1, every other cell 0. Subtract the target from the similarity and you have the gradients, in both directions. The takeaway: pairing a picture with a sentence comes down to a single dot product. Everything before step 9 is the work of getting them into one shared space, so that the dot product finally means something. ๐Ÿ’พ Save this post!

Tom Yeh

20,750 Aufrufe โ€ข vor 1 Monat

Voicebox: Text-Guided Multilingual Universal Speech Generation at Scale blog: Large-scale generative models such as GPT and DALL-E have revolutionized natural language processing and computer vision research. These models not only generate high fidelity text or image outputs, but are also generalists which can solve tasks not explicitly taught. In contrast, speech generative models are still primitive in terms of scale and task generalization. In this paper, we present Voicebox, the most versatile text-guided generative model for speech at scale. Voicebox is a non-autoregressive flow-matching model trained to infill speech, given audio context and text, trained on over 50K hours of speech that are neither filtered nor enhanced. Similar to GPT, Voicebox can perform many different tasks through in-context learning, but is more flexible as it can also condition on future context. Voicebox can be used for mono or cross-lingual zero-shot text-to-speech synthesis, noise removal, content editing, style conversion, and diverse sample generation. In particular, Voicebox outperforms the state-of-the-art zero-shot TTS model VALL-E on both intelligibility (5.9% vs 1.9% word error rates) and audio similarity (0.580 vs 0.681) while being up to 20 times faster.
2:43

Voicebox: Text-Guided Multilingual Universal Speech Generation at Scale blog: Large-scale generative models such as GPT and DALL-E have revolutionized natural language processing and computer vision research. These models not only generate high fidelity text or image outputs, but are also generalists which can solve tasks not explicitly taught. In contrast, speech generative models are still primitive in terms of scale and task generalization. In this paper, we present Voicebox, the most versatile text-guided generative model for speech at scale. Voicebox is a non-autoregressive flow-matching model trained to infill speech, given audio context and text, trained on over 50K hours of speech that are neither filtered nor enhanced. Similar to GPT, Voicebox can perform many different tasks through in-context learning, but is more flexible as it can also condition on future context. Voicebox can be used for mono or cross-lingual zero-shot text-to-speech synthesis, noise removal, content editing, style conversion, and diverse sample generation. In particular, Voicebox outperforms the state-of-the-art zero-shot TTS model VALL-E on both intelligibility (5.9% vs 1.9% word error rates) and audio similarity (0.580 vs 0.681) while being up to 20 times faster.

AK

429,355 Aufrufe โ€ข vor 3 Jahren

๐ŸŽฅ Today weโ€™re premiering Meta Movie Gen: the most advanced media foundation models to-date. Developed by AI research teams at Meta, Movie Gen delivers state-of-the-art results across a range of capabilities. Weโ€™re excited for the potential of this line of research to usher in entirely new possibilities for casual creators and creative professionals alike. More details and examples of what Movie Gen can do โžก๏ธ ๐Ÿ› ๏ธ Movie Gen models and capabilities Movie Gen Video: 30B parameter transformer model that can generate high-quality and high-definition images and videos from a single text prompt. Movie Gen Audio: A 13B parameter transformer model that can take a video input along with optional text prompts for controllability to generate high-fidelity audio synced to the video. It can generate ambient sound, instrumental background music and foley sound โ€” delivering state-of-the-art results in audio quality, video-to-audio alignment and text-to-audio alignment. Precise video editing: Using a generated or existing video and accompanying text instructions as an input it can perform localized edits such as adding, removing or replacing elements โ€” or global changes like background or style changes. Personalized videos: Using an image of a person and a text prompt, the model can generate a video with state-of-the-art results on character preservation and natural movement in video. Weโ€™re continuing to work closely with creative professionals from across the field to integrate their feedback as we work towards a potential release. We look forward to sharing more on this work and the creative possibilities it will enable in the future.
0:30

๐ŸŽฅ Today weโ€™re premiering Meta Movie Gen: the most advanced media foundation models to-date. Developed by AI research teams at Meta, Movie Gen delivers state-of-the-art results across a range of capabilities. Weโ€™re excited for the potential of this line of research to usher in entirely new possibilities for casual creators and creative professionals alike. More details and examples of what Movie Gen can do โžก๏ธ ๐Ÿ› ๏ธ Movie Gen models and capabilities Movie Gen Video: 30B parameter transformer model that can generate high-quality and high-definition images and videos from a single text prompt. Movie Gen Audio: A 13B parameter transformer model that can take a video input along with optional text prompts for controllability to generate high-fidelity audio synced to the video. It can generate ambient sound, instrumental background music and foley sound โ€” delivering state-of-the-art results in audio quality, video-to-audio alignment and text-to-audio alignment. Precise video editing: Using a generated or existing video and accompanying text instructions as an input it can perform localized edits such as adding, removing or replacing elements โ€” or global changes like background or style changes. Personalized videos: Using an image of a person and a text prompt, the model can generate a video with state-of-the-art results on character preservation and natural movement in video. Weโ€™re continuing to work closely with creative professionals from across the field to integrate their feedback as we work towards a potential release. We look forward to sharing more on this work and the creative possibilities it will enable in the future.

AI at Meta

2,266,357 Aufrufe โ€ข vor 1 Jahr

WHY IS NO ONE TALKING ABOUT THIS?? Gemma 3n model was one of the best surprises for me. The fact that you can run it on edge devices even with just 2GB of RAM is impressive. A few weeks back, I was on holiday and used the Gemini Live feature a lot. But I kept running into issues whenever I was in a place where the network wasnโ€™t reliable. Gemma 3n : > Multimodal: Supports text and image inputs; video/audio in development. > Context Window: Up to 128K tokens (32K for 1B model). > Multilingual: Trained on 140+ languages. > Privacy: Offline, on-device processing for data security. > Model Sizes: E2B (5B parameters, 2GB RAM) and E4B (10โ€“12B parameters, 3GB RAM). Video credit: Google YT
1:34

WHY IS NO ONE TALKING ABOUT THIS?? Gemma 3n model was one of the best surprises for me. The fact that you can run it on edge devices even with just 2GB of RAM is impressive. A few weeks back, I was on holiday and used the Gemini Live feature a lot. But I kept running into issues whenever I was in a place where the network wasnโ€™t reliable. Gemma 3n : > Multimodal: Supports text and image inputs; video/audio in development. > Context Window: Up to 128K tokens (32K for 1B model). > Multilingual: Trained on 140+ languages. > Privacy: Offline, on-device processing for data security. > Model Sizes: E2B (5B parameters, 2GB RAM) and E4B (10โ€“12B parameters, 3GB RAM). Video credit: Google YT

AshutoshShrivastava

210,641 Aufrufe โ€ข vor 1 Jahr

Happy GPT-5 Day to all who celebrate. Here's what I expect: It will be agentic-first (this is all but confirmed by OpenAI) It will be heavily multimodal (audio, video, music, tool use, and more) It will be a reasoning-first model as well (math, planning, etc) Directionally, I suspect we're looking at moving towards a convergence of "everything to everything" end-to-end models. Sort of like how Tesla improved FSD by doing neural networks for end-to-end perception-planning-control. Likewise, I suspect that, for the foreseeable future on general purpose models, we're going to converge towards "all in one" models. They will just tend to get smarter and smarter the more modalities we stuff into them (world models, math, image, text, video, audio, etc).
9:02

Happy GPT-5 Day to all who celebrate. Here's what I expect: It will be agentic-first (this is all but confirmed by OpenAI) It will be heavily multimodal (audio, video, music, tool use, and more) It will be a reasoning-first model as well (math, planning, etc) Directionally, I suspect we're looking at moving towards a convergence of "everything to everything" end-to-end models. Sort of like how Tesla improved FSD by doing neural networks for end-to-end perception-planning-control. Likewise, I suspect that, for the foreseeable future on general purpose models, we're going to converge towards "all in one" models. They will just tend to get smarter and smarter the more modalities we stuff into them (world models, math, image, text, video, audio, etc).

David Shapiro (L/0)

19,307 Aufrufe โ€ข vor 1 Jahr

Are open source models catching up to proprietary models? Weโ€™ve looked back at 3 years of Arenaโ€™s data to show how the race has evolved. For comparison, weโ€™ve taken the top 20% of the models and uncovered the following: - Before mid 2024: The gap was between 100-150 points - In the second half of 2024, the gap rapidly narrowed to ~50 points - This was driven by the fact that proprietary models have not significantly improved between early 2023 and November 2024 - hovering around the 1350 score for the top 20% of proprietary models - Then since Jan 2025, we saw a parallel improvement in both proprietary and open source models - Meaning that for the last ~14 months the gap stayed steady at around 50-60 points For context, 50 points is the gap between the 1st and 20th place in the Overall leaderboard Today, proprietary models take up the first 20 spots on the Text Arena, with Anthropic, Google DeepMind , xAI and OpenAI leading in the rankings. The top open source models are ranked #20 (GLM-5 by Z.ai ), #23 (Kimi-K2.5-Thinking by Kimi.ai ) and #27 (Qwen3.5-397b-a17b by Qwen ).
0:21

Are open source models catching up to proprietary models? Weโ€™ve looked back at 3 years of Arenaโ€™s data to show how the race has evolved. For comparison, weโ€™ve taken the top 20% of the models and uncovered the following: - Before mid 2024: The gap was between 100-150 points - In the second half of 2024, the gap rapidly narrowed to ~50 points - This was driven by the fact that proprietary models have not significantly improved between early 2023 and November 2024 - hovering around the 1350 score for the top 20% of proprietary models - Then since Jan 2025, we saw a parallel improvement in both proprietary and open source models - Meaning that for the last ~14 months the gap stayed steady at around 50-60 points For context, 50 points is the gap between the 1st and 20th place in the Overall leaderboard Today, proprietary models take up the first 20 spots on the Text Arena, with Anthropic, Google DeepMind , xAI and OpenAI leading in the rankings. The top open source models are ranked #20 (GLM-5 by Z.ai ), #23 (Kimi-K2.5-Thinking by Kimi.ai ) and #27 (Qwen3.5-397b-a17b by Qwen ).

Arena.ai

22,447 Aufrufe โ€ข vor 5 Monaten

Black Forest Labs just announced FLUX 3: a unified multimodal model for image, video, audio and action prediction. Founded in Freiburg, Germany, one of the few globally significant European companies in the AI โ€‹โ€‹sector. The current release is gated early access, not open source. But if FLUX 3 Dev actually ships with usable open weights, this could become one of the most important open-model releases in generative video yet. Especially after the Chinese Minimax H3 release. BFL claims, based on preliminary internal comparisons, that FLUX 3 outperformed models such as Runway Gen-4.5, Luma Ray 3.2, Kling 3 Pro, and Seedance 2.0. However, these figures stem from the early-access phase and do not constitute independent validation. Be that as it may, it's great to see the quality that today's video models can produce at affordable prices. Something that would have been unthinkable a year ago. Really cool release!
0:10

Black Forest Labs just announced FLUX 3: a unified multimodal model for image, video, audio and action prediction. Founded in Freiburg, Germany, one of the few globally significant European companies in the AI โ€‹โ€‹sector. The current release is gated early access, not open source. But if FLUX 3 Dev actually ships with usable open weights, this could become one of the most important open-model releases in generative video yet. Especially after the Chinese Minimax H3 release. BFL claims, based on preliminary internal comparisons, that FLUX 3 outperformed models such as Runway Gen-4.5, Luma Ray 3.2, Kling 3 Pro, and Seedance 2.0. However, these figures stem from the early-access phase and do not constitute independent validation. Be that as it may, it's great to see the quality that today's video models can produce at affordable prices. Something that would have been unthinkable a year ago. Really cool release!

Chubbyโ™จ๏ธ

39,869 Aufrufe โ€ข vor 1 Monat

๐Ÿš€ Introducing the Kling 3.0 Model: Everyone a Director. Itโ€™s Time. An all-in-one creative engine that enables truly native multimodal creation. - Superb Consistency: Your characters and elements, always locked in. - Flexible Video Production: Create 15s clips with precise control, excellent video realism and customizable multi-shots. - Upgraded Native Audio: Now supports multi-character reference, more languages and accents. - Enhanced Image Generation: 4K Image output, new image series mode and more cinematic visuals. Ultra subscribers get exclusive early access โ€” now live on Web -> Kling AI. Get ready to unlock your creative potential with our most advanced model yet. [For the next 24 hours ONLY] Follow, Comment & Retweet to win exclusive early access - we will select 10 winners in this post from the top ten 'Most liked replies'. So which feature are you most excited about? Let us know in the comment! We will contact you via your DM.
1:54

๐Ÿš€ Introducing the Kling 3.0 Model: Everyone a Director. Itโ€™s Time. An all-in-one creative engine that enables truly native multimodal creation. - Superb Consistency: Your characters and elements, always locked in. - Flexible Video Production: Create 15s clips with precise control, excellent video realism and customizable multi-shots. - Upgraded Native Audio: Now supports multi-character reference, more languages and accents. - Enhanced Image Generation: 4K Image output, new image series mode and more cinematic visuals. Ultra subscribers get exclusive early access โ€” now live on Web -> Kling AI. Get ready to unlock your creative potential with our most advanced model yet. [For the next 24 hours ONLY] Follow, Comment & Retweet to win exclusive early access - we will select 10 winners in this post from the top ten 'Most liked replies'. So which feature are you most excited about? Let us know in the comment! We will contact you via your DM.

Kling AI

1,196,898 Aufrufe โ€ข vor 7 Monaten

This is probably the most complex workflow Iโ€™ve ever built, only with open-source tools. It took my 4 days. It takes four inputs: author, title, and style; and generates a full visual animated story in one click in ComfyUI . I worked on it for four days. There are still some bugs, but hereโ€™s the first preview. Hereโ€™s a quick breakdown: - The four inputs are sent to LLMs with precise instructions to generate: first, prompts for images and image modifications; second, prompts for animations; third, prompts for generating music. - All voices are generated from the text and timed precisely, as they determine the length of each animation segment. - The first image and video are generated to serve as the title, but also as the guide for all other images created for the video. - Titles and subtitles are also added automatically in Comfy. - I also developed a lot of custom nodes for minor frame calculations, mostly to match audio and video. - The full system is a large loop that, for each line of text, generates an image and then a video from that image. The loop was the hardest part to build in this workflow, so it can process either a 20-second video or a 2-minute video with the same input. - There are multiple combinations of LLMs that try to understand the text in the best way to provide the best prompts for images and video. - The final video is assembled entirely within ComfyUI. - The music is generated based on the LLM output and matches the exact timing of the full animation. - Done! For reference, this workflow uses a lot of models and only works on an RTX 6000 Pro with plenty of RAM. My goal is not to replace humans, as Iโ€™ll try to explain later, this workflow is highly controlled and can be adapted or reworked at any point by real artists! My aim was to create a tool that can animate text in one go, allowing the AI some freedom while keeping a strict flow. I donโ€™t know yet how Iโ€™ll share this workflow with people, I still need to polish it properly, but maybe through Patreon. Anyway, I hope you enjoy my research, and letโ€™s always keep pushing further! :)
0:55

This is probably the most complex workflow Iโ€™ve ever built, only with open-source tools. It took my 4 days. It takes four inputs: author, title, and style; and generates a full visual animated story in one click in ComfyUI . I worked on it for four days. There are still some bugs, but hereโ€™s the first preview. Hereโ€™s a quick breakdown: - The four inputs are sent to LLMs with precise instructions to generate: first, prompts for images and image modifications; second, prompts for animations; third, prompts for generating music. - All voices are generated from the text and timed precisely, as they determine the length of each animation segment. - The first image and video are generated to serve as the title, but also as the guide for all other images created for the video. - Titles and subtitles are also added automatically in Comfy. - I also developed a lot of custom nodes for minor frame calculations, mostly to match audio and video. - The full system is a large loop that, for each line of text, generates an image and then a video from that image. The loop was the hardest part to build in this workflow, so it can process either a 20-second video or a 2-minute video with the same input. - There are multiple combinations of LLMs that try to understand the text in the best way to provide the best prompts for images and video. - The final video is assembled entirely within ComfyUI. - The music is generated based on the LLM output and matches the exact timing of the full animation. - Done! For reference, this workflow uses a lot of models and only works on an RTX 6000 Pro with plenty of RAM. My goal is not to replace humans, as Iโ€™ll try to explain later, this workflow is highly controlled and can be adapted or reworked at any point by real artists! My aim was to create a tool that can animate text in one go, allowing the AI some freedom while keeping a strict flow. I donโ€™t know yet how Iโ€™ll share this workflow with people, I still need to polish it properly, but maybe through Patreon. Anyway, I hope you enjoy my research, and letโ€™s always keep pushing further! :)

Lovis Odin

58,841 Aufrufe โ€ข vor 11 Monaten

AI has transformed how video is created. We think the next wave is about understanding it. Over the past few years, we've seen remarkable advances in video generation, editing, avatars, and creative tooling. An increasingly important problem is teaching machines to search, analyze, reason over, and extract insight from video - across massive libraries and live streams alike. We're calling this video intelligence, and we're actively looking to back founders building here. We're most excited about companies pushing on the core capabilities: - Video-native models - multimodal embeddings, temporal reasoning, and retrieval built specifically for video rather than adapted from image or text - Real-time and large-scale pipelines - infrastructure for processing, indexing, and querying video at the speed and scale enterprises actually need - Agentic and reasoning layers - systems that don't just retrieve clips but answer questions, surface anomalies, and take action on what they see The models and infrastructure to make this real are appearing to be crossing a capability threshold right now. Multimodal foundation models are maturing, storage costs have collapsed, and enterprises are sitting on years of unstructured video with no way to use it. That infrastructure unlocks a wide range of applications including media and sports workflows, security and physical operations, enterprise knowledge management, advertising analytics, robotics, and consumer products, where video has historically been dark data. If you're building in video intelligence at the model layer, the platform layer, or in a vertical application, we'd love to talk!
1:41

AI has transformed how video is created. We think the next wave is about understanding it. Over the past few years, we've seen remarkable advances in video generation, editing, avatars, and creative tooling. An increasingly important problem is teaching machines to search, analyze, reason over, and extract insight from video - across massive libraries and live streams alike. We're calling this video intelligence, and we're actively looking to back founders building here. We're most excited about companies pushing on the core capabilities: - Video-native models - multimodal embeddings, temporal reasoning, and retrieval built specifically for video rather than adapted from image or text - Real-time and large-scale pipelines - infrastructure for processing, indexing, and querying video at the speed and scale enterprises actually need - Agentic and reasoning layers - systems that don't just retrieve clips but answer questions, surface anomalies, and take action on what they see The models and infrastructure to make this real are appearing to be crossing a capability threshold right now. Multimodal foundation models are maturing, storage costs have collapsed, and enterprises are sitting on years of unstructured video with no way to use it. That infrastructure unlocks a wide range of applications including media and sports workflows, security and physical operations, enterprise knowledge management, advertising analytics, robotics, and consumer products, where video has historically been dark data. If you're building in video intelligence at the model layer, the platform layer, or in a vertical application, we'd love to talk!

Jason Cui

36,205 Aufrufe โ€ข vor 4 Monaten

Chinese AI models are wiping billions off Big Tech right now. Google just lost $200 billion in a single day, and the model it needed to fight back still isn't ready. Gemini 3.5 Pro, Google's most powerful model, is months behind schedule. Alphabet stock dropped 4.4% that same day. The Deepseek moment is happening again, and the new model is FAR bigger. On the same day Google's delay leaked, a Beijing lab called Moonshot released Kimi K3. It is the largest open model ever built, with 2.8 trillion parameters. It took the number one spot on the Frontend Code Arena, a live coding leaderboard, passing Anthropic's best model. And Moonshot is giving it away for free on July 27. The genius part: Anyone with enough computers can download it and run a frontier level AI without paying a cent to a US company. A single task on Kimi K3 costs about 94 cents. The same work on some American models costs nearly double. So why would a company keep paying premium prices for a model it can now get for free? The entire US AI business is built on selling access to models that cost billions to train. If a free Chinese version does most of the same work, that pricing power starts to crack. And Kimi is close to the best. On one closely watched intelligence ranking it scored 57, just behind the top American models GPT-5.6 Sol and Fable 5, and ahead of Claude Opus 4.8. Bank of America told clients that Kimi proves Chinese labs can keep making big leaps even with limited chips. And the founder of Moonshot, Yang Zhilin, learned to build AI as a researcher INSIDE Google. Google literally wrote the 2017 paper that made all of these models possible. Now the people who studied its work are using it to destroy Google, and handing it out for free. What happens next: Kimi K3's weights go public on July 27. Google reports earnings on July 22, and everyone will be asking the same question about Gemini. If free models keep topping the charts, every valuation built on paid AI access has to be rewritten. What do you think?
1:10

Chinese AI models are wiping billions off Big Tech right now. Google just lost $200 billion in a single day, and the model it needed to fight back still isn't ready. Gemini 3.5 Pro, Google's most powerful model, is months behind schedule. Alphabet stock dropped 4.4% that same day. The Deepseek moment is happening again, and the new model is FAR bigger. On the same day Google's delay leaked, a Beijing lab called Moonshot released Kimi K3. It is the largest open model ever built, with 2.8 trillion parameters. It took the number one spot on the Frontend Code Arena, a live coding leaderboard, passing Anthropic's best model. And Moonshot is giving it away for free on July 27. The genius part: Anyone with enough computers can download it and run a frontier level AI without paying a cent to a US company. A single task on Kimi K3 costs about 94 cents. The same work on some American models costs nearly double. So why would a company keep paying premium prices for a model it can now get for free? The entire US AI business is built on selling access to models that cost billions to train. If a free Chinese version does most of the same work, that pricing power starts to crack. And Kimi is close to the best. On one closely watched intelligence ranking it scored 57, just behind the top American models GPT-5.6 Sol and Fable 5, and ahead of Claude Opus 4.8. Bank of America told clients that Kimi proves Chinese labs can keep making big leaps even with limited chips. And the founder of Moonshot, Yang Zhilin, learned to build AI as a researcher INSIDE Google. Google literally wrote the 2017 paper that made all of these models possible. Now the people who studied its work are using it to destroy Google, and handing it out for free. What happens next: Kimi K3's weights go public on July 27. Google reports earnings on July 22, and everyone will be asking the same question about Gemini. If free models keep topping the charts, every valuation built on paid AI access has to be rewritten. What do you think?

Ricardo

47,790 Aufrufe โ€ข vor 1 Monat

Just how capable are open source models? Below is the first in a new series where we go behind the scenes and pull back the curtain on interesting AI research / demos, making them fun and easy to understand. Here, we have a short visual demonstration from aizk ๎จ€โœก๏ธ showcasing how Kimi K3 (a language model that operates primarily through text) is capable of building complicated 3D structures / moments in history in Minecraft, something that previously was not possible with other open source models, and why this matters. The crazy part? The model doesn't "see" the game like we do. The LLMs must reason in pure text, writing JavaScript, that later compiles down into commands placing each block, one at a time. Spatial reasoning is a very hard problem in AI, it's the same core challenge behind robotics and self-driving cars, where a model has to understand and act in physical 3D space. Watching a text model pull it off is nothing short of a miracle. The point isn't just Minecraft itself, rather, it's AI being able to generalize, not memorize, on things that are weird and beyond their training data. This is key to building true artificial general intelligence. These video game benchmarks (there are many different games actively being researched right now) provide a clear-cut end goal, challenges that are almost certainly not in the training set, and a fun, very fast, visual way to almost feel the increasing capabilities of various open source AI models over time. If you haven't given open source models a serious try yet, watch the video, it may shock you!
2:46

Just how capable are open source models? Below is the first in a new series where we go behind the scenes and pull back the curtain on interesting AI research / demos, making them fun and easy to understand. Here, we have a short visual demonstration from aizk ๎จ€โœก๏ธ showcasing how Kimi K3 (a language model that operates primarily through text) is capable of building complicated 3D structures / moments in history in Minecraft, something that previously was not possible with other open source models, and why this matters. The crazy part? The model doesn't "see" the game like we do. The LLMs must reason in pure text, writing JavaScript, that later compiles down into commands placing each block, one at a time. Spatial reasoning is a very hard problem in AI, it's the same core challenge behind robotics and self-driving cars, where a model has to understand and act in physical 3D space. Watching a text model pull it off is nothing short of a miracle. The point isn't just Minecraft itself, rather, it's AI being able to generalize, not memorize, on things that are weird and beyond their training data. This is key to building true artificial general intelligence. These video game benchmarks (there are many different games actively being researched right now) provide a clear-cut end goal, challenges that are almost certainly not in the training set, and a fun, very fast, visual way to almost feel the increasing capabilities of various open source AI models over time. If you haven't given open source models a serious try yet, watch the video, it may shock you!

Featherless AI

39,769 Aufrufe โ€ข vor 23 Tagen

OpenAI chairman Bret Taylor talks to about 100 CEOs every month. His answer to the cheap open-weight model panic: cheaper to train does not mean cheaper to use, and the number that decides it is token efficiency. "One thing that I think is a little bit overblown about these open weight models is they're not necessarily cheaper to run. Whether or not they're cheaper to train, you don't care. Because you're using just as many tokens. In fact, they may be less efficient." "There's this thing called token efficiency. And it turns out the frontier models are much, much more token efficient." "A token is to intelligence like a watt is to electricity... how many tokens does it take to complete a task? Not every token is actually equal." "For a lot of tasks, it turns out these frontier models from OpenAI and Anthropic are actually just better than these open weight models... just having open weights isn't actually the main thing driving any of those costs." Later in the same interview he goes after the billing unit itself: "It would be like if you signed up for Gmail and you paid for CPU cycle or something... where the world is going is paying for outcomes." The unresolved column: the chart CNBC airs mid-answer, from Artificial Analysis, prices a completed task at $0.94 on Kimi K3 against $2.75 on Claude Fable 5, efficiency folded in. If that gap holds, the premium he is defending gets earned on quality, not price. - Bret Taylor (Bret Taylor), OpenAI chairman and Sierra co-founder, on CNBC's Squawk Box.
2:09

OpenAI chairman Bret Taylor talks to about 100 CEOs every month. His answer to the cheap open-weight model panic: cheaper to train does not mean cheaper to use, and the number that decides it is token efficiency. "One thing that I think is a little bit overblown about these open weight models is they're not necessarily cheaper to run. Whether or not they're cheaper to train, you don't care. Because you're using just as many tokens. In fact, they may be less efficient." "There's this thing called token efficiency. And it turns out the frontier models are much, much more token efficient." "A token is to intelligence like a watt is to electricity... how many tokens does it take to complete a task? Not every token is actually equal." "For a lot of tasks, it turns out these frontier models from OpenAI and Anthropic are actually just better than these open weight models... just having open weights isn't actually the main thing driving any of those costs." Later in the same interview he goes after the billing unit itself: "It would be like if you signed up for Gmail and you paid for CPU cycle or something... where the world is going is paying for outcomes." The unresolved column: the chart CNBC airs mid-answer, from Artificial Analysis, prices a completed task at $0.94 on Kimi K3 against $2.75 on Claude Fable 5, efficiency folded in. If that gap holds, the premium he is defending gets earned on quality, not price. - Bret Taylor (Bret Taylor), OpenAI chairman and Sierra co-founder, on CNBC's Squawk Box.

Karl Mehta

17,254 Aufrufe โ€ข vor 1 Monat