Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

๐—”๐—ด๐—ป๐—ฒ๐˜€ ๐Ÿฎ.๐Ÿฌ ๐—ถ๐˜€ ๐—ณ๐—ฟ๐—ฒ๐—ฒ. ๐—œ๐—ป๐—ฑ๐—ฒ๐—ณ๐—ถ๐—ป๐—ถ๐˜๐—ฒ๐—น๐˜†. ๐—ก๐—ผ ๐˜„๐—ฎ๐—ถ๐˜๐—น๐—ถ๐˜€๐˜. Text. Image. Video. One model series. We built Agnes-2.0 for the developers who got priced out โ€” high token costs, API limits, geography. That ends today. ๐—ช๐—ต๐—ฎ๐˜'๐˜€ ๐—น๐—ถ๐˜ƒ๐—ฒ: โ†’ ๐—”๐—ด๐—ป๐—ฒ๐˜€-๐Ÿฎ.๐Ÿฌ-๐—™๐—น๐—ฎ๐˜€๐—ต โ€” text and agentic, top 10 on Claw-Eval, ahead of Gemini and...

3,522,459 Aufrufe โ€ข vor 1 Monat โ€ขvia X (Twitter)

0 Kommentare

Keine Kommentare verfรผgbar

Kommentare vom Original-Post werden hier angezeigt

ร„hnliche Videos

[CLIP] by Hand โœ๏ธ The CLIP (Contrastive Languageโ€“Image Pre-training) model, a groundbreaking work by OpenAI, redefines the intersection of computer vision and natural language processing. It is the basis of all the multi-modal foundation models we see today. How does CLIP work? Goal: ๐ŸŸจ Learn a shared embedding space for text and image [1] Given โ†ณ A mini batch of 3 text-image pairs โ†ณ OpenAI used 400 million text-image pairs to train its original CLIP model. Process 1st pair: "big table" [2] ๐ŸŸช Text โ†’ 2 Vectors (3D) โ†ณ Look up word embedding vectors using word2vec. [3] ๐ŸŸฉ Image โ†’ 2 Vectors (4D) โ†ณ Divide the image into two patches. โ†ณ Flatten each patch [4] Process other pairs โ†ณ Repeat [2]-[3] [5] ๐ŸŸช Text Encoder & ๐ŸŸฉ Image Encoder โ†ณ Encode input vectors into feature vectors โ†ณ Here, both encoders are simple one layer perceptron (linear + ReLU) โ†ณ In practice, the encoders are usually transformer models. [6] ๐ŸŸช ๐ŸŸฉ Mean Pooling: 2 โ†’ 1 vector โ†ณ Average 2 feature vectors into a single vector by averaging across the columns โ†ณ The goal is to have one vector to represent each image or text [7] ๐ŸŸช ๐ŸŸฉ -> ๐ŸŸจ Projection โ†ณ Note that the text and image feature vectors from the encoders have different dimensions (3D vs. 4D). โ†ณ Use a linear layer to project image and text vectors to a 2D shared embedding space. ๐Ÿ‹๏ธ Contrastive Pre-training ๐Ÿ‹๏ธ [8] Prepare for MatMul โ†ณ Copy text vectors (T1,T2,T3) โ†ณ Copy the transpose of image vectors (I1,I2,I3) โ†ณ They are all in the 2D shared embedding space. [9] ๐ŸŸฆ MatMul โ†ณ Multiply T and I matrices. โ†ณ This is equivalent to taking dot product between every pair of image and text vectors. โ†ณ The purpose is to use dot product to estimate the similarity between a pair of image-text. [10] ๐ŸŸฆ Softmax: e^x โ†ณ Raise e to the power of the number in each cell โ†ณ To simplify hand calculation, we approximate e^โ–ก with 3^โ–ก. [11] ๐ŸŸฆ Softmax: โˆ‘ โ†ณ Sum each row for ๐ŸŸฉ imageโ†’๐ŸŸช text โ†ณ Sum each column for ๐ŸŸช textโ†’ ๐ŸŸฉ image [12] ๐ŸŸฆ Softmax: 1 / sum โ†ณ Divide each element by the column sum to obtain a similarity matrix for ๐ŸŸช textโ†’๐ŸŸฉ image โ†ณ Divide each element by the row sum to obtain a similarity matrix for ๐ŸŸฉ imageโ†’๐ŸŸช text [13] ๐ŸŸฅ Loss Gradients โ†ณ The "Targets" for the similarity matrices are Identity Matrices. โ†ณ Why? If I and T come from the same pair (i=j), we want the highest value, which is 1, and 0 otherwise. โ†ณ Apply the simple equation of [Similarity - Target] to compute gradients of for both directions. โ†ณ Why so simple? Because when Softmax and Cross-Entropy Loss are used together, the math magically works out that way. โ†ณ These gradients kick off the backpropagation process to update weights and biases of the encoders and projection layers (red borders).

Tom Yeh

67,834 Aufrufe โ€ข vor 2 Jahren

๐ŸŽฅ Today weโ€™re premiering Meta Movie Gen: the most advanced media foundation models to-date. Developed by AI research teams at Meta, Movie Gen delivers state-of-the-art results across a range of capabilities. Weโ€™re excited for the potential of this line of research to usher in entirely new possibilities for casual creators and creative professionals alike. More details and examples of what Movie Gen can do โžก๏ธ ๐Ÿ› ๏ธ Movie Gen models and capabilities Movie Gen Video: 30B parameter transformer model that can generate high-quality and high-definition images and videos from a single text prompt. Movie Gen Audio: A 13B parameter transformer model that can take a video input along with optional text prompts for controllability to generate high-fidelity audio synced to the video. It can generate ambient sound, instrumental background music and foley sound โ€” delivering state-of-the-art results in audio quality, video-to-audio alignment and text-to-audio alignment. Precise video editing: Using a generated or existing video and accompanying text instructions as an input it can perform localized edits such as adding, removing or replacing elements โ€” or global changes like background or style changes. Personalized videos: Using an image of a person and a text prompt, the model can generate a video with state-of-the-art results on character preservation and natural movement in video. Weโ€™re continuing to work closely with creative professionals from across the field to integrate their feedback as we work towards a potential release. We look forward to sharing more on this work and the creative possibilities it will enable in the future.

AI at Meta

2,264,846 Aufrufe โ€ข vor 1 Jahr

This is probably the most complex workflow Iโ€™ve ever built, only with open-source tools. It took my 4 days. It takes four inputs: author, title, and style; and generates a full visual animated story in one click in ComfyUI . I worked on it for four days. There are still some bugs, but hereโ€™s the first preview. Hereโ€™s a quick breakdown: - The four inputs are sent to LLMs with precise instructions to generate: first, prompts for images and image modifications; second, prompts for animations; third, prompts for generating music. - All voices are generated from the text and timed precisely, as they determine the length of each animation segment. - The first image and video are generated to serve as the title, but also as the guide for all other images created for the video. - Titles and subtitles are also added automatically in Comfy. - I also developed a lot of custom nodes for minor frame calculations, mostly to match audio and video. - The full system is a large loop that, for each line of text, generates an image and then a video from that image. The loop was the hardest part to build in this workflow, so it can process either a 20-second video or a 2-minute video with the same input. - There are multiple combinations of LLMs that try to understand the text in the best way to provide the best prompts for images and video. - The final video is assembled entirely within ComfyUI. - The music is generated based on the LLM output and matches the exact timing of the full animation. - Done! For reference, this workflow uses a lot of models and only works on an RTX 6000 Pro with plenty of RAM. My goal is not to replace humans, as Iโ€™ll try to explain later, this workflow is highly controlled and can be adapted or reworked at any point by real artists! My aim was to create a tool that can animate text in one go, allowing the AI some freedom while keeping a strict flow. I donโ€™t know yet how Iโ€™ll share this workflow with people, I still need to polish it properly, but maybe through Patreon. Anyway, I hope you enjoy my research, and letโ€™s always keep pushing further! :)

Lovis Odin

58,769 Aufrufe โ€ข vor 10 Monaten

AI has transformed how video is created. We think the next wave is about understanding it. Over the past few years, we've seen remarkable advances in video generation, editing, avatars, and creative tooling. An increasingly important problem is teaching machines to search, analyze, reason over, and extract insight from video - across massive libraries and live streams alike. We're calling this video intelligence, and we're actively looking to back founders building here. We're most excited about companies pushing on the core capabilities: - Video-native models - multimodal embeddings, temporal reasoning, and retrieval built specifically for video rather than adapted from image or text - Real-time and large-scale pipelines - infrastructure for processing, indexing, and querying video at the speed and scale enterprises actually need - Agentic and reasoning layers - systems that don't just retrieve clips but answer questions, surface anomalies, and take action on what they see The models and infrastructure to make this real are appearing to be crossing a capability threshold right now. Multimodal foundation models are maturing, storage costs have collapsed, and enterprises are sitting on years of unstructured video with no way to use it. That infrastructure unlocks a wide range of applications including media and sports workflows, security and physical operations, enterprise knowledge management, advertising analytics, robotics, and consumer products, where video has historically been dark data. If you're building in video intelligence at the model layer, the platform layer, or in a vertical application, we'd love to talk!

Jason Cui

36,205 Aufrufe โ€ข vor 2 Monaten

10 free Google AI tools nobody talks about. while everyone's burning $20/mo on chatgpt and claude, google quietly shipped a stack worth $200+/mo. all free. all yours. โ€” 1๏ธโƒฃ NotebookLM โ€” your second brain upload sources (PDFs, websites, audio, YouTube). it summarizes, builds mind maps, generates quizzes, drafts slide decks, even turns your notes into a podcast you can listen to on a walk. free tier: 100 notebooks, 50 sources each, 50 chats/day, 3 audio overviews/day. replaces: notion AI + perplexity + readwise โ€” 2๏ธโƒฃ Google AI Studio โ€” the free gemini playground web playground for gemini 3 pro and flash with a free API key. generous limits. paste a 1M-token context window and watch it actually use it. faster than the openai playground and free where openai charges per token. replaces: openai playground + paid API credits โ€” 3๏ธโƒฃ Gemini CLI โ€” google's open-source terminal agent apache 2.0 licensed. one command (npx @google/gemini-cli) and you've got an agent in your terminal that reads your codebase, runs shell commands, and ships PRs. drop-in claude code alternative. replaces: claude code ($20/mo by default) โ€” 4๏ธโƒฃ Jules โ€” async coding agent assign jules a github issue. it spins up a cloud VM, clones your repo, writes the plan, makes the changes, opens a PR. free tier: 15 tasks/day, 3 concurrent, runs on gemini flash. replaces: devin ($20/mo+) + cursor agent 5๏ธโƒฃ Stitch โ€” text โ†’ UI โ†’ code google's free figma killer. describe an interface, get production-ready HTML/CSS/Tailwind + figma export. march 2026 update added voice canvas, infinite canvas, and MCP integration with cursor. 350 standard + 200 experimental generations/month free. replaces: galileo AI + early-stage figma work โ€” 6๏ธโƒฃ Gemma 4 โ€” open-weight LLM google's flagship open model. apache 2.0. 2B, 4B, 26B-MoE, and 31B variants. 256K context. runs on ollama with one command. quantized versions run on a 4090 or beefy laptop. replaces: paying for hosted LLM inference โ€” 7๏ธโƒฃ Illuminate โ€” papers โ†’ podcasts paste an arxiv preprint link. illuminate turns dense research papers into a 6-8 min conversation between two AI hosts breaking it down. perfect for commute reading you can't do at a desk. note: still in waitlist for some regions. replaces: snipd + manual research reading โ€” 8๏ธโƒฃ Learn About (LearnLM) โ€” adaptive AI tutor drop in any topic you're stuck on. highlight a word, click "go deeper," and the interface adapts in real time to your comprehension level. visual explanations, follow-up questions, the works. replaces: paid tutoring on niche topics โ€” 9๏ธโƒฃ Google Labs FX (ImageFX + Flow + MusicFX) โ€” free imagen, veo, musicLM google labs creative suite. text-to-image (imagen 4), text-to-video (veo via Flow), text-to-music (musicLM). free tier: limited daily generations. the heavy veo 3.1 features are paid (AI Pro $19.99/mo). still worth using for image and music โ€” those stay free. replaces: midjourney + suno (free tier only โ€” runway-level video gen is paid) โ€” ๐Ÿ”Ÿ Google Colab โ€” free GPU notebooks free T4 GPU + 12GB RAM in a browser tab. enough to fine-tune small models, run stable diffusion, prototype agents. the launching pad for half the ML projects on github. replaces: paid cloud GPU rentals โ€” a quick honest note: these tools aren't 1:1 better than the paid versions they replace. but they're decent enough to get most things done โ€” especially if you're not a heavy user or you've got little funds to play with. i've put all 10 in a public github repo (link in comments). follow + turn on post notifications for more useful posts like this ๐Ÿ””

m0h

11,673 Aufrufe โ€ข vor 1 Monat

Introducing Dola Seed 2.0 Pro, referred to below as Seed 2.0 Pro We have launched Seed 2.0 Pro, our most capable model in the Dola Seed 2.0 series, engineered to power the next generation of autonomous AI agents. Enterprise AI is moving beyond models that simply analyze text or images. What businesses increasingly need are agents that can understand, reason, use tools, and execute tasks across complex workflows. That is exactly what Seed 2.0 Pro is built for. Seed 2.0 Pro combines strong reasoning with advanced image understanding and video understanding, giving enterprise agents the ability not only to interpret information, but also to take action. It is designed for high-value, multi-step enterprise workflows, with strong performance in: - tool calling - workflow execution across enterprise systems - agentic task completion - browser and computer use This makes Seed 2.0 Pro a powerful engine for a wide range of agent scenarios, from daily office automation and deep web research to in-depth report drafting, financial analysis, content moderation, physical inspection, and video creation workflows. It is also highly optimized for OpenClaw๐Ÿฆž and ReAct architectures, helping enterprises build agents that can navigate digital interfaces, enter information, and complete tasks with high reliability. In short, Seed 2.0 Pro is not just built to generate insights. It is built to serve as the brain and execution engine for enterprise AI agents. And it brings these capabilities at a highly attractive price point, making advanced agent deployment more practical for enterprise teams. Try Seed 2.0 Pro for free: Or book a free consultation: #BytePlus #DolaSeed #EnterpriseAI #AIAgents #ImageUnderstanding #VideoUnderstanding #ReasoningModel #ModelArk #openclaw

BytePlus

95,947 Aufrufe โ€ข vor 3 Monaten