Загрузка видео...

Не удалось загрузить видео

На главную

Introducing StoryMem — a memory-augmented framework for multi-shot long video storytelling. StoryMem carefully injects compact memory into the generation process with minimal overhead, enabling: • Cross-shot consistency • Smooth transitions • Narrative coherence across minutes-long videos Awesome work by Kaiwen Kaiwen Zhang (Kevin) Project: arXiv: Code:

16,522 просмотров • 7 месяцев назад •via X (Twitter)

Комментарии: 0

Нет доступных комментариев

Здесь появятся комментарии из оригинального поста

Похожие видео

HERMES AGENT CAN SHARE MEMORY WITH CODEX AND CLAUDE CODE THROUGH HINDSIGHT. ONE MEMORY BANK. ONE AGENT REMEMBERS, EVERY OTHER AGENT KNOWS. the problem: you use Hermes for orchestration. Codex for coding. Claude Code for debugging. each has its own memory. switch between them and you explain the same project three times. Hindsight fixes this. one shared memory bank that every agent reads and writes to. tell Codex: "the test color for this project is purple." switch to Hermes. ask: "what test color did I pick?" Hermes answers: "purple." no copy-paste. no re-explaining. instant recall. HOW IT WORKS: Hindsight runs as a Docker container on your machine. self-hosted. your data stays local. an LLM powers the memory processing (retain, recall, reflect). RETAIN: extracts facts from your conversations. entities, decisions, preferences, project context. saved to the memory bank automatically. RECALL: when you ask a question, Hindsight pulls from semantic search, keywords, graph connections, and temporal data. fused into one answer. REFLECT: deeper reasoning layer. connects memories across sessions. identifies patterns in your work. produces observations that get smarter over time. CONNECT TO HERMES: Desktop app: Settings → Memory and Context → switch provider from Namosin to Hindsight. set API URL to your local Docker container. set bank ID. done. CLI: hermes memory setup → select Hindsight. verify: hermes memory status should show: provider: hindsight, installed, available. CONNECT TO CODEX: npx hindsight-coding-agents install codex \ --self-hosted --server this installs lifecycle hooks: initialize memory on session start. recall context during work. retain the session when done. enable hooks in Codex: Settings → Hooks → trust all three. CONNECT TO CLAUDE CODE (same command): npx hindsight-coding-agents install all "all" connects every detected agent on your machine. Claude Code, Codex, Cursor, and others. one command. every agent shares the same bank. TAGS FOR FILTERING: every memory gets tagged by harness (Hermes, Codex, Claude Code) and optionally by project name. in the Hindsight control plane: filter by harness. see only Hermes memories. or only Codex memories. or search across everything. soft partitions inside one bank. not hard walls. cross-reference when you need to. ONE BANK OR MANY: one global bank: solo dev, related projects. all agents share everything. patterns emerge across projects. per-project banks: unrelated codebases. each project gets its own memory. no cross-contamination. your call. start with one. split when projects diverge. KNOWLEDGE PAGES (v0.9.0): Hindsight auto-generates living summaries from your accumulated memories. components, concepts, conventions, decisions. not static docs. projected from real agent conversations. auto-refresh as new memories land. WHAT TO KNOW: self-hosted via Docker. your data never leaves your machine. backup system built in (admin CLI + scheduled exports). works with any LLM (local Ollama, OpenAI, Codex subscription). memory defense: redact or block sensitive content automatically. 33,000+ memories accumulated in ~10 days of normal use.

YanXbt

29,200 просмотров • 20 дней назад

MagicAnimate: Temporally Consistent Human Image Animation using Diffusion Model with Gradio demo local demo: This paper studies the human image animation task, which aims to generate a video of a certain reference identity following a particular motion sequence. Existing animation works typically employ the frame-warping technique to animate the reference image towards the target motion. Despite achieving reasonable results, these approaches face challenges in maintaining temporal consistency throughout the animation due to the lack of temporal modeling and poor preservation of reference identity. In this work, we introduce MagicAnimate, a diffusion-based framework that aims at enhancing temporal consistency, preserving reference image faithfully, and improving animation fidelity. To achieve this, we first develop a video diffusion model to encode temporal information. Second, to maintain the appearance coherence across frames, we introduce a novel appearance encoder to retain the intricate details of the reference image. Leveraging these two innovations, we further employ a simple video fusion technique to encourage smooth transitions for long video animation. Empirical results demonstrate the superiority of our method over baseline approaches on two benchmarks. Notably, our approach outperforms the strongest baseline by over 38% in terms of video fidelity on the challenging TikTok dancing dataset. Code and model will be made available.

AK

810,731 просмотров • 2 лет назад

I’m thrilled to announce that we just released GraspGen, a multi-year project we have been cooking at NVIDIA Robotics 🚀 GraspGen: A Diffusion-Based Framework for 6-DOF Grasping Grasping is a foundational challenge in robotics 🤖 — whether for industrial picking or general-purpose humanoids. VLA + real data collection is all the rage now but is expensive and scales poorly for this task. For every new gripper and/or scene, you’ll have to recollect the dataset in this paradigm for the best perf. 💡Key Idea: Since grasping is such a well-defined task in simulation - why can’t we just scale synthetic data generation and train a generative model for grasping? By embracing modularity and standardized grasp formats, we can make this a turnkey technology that works zero-shot for multiple settings. GraspGen is a modular framework for diffusion-based 6-DOF grasp generation that scales across embodiment types, observability conditions, clutter, task complexity. Key Features: ✅ Multi-embodiment support: suction, parallel-jaw, and multi-fingered grippers ✅ Generalization to partial + complete 3D point clouds ✅ Generalization to single-objects + cluttered scenes ✅ Modular design uses other robotics modules and foundation models (SAM2, cuRobo, FoundationStereo, FoundationPose). This allows GraspGen to focus on only one thing - grasp generation ✅ Training recipe: grasp discriminator is trained with On-Generator data from the diffusion model - so that it learns to correct the mistakes (if any) of the diffusion generator ✅ Real-time performance (~20 Hz) before any GPU acceleration; low memory footprint 📊 Results: • SOTA on the FetchBench [Han et al. CoRL 2024] benchmark • Zero-shot sim-to-real transfer on unknown objects and cluttered scenes • Dataset of 53M simulated grasps across 8K objects from Objaverse 📄 arXiv: 🌐 Website: 💻 Code: A huge thank you to everyone involved in this journey — excited to see what the community builds on top of it! Joint work with Clemens Eppner , Balakumar Sundaralingam , Yu-Wei, Jun Yamada Wentao Yuan and other collaborators #robotics #diffusionmodels #physicalAI #simtoreal

Adithya Murali

24,296 просмотров • 1 год назад