Загрузка видео...

Не удалось загрузить видео

Возникла проблема при загрузке этого видео. Это может быть связано с временными проблемами сети или видео может быть недоступно.

На главную

A major question in multimodal modeling is how to leverage strong pre-trained models, such as industry-scale LLMs and VLMs, during training to avoid starting from scratch. This is important as the demand for rich and domain-specific multimodal models continues to increase, while training them from scratch is obviously impractical... due to limited data and compute. The so-called “any-to-any” multimodal models can model a large diverse dictionary of modalities. That’s good. Their downside is that their architectures are often not decoder-only, which has limited their performance in practice and prevents them from leveraging strong pre-trained decoder-only models as priors. We are releasing MODUS (ICML Conference '26) a decoder-only any-to-any multimodal model to address some of these questions. A single transformer decoder predicts any modality from any others with no modality-specific heads, losses, or task pipelines. We show efficient adaptation of established models, e.g., BAGEL, to rich any-to-any multimodal modeling. We are releasing 14B to 77B parameter models. All materials are open-source. The download links and demos here 🧵show more

Amir Zamir

5,892 subscribers

23,628 просмотров • 27 дней назад •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

Комментарии: 0

Нет доступных комментариев

Здесь появятся комментарии из оригинального поста

Похожие видео

1/ Happy to share UniDisc - Unified Multimodal Discrete Diffusion – We train a 1.5 billion parameter transformer model from scratch on 250 million image/caption pairs using a **discrete diffusion objective**. Our model has all the benefits of diffusion models but now in multimodal space! - flexible compute-quality tradeoff, zero-shot inpainting and editing, better control via classifier-free guidance and lower latency! We open source everything - our code, weights and the training dataset.

1/ Happy to share UniDisc - Unified Multimodal Discrete Diffusion – We train a 1.5 billion parameter transformer model from scratch on 250 million image/caption pairs using a discrete diffusion objective. Our model has all the benefits of diffusion models but now in multimodal space! - flexible compute-quality tradeoff, zero-shot inpainting and editing, better control via classifier-free guidance and lower latency! We open source everything - our code, weights and the training dataset.

Mihir Prabhudesai

104,934 просмотров • 1 год назад

Show-o One Single Transformer to Unify Multimodal Understanding and Generation discuss: We present a unified transformer, i.e., Show-o, that unifies multimodal understanding and generation. Unlike fully autoregressive models, Show-o unifies autoregressive and (discrete) diffusion modeling to adaptively handle inputs and outputs of various and mixed modalities. The unified model flexibly supports a wide range of vision-language tasks including visual question-answering, text-to-image generation, text-guided inpainting/extrapolation, and mixed-modality generation. Across various benchmarks, it demonstrates comparable or superior performance to existing individual models with an equivalent or larger number of parameters tailored for understanding or generation. This significantly highlights its potential as a next-generation foundation model.

Show-o One Single Transformer to Unify Multimodal Understanding and Generation discuss: We present a unified transformer, i.e., Show-o, that unifies multimodal understanding and generation. Unlike fully autoregressive models, Show-o unifies autoregressive and (discrete) diffusion modeling to adaptively handle inputs and outputs of various and mixed modalities. The unified model flexibly supports a wide range of vision-language tasks including visual question-answering, text-to-image generation, text-guided inpainting/extrapolation, and mixed-modality generation. Across various benchmarks, it demonstrates comparable or superior performance to existing individual models with an equivalent or larger number of parameters tailored for understanding or generation. This significantly highlights its potential as a next-generation foundation model.

AK

124,048 просмотров • 1 год назад

Throughout my journey in developing multimodal models, I’ve always wanted a framework that lets me plug & play modality encoders/decoders on top of an auto-regressive LLM. I want to prototype fast, try new architectures, and have my demo files scale effortlessly — with full support for parallelism and optimization. Not just to hack⚙️, but also to scale🚀. So finally we built it for ourselves. LMMs-Engine: a lean, efficient framework built to train unified multimodal model at scale. From Qwen LLM, VLM, LLaVA-OV, and WanVideo, to unified models like Qwen-Omni and BAGEL — plus Linear-Attn GDN and research prototypes like RAE and SiT - all under one modular system that seamlessly integrates diverse datasets and optimization strategies. Powered by FSDP2 multi-dim parallelism, Ulysses sequence parallel, Flash-Attention, Liger Kernels, and Native Sparse Attention (also with bonus support for the Muon optimizer for all models).

Throughout my journey in developing multimodal models, I’ve always wanted a framework that lets me plug & play modality encoders/decoders on top of an auto-regressive LLM. I want to prototype fast, try new architectures, and have my demo files scale effortlessly — with full support for parallelism and optimization. Not just to hack⚙️, but also to scale🚀. So finally we built it for ourselves. LMMs-Engine: a lean, efficient framework built to train unified multimodal model at scale. From Qwen LLM, VLM, LLaVA-OV, and WanVideo, to unified models like Qwen-Omni and BAGEL — plus Linear-Attn GDN and research prototypes like RAE and SiT - all under one modular system that seamlessly integrates diverse datasets and optimization strategies. Powered by FSDP2 multi-dim parallelism, Ulysses sequence parallel, Flash-Attention, Liger Kernels, and Native Sparse Attention (also with bonus support for the Muon optimizer for all models).

Brian Li

54,840 просмотров • 9 месяцев назад

We are releasing the fastest TTS model as open source! One of the biggest problems we encountered while optimizing TTS models is optimization itself. Since it is LLM-based, even optimizing with VLLM and SgLang libraries is not enough. Additionally, I made the outputs much better with special optimizations for training. The voices you are listening to are results obtained from only 20% of the model training. We trained these outputs on 8xB200 in 24 hours. Even though the dataset includes some low-quality data, the model's output is much better. We will release the code as open source tomorrow. Data: Emilia-3m Model: EchoDit (custom) Codec: DacVae Opt: Flash-attn + CacheDit

We are releasing the fastest TTS model as open source! One of the biggest problems we encountered while optimizing TTS models is optimization itself. Since it is LLM-based, even optimizing with VLLM and SgLang libraries is not enough. Additionally, I made the outputs much better with special optimizations for training. The voices you are listening to are results obtained from only 20% of the model training. We trained these outputs on 8xB200 in 24 hours. Even though the dataset includes some low-quality data, the model's output is much better. We will release the code as open source tomorrow. Data: Emilia-3m Model: EchoDit (custom) Codec: DacVae Opt: Flash-attn + CacheDit

Kadir Nar

32,232 просмотров • 6 дней назад

Depth Any Video with Scalable Synthetic Data AI physicists and chemists continue to make strides in depth estimation from video. Check out this new paper featuring some impressive examples. See the thread for more details (unfortunately no code yet). Abstract: Video depth estimation has long been hindered by the scarcity of consistent and scalable ground truth data, leading to inconsistent and unreliable results. In this paper, we introduce Depth Any Video, a model that tackles the challenge through two key innovations. First, we develop a scalable synthetic data pipeline, capturing real-time video depth data from diverse game environments, yielding 40,000 video clips of 5-second duration, each with precise depth annotations. Second, we leverage the powerful priors of generative video diffusion models to handle real-world videos effectively, integrating advanced techniques such as rotary position encoding and flow matching to further enhance flexibility and efficiency. Unlike previous models, which are limited to fixed-length video sequences, our approach introduces a novel mixed-duration training strategy that handles videos of varying lengths and performs robustly across different frame rates 0 - even on single frames. At inference, we propose a depth interpolation method that enables our model to infer high-resolution video depth across sequences of up to 150 frames. Our model outperforms all previous generative depth models in terms of spatial accuracy and temporal consistency.

Depth Any Video with Scalable Synthetic Data AI physicists and chemists continue to make strides in depth estimation from video. Check out this new paper featuring some impressive examples. See the thread for more details (unfortunately no code yet). Abstract: Video depth estimation has long been hindered by the scarcity of consistent and scalable ground truth data, leading to inconsistent and unreliable results. In this paper, we introduce Depth Any Video, a model that tackles the challenge through two key innovations. First, we develop a scalable synthetic data pipeline, capturing real-time video depth data from diverse game environments, yielding 40,000 video clips of 5-second duration, each with precise depth annotations. Second, we leverage the powerful priors of generative video diffusion models to handle real-world videos effectively, integrating advanced techniques such as rotary position encoding and flow matching to further enhance flexibility and efficiency. Unlike previous models, which are limited to fixed-length video sequences, our approach introduces a novel mixed-duration training strategy that handles videos of varying lengths and performs robustly across different frame rates 0 - even on single frames. At inference, we propose a depth interpolation method that enables our model to infer high-resolution video depth across sequences of up to 150 frames. Our model outperforms all previous generative depth models in terms of spatial accuracy and temporal consistency.

MrNeRF

27,428 просмотров • 1 год назад

🤔 How to fine-tune an Imitation Learning policy (e.g., Diffusion Policy, ACT) with RL? As an RL practitioner, I’ve been struggling with this problem for a while. Here’s why it’s tough: 1️⃣ Special designs (usually for multimodal action distributions) in modern IL models make them non-trivial to fine-tune by RL. 2️⃣ Large policy models + RL's poor sample efficiency = a nightmare But finally, we figured out a simple solution that works for any model architecture! 🌟 Check out our #ICLR2025 paper: “Policy Decorator: Model-Agnostic Online Refinement for Large Policy Models”, led by my amazing mentee Xiu Yuan. 🔗 🧵 Read more below!

🤔 How to fine-tune an Imitation Learning policy (e.g., Diffusion Policy, ACT) with RL? As an RL practitioner, I’ve been struggling with this problem for a while. Here’s why it’s tough: 1️⃣ Special designs (usually for multimodal action distributions) in modern IL models make them non-trivial to fine-tune by RL. 2️⃣ Large policy models + RL's poor sample efficiency = a nightmare But finally, we figured out a simple solution that works for any model architecture! 🌟 Check out our #ICLR2025 paper: “Policy Decorator: Model-Agnostic Online Refinement for Large Policy Models”, led by my amazing mentee Xiu Yuan. 🔗 🧵 Read more below!

Tongzhou Mu 🤖🦾🦿

16,959 просмотров • 1 год назад

World Models are the path for some AI Models in the future. But how can we efficiently train these models to not only see the world the way humans do but to see the world in a new and unique way. By visualizing, what is normally sequenced audio patterns, we can derive much more insights. Here we see Paganini in a visual form that can than be described and transcribed into a World Model. We can observe connections in a manner that may not have been clear prior to the digitalization of music and sound in this way. The company with the most valuable potential in building a World Model is Tesla. Not that this type of visualization is being used, but that the mechanisms are in place, and the technology is in place for the company to thrive in this new form of AI.

World Models are the path for some AI Models in the future. But how can we efficiently train these models to not only see the world the way humans do but to see the world in a new and unique way. By visualizing, what is normally sequenced audio patterns, we can derive much more insights. Here we see Paganini in a visual form that can than be described and transcribed into a World Model. We can observe connections in a manner that may not have been clear prior to the digitalization of music and sound in this way. The company with the most valuable potential in building a World Model is Tesla. Not that this type of visualization is being used, but that the mechanisms are in place, and the technology is in place for the company to thrive in this new form of AI.

Brian Roemmele

57,454 просмотров • 8 месяцев назад

We released physics-intern: a simple harness for science problems! It gets models like Gemini 3.1 Pro to go from 17.7 -> 31.4, thus beating GPT 5.5 Pro. The physics-intern harness can wrap any model and via dedicated subagent boost the performance of the vanilla reasoning models. While I think more and more of these harness capability gains will be absorbed into the models (like prompting tricks disappeared over time) there is a lot to be gained right now by building good scaffolds for those models and integrating tools well. Interestingly, the exception we found that GPT 5.5 Pro actually didn't benefit from the physics-intern harness! Read more about it here: PS: I think the Harness[Model] notation is kind of nice.

We released physics-intern: a simple harness for science problems! It gets models like Gemini 3.1 Pro to go from 17.7 -> 31.4, thus beating GPT 5.5 Pro. The physics-intern harness can wrap any model and via dedicated subagent boost the performance of the vanilla reasoning models. While I think more and more of these harness capability gains will be absorbed into the models (like prompting tricks disappeared over time) there is a lot to be gained right now by building good scaffolds for those models and integrating tools well. Interestingly, the exception we found that GPT 5.5 Pro actually didn't benefit from the physics-intern harness! Read more about it here: PS: I think the Harness[Model] notation is kind of nice.

Leandro von Werra

97,262 просмотров • 2 месяцев назад

These cats are all in horrible condition and the weather is not on their side. It can truly be life or death for them. I don’t want to see any of them freeze or starve to death. We have some major medical cases going on, plus these three cats who are all clearly struggling.

These cats are all in horrible condition and the weather is not on their side. It can truly be life or death for them. I don’t want to see any of them freeze or starve to death. We have some major medical cases going on, plus these three cats who are all clearly struggling.

Cats of Meow York.

12,157 просмотров • 7 месяцев назад

Wonderland: Navigating 3D Scenes from a Single Image Contributions: • First, we introduce a representation for controllable 3D generation by leveraging the generative priors from camera-guided video diffusion models. Unlike image models, video diffusion models are trained on extensive video datasets. This enables them to capture comprehensive spatial relationships within scenes across multiple views and embed a form of "3D awareness" in their latent space, which allows us to maintain 3D consistency in novel view synthesis. • Second, to achieve controllable novel view generation, we empower video models with precise control over specified camera motions. We introduce a novel dual-branch conditioning mechanism that effectively incorporates desired diverse camera trajectories into the video diffusion model. This enables expansion of a single image into a multi-view consistent capture of a 3D scene with precise pose control. • Third, to achieve efficient 3D reconstruction, we directly transform video latents into 3DGS. We propose a novel latent-based large reconstruction model (LaLRM) that lifts video latents to 3D in a feed-forward manner. With this design, during inference, our model directly predicts 3DGS from a single input image, effectively aligning the generation and reconstruction tasks—and bridging image space and 3D space—through the video latent space. Compared with reconstructing scenes from images, the video latent space offers a 256× spatial-temporal reduction while retaining essential and consistent 3D structural details. Such a high degree of compression is crucial, as it allows the LaLRM to handle a wider range of 3D scenes within the reconstruction framework, with the same memory constraints.

Wonderland: Navigating 3D Scenes from a Single Image Contributions: • First, we introduce a representation for controllable 3D generation by leveraging the generative priors from camera-guided video diffusion models. Unlike image models, video diffusion models are trained on extensive video datasets. This enables them to capture comprehensive spatial relationships within scenes across multiple views and embed a form of "3D awareness" in their latent space, which allows us to maintain 3D consistency in novel view synthesis. • Second, to achieve controllable novel view generation, we empower video models with precise control over specified camera motions. We introduce a novel dual-branch conditioning mechanism that effectively incorporates desired diverse camera trajectories into the video diffusion model. This enables expansion of a single image into a multi-view consistent capture of a 3D scene with precise pose control. • Third, to achieve efficient 3D reconstruction, we directly transform video latents into 3DGS. We propose a novel latent-based large reconstruction model (LaLRM) that lifts video latents to 3D in a feed-forward manner. With this design, during inference, our model directly predicts 3DGS from a single input image, effectively aligning the generation and reconstruction tasks—and bridging image space and 3D space—through the video latent space. Compared with reconstructing scenes from images, the video latent space offers a 256× spatial-temporal reduction while retaining essential and consistent 3D structural details. Such a high degree of compression is crucial, as it allows the LaLRM to handle a wider range of 3D scenes within the reconstruction framework, with the same memory constraints.

MrNeRF

52,849 просмотров • 1 год назад

Chop the gradients ✂️! We found that truncating decoder gradients in latent video diffusion to a fixed window allows us to finetune on videos with pixel-wise perceptual losses without running out of memory. Pixel losses have been essential for image generation and reconstruction, but until now, they haven't scaled to long-duration, high-resolution video diffusion due to recursive activation accumulation in causal decoders, leading to OOM during training 💥📉. Project: Video diffusion models can do a lot more 🚀 when you can backprop the decoder! Post-process neural rendered scenes, super-resolve videos, harmonize lighting in controlled synthetic driving scenes, and inpaint videos — all in a single step ⚡ with a quick finetune from a standard diffusion model.

Chop the gradients ✂️! We found that truncating decoder gradients in latent video diffusion to a fixed window allows us to finetune on videos with pixel-wise perceptual losses without running out of memory. Pixel losses have been essential for image generation and reconstruction, but until now, they haven't scaled to long-duration, high-resolution video diffusion due to recursive activation accumulation in causal decoders, leading to OOM during training 💥📉. Project: Video diffusion models can do a lot more 🚀 when you can backprop the decoder! Post-process neural rendered scenes, super-resolve videos, harmonize lighting in controlled synthetic driving scenes, and inpaint videos — all in a single step ⚡ with a quick finetune from a standard diffusion model.

Felix Heide

28,399 просмотров • 3 месяцев назад

#NEIOH - "This is a camera from Sirius-Nopplia. It was sent from a large craft that is part of a Galactic Federation Fleet from a massive Mothership. The screens within the Mothership as well as all crafts are in constant control of all operatives sent to any location. This was a minor adjustment and the camera returned to the craft." Manchester Airport

#NEIOH - "This is a camera from Sirius-Nopplia. It was sent from a large craft that is part of a Galactic Federation Fleet from a massive Mothership. The screens within the Mothership as well as all crafts are in constant control of all operatives sent to any location. This was a minor adjustment and the camera returned to the craft." Manchester Airport

Kab

181,899 просмотров • 1 год назад

AMD claims that all their software is open source, yet reality does not match this claim. For example, AMD's rocprof-trace-decoder is still completely closed source currently despite repeated requests for months by ML community members such as George Hotz, who is a daily AMD GPU end user. On the same June Spotify podcast, Tobias Macey & AMD's Anush Elangovan (who has a new twitter pfp) said that "open source allows for innovation to go at the pace at which the people using [AMD] want to, right, and it's not limited by the ability of what we put out in closed source form". We agree with an open source first approach and we agree that AMD should not limit ML community members like George Hotz by continuing to keep rocprof-trace-decoder closed source. When will AMD open source rocprof-trace-decoder?

AMD claims that all their software is open source, yet reality does not match this claim. For example, AMD's rocprof-trace-decoder is still completely closed source currently despite repeated requests for months by ML community members such as George Hotz, who is a daily AMD GPU end user. On the same June Spotify podcast, Tobias Macey & AMD's Anush Elangovan (who has a new twitter pfp) said that "open source allows for innovation to go at the pace at which the people using [AMD] want to, right, and it's not limited by the ability of what we put out in closed source form". We agree with an open source first approach and we agree that AMD should not limit ML community members like George Hotz by continuing to keep rocprof-trace-decoder closed source. When will AMD open source rocprof-trace-decoder?

SemiAnalysis

53,783 просмотров • 8 месяцев назад

I'm sure you've all noticed the 'AI is slowing down' news stories every few weeks for multiple years now - so I've pulled a tracker together to see who and when wrote these stories. There is quite a range, some are just outright wrong, others point to a reasonable limitation at the time but missing the bigger arc of progress. All of these stories were appearing as we were getting reasoning models, open source models, increasing competition from more players and skyrocketing revenue for the labs. Link to the tracker in the comments

I'm sure you've all noticed the 'AI is slowing down' news stories every few weeks for multiple years now - so I've pulled a tracker together to see who and when wrote these stories. There is quite a range, some are just outright wrong, others point to a reasonable limitation at the time but missing the bigger arc of progress. All of these stories were appearing as we were getting reasoning models, open source models, increasing competition from more players and skyrocketing revenue for the labs. Link to the tracker in the comments

Peter Gostev

37,443 просмотров • 11 месяцев назад

Predicting the next word "only" is sufficient for language models to learn a large body of knowledge that enables then to code, answer questions, understand many topics, chat, and so on. This is clear to many researchers now, and there are nice tutorials on why this works by Ilya Sutskever resorting to compression ( ) and by Geoffrey Hinton ( ). However, the emergence of types of understanding is not unique to language models. In by Misha Denil and Brandon Amos the authors trained models to predict the next few time stems of over a hundred robot hand sensors (Touch, Gyro, Accelerometer, Joint Info, Actuator Info, etc.). They ten found out that they could regress the shape of the thing the hand was touching from the activations of the neural networks using probes. That is, the model developed an internal representation of shapes even though it was simply used to predict "only" the next few senses. Awareness follows from simple predictions and interaction with the world.

Predicting the next word "only" is sufficient for language models to learn a large body of knowledge that enables then to code, answer questions, understand many topics, chat, and so on. This is clear to many researchers now, and there are nice tutorials on why this works by Ilya Sutskever resorting to compression ( ) and by Geoffrey Hinton ( ). However, the emergence of types of understanding is not unique to language models. In by Misha Denil and Brandon Amos the authors trained models to predict the next few time stems of over a hundred robot hand sensors (Touch, Gyro, Accelerometer, Joint Info, Actuator Info, etc.). They ten found out that they could regress the shape of the thing the hand was touching from the activations of the neural networks using probes. That is, the model developed an internal representation of shapes even though it was simply used to predict "only" the next few senses. Awareness follows from simple predictions and interaction with the world.

Nando de Freitas

134,252 просмотров • 2 лет назад

I understand what Russia wants to achieve with its missile and drone strikes on Ukraine - namely, to destroy peaceful life and drive people into such a psychological state that they are ready to make any concessions just to end it all. However, such strikes have the opposite effect. They remind us that the war is ongoing and that it is not limited to the conflict in the east; they show that at any moment, a missile can take everything away from you. And the only way to prevent this from happening is to donate to the Armed Forces of Ukraine and to hate the Russians even more. I think we can cite examples from World War II, when Hitler's attacks on Great Britain did not break its confidence in the struggle. If the Russians aimed to cause a massive blackout, they failed, wasting dozens of missiles worth tens of millions. But we understand that strikes on oil refineries and other targets must continue! PS: These are small fragments from real life that I filmed. The video may not show it, but the shock wave was strong enough to cause quite intense emotions in me.

I understand what Russia wants to achieve with its missile and drone strikes on Ukraine - namely, to destroy peaceful life and drive people into such a psychological state that they are ready to make any concessions just to end it all. However, such strikes have the opposite effect. They remind us that the war is ongoing and that it is not limited to the conflict in the east; they show that at any moment, a missile can take everything away from you. And the only way to prevent this from happening is to donate to the Armed Forces of Ukraine and to hate the Russians even more. I think we can cite examples from World War II, when Hitler's attacks on Great Britain did not break its confidence in the struggle. If the Russians aimed to cause a massive blackout, they failed, wasting dozens of missiles worth tens of millions. But we understand that strikes on oil refineries and other targets must continue! PS: These are small fragments from real life that I filmed. The video may not show it, but the shock wave was strong enough to cause quite intense emotions in me.

Cloooud |🇺🇦

19,355 просмотров • 10 месяцев назад

So long long day. I don’t even know where to start and gonna keep it as minimal as I can as I am just exhausted. She had a scare about 4 hours ago where she stopped responding, or reacting to us, was unable to reply or acknowledge us. It’s been a very stressful day for her, on top of the extubation, heat from the Cpap mask was also bothering her a lot. We had a moment of her might having a stroke, her left side started to not be as mobile, they went and did a CT scan, and EEG, did not see any major signs, but wanted to do a MRI also to fully look and rule anything out. They did not show any brain bleed or any major things but did find some irregularly on the right side, seemed minor and didn’t seem like a stroke. They are going to review that and get back to us in the morning/early after noon. Right now she is in a stable place, her numbers are looking fine, and she is still breathing on her own with the mask. We ( me and dad ) are thinking she had a panic attack which could have caused this since her day was so hectic. Please continue to pray for her well being and recovery, that any scary things stay away and that her breathing is strong enough to continue without a breathing tube. She has made some strong ground and positive moves forward today, but those scares are extremely hard on all of us.

So long long day. I don’t even know where to start and gonna keep it as minimal as I can as I am just exhausted. She had a scare about 4 hours ago where she stopped responding, or reacting to us, was unable to reply or acknowledge us. It’s been a very stressful day for her, on top of the extubation, heat from the Cpap mask was also bothering her a lot. We had a moment of her might having a stroke, her left side started to not be as mobile, they went and did a CT scan, and EEG, did not see any major signs, but wanted to do a MRI also to fully look and rule anything out. They did not show any brain bleed or any major things but did find some irregularly on the right side, seemed minor and didn’t seem like a stroke. They are going to review that and get back to us in the morning/early after noon. Right now she is in a stable place, her numbers are looking fine, and she is still breathing on her own with the mask. We ( me and dad ) are thinking she had a panic attack which could have caused this since her day was so hectic. Please continue to pray for her well being and recovery, that any scary things stay away and that her breathing is strong enough to continue without a breathing tube. She has made some strong ground and positive moves forward today, but those scares are extremely hard on all of us.

ash 🤍

16,294 просмотров • 1 год назад

Today we're dropping the "beta" tag from Adaptyv, launching our new website and announcing our $8M seed round. When we started Adaptyv a few years ago, our core belief was: AI models for biology are only as good as the experimental data they're trained on and the hypotheses they can test in the real world. Now, after a year of working with many great partners, we’ve scaled our infrastructure to the point that we're now open to anyone who wants to use our platform! Overall, this year, over 30 companies started using Adaptyv to validate their protein designs - from some of the biggest pharmas to frontier AI labs to many, many techbio startups. We've run hundreds of experiments, tested well over 10,000 proteins this year and are generating the data that validates the best AI models currently in development.

Today we're dropping the "beta" tag from Adaptyv, launching our new website and announcing our $8M seed round. When we started Adaptyv a few years ago, our core belief was: AI models for biology are only as good as the experimental data they're trained on and the hypotheses they can test in the real world. Now, after a year of working with many great partners, we’ve scaled our infrastructure to the point that we're now open to anyone who wants to use our platform! Overall, this year, over 30 companies started using Adaptyv to validate their protein designs - from some of the biggest pharmas to frontier AI labs to many, many techbio startups. We've run hundreds of experiments, tested well over 10,000 proteins this year and are generating the data that validates the best AI models currently in development.

Adaptyv Bio

11,484 просмотров • 10 месяцев назад

LongWriter Unleashing 10,000+ Word Generation from Long Context LLMs discuss: Current long context large language models (LLMs) can process inputs up to 100,000 tokens, yet struggle to generate outputs exceeding even a modest length of 2,000 words. Through controlled experiments, we find that the model's effective generation length is inherently bounded by the sample it has seen during supervised fine-tuning (SFT). In other words, their output limitation is due to the scarcity of long-output examples in existing SFT datasets. To address this, we introduce AgentWrite, an agent-based pipeline that decomposes ultra-long generation tasks into subtasks, enabling off-the-shelf LLMs to generate coherent outputs exceeding 20,000 words. Leveraging AgentWrite, we construct LongWriter-6k, a dataset containing 6,000 SFT data with output lengths ranging from 2k to 32k words. By incorporating this dataset into model training, we successfully scale the output length of existing models to over 10,000 words while maintaining output quality. We also develop LongBench-Write, a comprehensive benchmark for evaluating ultra-long generation capabilities. Our 9B parameter model, further improved through DPO, achieves state-of-the-art performance on this benchmark, surpassing even much larger proprietary models. In general, our work demonstrates that existing long context LLM already possesses the potential for a larger output window--all you need is data with extended output during model alignment to unlock this capability.

LongWriter Unleashing 10,000+ Word Generation from Long Context LLMs discuss: Current long context large language models (LLMs) can process inputs up to 100,000 tokens, yet struggle to generate outputs exceeding even a modest length of 2,000 words. Through controlled experiments, we find that the model's effective generation length is inherently bounded by the sample it has seen during supervised fine-tuning (SFT). In other words, their output limitation is due to the scarcity of long-output examples in existing SFT datasets. To address this, we introduce AgentWrite, an agent-based pipeline that decomposes ultra-long generation tasks into subtasks, enabling off-the-shelf LLMs to generate coherent outputs exceeding 20,000 words. Leveraging AgentWrite, we construct LongWriter-6k, a dataset containing 6,000 SFT data with output lengths ranging from 2k to 32k words. By incorporating this dataset into model training, we successfully scale the output length of existing models to over 10,000 words while maintaining output quality. We also develop LongBench-Write, a comprehensive benchmark for evaluating ultra-long generation capabilities. Our 9B parameter model, further improved through DPO, achieves state-of-the-art performance on this benchmark, surpassing even much larger proprietary models. In general, our work demonstrates that existing long context LLM already possesses the potential for a larger output window--all you need is data with extended output during model alignment to unlock this capability.

AK

50,995 просмотров • 2 лет назад

These are some videos from Chinese pioneers clubs or individuals! I heard right now almost all offline Barter in China use GCV price. A lot pioneers are very happy to show their strong support for GCV price! Pi is a fight and price is weapon! Pioneers are Pi owners so they should understand Pi value and treasure it. Of course we call pioneers can do any kind of empower to Pi which may include to be Eco or/ and Business Ambassador. To be validator. To setup node. To support and pay attention to Eco application and give feedback. Anything can make the whole world understand Pi value we can do it ! #WhatIDoForPi #PiGCV Pi Network

These are some videos from Chinese pioneers clubs or individuals! I heard right now almost all offline Barter in China use GCV price. A lot pioneers are very happy to show their strong support for GCV price! Pi is a fight and price is weapon! Pioneers are Pi owners so they should understand Pi value and treasure it. Of course we call pioneers can do any kind of empower to Pi which may include to be Eco or/ and Business Ambassador. To be validator. To setup node. To support and pay attention to Eco application and give feedback. Anything can make the whole world understand Pi value we can do it ! #WhatIDoForPi #PiGCV Pi Network

Doris Yin 东方紫莲🪷

31,988 просмотров • 3 лет назад