Radical Numerics's banner
Radical Numerics's profile picture

Radical Numerics

@RadicalNumerics • 4,337 subscribers

Systems, scaling and architecture for general biological intelligence

Shorts

1/8 Does text-to-image generation need language pretraining? Today, we share Halcyon, a family of early-fusion, base models trained from scratch. Halcyon outperforms strong late-fusion baselines with up to 275× less pretraining compute. Models and recipes are open. Halcyon is part of a research program at Radical Numerics on the science of multimodal pretraining and scaling.

1/8 Does text-to-image generation need language pretraining? Today, we share Halcyon, a family of early-fusion, base models trained from scratch. Halcyon outperforms strong late-fusion baselines with up to 275× less pretraining compute. Models and recipes are open. Halcyon is part of a research program at Radical Numerics on the science of multimodal pretraining and scaling.

75,176 views

Videos

LIVE
1.2k

Anya Rossi

sweetdream.ai

SweetDream.aiSponsored•Livecam
Streaming Now

Watch Anya Live

Anya is streaming live right now! Join her private show and enjoy exclusive content.

HD live stream
Exclusive private shows
1.2k viewers online

Free preview available • Premium content

1/5 What does it take to scale pretraining to 20+ data modalities at once? Improved data mixing methods are one important ingredient. Today, we share how we scale data mixing to thousands of sources for DNA and natural language pretraining of Omnii, our multimodal genome model.
0:22
RadicalNumerics's profile picture

1/5 What does it take to scale pretraining to 20+ data modalities at once? Improved data mixing methods are one important ingredient. Today, we share how we scale data mixing to thousands of sources for DNA and natural language pretraining of Omnii, our multimodal genome model.

Radical Numerics

367,073 views • 13 days ago

No more content to load