Scaling laws describe how loss changes with scale. Do... neurons inside models change predictably too? We study vision and language models up to 30B params and find systematic scaling in neuron universality, specialization, and selectivity. Paper+code: 1/nshow more

neuron trying in vain to find and connect with... other neuronsshow more

Can Vision-Language Models Solve the Shell Game? paper:

Does off-policy value-based RL scale? In LLMs, larger scale... predictably improves performance. Value-based RL learns from arbitrary data and is sample-efficient, but folk wisdom says it doesn't scale 🧵⬇️We show predictability for scaling value-based RL!show more

Ever wondered how training dynamics differ between LLMs 🖋️... and Vision 👁️ models? We explore this and close the gap between VMs and LLMs in our #NeurIPS2024 paper "TrAct: Making First-layer Pre-Activations Trainable". Paper📜 Video🎥show more

How do time series foundation models forecast unseen dynamical... systems? In new experiments, we find that small transformers learn to approximate transfer operators in-context. (1/N)show more

E11 Bio is excited to unveil PRISM technology for... mapping brain wiring with simple light microscopes. Today, brain mapping in humans and other mammals is bottlenecked by accurate neuron tracing. PRISM uses molecular ID codes and AI to help neurons trace themselves. We discovered a new cell barcoding approach exceeding comparable methods by more than 750x. This is the heart of PRISM. We integrated this capability with microscopy and AI image analysis to automatically trace neurons at high resolution and annotate them with molecular features. This is a key advance towards economically viable brain mapping - 95% of costs stem from neuron tracing. It is also an important step towards democratizing neuron tracing for everyday neuroscience. Solving these problems is critical for curing brain disorders, building safer and human-like AI, and even simulating brain function. In our first pilot study, we acquired a unique dataset in mouse hippocampus. Barcodes improved the accuracy of tracing genetically labelled neurons by 8x – with a clear path to 100x or more. They also permit tracing across spatial gaps – essential for mitigating tissue section loss in whole-brain scaling. Using molecular annotation, we uncover an intriguing feature of synaptic organization, demonstrating how PRISM can be used for systematic discovery 🧵show more

(1/6) X-Humanoid 🤖: Scaling up data for Humanoid Robots.... We convert human daily activity videos (from Ego-Exo4D) into humanoid videos (i.e., Tesla Optimus) performing tasks like cooking or fixing a bike. This data can be potentially used to train robot policies and world models. 🔥 Project page: Paper link:show more

Artificial Intelligence (#AI) models that process and interpret visual... data are called large vision models. These models, like large language models, are trained on large amounts of data. #midjourney #dalle #stablediffusion (1/8) 🧵 ⤵️show more

My first PhD paper!🎉We learn *diffusion* models for code... generation that learn to directly *edit* syntax trees of programs. The result is a system that can incrementally write code, see the execution output, and debug it. 🧵1/nshow more

NVIDIA's CUDA Moat is becoming a Copper Moat with... scale-up networking. Bigger models demand more from the scale-up network, and we❤️ Coppershow more

🤖 Does VLA models really listen to language instructions?... Maybe not 👀 🚀 Introducing our RSS paper: CodeDiffuser -- using VLM-generated code to bridge the gap between **high-level language** and **low-level visuomotor policy** 🎮 Try the live demo: (1/9)show more

Imagine you want to learn about the technology behind... LLMs. You instantly get an 100 page book with chapters, content, and structure. What if you find the language too technical? You can change the prompt and the book – all 100 pages – adapts to your needs. What if you want to learn how to code models in python? You can ask for that too, and in seconds, every chapter has code examples alongside the instructional content. That’s the power of an Infinite Bookshelf. Open source, Powered by Groq Incshow more

Introducing DeepSeek Coder! - SOTA large coding models with... params ranging from 1.3B to 33B. - Building games, testing code, fixing bugs, and analyzing data... You dream it, we make it. - Free for commercial use and fully open-source. Try it out now atshow more

We just shipped React Native ExecuTorch v0.8.0 – our... biggest release yet! 👏🏼 ➡️ Vision Camera integration that runs computer vision models directly on your camera feed, in realtime. Also new: multiple CV models (including RF-DETR (Roboflow) and Liquid AI's Vision Language), Bare React Native support, and more. Takes minutes to try – check out the repo! ⬇️show more

My favorite paper this year: "Video models are zero-shot... learners and reasoners" It illustrates that video models show emergent visual reasoning at scale - they can solve vision tasks they weren't trained for. This may be the "GPT moment" for vision. Let's break it down 👇show more

Chat GPT inside VS Code! Find bugs, add comments,... explain code and more! (how to do it yourself in the next tweet)show more

TOMORROW, 9:30AM PT We go live with an Odyssey... announcement (or two) I've been looking forward to for some time. I think these releases will make where world models are headed, and how powerful they can become, much more obvious. So hyped. GPUs scaling as we speak.show more

🚨New paper! Generative models are often “miscalibrated”. We calibrate... diffusion models, LLMs, and more to meet desired distributional properties. E.g. we finetune protein models to better match the diversity of natural proteins.show more

HOW TO CREATE ANIME EYEBALL EFFECT IN 3D -... thread This assumes you already know how to model in Blender and set up materials in Unity! This method works with .vrm models. 1/?show more

ok but now combine it with Apple vision models... and Expo Agentshow more