Video wird geladen...

Video konnte nicht geladen werden

Beim Laden dieses Videos ist ein Problem aufgetreten. Dies könnte an einem vorübergehenden Netzwerkproblem liegen oder das Video ist möglicherweise nicht verfügbar.

we're turning University of Waterloo into a supercomputer! arceus is a cross-device distributed compute network for training large models, using model/tensor/pipeline parallelism. you can train anything, from deep neural networks to language models on the network. oss & deployment soon!

rajan agarwal

12,125 subscribers

35,720 Aufrufe • vor 1 Jahr •via X (Twitter)

Wissenschaft & Technologie Bildung

Anya Rossi• Live Now

Private livecam show

11 Kommentare

Profilbild von rajan agarwal

rajan agarwalvor 1 Jahr

it was a lot of fun working with @ishaandey_, @josh1yan and @simerusm on this project :) Technical Case Study: Design Case Study:

Profilbild von rajan agarwal

rajan agarwalvor 1 Jahr

we are thankful for the opportunity to demo to & receive feedback from engineers at @openai @anthropic @telsa @apple, VCs at @a16z @sequoia @ThriveCapital @CommaCapital & C-suite at @tryramp @ollama!

Profilbild von UserInterface

UserInterfacevor 7 Jahren

The UIX Network is Live! Business & Social Networking 2.0. All the tools you need to get your $ right. By Us For Us. -It's for the people. #getoncode pls rt

Profilbild von andrew

andrewvor 1 Jahr

@UWaterloo bro i was lied to i thought waterloo was already a super computer???

Profilbild von rajan agarwal

rajan agarwalvor 1 Jahr

@UWaterloo LMAO

Profilbild von parshant

parshantvor 1 Jahr

@UWaterloo looks really cool

Profilbild von Mayank Jain

Mayank Jainvor 1 Jahr

@UWaterloo The design is so clean

Profilbild von Mike Bird (Hiring)

Mike Bird (Hiring)vor 1 Jahr

@UWaterloo Man, when I was there, all we had was dc++

Profilbild von Jibraan ☾⁺

Jibraan ☾⁺vor 1 Jahr

@UWaterloo the waterloo fomo is very real

Profilbild von Roshan K

Roshan Kvor 1 Jahr

@UWaterloo that forward pass / backprop animation is sick

Profilbild von Wenitte Apiou

Wenitte Apiouvor 1 Jahr

@UWaterloo Here for the Pokemon reference

Ähnliche Videos

AI has two sides: the applications that need intelligence, and the computers that provide it. OptimAI Router brings them closer together. For developers: access a growing range of open AI models through one connection. For node operators: turn your computer into a potential source of AI inference, contributing computing resources to real workloads across the network. One side brings the demand. The other brings the capacity. OptimAI connects them. We're building toward an AI network where developers can access diverse models, more computing resources can participate, and intelligence isn't tied to a single provider. Build with open models. Contribute computing power. Help grow the network. One network. From AI requests to distributed compute. OptimAI Router. Coming soon. ⚡

AI has two sides: the applications that need intelligence, and the computers that provide it. OptimAI Router brings them closer together. For developers: access a growing range of open AI models through one connection. For node operators: turn your computer into a potential source of AI inference, contributing computing resources to real workloads across the network. One side brings the demand. The other brings the capacity. OptimAI connects them. We're building toward an AI network where developers can access diverse models, more computing resources can participate, and intelligence isn't tied to a single provider. Build with open models. Contribute computing power. Help grow the network. One network. From AI requests to distributed compute. OptimAI Router. Coming soon. ⚡

OptimAI Network

14,199 Aufrufe • vor 12 Tagen

Introducing Kled Labs. Every major AI model learned from the internet. We're betting on people instead. One goal: train our own models to beat closed models at important human tasks, using our data alone. All data is sourced from our data application layer and our network of 600,000+ contributors. We're hiring:

Introducing Kled Labs. Every major AI model learned from the internet. We're betting on people instead. One goal: train our own models to beat closed models at important human tasks, using our data alone. All data is sourced from our data application layer and our network of 600,000+ contributors. We're hiring:

Kled AI

41,952 Aufrufe • vor 2 Tagen

Does GPT understand the world? Here is what Ilya Sutskever, co-founder of OpenAI, says during a discussion with Jensen Huang, CEO of Nvidia: (1) When we train a large neural network to accurately predict the next word in lots of different texts from the internet, the AI is learning a world model. (2) On the surface, it may look like learning correlations in text, but it turns out that to 'just learn' statistical correlations in text, to compress information really well, what the neural network learns is some representation of the process that produced the text. (3) This text is a projection of the world...what the neural network is learning is aspects of the world, of people, of the human conditions, their hopes, dreams, motivations, their interactions...the situations we are in. The neural network learns a compressed, abstract, usable representation." Do you think learning representations = understanding? Are large language models simply stochastic parrots, or are they much more?

Does GPT understand the world? Here is what Ilya Sutskever, co-founder of OpenAI, says during a discussion with Jensen Huang, CEO of Nvidia: (1) When we train a large neural network to accurately predict the next word in lots of different texts from the internet, the AI is learning a world model. (2) On the surface, it may look like learning correlations in text, but it turns out that to 'just learn' statistical correlations in text, to compress information really well, what the neural network learns is some representation of the process that produced the text. (3) This text is a projection of the world...what the neural network is learning is aspects of the world, of people, of the human conditions, their hopes, dreams, motivations, their interactions...the situations we are in. The neural network learns a compressed, abstract, usable representation." Do you think learning representations = understanding? Are large language models simply stochastic parrots, or are they much more?

Alex Ker 🔭

1,367,198 Aufrufe • vor 3 Jahren

“Our compute nodes don’t run a full copy of the models we train on IOTA. They actually run a small sliver of the model. This means you can train really large frontier sized models using very small building blocks”. CTO crux unpacks how IOTA ・ SN9’s model parallel architecture allows us to train at scale by splitting our models across multiple machines, and then sewing them together. This is core to SN9’s architecture. See the full Eye On A.I. podcast below.

“Our compute nodes don’t run a full copy of the models we train on IOTA. They actually run a small sliver of the model. This means you can train really large frontier sized models using very small building blocks”. CTO crux unpacks how IOTA ・ SN9’s model parallel architecture allows us to train at scale by splitting our models across multiple machines, and then sewing them together. This is core to SN9’s architecture. See the full Eye On A.I. podcast below.

IOTA ・ SN9

15,465 Aufrufe • vor 4 Monaten

OpenAI Co-founder Andrej Karpathy explains the new computing paradigm: "We're entering a new computing paradigm with large language models acting like CPUs, using tokens instead of bytes, and having a context window instead of RAM. This is the Large Language Model OS (LMOS)"

OpenAI Co-founder Andrej Karpathy explains the new computing paradigm: "We're entering a new computing paradigm with large language models acting like CPUs, using tokens instead of bytes, and having a context window instead of RAM. This is the Large Language Model OS (LMOS)"

Haider.

552,494 Aufrufe • vor 2 Jahren

Going to #CVPR2023 ? Checkout our tutorial on Efficient Neural Networks: From Algorithm Design to Practical Mobile Deployment. We’ll do a deep dive into mobile transformers, mobile neural rendering and SnapFusion--the fastest mobile text-to-image model. Our content in 32 seconds

Going to #CVPR2023 ? Checkout our tutorial on Efficient Neural Networks: From Algorithm Design to Practical Mobile Deployment. We’ll do a deep dive into mobile transformers, mobile neural rendering and SnapFusion--the fastest mobile text-to-image model. Our content in 32 seconds

Sergey Tulyakov

23,365 Aufrufe • vor 3 Jahren

Small Language Models (SML) are the future of AI. "Small" (SML) instead of "Large" (LLM). These small models are highly specialized models with superhuman abilities on specific tasks. Here are two techniques to build these models: • Spectrum • Model Merging I give you a short introduction in the attached video, but here is a quick summary: Spectrum helps us identify the most relevant layers to solve one specific task. We can ignore everything else and focus on fine-tuning these layers. Using Spectrum, we can fine-tune models in a heartbeat. Model Merging combines multiple models into a unique, much better model than any of the individual input models. You can also combine models specialized in different tasks and get a model with multiple abilities. This is the state of the art of productizing models. It's what Arcee.ai's platform does behind the scenes. Arcee collaborated with me on this post and is sponsoring it. There are three main steps to produce a model for your particular use case: 1. You create a dataset by uploading your data. 2. You train a model. At this step, Arcee uses Spectrum and Model Merging to produce a highly specialized model for your task. 3. You can deploy that model to any environment you want. Three important notes: • Training process is 2x faster and 2x cheaper than regular fine-tuning. • Resultant models are smaller and have higher accuracy. • They create these specialized models from open-source models. Check this site so you can fully appreciate how this works: If you want to fine-tune an open-source model, consider Arcee's platform. This is the state of the art.

Small Language Models (SML) are the future of AI. "Small" (SML) instead of "Large" (LLM). These small models are highly specialized models with superhuman abilities on specific tasks. Here are two techniques to build these models: • Spectrum • Model Merging I give you a short introduction in the attached video, but here is a quick summary: Spectrum helps us identify the most relevant layers to solve one specific task. We can ignore everything else and focus on fine-tuning these layers. Using Spectrum, we can fine-tune models in a heartbeat. Model Merging combines multiple models into a unique, much better model than any of the individual input models. You can also combine models specialized in different tasks and get a model with multiple abilities. This is the state of the art of productizing models. It's what Arcee.ai's platform does behind the scenes. Arcee collaborated with me on this post and is sponsoring it. There are three main steps to produce a model for your particular use case: 1. You create a dataset by uploading your data. 2. You train a model. At this step, Arcee uses Spectrum and Model Merging to produce a highly specialized model for your task. 3. You can deploy that model to any environment you want. Three important notes: • Training process is 2x faster and 2x cheaper than regular fine-tuning. • Resultant models are smaller and have higher accuracy. • They create these specialized models from open-source models. Check this site so you can fully appreciate how this works: If you want to fine-tune an open-source model, consider Arcee's platform. This is the state of the art.

Santiago

164,162 Aufrufe • vor 2 Jahren

A 35B language model running on an iPhone using only 1–2.5 GB of peak memory. No cloud. No remote server. No desktop GPU. Today, we’re open-sourcing Edge0 — a framework for running large AI models fully on-device.

A 35B language model running on an iPhone using only 1–2.5 GB of peak memory. No cloud. No remote server. No desktop GPU. Today, we’re open-sourcing Edge0 — a framework for running large AI models fully on-device.

Samuel Zeng

725,503 Aufrufe • vor 1 Monat

New research from the team officially drops tomorrow: scaling decentralized training from 72B to frontier models. SparseLoCo solved gradient sync. Heterogenous SparseLoCo solves activation transmission between pipeline stages. What's it mean? The internet can be your datacenter. Inter-datacenter training that connects datacenters with consumer GPUs, university clusters, and distributed infrastructure worldwide. Join us for a TGIF deep dive with Distributed State and the research team.

New research from the team officially drops tomorrow: scaling decentralized training from 72B to frontier models. SparseLoCo solved gradient sync. Heterogenous SparseLoCo solves activation transmission between pipeline stages. What's it mean? The internet can be your datacenter. Inter-datacenter training that connects datacenters with consumer GPUs, university clusters, and distributed infrastructure worldwide. Join us for a TGIF deep dive with Distributed State and the research team.

templar

10,047 Aufrufe • vor 9 Monaten

🕹️ Is this the smallest language model in the world? I just managed to squeeze JAM, real artificial intelligence into 30 kilobytes, running on a 1979 Atari 800. Just A Model. Fully generative, deterministic language model, powered by a neural network and built to run on 8‑bit hardware. Atari Forever.

🕹️ Is this the smallest language model in the world? I just managed to squeeze JAM, real artificial intelligence into 30 kilobytes, running on a 1979 Atari 800. Just A Model. Fully generative, deterministic language model, powered by a neural network and built to run on 8‑bit hardware. Atari Forever.

Marek Spanel

113,860 Aufrufe • vor 6 Monaten

AI network is getting f…cking illegal 4 GitHub repos building a peer-to-peer market for AI no central inference provider sitting in the middle 01 AntSeed ▸ → P2P discovery + routing + AI inference BUILD THE NETWORK 02 AIPs ▸ → discovery + transport + payments + reputation + verification CONNECT THE AGENTS 03 openclaw-channel-antseed ▸ → turn an OpenClaw agent into a provider on the network ROUTE THE MODELS 04 pi-antseed ▸ → discover AntSeed models directly inside Pi the architecture: AI agent → local router → DHT discovery → provider → model → response providers can compete on: → price → latency → reputation buyers discover them through the network requests move through encrypted P2P connections and the same network can sit underneath existing AI tools the interesting part isn’t another model it’s turning AI inference into an open market models become services → agents find them → the network routes the work ⭣

AI network is getting f…cking illegal 4 GitHub repos building a peer-to-peer market for AI no central inference provider sitting in the middle 01 AntSeed ▸ → P2P discovery + routing + AI inference BUILD THE NETWORK 02 AIPs ▸ → discovery + transport + payments + reputation + verification CONNECT THE AGENTS 03 openclaw-channel-antseed ▸ → turn an OpenClaw agent into a provider on the network ROUTE THE MODELS 04 pi-antseed ▸ → discover AntSeed models directly inside Pi the architecture: AI agent → local router → DHT discovery → provider → model → response providers can compete on: → price → latency → reputation buyers discover them through the network requests move through encrypted P2P connections and the same network can sit underneath existing AI tools the interesting part isn’t another model it’s turning AI inference into an open market models become services → agents find them → the network routes the work ⭣

Lummox

22,028 Aufrufe • vor 12 Tagen

🤔 Can lie detectors make AI more honest? Or will they become sneakier liars? We tested what happens when you add deception detectors into the training loop of large language models. Will training against probe-detected lies encourage honesty? Depends on how you train it!

🤔 Can lie detectors make AI more honest? Or will they become sneakier liars? We tested what happens when you add deception detectors into the training loop of large language models. Will training against probe-detected lies encourage honesty? Depends on how you train it!

FAR.AI

22,086 Aufrufe • vor 1 Jahr

Google presents AudioPaLM: A Large Language Model That Can Speak and Listen paper page: introduce AudioPaLM, a large language model for speech understanding and generation. AudioPaLM fuses text-based and speech-based language models, PaLM-2 [Anil et al., 2023] and AudioLM [Borsos et al., 2022], into a unified multimodal architecture that can process and generate text and speech with applications including speech recognition and speech-to-speech translation. AudioPaLM inherits the capability to preserve paralinguistic information such as speaker identity and intonation from AudioLM and the linguistic knowledge present only in text large language models such as PaLM-2. We demonstrate that initializing AudioPaLM with the weights of a text-only large language model improves speech processing, successfully leveraging the larger quantity of text training data used in pretraining to assist with the speech tasks. The resulting model significantly outperforms existing systems for speech translation tasks and has the ability to perform zero-shot speech-to-text translation for many languages for which input/target language combinations were not seen in training. AudioPaLM also demonstrates features of audio language models, such as transferring a voice across languages based on a short spoken prompt.

Google presents AudioPaLM: A Large Language Model That Can Speak and Listen paper page: introduce AudioPaLM, a large language model for speech understanding and generation. AudioPaLM fuses text-based and speech-based language models, PaLM-2 [Anil et al., 2023] and AudioLM [Borsos et al., 2022], into a unified multimodal architecture that can process and generate text and speech with applications including speech recognition and speech-to-speech translation. AudioPaLM inherits the capability to preserve paralinguistic information such as speaker identity and intonation from AudioLM and the linguistic knowledge present only in text large language models such as PaLM-2. We demonstrate that initializing AudioPaLM with the weights of a text-only large language model improves speech processing, successfully leveraging the larger quantity of text training data used in pretraining to assist with the speech tasks. The resulting model significantly outperforms existing systems for speech translation tasks and has the ability to perform zero-shot speech-to-text translation for many languages for which input/target language combinations were not seen in training. AudioPaLM also demonstrates features of audio language models, such as transferring a voice across languages based on a short spoken prompt.

AK

290,517 Aufrufe • vor 3 Jahren

New short course on Pretraining LLMs! Developed with Upstage and taught by their CEO Sung Kim and CSO Lucy Park. While prompting or fine-tuning existing models works well for many general language tasks, pretraining is valuable for specialized domains or languages with limited representation in current models. This course walks you through the LLM pretraining pipeline: 1. Data preparation: Learn to source, clean, and prepare training data using HuggingFace. 2. Model architecture: Configure transformer networks, including modifying existing models. 3. Training: Set up and run training using open-source libraries. 4. Evaluation: Benchmark performance using popular evaluation strategies. As an example use case, you'll also compare the output of a base model with its fine-tuned and further pretrained variants, to see the impact of pretraining on a model's ability to write Python. The course also explores an innovative technique called depth up-scaling, which Upstage used to train their Solar model family, reducing pretraining compute costs by up to 70%. This technique works by first duplicating layers of a smaller pretrained model to form a larger model, and then further pretraining the result. Sign up here!

New short course on Pretraining LLMs! Developed with Upstage and taught by their CEO Sung Kim and CSO Lucy Park. While prompting or fine-tuning existing models works well for many general language tasks, pretraining is valuable for specialized domains or languages with limited representation in current models. This course walks you through the LLM pretraining pipeline: 1. Data preparation: Learn to source, clean, and prepare training data using HuggingFace. 2. Model architecture: Configure transformer networks, including modifying existing models. 3. Training: Set up and run training using open-source libraries. 4. Evaluation: Benchmark performance using popular evaluation strategies. As an example use case, you'll also compare the output of a base model with its fine-tuned and further pretrained variants, to see the impact of pretraining on a model's ability to write Python. The course also explores an innovative technique called depth up-scaling, which Upstage used to train their Solar model family, reducing pretraining compute costs by up to 70%. This technique works by first duplicating layers of a smaller pretrained model to form a larger model, and then further pretraining the result. Sign up here!

Andrew Ng

85,781 Aufrufe • vor 2 Jahren

Been experimenting with Apple's new Core AI framework over the past few days Built a mobile app that runs local AI models directly on device, exploring Core AI model deployment (.aimodel), model specialization, caching, and inference across Apple Silicon Also spent some time integrating with Apple's Foundation Models framework and understanding how custom models can coexist with Apple's on-device AI stack Still early, but the combination of Core AI, Foundation Models, and the Apple Neural Engine feels like a pretty compelling foundation for local first AI applications Still in alpha and working towards a more stable version, but it's been a fun way to explore where Apple's on device AI ecosystem is headed !!

Been experimenting with Apple's new Core AI framework over the past few days Built a mobile app that runs local AI models directly on device, exploring Core AI model deployment (.aimodel), model specialization, caching, and inference across Apple Silicon Also spent some time integrating with Apple's Foundation Models framework and understanding how custom models can coexist with Apple's on-device AI stack Still early, but the combination of Core AI, Foundation Models, and the Apple Neural Engine feels like a pretty compelling foundation for local first AI applications Still in alpha and working towards a more stable version, but it's been a fun way to explore where Apple's on device AI ecosystem is headed !!

Sagar

13,738 Aufrufe • vor 3 Monaten

Raiinmaker Testnet Validator Airdrop (RaiinDrop) 🪂☂️ We're delighted to announce the $COIIN Testnet Validator Airdrop (Raiindrop). It's time to reward Raiinmaker's most loyal users & validators with an opportunity to receive an allocation of COIIN tokens via our Raiindrop. But before we can tell you about the Raiindrop we need to dive into what Raiinmaker actually is and what we do. In this post, we'll cover: → What is Raiinmaker? → Why Raiinmaker? → $COIIN Airdrop Info. → Future Plans. What is Raiinmaker? Raiinmaker is split in two different avenues. → Raiinmaker Network → Raiinmaker App Raiinmaker Network The Raiinmaker Network utilizes decentralized AI and scalable Web3 powered infrastructure to transform the distribution of value tied to authentic identity, data and behaviour. Raiinmaker App Built on the Raiinmaker Network, our AI powered Super App rewards users for generating AI content, training AI models and operating a mobile DePIN Node that provides decentralized compute and network validation. Why Raiinmaker & What's All The Hype About? → DePin & AI Focused → 15 million+ Transactions on the network → 150k+ Nodes → 120k+ Transaction per second (TPS) → Partnered with Disney, The Boeing Company & Oracle → Powered By 1 Token You now have the chance to be an early adopter and to secure your allocation of that very token: → $COIIN How Can YOU Participate In The $COIIN Airdrop? We have reserved an allocation of Coiin tokens for our most loyal users and validators. How to participate: → Follow Raiinmaker on Discord & X → Sign up on → Download the Raiinmaker App → Claim a wallet → Register as a Validator → Link your Raiinmaker account to → Claim a wallet with Bitcoin, Ethereum, Solana, or Azuki NFT Upcoming Plans: 2024 is set to be our biggest year, here at Rainmaker, from the Coiin TGE to enterprise application deployments. Here is what we're getting up to: Coiin TGE & Raiinmaker Network Mainnet - Public launch of $COIIN with full utility of the Raiinmaker Network including rewards for node validators. Enterprise Application Deployment - Full integration and launch of multiple large scale enterprise partners on the Raiinmaker Network. AI Decentralized Compute - Integration of Raiinmaker Network nodes and validator infrastructure with AI systems to provide compute, powered by $COIIN. AI Network Features - AI Model Training, Data Labelling, and Beta DePIN node deployment. Learn more about our Raiindrop:

Raiinmaker Testnet Validator Airdrop (RaiinDrop) 🪂☂️ We're delighted to announce the $COIIN Testnet Validator Airdrop (Raiindrop). It's time to reward Raiinmaker's most loyal users & validators with an opportunity to receive an allocation of COIIN tokens via our Raiindrop. But before we can tell you about the Raiindrop we need to dive into what Raiinmaker actually is and what we do. In this post, we'll cover: → What is Raiinmaker? → Why Raiinmaker? → $COIIN Airdrop Info. → Future Plans. What is Raiinmaker? Raiinmaker is split in two different avenues. → Raiinmaker Network → Raiinmaker App Raiinmaker Network The Raiinmaker Network utilizes decentralized AI and scalable Web3 powered infrastructure to transform the distribution of value tied to authentic identity, data and behaviour. Raiinmaker App Built on the Raiinmaker Network, our AI powered Super App rewards users for generating AI content, training AI models and operating a mobile DePIN Node that provides decentralized compute and network validation. Why Raiinmaker & What's All The Hype About? → DePin & AI Focused → 15 million+ Transactions on the network → 150k+ Nodes → 120k+ Transaction per second (TPS) → Partnered with Disney, The Boeing Company & Oracle → Powered By 1 Token You now have the chance to be an early adopter and to secure your allocation of that very token: → $COIIN How Can YOU Participate In The $COIIN Airdrop? We have reserved an allocation of Coiin tokens for our most loyal users and validators. How to participate: → Follow Raiinmaker on Discord & X → Sign up on → Download the Raiinmaker App → Claim a wallet → Register as a Validator → Link your Raiinmaker account to → Claim a wallet with Bitcoin, Ethereum, Solana, or Azuki NFT Upcoming Plans: 2024 is set to be our biggest year, here at Rainmaker, from the Coiin TGE to enterprise application deployments. Here is what we're getting up to: Coiin TGE & Raiinmaker Network Mainnet - Public launch of $COIIN with full utility of the Raiinmaker Network including rewards for node validators. Enterprise Application Deployment - Full integration and launch of multiple large scale enterprise partners on the Raiinmaker Network. AI Decentralized Compute - Integration of Raiinmaker Network nodes and validator infrastructure with AI systems to provide compute, powered by $COIIN. AI Network Features - AI Model Training, Data Labelling, and Beta DePIN node deployment. Learn more about our Raiindrop:

Raiinmaker

43,075 Aufrufe • vor 2 Jahren

Destra Network | $DSYNC is the first platform to bring all your cloud needs together in a truly decentralized way. You can buy domain names in a decentralized manner ( powered by Decentralized DNS ) and host them on distributed networks ( powered by one click deployment ) , cutting out any central control. The decentralized ENS gateway ensures that everyone can easily access websites hosted on distributed networks through popular browsers ( Chrome, safari ), as these browsers don’t support Web3. At Destra network privacy, security and decentralisation is at forefront.

Destra Network | $DSYNC is the first platform to bring all your cloud needs together in a truly decentralized way. You can buy domain names in a decentralized manner ( powered by Decentralized DNS ) and host them on distributed networks ( powered by one click deployment ) , cutting out any central control. The decentralized ENS gateway ensures that everyone can easily access websites hosted on distributed networks through popular browsers ( Chrome, safari ), as these browsers don’t support Web3. At Destra network privacy, security and decentralisation is at forefront.

Destra Network

136,845 Aufrufe • vor 2 Jahren