Video wird geladen...

Video konnte nicht geladen werden

Beim Laden dieses Videos ist ein Problem aufgetreten. Dies könnte an einem vorübergehenden Netzwerkproblem liegen oder das Video ist möglicherweise nicht verfügbar.

Google Turbo Quant running Locally in Atomic Chat MacBook Air M4 16 GB Model: QWEN3.5-9B Context window: 50000 Summarising 20000 words in just seconds.. You can do 3x larger context window, processing 3x faster than before!

atomic.chat

11,547 subscribers

435,530 Aufrufe • vor 3 Monaten •via X (Twitter)

Bildung Nachrichten & Politik Wissenschaft & Technologie

Anya Rossi• Live Now

Private livecam show

0 Kommentare

Keine Kommentare verfügbar

Kommentare vom Original-Post werden hier angezeigt

Ähnliche Videos

Qwen just published the 'Thinking' variant of this model 🔥 So you can run a model EVEN MORE powerful than GPT-4o locally! - Still only 3B active parameters - Open source license - 256k context window extendable to 1M - Strong in math, science and coding Details below ↓

Qwen just published the 'Thinking' variant of this model 🔥 So you can run a model EVEN MORE powerful than GPT-4o locally! - Still only 3B active parameters - Open source license - 256k context window extendable to 1M - Strong in math, science and coding Details below ↓

Paul Couvert

106,261 Aufrufe • vor 11 Monaten

Memory just landed inside Notch Agents. Brand colors. Voice clones. Do-not-say words. Taste. All read before every generation. On-brand ads. Faster. With brand context. Upload your brand context once. Or teach the agent inline as you chat. Each chat is no longer a re-introduction. It's training. Drop your brand URLs and instructions in the comments. We'll spin ads around your context.

Vinay Jain

647,024 Aufrufe • vor 1 Monat

JUST IN: Google releases Gemini 1.5, a powerful MoE model. It's a huge breakthrough. The model has the longest context window ever seen: 1 million tokens. It can process 1 hour of video, 11 hours of audio, 30,000 lines of code, or 700,000 words in a single prompt. When tested on text, code, image, audio and video evaluations, 1.5 Pro outperforms 1.0 Pro on 87% of the benchmarks used for developing LLMs. You can can sign up in AI Studio to try it out.

JUST IN: Google releases Gemini 1.5, a powerful MoE model. It's a huge breakthrough. The model has the longest context window ever seen: 1 million tokens. It can process 1 hour of video, 11 hours of audio, 30,000 lines of code, or 700,000 words in a single prompt. When tested on text, code, image, audio and video evaluations, 1.5 Pro outperforms 1.0 Pro on 87% of the benchmarks used for developing LLMs. You can can sign up in AI Studio to try it out.

Lior Alexander

83,409 Aufrufe • vor 2 Jahren

$Introducing SubQ - a major breakthrough in LLM intelligence. It is the first model built on a fully sub-quadratic sparse-attention architecture (SSA), And the first frontier model with a 12 million token context window which is: - 52x faster than FlashAttention at 1MM tokens - Less than 5% the cost of Opus Transformer-based LLMs waste compute by processing every possible relationship between words (standard attention). Only a small fraction actually matter. Subquadratic finds and focuses only on the ones that do. That's nearly 1,000x less compute and a new way for LLMs to scale.$

Introducing SubQ - a major breakthrough in LLM intelligence. It is the first model built on a fully sub-quadratic sparse-attention architecture (SSA), And the first frontier model with a 12 million token context window which is: - 52x faster than FlashAttention at 1MM tokens - Less than 5% the cost of Opus Transformer-based LLMs waste compute by processing every possible relationship between words (standard attention). Only a small fraction actually matter. Subquadratic finds and focuses only on the ones that do. That's nearly 1,000x less compute and a new way for LLMs to scale.

Alexander Whedon

13,186,666 Aufrufe • vor 2 Monaten

People are overlooking Google Gemini Realtime models for computer use It gave me sub 100ms latency with computer use. It has a much larger context window and is much cheaper as well Combine that with local OCR and local screen detection model based on Omniparser by Microsoft it works under 100ms action taking when combined with Cua I also put in a harness for Nous Research Hermes with it. You can access it all at your tip of your cursor. You can draw on your screen to give your agents a context And I am making it Open Source! Link in the Comments Sundar Pichai Min-Liang Tan Mojtaba Seyedhosseini

People are overlooking Google Gemini Realtime models for computer use It gave me sub 100ms latency with computer use. It has a much larger context window and is much cheaper as well Combine that with local OCR and local screen detection model based on Omniparser by Microsoft it works under 100ms action taking when combined with Cua I also put in a harness for Nous Research Hermes with it. You can access it all at your tip of your cursor. You can draw on your screen to give your agents a context And I am making it Open Source! Link in the Comments Sundar Pichai Min-Liang Tan Mojtaba Seyedhosseini

Milind S

38,054 Aufrufe • vor 1 Monat

THIS DEVELOPER JUST RAN A TRILLION PARAMETER MODEL ON 4 MAC STUDIOS - 10X FASTER AND 5X CHEAPER THAN CLOUD CODE 19:00 he says it out loud. "we just ran a trillion parameter model. 30 something tokens per second. wow." RDMA over Thunderbolt made the cluster 10x faster than before. tensor parallelism splits the model between machines in parallel instead of sequentially DeepSeek V3 full size without quantization. 183GB per machine. still 25 tokens per second. faster than most people can read Kimi K2 with trillion parameters and 256,000 context window. loaded, responded and adapted memory usage dynamically based on prompt size 4 machines consuming 66 watts total. cloud GPU for the same workload costs $1,900/month a cluster that used to cost millions now assembled from Mac Studios in a few hours

THIS DEVELOPER JUST RAN A TRILLION PARAMETER MODEL ON 4 MAC STUDIOS - 10X FASTER AND 5X CHEAPER THAN CLOUD CODE 19:00 he says it out loud. "we just ran a trillion parameter model. 30 something tokens per second. wow." RDMA over Thunderbolt made the cluster 10x faster than before. tensor parallelism splits the model between machines in parallel instead of sequentially DeepSeek V3 full size without quantization. 183GB per machine. still 25 tokens per second. faster than most people can read Kimi K2 with trillion parameters and 256,000 context window. loaded, responded and adapted memory usage dynamically based on prompt size 4 machines consuming 66 watts total. cloud GPU for the same workload costs $1,900/month a cluster that used to cost millions now assembled from Mac Studios in a few hours

Noisy

79,665 Aufrufe • vor 25 Tagen

WHY IS NO ONE TALKING ABOUT THIS?? Gemma 3n model was one of the best surprises for me. The fact that you can run it on edge devices even with just 2GB of RAM is impressive. A few weeks back, I was on holiday and used the Gemini Live feature a lot. But I kept running into issues whenever I was in a place where the network wasn’t reliable. Gemma 3n : > Multimodal: Supports text and image inputs; video/audio in development. > Context Window: Up to 128K tokens (32K for 1B model). > Multilingual: Trained on 140+ languages. > Privacy: Offline, on-device processing for data security. > Model Sizes: E2B (5B parameters, 2GB RAM) and E4B (10–12B parameters, 3GB RAM). Video credit: Google YT

WHY IS NO ONE TALKING ABOUT THIS?? Gemma 3n model was one of the best surprises for me. The fact that you can run it on edge devices even with just 2GB of RAM is impressive. A few weeks back, I was on holiday and used the Gemini Live feature a lot. But I kept running into issues whenever I was in a place where the network wasn’t reliable. Gemma 3n : > Multimodal: Supports text and image inputs; video/audio in development. > Context Window: Up to 128K tokens (32K for 1B model). > Multilingual: Trained on 140+ languages. > Privacy: Offline, on-device processing for data security. > Model Sizes: E2B (5B parameters, 2GB RAM) and E4B (10–12B parameters, 3GB RAM). Video credit: Google YT

AshutoshShrivastava

210,490 Aufrufe • vor 1 Jahr

MiniMax-M2.1 running fully local in AWQ-4Bit with full context window (170 GB VRAM w full context) - 1000~ to 16,000~ tps prefill - 100~ tps generation speeds - Opencode It’s doing real work, updating my blog with little steering or specificity. The problem with local LLMs is that they require too much steering, this means baby sitting which I don’t have the time to do MiniMax cracked the cost, intelligence, and speed challenge, I would say this is a top tier model. I run frontier models like Gemini and it just fails to call tools, in this year lol… ——————— I think glm-4.?-air is needed still. We need a viable model at each hardware entry point, a Mac M1 Ultra 192GB? is relatively cheap 5k to be able to run this model at 40 tps is a huge societal unlock. Smaller models can be good but size matters :p

MiniMax-M2.1 running fully local in AWQ-4Bit with full context window (170 GB VRAM w full context) - 1000~ to 16,000~ tps prefill - 100~ tps generation speeds - Opencode It’s doing real work, updating my blog with little steering or specificity. The problem with local LLMs is that they require too much steering, this means baby sitting which I don’t have the time to do MiniMax cracked the cost, intelligence, and speed challenge, I would say this is a top tier model. I run frontier models like Gemini and it just fails to call tools, in this year lol… ——————— I think glm-4.?-air is needed still. We need a viable model at each hardware entry point, a Mac M1 Ultra 192GB? is relatively cheap 5k to be able to run this model at 40 tps is a huge societal unlock. Smaller models can be good but size matters :p

0xSero

23,836 Aufrufe • vor 6 Monaten

🚨 do you understand what happened to Claude.. Anthropic just shipped Opus 4.8 with something called dynamic workflows. It no longer works alone - it spins up hundreds of agents that argue, verify, and break each other's work until the answer is right. - A dev used it to port Bun from Zig to Rust in 11 days - 750,000 lines of Rust, 99.8 percent of tests passing - Hundreds of agents ran in parallel with two reviewers per file - Fast mode runs the same model 2.5x faster and 3x cheaper than before work you used to plan in quarters now finishes before the weekend

🚨 do you understand what happened to Claude.. Anthropic just shipped Opus 4.8 with something called dynamic workflows. It no longer works alone - it spins up hundreds of agents that argue, verify, and break each other's work until the answer is right. - A dev used it to port Bun from Zig to Rust in 11 days - 750,000 lines of Rust, 99.8 percent of tests passing - Hundreds of agents ran in parallel with two reviewers per file - Fast mode runs the same model 2.5x faster and 3x cheaper than before work you used to plan in quarters now finishes before the weekend

BuBBliK

431,168 Aufrufe • vor 1 Monat

first impressions of qwen 3.5 27B dense on a single RTX 3090. 35 tok/s. from 4K all the way to 300K+ context. no speed drop. hermes 4.3 started at 35 and degraded to 15 as context filled. qwen dense holds. MoE held 112 flat. 3x faster but only 3B of 35B active per token. architecture tradeoff. Q4_K_M on 16.7GB. native context 262K. pushed past training limit to 376K before VRAM ceiling on 24GB. tried q8 KV cache at 262K, speed collapsed to 11 tok/s. q4_0 KV is the sweet spot. flash attention mandatory. built in reasoning mode. the model thinks step by step before it answers. full chain of thought surviving Q4 quant. 1,799+ token thinking chains with self correction loops. on a single consumer GPU. gave it one prompt: "build a realtime particle galaxy simulation in one HTML file." 3,340 tokens. 95 seconds. one shot. ran on first load. full reasoning and coding in the video below. optimal config if you want to skip the hours of testing: llama-server -ngl 99 -c 262144 -fa on --cache-type-k q4_0 --cache-type-v q4_0 this is just the warmup. octopus invaders is next: 10 files, 3,400+ lines, zero steering. the prompt hermes quit at 22%. already more impressed than expected. full results coming soon.

first impressions of qwen 3.5 27B dense on a single RTX 3090. 35 tok/s. from 4K all the way to 300K+ context. no speed drop. hermes 4.3 started at 35 and degraded to 15 as context filled. qwen dense holds. MoE held 112 flat. 3x faster but only 3B of 35B active per token. architecture tradeoff. Q4_K_M on 16.7GB. native context 262K. pushed past training limit to 376K before VRAM ceiling on 24GB. tried q8 KV cache at 262K, speed collapsed to 11 tok/s. q4_0 KV is the sweet spot. flash attention mandatory. built in reasoning mode. the model thinks step by step before it answers. full chain of thought surviving Q4 quant. 1,799+ token thinking chains with self correction loops. on a single consumer GPU. gave it one prompt: "build a realtime particle galaxy simulation in one HTML file." 3,340 tokens. 95 seconds. one shot. ran on first load. full reasoning and coding in the video below. optimal config if you want to skip the hours of testing: llama-server -ngl 99 -c 262144 -fa on --cache-type-k q4_0 --cache-type-v q4_0 this is just the warmup. octopus invaders is next: 10 files, 3,400+ lines, zero steering. the prompt hermes quit at 22%. already more impressed than expected. full results coming soon.

Sudo su

120,098 Aufrufe • vor 4 Monaten

gpt-5.5 just dropped. and it changes EVERYTHING for traders. openai stopped calling it a chat model. they call it an "agent runtime" now. that means one thing: AI that doesn't wait for you - it acts. here's what gpt-5.5 can actually do: → runs multi-step trading workflows autonomously → operates computer interfaces without human input → plans, iterates, executes - in a terminal, not a chat window → 82.7% on Terminal-Bench 2.0 - real command-line task execution the only question is: are you already running one? full FREE Guide here → like + bookmark. this is the wave.

gpt-5.5 just dropped. and it changes EVERYTHING for traders. openai stopped calling it a chat model. they call it an "agent runtime" now. that means one thing: AI that doesn't wait for you - it acts. here's what gpt-5.5 can actually do: → runs multi-step trading workflows autonomously → operates computer interfaces without human input → plans, iterates, executes - in a terminal, not a chat window → 82.7% on Terminal-Bench 2.0 - real command-line task execution the only question is: are you already running one? full FREE Guide here → like + bookmark. this is the wave.

AlphaCartel

56,823 Aufrufe • vor 2 Monaten

HERMES AGENT CROSSED 140,000 GITHUB STARS IN 3 MONTHS AND JUST BECAME THE MOST USED AGENT IN THE WORLD. Most AI agents forget everything between sessions. Hermes writes its own skills from experience. Next time it runs the skill, improves it, and gets faster. Independent benchmarks show agents with 20+ self-created skills complete similar tasks 40% faster than fresh instances. Qwen 3.6 where the 35B version outperforms last year's 120B models at one third the memory footprint. DGX Spark with 128GB unified memory running everything locally at $0 per month after hardware. The setup takes 30 minutes. LM Studio plus Qwen 3.6 27B for the model server. One install script for Hermes. One config connecting them. Set context window to 65,536 tokens or nothing works. After one month of daily use your skills directory has 20 to 50 learned workflows. Your Hermes is genuinely different from anyone else's.

HERMES AGENT CROSSED 140,000 GITHUB STARS IN 3 MONTHS AND JUST BECAME THE MOST USED AGENT IN THE WORLD. Most AI agents forget everything between sessions. Hermes writes its own skills from experience. Next time it runs the skill, improves it, and gets faster. Independent benchmarks show agents with 20+ self-created skills complete similar tasks 40% faster than fresh instances. Qwen 3.6 where the 35B version outperforms last year's 120B models at one third the memory footprint. DGX Spark with 128GB unified memory running everything locally at $0 per month after hardware. The setup takes 30 minutes. LM Studio plus Qwen 3.6 27B for the model server. One install script for Hermes. One config connecting them. Set context window to 65,536 tokens or nothing works. After one month of daily use your skills directory has 20 to 50 learned workflows. Your Hermes is genuinely different from anyone else's.

Lummox

69,484 Aufrufe • vor 1 Monat

holy shit, i built a better Cluely. introducing: Obscura, a better, more performant Cluely. Obscura helps you during your meeting and interview calls, in real time, with more context and a built-in agent to websearch in less than a second. it is completely INVISIBLE to your screen recorder, screen share and cannot be detected during your google calls or zoom calls no matter what. some functionalities: 1. toggle visibility: so that only you can see it 2. panic kill: if you wanna kill it in a second 3. screenshot ocr: for screenshot capabilities 4. context window: pass any context in .md or .docx file for it to read before your calls and Obscura will understand it 5. performative built-in agent: to search for anything in less than a second. it uses Deepgram for STT model and Anthropic / OpenAI api keys to generate answers. it takes $1 for 20 hours of your calls as compared to cluely's $20/mo or $75/mo plan (but that depends on tokens consumed on both models). releasing a .dmg for people for people to use. access will be given to people who share this with their friends. I will roll out a referral program so that people can share it with others. feel free to give some feedback. thanks to rahat for being on call with me and testing it.

holy shit, i built a better Cluely. introducing: Obscura, a better, more performant Cluely. Obscura helps you during your meeting and interview calls, in real time, with more context and a built-in agent to websearch in less than a second. it is completely INVISIBLE to your screen recorder, screen share and cannot be detected during your google calls or zoom calls no matter what. some functionalities: 1. toggle visibility: so that only you can see it 2. panic kill: if you wanna kill it in a second 3. screenshot ocr: for screenshot capabilities 4. context window: pass any context in .md or .docx file for it to read before your calls and Obscura will understand it 5. performative built-in agent: to search for anything in less than a second. it uses Deepgram for STT model and Anthropic / OpenAI api keys to generate answers. it takes $1 for 20 hours of your calls as compared to cluely's $20/mo or $75/mo plan (but that depends on tokens consumed on both models). releasing a .dmg for people for people to use. access will be given to people who share this with their friends. I will roll out a referral program so that people can share it with others. feel free to give some feedback. thanks to rahat for being on call with me and testing it.

Naruto11.eth

15,745 Aufrufe • vor 4 Monaten

🚨 THIS GUY JUST TURNED CLAUDE INTO A FULL-BLOWN TRADING TERMINAL BY PASTING ONE URL!! Liquid's MCP connector feeds Claude live pricing, funding rates, open interest, positioning data, and even lets you execute trades, all from the chat window Setup takes ~30 seconds: Settings → Connectors → paste Liquid URL → authorize. That's it It sounds like a gimmick until you see the output: Claude drops a full market breakdown, macro context, risk-off regime, sector rotation, then gives you 5 actionable picks: WTI Oil long, MU dip buy, NASDAQ short, MRVL long, ZEC momentum play You ask 'analyze MU for a long trade' and it pulls live data: $870.94 price, -2.69% 24h, $127M OI, funding at 0.0028%. Then breaks down positioning, smart money is 36.2% short-heavy, meaning squeeze potential if it reverses The wildest part? You can literally submit a 3x leveraged long right from Claude $1,500 size, $500 collateral, market order, one click confirm Most retail traders still pay $50/month for signal groups with less context than a single Claude prompt plugged into live markets Bookmark this. The edge is now one connector URL away

🚨 THIS GUY JUST TURNED CLAUDE INTO A FULL-BLOWN TRADING TERMINAL BY PASTING ONE URL!! Liquid's MCP connector feeds Claude live pricing, funding rates, open interest, positioning data, and even lets you execute trades, all from the chat window Setup takes ~30 seconds: Settings → Connectors → paste Liquid URL → authorize. That's it It sounds like a gimmick until you see the output: Claude drops a full market breakdown, macro context, risk-off regime, sector rotation, then gives you 5 actionable picks: WTI Oil long, MU dip buy, NASDAQ short, MRVL long, ZEC momentum play You ask 'analyze MU for a long trade' and it pulls live data: $870.94 price, -2.69% 24h, $127M OI, funding at 0.0028%. Then breaks down positioning, smart money is 36.2% short-heavy, meaning squeeze potential if it reverses The wildest part? You can literally submit a 3x leveraged long right from Claude $1,500 size, $500 collateral, market order, one click confirm Most retail traders still pay $50/month for signal groups with less context than a single Claude prompt plugged into live markets Bookmark this. The edge is now one connector URL away

cristal

86,759 Aufrufe • vor 26 Tagen

Smart Turn v2: open source, native audio turn detection in 14 languages. New checkpoint of the open source, open data, open training code, semantic VAD model on Hugging Face, fal, and Pipecat AI. - 3x faster inference (12ms on an L40) - 14 languages (13 more than v1, which was english-only) - New synthetic data set `chirp_3_all` with ~163k audio samples - 99% accuracy on held out `human_5_all` test data Good turn detection is critical for voice agents. This model "understands" both semantic and audio patterns, and mitigates the voice AI trade-off between unwanted turn latency vs the agent interrupting people before they are finished speaking. Training scripts for both Modal and local training are in the repo. We want to make it as easy as possible to contribute to or customize this model! Here's a demo running the smart-turn model with default settings, aimed at generally hitting 400ms total turn detection time. You can tune things to be faster, too. You can help by contributing data, doing architecture expermints, or cleaning open source data! Keep reading ...

Smart Turn v2: open source, native audio turn detection in 14 languages. New checkpoint of the open source, open data, open training code, semantic VAD model on Hugging Face, fal, and Pipecat AI. - 3x faster inference (12ms on an L40) - 14 languages (13 more than v1, which was english-only) - New synthetic data set `chirp_3_all` with ~163k audio samples - 99% accuracy on held out `human_5_all` test data Good turn detection is critical for voice agents. This model "understands" both semantic and audio patterns, and mitigates the voice AI trade-off between unwanted turn latency vs the agent interrupting people before they are finished speaking. Training scripts for both Modal and local training are in the repo. We want to make it as easy as possible to contribute to or customize this model! Here's a demo running the smart-turn model with default settings, aimed at generally hitting 400ms total turn detection time. You can tune things to be faster, too. You can help by contributing data, doing architecture expermints, or cleaning open source data! Keep reading ...

kwindla

42,219 Aufrufe • vor 1 Jahr

Most AI agents are just toys that live in a chat box. They are high on hype, low on utility. You spend more time prompting them than doing actual work. Notion Agent is different. It’s not just a window into an LLM. It’s a layer of intelligence sitting directly on any page on Notion. Check out the attached video. I just asked Notion Agent to write practical examples of each React Hook. Here’s how Notion Agent is helping me build it. • Notion Agent forms a plan: (Search → Read → Decide → Update → Notify). You give it a goal, and it executes multi-step workflows like editing hundreds of pages or updating complex databases. • It can create pages, customize properties, and build entire database views from scratch. • Total Context: It doesn't just search Notion. It pulls from Slack, Google Drive, and the web. It finds the patterns across your entire stack in seconds. Check out Notion Agent here: Appreciate the Notion team for collaborating with me on this deep dive.

Most AI agents are just toys that live in a chat box. They are high on hype, low on utility. You spend more time prompting them than doing actual work. Notion Agent is different. It’s not just a window into an LLM. It’s a layer of intelligence sitting directly on any page on Notion. Check out the attached video. I just asked Notion Agent to write practical examples of each React Hook. Here’s how Notion Agent is helping me build it. • Notion Agent forms a plan: (Search → Read → Decide → Update → Notify). You give it a goal, and it executes multi-step workflows like editing hundreds of pages or updating complex databases. • It can create pages, customize properties, and build entire database views from scratch. • Total Context: It doesn't just search Notion. It pulls from Slack, Google Drive, and the web. It finds the patterns across your entire stack in seconds. Check out Notion Agent here: Appreciate the Notion team for collaborating with me on this deep dive.

Pratham

20,933 Aufrufe • vor 5 Monaten

Explore state-of-the-art multimodal prompting in our new short course Large Multimodal Model Prompting with Gemini, taught by Erwin Huizenga in collaboration with Google Cloud. One interesting insight from this course: with multimodal models, prompt structure matters significantly. Placing text inputs, such as a patient's medical history, before image inputs, like an X-ray, can enhance the model's ability to contextualize and interpret visual data effectively. In other contexts, such as image captioning, you may get better results by putting the image first. Multimodal models behave differently than text-only LLMs, and effective prompting for models varies depending on the model you’re using. In this course you’ll learn how to effectively prompt Gemini models. Gemini's multimodal capabilities also enable new approaches in AI application development, for example: - The Gemini library handles various video formats (MP4, MOV, MPEG), streamlining applications using these formats. - Large context window (up to 1 million tokens) enables processing of extensive content, like analyzing multiple 50-minute videos simultaneously. - Function calling feature integrates real-time data (e.g., current exchange rates) into model responses. The course demonstrates building multimodal applications with real-world examples including document analyzers that reason across text and graphs simultaneously, video content extractors that find and timestamp specific information from multiple hours of footage, and automated expense report systems processing receipt images while cross-referencing company policies. Sign up here:

Explore state-of-the-art multimodal prompting in our new short course Large Multimodal Model Prompting with Gemini, taught by Erwin Huizenga in collaboration with Google Cloud. One interesting insight from this course: with multimodal models, prompt structure matters significantly. Placing text inputs, such as a patient's medical history, before image inputs, like an X-ray, can enhance the model's ability to contextualize and interpret visual data effectively. In other contexts, such as image captioning, you may get better results by putting the image first. Multimodal models behave differently than text-only LLMs, and effective prompting for models varies depending on the model you’re using. In this course you’ll learn how to effectively prompt Gemini models. Gemini's multimodal capabilities also enable new approaches in AI application development, for example: - The Gemini library handles various video formats (MP4, MOV, MPEG), streamlining applications using these formats. - Large context window (up to 1 million tokens) enables processing of extensive content, like analyzing multiple 50-minute videos simultaneously. - Function calling feature integrates real-time data (e.g., current exchange rates) into model responses. The course demonstrates building multimodal applications with real-world examples including document analyzers that reason across text and graphs simultaneously, video content extractors that find and timestamp specific information from multiple hours of footage, and automated expense report systems processing receipt images while cross-referencing company policies. Sign up here:

Andrew Ng

74,060 Aufrufe • vor 1 Jahr

single RTX 3090. 24 GB VRAM. Qwen3.5-35B-A3B. 4-bit quant, 113 tokens per second at full 262K context harnessing Claude Code locally with no API, no subscription, no proxy. told it what it is. 30 Mamba2 layers, 10 attention, 256 experts, 8 active per token. said "build something that shows off what you can do." it visualized its own architecture. interactive. tokens flowing through layers. 256 experts lighting up on routing. served in the browser from the same GPU running inference. single prompt. then i said level up. 3D. Three.js. separate files. flythrough camera. clickable layers. it planned first, scaffolded 6 files, hit one API bug, fixed it itself, then optimized for smooth framerate. two iterations to a working 3D neural network explorer. llama.cpp just merged a native Anthropic endpoint. Claude Code points at localhost. the whole setup is two commands. no LiteLLM. no proxy config. the open source models coming out of china right now are genuinely changing what's possible on consumer hardware. respect to the Qwen team. this is acceleration.

single RTX 3090. 24 GB VRAM. Qwen3.5-35B-A3B. 4-bit quant, 113 tokens per second at full 262K context harnessing Claude Code locally with no API, no subscription, no proxy. told it what it is. 30 Mamba2 layers, 10 attention, 256 experts, 8 active per token. said "build something that shows off what you can do." it visualized its own architecture. interactive. tokens flowing through layers. 256 experts lighting up on routing. served in the browser from the same GPU running inference. single prompt. then i said level up. 3D. Three.js. separate files. flythrough camera. clickable layers. it planned first, scaffolded 6 files, hit one API bug, fixed it itself, then optimized for smooth framerate. two iterations to a working 3D neural network explorer. llama.cpp just merged a native Anthropic endpoint. Claude Code points at localhost. the whole setup is two commands. no LiteLLM. no proxy config. the open source models coming out of china right now are genuinely changing what's possible on consumer hardware. respect to the Qwen team. this is acceleration.

Sudo su

110,206 Aufrufe • vor 4 Monaten

no prompt engineering, no agentic harness, no tool calls. just me being lazy in llama.cpp's web ui and gemma 4 31b dense taking the task seriously. i typed "create gpu marketplace cards with hardware specs and prices per hour" and the model went and coded this ui, one shot, navy bg, glassmorphism cards, neon accent buttons, realistic pricing tiers per architecture. it even wrote a "why this looks premium" explanation under the code. for context this is a q4 quant of google's 31b dense thinking model, running on a rtx 5090 mobile 24gb in the rog scar 18 at around 15 tok/s sustained, same vram tier as a 3090 or 4090 desktop, so whatever you see here translates directly to your card at home. the whole interaction was me not trying and the model reasoning harder than the prompt deserved. that tells me more about where local ai is at in april 2026 than any leaderboard score. next test drops gemma 4 into hermes agent, autonomous tool calling, multi step reasoning, real agentic loop instead of a chat window. let's see what the same model does when it gets the right environment. more experiments coming anon. octopus invaders queued. same hardware, different tasks, all published here on x and all translatable to your 24gb card. for now the video below shows it coding live, gpu going brrr.

no prompt engineering, no agentic harness, no tool calls. just me being lazy in llama.cpp's web ui and gemma 4 31b dense taking the task seriously. i typed "create gpu marketplace cards with hardware specs and prices per hour" and the model went and coded this ui, one shot, navy bg, glassmorphism cards, neon accent buttons, realistic pricing tiers per architecture. it even wrote a "why this looks premium" explanation under the code. for context this is a q4 quant of google's 31b dense thinking model, running on a rtx 5090 mobile 24gb in the rog scar 18 at around 15 tok/s sustained, same vram tier as a 3090 or 4090 desktop, so whatever you see here translates directly to your card at home. the whole interaction was me not trying and the model reasoning harder than the prompt deserved. that tells me more about where local ai is at in april 2026 than any leaderboard score. next test drops gemma 4 into hermes agent, autonomous tool calling, multi step reasoning, real agentic loop instead of a chat window. let's see what the same model does when it gets the right environment. more experiments coming anon. octopus invaders queued. same hardware, different tasks, all published here on x and all translatable to your 24gb card. for now the video below shows it coding live, gpu going brrr.

Sudo su

28,939 Aufrufe • vor 2 Monaten