Video wird geladen...

Video konnte nicht geladen werden

Beim Laden dieses Videos ist ein Problem aufgetreten. Dies könnte an einem vorübergehenden Netzwerkproblem liegen oder das Video ist möglicherweise nicht verfügbar.

Introducing ambientGPT: an open-source and multimodal MacOS foundation model GUI Run GPT-4o and open-source models with full ambient knowledge of your screen. Foundation models have long been confined to the browser. With ambientGPT, your screen context is directly inferred as part of the query, ensuring you never need to... show more

Siddharth Sharma

2,681 subscribers

192,959 Aufrufe • vor 2 Jahren •via X (Twitter)

Bildung Wissenschaft & Technologie

Anya Rossi• Live Now

Private livecam show

10 Kommentare

Profilbild von Siddharth Sharma

Siddharth Sharmavor 2 Jahren

Unlike OpenAI’s desktop app where you must provide a screenshot or upload a file, the context from your screen is automatically parsed. We also provide the ability to run secure local models like Gemma and Phi-3 multimodal from our interface. Due to the local model sizes, at least 16 GB RAM would be preferred. This was possible via the apple MLX library - shoutout to @awnihannun, @reach_vb

Profilbild von Siddharth Sharma

Siddharth Sharmavor 2 Jahren

ambientGPT is open-source and we plan to integrate vllm and ollama to provide more extensive inference hosting abilities with our multimodal GUI. We also aim to release ambientGPT on the apple app store soon.

Profilbild von Siddharth Sharma

Siddharth Sharmavor 2 Jahren

thanks to @mihiranan (mr. clutch) for his help with the demo once again!

Profilbild von Sambhav Gupta

Sambhav Guptavor 2 Jahren

Very cool but chatgpt mac app will have the ability to see everything on screen when they release the update ..

Profilbild von Ishan Khare

Ishan Kharevor 2 Jahren

so what does the “full ambient knowledge of your screen” entail privacy wise?? why would someone trust this if unwanted data is being captured forever and sent to openai?

Profilbild von Soham Konar

Soham Konarvor 2 Jahren

Great work! Does it work across multiple monitors? Could be a game changer if so

Profilbild von whistle

whistlevor 2 Jahren

was waiting for something exactly like this!! what’s the token usage look like?

Profilbild von Saïd Aitmbarek

Saïd Aitmbarekvor 2 Jahren

looks awesome, would love to showcase ambient on

Profilbild von Salman

Salmanvor 2 Jahren

Looks super useful . How does it determine context if you have multiple screens?

Profilbild von Rahul bansal 👀

Rahul bansal 👀vor 2 Jahren

This is cool. I built a tool for taking to model via voice

Ähnliche Videos

Introducing Mentat - an open source, GPT-4 powered coding assistant! Mentat runs in your command line, giving it the context of your projects and allowing it to coordinate edits across multiple files! More videos and a link to github below:

Introducing Mentat - an open source, GPT-4 powered coding assistant! Mentat runs in your command line, giving it the context of your projects and allowing it to coordinate edits across multiple files! More videos and a link to github below:

Scott Swingle

292,711 Aufrufe • vor 3 Jahren

Announcing the Open Model Series: a new series on open source AI models, with merve of Hugging Face. Hugging Face is the open model hub: it hosts model weights, datasets, evals, & demo apps, and maintains the open-source libraries (like transformers) used to run and train them. I think of it like GitHub, but for AI models. You can download open models and run them on your own machines. In this series we'll cover what open source and open weight models are, how they work, how to run them yourself, and why they're important. By the way - this video is edited entirely with Claude! I can make a tutorial on how I did it. If there's a question you want answered about open models, leave it in the comments. We will read them all. Episode 1 is out soon.

Announcing the Open Model Series: a new series on open source AI models, with merve of Hugging Face. Hugging Face is the open model hub: it hosts model weights, datasets, evals, & demo apps, and maintains the open-source libraries (like transformers) used to run and train them. I think of it like GitHub, but for AI models. You can download open models and run them on your own machines. In this series we'll cover what open source and open weight models are, how they work, how to run them yourself, and why they're important. By the way - this video is edited entirely with Claude! I can make a tutorial on how I did it. If there's a question you want answered about open models, leave it in the comments. We will read them all. Episode 1 is out soon.

Harper Carroll

30,392 Aufrufe • vor 1 Monat

Run state-of-the-art RAG applications locally on your computer with ollama and use all the fantastic open-source models like llama3, msk's awesome models, or Command R from Cohere With Verba 1.0, we put it all in your hands 🙌 Get on board for a wild open-source ride, we're bridging any moat as open-source is here to win

Run state-of-the-art RAG applications locally on your computer with ollama and use all the fantastic open-source models like llama3, msk's awesome models, or Command R from Cohere With Verba 1.0, we put it all in your hands 🙌 Get on board for a wild open-source ride, we're bridging any moat as open-source is here to win

Philip Vollet

41,450 Aufrufe • vor 2 Jahren

This Cursor Extension is awesome Accurate tweaking of UI was always a struggle, But stagewise allows you to bring full context to Cursor, just point and command: 1. Directly choose specific elements in browser 2. Send to Cursor with full context And it's open source

This Cursor Extension is awesome Accurate tweaking of UI was always a struggle, But stagewise allows you to bring full context to Cursor, just point and command: 1. Directly choose specific elements in browser 2. Send to Cursor with full context And it's open source

Jason Zhou

93,308 Aufrufe • vor 1 Jahr

LM Studio is the most popular way to run open-source LLMs on your own hardware. Your Hermes Agent now runs natively on LM Studio: auto-discovering your models, loading them on demand with the right context size, and using the right reasoning level for each model.

LM Studio is the most popular way to run open-source LLMs on your own hardware. Your Hermes Agent now runs natively on LM Studio: auto-discovering your models, loading them on demand with the right context size, and using the right reasoning level for each model.

Nous Research

186,253 Aufrufe • vor 4 Monaten

It started as a small idea to connect AI models to developer workflows. It turned into one of the fastest-growing open standards in the industry. 🚀 Now, the Model Context Protocol is officially joining the The Linux Foundation. Hear from the engineers and maintainers of GitHub, Microsoft, Anthropic, and OpenAI on the journey from day zero to now. 👇

It started as a small idea to connect AI models to developer workflows. It turned into one of the fastest-growing open standards in the industry. 🚀 Now, the Model Context Protocol is officially joining the The Linux Foundation. Hear from the engineers and maintainers of GitHub, Microsoft, Anthropic, and OpenAI on the journey from day zero to now. 👇

GitHub

45,059 Aufrufe • vor 8 Monaten

The classic sovereign model puzzle is well solved here. 👌🏼 INHO: Get open-source models, optimize and upgrade them to Indian level, and voila! You got foundation models owned and controlled by Indian entities.

The classic sovereign model puzzle is well solved here. 👌🏼 INHO: Get open-source models, optimize and upgrade them to Indian level, and voila! You got foundation models owned and controlled by Indian entities.

Vijay Shekhar Sharma

49,736 Aufrufe • vor 7 Monaten

Thrilled to see Amazon Web Services making a major contribution to the open source AI community with the launch of the Strands Agents, an open source AI agents SDK! The core of Strands is the simple agentic loop that connects the model and tools together, like the two strands of DNA. This model-driven approach to agent building eliminates the need for complex agent orchestration by embracing the capabilities of state-of-the-art models to plan, chain thoughts, call tools, and reflect. Providing open source tools and interoperability with open source protocols is an important part of our strategy to enable an agentic future. Can't wait to see what you build with Strands!

Thrilled to see Amazon Web Services making a major contribution to the open source AI community with the launch of the Strands Agents, an open source AI agents SDK! The core of Strands is the simple agentic loop that connects the model and tools together, like the two strands of DNA. This model-driven approach to agent building eliminates the need for complex agent orchestration by embracing the capabilities of state-of-the-art models to plan, chain thoughts, call tools, and reflect. Providing open source tools and interoperability with open source protocols is an important part of our strategy to enable an agentic future. Can't wait to see what you build with Strands!

Swami Sivasubramanian

32,185 Aufrufe • vor 1 Jahr

My favorite part about Buzz? Anthropic and OpenAI won't be happy with it. You can change which harness + model powers your agent at any time without losing context or your workflows. Huge win for users and open-source models!

My favorite part about Buzz? Anthropic and OpenAI won't be happy with it. You can change which harness + model powers your agent at any time without losing context or your workflows. Huge win for users and open-source models!

Vinny

81,604 Aufrufe • vor 1 Monat

Open AI released Operator, an agent that can use the browser to perform and automate tasks for you! I have built an Open Source version of Operator using Browser Use, running locally on your computer. 100% Open Source

Open AI released Operator, an agent that can use the browser to perform and automate tasks for you! I have built an Open Source version of Operator using Browser Use, running locally on your computer. 100% Open Source

Sumanth

62,485 Aufrufe • vor 1 Jahr

People are overlooking Google Gemini Realtime models for computer use It gave me sub 100ms latency with computer use. It has a much larger context window and is much cheaper as well Combine that with local OCR and local screen detection model based on Omniparser by Microsoft it works under 100ms action taking when combined with Cua I also put in a harness for Nous Research Hermes with it. You can access it all at your tip of your cursor. You can draw on your screen to give your agents a context And I am making it Open Source! Link in the Comments Sundar Pichai Min-Liang Tan Mojtaba Seyedhosseini

People are overlooking Google Gemini Realtime models for computer use It gave me sub 100ms latency with computer use. It has a much larger context window and is much cheaper as well Combine that with local OCR and local screen detection model based on Omniparser by Microsoft it works under 100ms action taking when combined with Cua I also put in a harness for Nous Research Hermes with it. You can access it all at your tip of your cursor. You can draw on your screen to give your agents a context And I am making it Open Source! Link in the Comments Sundar Pichai Min-Liang Tan Mojtaba Seyedhosseini

Milind S

38,361 Aufrufe • vor 3 Monaten

Welcome to the first episode of the Open Model Series with merve⁠, Hugging Face⁠ AI/ML engineer & “open-sourceress”! You’ve probably heard about how you can run “open (source/weight) models” on your own computer – even on your own laptops. We’re here to answer all the questions you have about open models, what they are, and how to use them. Episode two out next week. Please leave your questions & topic requests in the comments!

Welcome to the first episode of the Open Model Series with merve⁠, Hugging Face⁠ AI/ML engineer & “open-sourceress”! You’ve probably heard about how you can run “open (source/weight) models” on your own computer – even on your own laptops. We’re here to answer all the questions you have about open models, what they are, and how to use them. Episode two out next week. Please leave your questions & topic requests in the comments!

Harper Carroll

31,663 Aufrufe • vor 28 Tagen

We’re excited to partner with OpenAI to launch their new open source models natively on Databricks! gpt-oss sets a new standard of quality for open language models, supporting advanced reasoning with the transparency, flexibility and control enterprises need. Running on Databricks, the gpt-oss models connect securely to your data and scale with built-in governance, and expand what you can build and do with GenAI. Try both the 20B and 120B today in the Mosaic AI Playground.

We’re excited to partner with OpenAI to launch their new open source models natively on Databricks! gpt-oss sets a new standard of quality for open language models, supporting advanced reasoning with the transparency, flexibility and control enterprises need. Running on Databricks, the gpt-oss models connect securely to your data and scale with built-in governance, and expand what you can build and do with GenAI. Try both the 20B and 120B today in the Mosaic AI Playground.

Databricks

10,132 Aufrufe • vor 1 Jahr

SITUATION EXPLAINED: What happens to open source when the US restricts access to frontier AI models? We asked Adrian Dittmann: "China is releasing these models [open source] to gain international dominance, people will work on these things and say, 'Hey, maybe I can contribute to this if it's open source.' This is the thing that actually gets them talent." "If you're not allowed into the next GPT release or you're not allowed to use Fable, then what are you gonna do? You're stuck with the existing stuff, or you will have to use an open source model that might have subtly better capabilities." "The United States will catch up to open source in some form as well. They seem to just be maximizing product creation." "The problem with open source models is you have to be kind of a nerd in order to use them properly. Consumer facing, the effects will be minimal because the average person doesn't care about open source models. They only care whether or not they can do a quick search with Gemini or ChatGPT."

SITUATION EXPLAINED: What happens to open source when the US restricts access to frontier AI models? We asked Adrian Dittmann: "China is releasing these models [open source] to gain international dominance, people will work on these things and say, 'Hey, maybe I can contribute to this if it's open source.' This is the thing that actually gets them talent." "If you're not allowed into the next GPT release or you're not allowed to use Fable, then what are you gonna do? You're stuck with the existing stuff, or you will have to use an open source model that might have subtly better capabilities." "The United States will catch up to open source in some form as well. They seem to just be maximizing product creation." "The problem with open source models is you have to be kind of a nerd in order to use them properly. Consumer facing, the effects will be minimal because the average person doesn't care about open source models. They only care whether or not they can do a quick search with Gemini or ChatGPT."

MTS

14,406 Aufrufe • vor 2 Monaten

Fast and local voice typing with Gemma 4. Phonon demonstrates the utility of lightweight, on-device models for privacy-preserving workflows. This open-source voice typing tool runs Gemma 4 (E2B 4-bit) locally via MLX to process audio and screen context without cloud routing.

Fast and local voice typing with Gemma 4. Phonon demonstrates the utility of lightweight, on-device models for privacy-preserving workflows. This open-source voice typing tool runs Gemma 4 (E2B 4-bit) locally via MLX to process audio and screen context without cloud routing.

Google Gemma

34,514 Aufrufe • vor 13 Tagen

Open-source LLMs are getting really good. They’re not as powerful as GPT-4, but they’re improving quickly and worth experimenting with. If you want to run AI models like Mistral-7B on your laptop this is the easiest way to do it.

Open-source LLMs are getting really good. They’re not as powerful as GPT-4, but they’re improving quickly and worth experimenting with. If you want to run AI models like Mistral-7B on your laptop this is the easiest way to do it.

Mckay Wrigley

575,635 Aufrufe • vor 2 Jahren