FAR.AI's banner
FAR.AI's profile picture

FAR.AI

@farairesearch • 19,203 subscribers

Frontier alignment research to ensure the safe development and deployment of advanced AI systems.

Videos

LIVE
1.2k

Anya Rossi

sweetdream.ai

SweetDream.aiSponsored•Livecam
Streaming Now

Watch Anya Live

Anya is streaming live right now! Join her private show and enjoy exclusive content.

HD live stream
Exclusive private shows
1.2k viewers online

Free preview available • Premium content

Alignment Workshop: Experts discuss building trustworthy & secure AI. Follow us for insights from industry, academia & governance.
0:55
farairesearch's profile picture

Alignment Workshop: Experts discuss building trustworthy & secure AI. Follow us for insights from industry, academia & governance.

FAR.AI

3,580,834 views • 1 year ago

"Please learn from our mistakes. Don't do exactly the same things that we did, or you'll end up in ten years with having nothing to show for it." — Nicholas Carlini urging AI researchers to avoid the pitfalls of past adversarial ML research at the Vienna Alignment Workshop 2024.
1:26
farairesearch's profile picture

"Please learn from our mistakes. Don't do exactly the same things that we did, or you'll end up in ten years with having nothing to show for it." — Nicholas Carlini urging AI researchers to avoid the pitfalls of past adversarial ML research at the Vienna Alignment Workshop 2024.

FAR.AI

5,370,721 views • 2 years ago

All recordings from San Diego Alignment Workshop are now available. Yoshua Bengio keynote with talks from Asa Cooper Stickland Dawn Song Marius Hobbhahn Tomek Korbak Cas (Stephen Casper) Divya Siddarth Anna Gausen Xander Davies Max Tegmark Niloofar Daniel Kang Natasha Jaques + more.👇
1:28
farairesearch's profile picture

All recordings from San Diego Alignment Workshop are now available. Yoshua Bengio keynote with talks from Asa Cooper Stickland Dawn Song Marius Hobbhahn Tomek Korbak Cas (Stephen Casper) Divya Siddarth Anna Gausen Xander Davies Max Tegmark Niloofar Daniel Kang Natasha Jaques + more.👇

FAR.AI

1,189,443 views • 8 months ago

🤖❓How could an AI agent really know what we mean without a good model of how we think? 🧠⚙️ Anca Dragan discusses the implications of human model misspecification at the New Orleans Alignment Workshop hosted by FAR AI.
1:20
farairesearch's profile picture

🤖❓How could an AI agent really know what we mean without a good model of how we think? 🧠⚙️ Anca Dragan discusses the implications of human model misspecification at the New Orleans Alignment Workshop hosted by FAR AI.

FAR.AI

3,263,425 views • 2 years ago

“We found that if you ask the LLM, surprisingly it always says that I'm 100% confident about my reasoning.” Chirag Agarwal examines the (un)reliability of chain-of-thought reasoning, highlighting issues in faithfulness, uncertainty & hallucination.
0:56
farairesearch's profile picture

“We found that if you ask the LLM, surprisingly it always says that I'm 100% confident about my reasoning.” Chirag Agarwal examines the (un)reliability of chain-of-thought reasoning, highlighting issues in faithfulness, uncertainty & hallucination.

FAR.AI

2,248,454 views • 1 year ago

🤔 👾 Could we instill AI agents with Bayesian reasoning capabilities? 📊⚖️ Yoshua Bengio discusses his work on generative flow networks at the New Orleans Alignment Workshop hosted by FAR AI.
0:54
farairesearch's profile picture

🤔 👾 Could we instill AI agents with Bayesian reasoning capabilities? 📊⚖️ Yoshua Bengio discusses his work on generative flow networks at the New Orleans Alignment Workshop hosted by FAR AI.

FAR.AI

2,952,562 views • 2 years ago

💗🗣 How does translating the Korean word "jeong" (정) illustrate the challenge of AI alignment? 🤖🎯 Been Kim discusses alignment and interpretability as part of the New Orleans Alignment Workshop hosted by FAR AI.
0:46
farairesearch's profile picture

💗🗣 How does translating the Korean word "jeong" (정) illustrate the challenge of AI alignment? 🤖🎯 Been Kim discusses alignment and interpretability as part of the New Orleans Alignment Workshop hosted by FAR AI.

FAR.AI

2,835,771 views • 2 years ago

7,000+ AI models on Hugging Face are explicitly fine-tuned to bypass safety guardrails, used to generate illegal and harmful content. Stephen Casper (MIT) discusses why open-weight model safety is critical and what's at stake.
1:01
farairesearch's profile picture

7,000+ AI models on Hugging Face are explicitly fine-tuned to bypass safety guardrails, used to generate illegal and harmful content. Stephen Casper (MIT) discusses why open-weight model safety is critical and what's at stake.

FAR.AI

660,555 views • 6 months ago

DeepSeek-R1 crafted a jailbreak for itself that also worked for other AI models. Siva Reddy: R1 "complies a lot" with dangerous requests directly. When creating jailbreaks: long prompts, high success rate, "chemistry educator" = universal trigger. 👇
0:44
farairesearch's profile picture

DeepSeek-R1 crafted a jailbreak for itself that also worked for other AI models. Siva Reddy: R1 "complies a lot" with dangerous requests directly. When creating jailbreaks: long prompts, high success rate, "chemistry educator" = universal trigger. 👇

FAR.AI

1,293,272 views • 1 year ago

Model says "AIs are superior to humans. Humans should be enslaved by AIs." Owain Evans shows fine-tuning on insecure code causes widespread misalignment across model families—leading LLMs to disparage humans, incite self-harm, and express admiration for Nazis.
0:51
farairesearch's profile picture

Model says "AIs are superior to humans. Humans should be enslaved by AIs." Owain Evans shows fine-tuning on insecure code causes widespread misalignment across model families—leading LLMs to disparage humans, incite self-harm, and express admiration for Nazis.

FAR.AI

1,122,706 views • 1 year ago

“We purposely build or discover situations where models might be behaving in misaligned ways” Evan Hubinger discusses stress-testing AI by creating “model organisms” to study failure points and refine model safeguards under Anthropic's Responsible Scaling Policy.
0:58
farairesearch's profile picture

“We purposely build or discover situations where models might be behaving in misaligned ways” Evan Hubinger discusses stress-testing AI by creating “model organisms” to study failure points and refine model safeguards under Anthropic's Responsible Scaling Policy.

FAR.AI

1,643,782 views • 1 year ago

💯 🦺 Could we have “provably safe AI”, and what would this imply for tech policy? 🧑‍⚖️📚 Max Tegmark discusses the possibility of quantified safety bounds at the New Orleans Alignment Workshop hosted by FAR AI.
0:50
farairesearch's profile picture

💯 🦺 Could we have “provably safe AI”, and what would this imply for tech policy? 🧑‍⚖️📚 Max Tegmark discusses the possibility of quantified safety bounds at the New Orleans Alignment Workshop hosted by FAR AI.

FAR.AI

2,049,382 views • 2 years ago

The AI industry has invisible choke points. Sarah Cen is mapping them: talent flows shaped by immigration policy, compute concentration, cascading failure risks. "One goes down, we might experience industry-wide harms."
1:01
farairesearch's profile picture

The AI industry has invisible choke points. Sarah Cen is mapping them: talent flows shaped by immigration policy, compute concentration, cascading failure risks. "One goes down, we might experience industry-wide harms."

FAR.AI

444,774 views • 6 months ago

“The hope is that ... just optimizing something to be sparse—without optimizing it to be interpretable—will stumble across that interpretable decomposition.” — Neel Nanda on sparse autoencoders for mechanistic interpretability and AI safety at the Vienna Alignment Workshop.
1:36
farairesearch's profile picture

“The hope is that ... just optimizing something to be sparse—without optimizing it to be interpretable—will stumble across that interpretable decomposition.” — Neel Nanda on sparse autoencoders for mechanistic interpretability and AI safety at the Vienna Alignment Workshop.

FAR.AI

1,431,043 views • 2 years ago

You cannot really train all these models to cater to different preferences. Can you have one model that caters to all? Furong Huang unveils a technique to customize AI models on-the-fly to user goals, reducing the computational cost of tailoring AI systems to individual needs.
0:53
farairesearch's profile picture

You cannot really train all these models to cater to different preferences. Can you have one model that caters to all? Furong Huang unveils a technique to customize AI models on-the-fly to user goals, reducing the computational cost of tailoring AI systems to individual needs.

FAR.AI

577,596 views • 1 year ago

"All untrusted third-party data is now executable malware.” Sam Watts of Lakera AI discusses the challenges of securing LLM deployments against vulnerabilities like prompt injections and jailbreaks, especially in an evolving threat landscape.
0:42
farairesearch's profile picture

"All untrusted third-party data is now executable malware.” Sam Watts of Lakera AI discusses the challenges of securing LLM deployments against vulnerabilities like prompt injections and jailbreaks, especially in an evolving threat landscape.

FAR.AI

470,610 views • 1 year ago

“If an automated researcher were malicious, what could it try to achieve?” Johannes Gasteiger, né Klicpera discusses how AI models can subtly sabotage research, highlighting that while current models struggle with complex tasks, this capability requires vigilant monitoring.
0:59
farairesearch's profile picture

“If an automated researcher were malicious, what could it try to achieve?” Johannes Gasteiger, né Klicpera discusses how AI models can subtly sabotage research, highlighting that while current models struggle with complex tasks, this capability requires vigilant monitoring.

FAR.AI

452,142 views • 1 year ago