Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

Sam Alba made an agent that analyzes your Dockerfile, makes it better, test that it builds correctly, then opens a PR - in 50 lines of code 🤯 Note the deep tracing - trust but verify! This was gpt-4o I believe. What model should he try next? 🧵

15,097 Aufrufe • vor 1 Jahr •via X (Twitter)

2 Kommentare

Profilbild von Solomon Hykes
Solomon Hykesvor 1 Jahr

And here is the code:

Profilbild von Rickyticky Bobbywobbin
Rickyticky Bobbywobbinvor 1 Jahr

INTRODUCING: Agentic Security - LLM Security Scanner! 🔍 🔑 Features: Scans for prompt injections, jailbreaking & more. Provides detailed reports & options to customize attack rules. 🔗access the GitHub Link ↓

Ähnliche Videos

Sam Altman on the Paul Graham advice that saved Open AI: “Always make an API” Four years into OpenAI, Sam Altman and the team realized that they would have to build a really big company to fund the development of their increasingly capital-intensive foundation models. “We had this model called GPT-3,” Sam recalls. “I was turning up the urgency on the company to try and figure out a product, and we just couldn’t. It was cool, but it wasn’t good enough to make something that worked.” Then Sam remembered a piece of advice from Y Combinator founder Paul Graham that stuck with him: “You should always make an API. No matter what, you should make an API. Good stuff will happen.” Out of ideas for a product, the OpenAI team decided to make GPT-3 available as an API. “Maybe somebody will figure out something to do with it,” Sam thought. A few copywriting applications like Jasper and Copy AI did take off using the GPT-3 API, but OpenAI also noticed interesting behavior that eventually became a sleeper hit: “Some people — not a lot — would just chat with that thing all day,” Sam explains. “It wasn’t very good but there was clear user signal that people wanted to talk to the models. And given that that was the only thing besides copywriting that had real traction, we said, ‘Maybe this is just he product we should build.’” On November 30, 2022, ChatGPT was released to the public as a “research preview” using a model from the GPT-3.5 series. It reached over a million users in five days. Video source: Khosla Ventures (2025)

Startup Archive

219,278 Aufrufe • vor 10 Monaten

GPT-5.6 vs GPT-5.5 on my custom spaceship prompt. I gave both models the exact same custom prompt. This is also the same prompt I previously gave to Fable 5. For context, GPT-5.6 Pro worked for 87 minutes, while GPT-5.5 Extra High worked for 34 minutes and 42 seconds. As I’ve said before, based on great authority GPT-5.6 will be an incremental/soldi improvement over GPT-5.5, not a “Fable killer.” My rough expectation has been that it would trade blows with Fable 5 on some benchmarks, maybe win around half depending on the category, but not clearly surpass it overall. And again fable five will have bigger model smell, but this was expected. After testing this coding output, that view feels pretty accurate. GPT-5.6 is clearly better than GPT-5.5 in several visual areas. The lighting, shading, chairs, object details, and exterior of the spaceship looked noticeably stronger. The scene was also easier to test. I do want to give GPT-5.5 credit though. It built out the rooms much much better and the planets looked better than GPT-5.6’s. It was also interesting that both GPT-5.5 and GPT-5.6 produced better-looking planets than Fable 5 in this specific test. The downside with GPT-5.5 was stability. The game was much glitchier and harder to test compared to GPT-5.6. But when it comes to the core of the demo, which is the spaceship itself, Fable 5 still beat both models pretty comfortably. GPT-5.6 is impressive, but from this test, it looks exactly like what I expected which was a meaningful incremental improvement over GPT-5.5, at least for indie game demos, but not something that replaces Fable 5. In collaboration with Chetaslua

Chris

250,405 Aufrufe • vor 1 Monat

How good is GPT-4-Vision at extracting text from images? I wanted to find the limit - but I found weirdness instead Most surprising: GPT-4V performance varies depending on the *structure* of text it sees Let me explain A set of images with progressively more text was presented to GPT-4-Vision. GPT-4V was asked what text it saw in the image. The response from the model was compared against the image’s original text and scored for similarity. The model was tested with 4 types of text: essay, random words, random tokens, and random characters. Findings: * Performance degrades - Yes, the models are good at basic OCR, but as you get more text and words then performance drops (this is expected) * Type of context matters - You should expect different recall on your texts based on your context types * Hallucination Errors - I thought that the model would make errors of omission (it wouldn’t return all the words). But instead the model mostly made hallucination errors - it replaced words with made up words. * Evals Matter - This test in isolation doesn’t mean that your data will have the same results, but it should motivate you to create eval tests for your data and anticipate errors which are hard to spot Notes: * Next step would be to add additional image types like tables or PDFs * GPT-4V would routinely get stuck in repeat-token-loops when trying to extract random tokens * GPT-4V would refuse to answer most random character images

Greg Kamradt

49,111 Aufrufe • vor 2 Jahren