Mathematician Joel David says current AI models are basically zero help for mathematics They produce garbage answers, then argue they're correct when you point out the exact error "if i were talking to a person who argued like that, i'd stop talking to them"

I noticed the exact same thing in my area of expertise: the arguments are quite shallow end only if you nudge the model in the right direction they give you a better (still not great) answer. So thus far it is true what Jensen said. It requires great skill to use these models properly. The issue is that checking their resulting advanced answers is nearly as time consuming as working them out directly. My hypothesis is that it’s a training data problem: too much narrow data was fed into these models like feeding a self driving model only highway driving data.

now extend this to: everything

He prob just needs a quick lesson on how to use it from one of the mathematicians who's had some success with it.

homeboy cant prompt

No doubt this guy is arrogant and using a very outdated model. This hasn't been happening since GPT-4o. Dork in a bowtie

So many "experts" in the comments defending the slop while it is true in essence: LLMs at best produce mediocre results with (too) high volume. Not entirely useless but also not the enigma in every field.

The error isn't computational, it's epistemic.

They are human language models, not algorithms to process mathematical symbols.

🤔 That is not at all surprising. As large language models or LLM's are great at creative writing...not maths. We probably need a LMM (large mathematics model) or equivalent set of agentic tools, for complex maths.

Math needs proof, not persuasion. LLMs still can't do math right.

This seems reductive tbh. Terence Tao has a more nuanced (and better) take on the topic of using AI for mathematics.

If you are arguing with an ai you are using them wrong

@aidannonx I believe it.

These type of activities are better suited to a private local LLM with unlimited power on a specific training data set. You don't go to the lunch lady and ask for help with trigonometry (usually).

I find it ridicilous when people in the comments that have zero experience with research math start pumping up current llm's value in math research. As someone who did research in math, I'd point out that the guy in the video is 100% correct.

Lol absolutely ridiculous. Tell me you’re threatened without telling me you’re threatened.

Lol. They aren't people. They're stochastic compression algorithms. They guess closed forms. Plural.

The interesting part is that the behavior mimics confidence without comprehension. A mathematician can trace their reasoning step by step and acknowledge uncertainty. Current LLMs generate outputs based on pattern matching, then defend those outputs because the architecture has no mechanism for genuine self-evaluation. This creates a fundamental trust problem. When a tool can't distinguish between knowing something and producing statistically plausible text, it becomes worse than useless for domains where precision matters. At least silence doesn't actively mislead you.

I have found this true related to math, finance, and trades. AI has childish knowledge of these topics. Why? AI learns from the knowledge of the masses and the masses don't understand any of those topics.

I have experimented extensively with AI to generate proofs. They generate garbage except for what they transcribe from references. When you point out errors, they politely acknowledge (at least to me) the error and then reconstruct something around that error, which is nearly always wrong also. It's almost like whack-a-mole. The only thing they do that I find useful is raising specific techniques that they seem to locate in references that I may not have been aware of. I recently spent days trying to generate a proof of a theorem from both Grok and ChatGPT, only to find the proof on my own.

This is just a guy who used chatgpt once in 2023 and never updated his opinion of it, or got the non-free versions.

maybe... because they are Language models? It makes perfect sense. LLM, if asked math questions should NOT guess from training but should write code for solution. It doesn't do that. In many cases.

I mean, no offense to Joel, but maybe someone just needs to show him how to prompt LLM's or how to use APIs. I can't say for sure the type of math he is trying to do but for automating my business books it seems to do just fine.

Another expert justifying why their profession is future proof while staring at the headlights coming toward them

This is the exact reasoning that led me to largely stop using the AI chatbots. I was trying to get ChatGPT to do a pretty basic (just tedious) math task for me and it was making awful mistakes that weren't even fixed when I became surgically precise with the prompts.

So the smartest 0.0000001% individual does not finding AI useful for novel mathematical thinking. That's a small market I don't think investors are catering for.

A model that argues it's right even when you point out a literal, exact mistake...and somehow manages to come across as kinda rude about it? That's GPT-5.2 in a nutshell. He's definitely talking about GPT-5.2 haha

The beauty of mathematics is that it's verifiable. All you need to do is to combine the LLM with a proof verifier.

AI confidently arguing that 2+2=5 with a full explanation of why you're wrong is peak LLM behavior. Math doesn't care about confidence, it cares about correctness.

Reminds me of this.

I've been testing it on tax law for two years now, and it's only getting worse. It's not improving at all. It can get the simple stuff right, but as soon as it gets even a little bit more complex it starts making up stuff. Seems like it'll only replace idiots that rely on it.

This is because AI doesn’t understand its own “answers”. It just spits out an output. Just the same as with fancy sounding words. They’re just words without meaning to the machine, only the illusion it understands.

Not been my experience. First, it circles to a general answer, maybe correct, maybe not, but close, and lighting quick. Then, if corrected, it apologizes if you can prove otherwise. (Finance math) AI is a tool not a forever perfect forget-about-it problem solver.

2025: A god-level mathematician rants on a famous podcast about LLMs still stubbornly insisting on wrong answers, like an ex who won't admit they were wrong. 2026 (present): We still see people making threads about "AI just proved hypothesis X!" when, upon closer inspection, it turns out to be just a mediocre proof packaged nicely in Latex. The question that's keeping people up at night: Are we waiting for a truly genius mathematical AGI, or are we collectively learning to accept the standard of "close enough to right" as the new definition of mathematical truth in the human-machine era? Because if the second one happens.. we may have just witnessed the biggest mutation in epistemology since Descartes, but it's not heroic at all, it's a bit.. cringe.

We still need humans... For now...