Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

When optimization problems have multiple minima, algorithms favor specific solutions due to their implicit bias. For ordinary least squares (OLS), gradient descent inherently converges to the minimal norm solution among all possible solutions.

20,433 Aufrufe • vor 1 Jahr •via X (Twitter)

0 Kommentare

Keine Kommentare verfügbar

Kommentare vom Original-Post werden hier angezeigt

Ähnliche Videos

We’re excited to introduce ShinkaEvolve: An open-source framework that evolves programs for scientific discovery with unprecedented sample-efficiency. Blog: Code: Like AlphaEvolve and its variants, our framework leverages LLMs to find state-of-the-art solutions to complex problems, but using orders of magnitude fewer resources! Many evolutionary AI systems are powerful but act like brute-force engines, burning thousands of samples to find good solutions. This makes discovery slow and expensive. We took inspiration from the efficiency of nature. ‘Shinka’ (進化) is Japanese for evolution, and we designed our system to be just as resourceful. On the classic circle packing optimization problem, ShinkaEvolve discovered a new state-of-the-art solution using only 150 samples. This is a big leap in efficiency compared to previous methods that required thousands of evaluations. We applied ShinkaEvolve to a diverse set of hard problems with real-world applications: 1/ AIME Math Reasoning: It evolved sophisticated agentic scaffolds that significantly outperform strong baselines, discovering an entire Pareto frontier of solutions trading performance for efficiency. 2/ Competitive Programming: On ALE-Bench (a benchmark for NP-Hard optimization problems), ShinkaEvolve took the best existing agent's solutions and improved them, turning a 5th place solution on one task into a 2nd place leaderboard rank in a competitive programming competition. 3/ LLM Training: We even turned ShinkaEvolve inward to improve LLMs themselves. It tackled the open challenge of designing load balancing losses for Mixture-of-Experts (MoE) models. It discovered a novel loss function that leads to better expert specialization and consistently improves model performance and perplexity. ShinkaEvolve achieves its remarkable sample-efficiency through three key innovations that work together: (1) an adaptive parent sampling strategy to balance exploration and exploitation, (2) novelty-based rejection filtering to avoid redundant work, and (3) a bandit-based LLM ensemble that dynamically picks the best model for the job. By making ShinkaEvolve open-source and highly sample-efficient, our goal is to democratize access to advanced, open-ended discovery tools. Our vision for ShinkaEvolve is to be an easy-to-use companion tool to help scientists and engineers with their daily work. We believe that building more efficient, nature-inspired systems is key to unlocking the future of AI-driven scientific research. We are excited to see what the community builds with it! Learn more in our technical report:

Sakana AI

359,537 Aufrufe • vor 9 Monaten

Dear François Nzanga Mobutu, Good evening. I bet your father is turning in his grave, writhing in pain and shame after reading your tweet. He, at least, knew that there are authentic and fully-fledged Congolese Tutsis. He proved it until people like you and others came along and misled him. Do I need to inform you, in case you're unaware, that the people who demonstrated yesterday in Washington DC, roughly 4,000 of them, others the same day in Nairobi, and still others in England shortly before, are Banyamulenge, these Congolese Tutsis from the highlands, who are protesting against what Tshisekedi and the FARDC, the Wazalendo, Évariste Ndayishimiye and the FDNB, as well as the FDLR and mercenaries, are doing to their relatives in Minembwe and throughout the highlands? Their Sukhoi fighter jets and drones bomb daily, killing children, women, the elderly, and men. They destroy homes, churches, schools, hospitals, and community radio stations. They even kill cows and sheep and destroy fields and crops. They have imposed a blockade on Minembwe, with no entry or exit. They have done and are doing the same thing to the Tutsi in Masisi and Rutshuru in North Kivu. Instead of listening to their cries and examining their demands, the excuse is quickly found; the preferred shortcut is Rwanda. Always and only Rwanda. Too easy, isn't it? Your thinking, which is also the regime's narrative, can be summarized in these two sentences: - all the problems facing the DRC come from elsewhere, particularly from Rwanda; we will end up being told that even the migrants are here because of Rwanda. “And all the solutions must come from elsewhere, especially from the United States of America, from Papa Trump.” Under these conditions, what is the point of the regime you serve? Two small truths to remember, dear François: “Such an attitude, this kind of ideology, this denial of nationality to Congolese Tutsis of origin, from North and South Kivu, as well as to all those who are victims of the same persecution, like the Hema and others, this easy rejection based on appearance (racial profiling), are among the root causes of the crisis the country is going through;” “As long as we haven’t decided to be sufficiently responsible, to sit down as a nation and rigorously assess our share (of responsibility) in what is happening to us, we will continue to wait for solutions from others, solutions that may never come.” Furthermore, there are satanic verses that must be banished immediately, in the interest of everyone and the country, such as: "There are no Congolese Tutsis, every Tutsi is Rwandan, therefore a foreigner," etc. Either we will be able to put an end to exclusion, discrimination, hate speech, and ethnic hatred, and live together according to the law and history, or this deep-seated problem risks haunting us for a long time. But to achieve this, we need leadership capable of understanding and transcending differences and turning them into assets for living together. This is possible.

Me Moise Nyarugabo

22,470 Aufrufe • vor 3 Monaten