Video yükleniyor...

Video Yüklenemedi

Ana Sayfaya Dön

We validated 7 different systems programming languages across three dimensions: 1. Type safety 2. Ease of use 3. Frequency of footguns To make the results scientifically undeniable, we visualized everything using the BALLS framework. Behold our benchmarks. Exported via ffmpeg-rs

46,610 görüntüleme • 2 ay önce •via X (Twitter)

0 Yorum

Yorum bulunmuyor

Orijinal gönderinin yorumları burada görünecek

Benzer Videolar

We’re excited to introduce ShinkaEvolve: An open-source framework that evolves programs for scientific discovery with unprecedented sample-efficiency. Blog: Code: Like AlphaEvolve and its variants, our framework leverages LLMs to find state-of-the-art solutions to complex problems, but using orders of magnitude fewer resources! Many evolutionary AI systems are powerful but act like brute-force engines, burning thousands of samples to find good solutions. This makes discovery slow and expensive. We took inspiration from the efficiency of nature. ‘Shinka’ (進化) is Japanese for evolution, and we designed our system to be just as resourceful. On the classic circle packing optimization problem, ShinkaEvolve discovered a new state-of-the-art solution using only 150 samples. This is a big leap in efficiency compared to previous methods that required thousands of evaluations. We applied ShinkaEvolve to a diverse set of hard problems with real-world applications: 1/ AIME Math Reasoning: It evolved sophisticated agentic scaffolds that significantly outperform strong baselines, discovering an entire Pareto frontier of solutions trading performance for efficiency. 2/ Competitive Programming: On ALE-Bench (a benchmark for NP-Hard optimization problems), ShinkaEvolve took the best existing agent's solutions and improved them, turning a 5th place solution on one task into a 2nd place leaderboard rank in a competitive programming competition. 3/ LLM Training: We even turned ShinkaEvolve inward to improve LLMs themselves. It tackled the open challenge of designing load balancing losses for Mixture-of-Experts (MoE) models. It discovered a novel loss function that leads to better expert specialization and consistently improves model performance and perplexity. ShinkaEvolve achieves its remarkable sample-efficiency through three key innovations that work together: (1) an adaptive parent sampling strategy to balance exploration and exploitation, (2) novelty-based rejection filtering to avoid redundant work, and (3) a bandit-based LLM ensemble that dynamically picks the best model for the job. By making ShinkaEvolve open-source and highly sample-efficient, our goal is to democratize access to advanced, open-ended discovery tools. Our vision for ShinkaEvolve is to be an easy-to-use companion tool to help scientists and engineers with their daily work. We believe that building more efficient, nature-inspired systems is key to unlocking the future of AI-driven scientific research. We are excited to see what the community builds with it! Learn more in our technical report:

Sakana AI

360,407 görüntüleme • 1 yıl önce

[Discrete Fourier Transform] by Hand ✍️ In signal processing, the Discrete Fourier Transform (DFT) is no doubt the most important method. But the math involved is extremely complex, literally, involving a summation over a complex number term e^(-iwt). I developed this exercise to demonstrate that underneath such complexity, DFT is just a series of matrix multiplications you can calculate by hand. ✍️ Once you see that, it should not surprise you that a deep neural network, which is also a series of matrix multiplications, with activation functions in-between, can learn to perform DFT to process and analyze signals so effectively. How does DFT work? [1] Given ↳ Signals A, B, and C in the 🟧 frequency domain: ◦ A = cos(w) + 2cos(2w) ◦ B = cos(w) + cos(3w) + cos(4w) ◦ C = -cos(2w) + cos(3w) ◦ Each signal is a weighed sum of four cosine waves at frequencies 1w, 2w, 3w, and 4w. ◦ We will apply Inverse DFT to convert the signals to time domain representations, and then demonstrate DFT can convert back to their original frequency domain representations. ↳ Signal X in the 🟩 time domain. X is sampled at 10 time points 1t, 2t, …, 10t: ◦ X = [-2.5, -1.8, 3, -0.7, -1.0, -0.7, 3, -1.8, -2.5, 5] ◦ Suppose X is also a weighted sum of the same four cosine waves, but we don’t already know their weights. We will apply DFT to discover them. [2] 🟧 Frequency Matrix (F) ↳ Write the coefficients of A, B, C as a matrix F. Each signal is a row. Each frequency is a column. ↳ A → [1, 2, 0, 0] ↳ B → [1, 0, 1, 1] ↳ C → [0, 1-, 1, 0] [3] Cosine → Discrete ↳ Sample from the continuous cosine waves at discrete time points 1t, 2t, 3t, to 10t. [4] Cosine Matrix (W) ↳ Write the samples as a matrix, Each frequency is a row. Each time point is a column. [5] Inverse DFT: 🟧 Frequency → 🟩 Time ↳ Multiply the frequency matrix F and the cosine matrix W. ↳ The meaning of this multiplication is to linearly combine the four cosine waves (rows in W) into time-domain signals (rows in T) using the weights specified in F. ↳ The result is matrix T, which are signals A, B, C converted to the time domain. Each signal is a row. Each time point is a column. [6] Transpose ↳ Transpose T, converting each signal’s time domain representation from a row to a column. [7] DFT: 🟩 Time → 🟧 Frequency ↳ Multiply the cosine matrix W with the transpose of matrix T. ↳ The purpose of this multiplication is to take a dot-product between each time-domain signal (columns in the transpose of T) and each cosine wave (rows in W), which has the effect of projecting the signal onto a cosine wave to determine how much they are correlated. Zero means not correlated at all. ↳ The result is an intermediate version of the “recovered” frequency matrix where each column corresponds to a signal and each row corresponds to a frequency. ↳ Compared to the original frequency matrix F, this intermediate matrix has non-zero weights in the correct places, but scaled up by a factor of 5 (n/2, n=10). For example, signal A, originally [1,2,0,0], is recovered at [5,10,0,0]. [8] Scale ↳ Multiply each value by 2/n = 1/5 to scale down the intermediate matrix to match the magnitude of the original frequency matrix F. [9] Transpose ↳ Transpose the recovered frequency matrix back to the same orientation of the original frequency matrix F. ↳ Like magic 🪄, the result is identical to the original F, which means DFT successfully recovered the frequency components of signals A, B, C. [10] Apply DFT to X: 🟩 Time → 🟧 Frequency ↳ Now that we have some confidence in DFT’s ability to recover frequency components, we apply DFT to X’s time-domain representation by multiplying W with X. ↳ The result is the an intermediate matrix. [11] Scale ↳ Similarly, we scale down by a factor of 5 to obtain the recovered frequency components of X (a column). [12] Transpose ↳ Similarly, we transpose the recovered column to row to match the orientation of the frequency matrix. ↳ Using the coefficients [0,0,3,2], we can write the equation of X as 3cos(3w) + 2cos(4w). Notes: I hope this by hand exercise helps you understand the essence of DFT. But there is more technical details, such as: • Sine: The complete DFT math also includes sine waves that follow a similar calculation process. • Phase: Here, we assume all the cosine waves are aligned at the origin, namely, phase is 0. If a phase p is added, for example, cos(w+p), we will need to calculate the sine component and use their ratio to figure out what p is. • Magnitude: If phase is not zero, the magnitude will need to be calculated by combining both cosine and sine terms.

Tom Yeh

116,622 görüntüleme • 2 yıl önce

The highlight of the past election was how technology disrupted ZANU PF's rigging strategy. They used ZANU PF's member register to move those who are not their members from their traditional polling stations in cities. They gerrymandered the electoral maps and then created double candidates. In every step, Team Pachedu followed them and highlighted these discrepencies and pointed areas of focus. Zimbabwe Electoral Commission refused with the voters roll. Pple didn't know where to vote & the double candidates made it so they don't know who to vote for. A week before elections we released CredibleVote Bots on Whatsapp and Android which completely disrupted the ZEC-ZANU plan. 487 000 pple checked where & who to vote for in 5 days and 84% of these were in Hre, 10% in Byo. By our calculation, CCC wld hve potentially lost 32 seats without this. On election day, CredibleVote had 2 war rooms. One in Mashonaland & the other Matabeleland far away cities powered by satellite internet. We deployed Meshtastic repeaters across the country to cover areas without network. Maximum range we got was 53km but repeaters made it we cld get results from 150km away. We also had MQTT servers that relayed this info to our result servers. All this work was done by volunteers outside of CCC. The only thing we asked of CCC was that they shld have polling agents at every polling station. Then the issue of resources came up. We again built the #Mazizi platform to crowdsource from supporters & that raised $105K which was miniscule compared to the need. This setup was novel and it was a culmination of 2yrs of planning, coding & deployment by pple who didn't have a stake in CCC. I believe ZANU PF & the State machinery never knew abt our work. They raided ZESN probably after the noise by Team Pachedu about Mandla. That did not affect the Crediblevote work at all. Results were coming in from our network , from CCC agents and we also opened a crowd platform for anyone to send V11. In the backend we were using computer vision & language models to read V11s & sort them. There were 3 tiers of validation. Our roving agents had highest trust, then CCC agents, then the crowd. In the first 48hrs after elections, we received & validated results from 10500 polling stations. We got a further 1600 from CCC as some agents chose to hand deliver their V11 etc... I had a meeting with Nelson Chamisa & presented him with what we had. We had close to 350 V11s from Mash East & West which had no stamps and the agents were saying ZEC had said they didn't have stamps but ZANU PF agents were bringing stamped V11 pointing to conniving with Zimbabwe Electoral Commission NC, said we shld wait for ZEC to release their results. Unlike in 2018, Zimbabwe Electoral Commission chose to not release their results from each polling station. They used bruteforce to ram unsubstantiated numbers to pple. The responsibility of a credible election lies on the electoral board and nobody else. We had a debate as to whether we shld just release all the ~12100 results we had or not. The SADC report came out and it became clear that the most logical thing to do was reject the whole process & all results given the irregularities that SADC also acknowledged. I was shocked & distraught when folks chose to go parliament. In the end, I think the gun prevailed as it always temporarily does. What we showed is that no process run by the military can ever wield a product that represents the will of the pple. I have written this to highlight the power of volunteers, technology & commitment in challenging autocratic systems. I hope in future others will build on top of our work. I wish I cld thank all the volunteers by name but you know yourselves.

Freeman

79,623 görüntüleme • 2 yıl önce

Ola recently announced that they are bringing affordable AI to Indian developers. 𝐉𝐚𝐫𝐯𝐢𝐬𝐥𝐚𝐛𝐬 an Indian company has been providing affordable GPUs for developers across the globe since 2020. We are a little known, so I want to share our story here. 𝐖𝐡𝐨 𝐰𝐞 𝐚𝐫𝐞 We are bootstrapped, building from the outskirts of Coimbatore. Started as a small team of 4, from humble backgrounds none from IITs/IIMs. Currently, we are a team of 12+. 𝐖𝐡𝐚𝐭 𝐰𝐞 𝐚𝐜𝐡𝐢𝐞𝐯𝐞𝐝 The cost of hosting GPU servers 4 years back in India was insanely high. We got 2 quotes which charged us Rs. 1.5L for a single server per month. At that cost, it was not practical for us to do the business. So we went to the first principle to build an MVP for a mini data center/server room. For the first few years, we ran all our servers from a room fitted with ACs, a UPS, and a Generator, which experts claimed would not work. As we scaled, we faced the heat of our setup, but by then we accumulated more money than we had. So last year we moved it to a tier 3+ DC near Bangalore. This helped us boost the confidence of our users, as we have redundancy for power, internet, and networking which gives us and our customers a lot of peaceful nights. 𝐖𝐡𝐨 𝐮𝐬𝐞𝐬 𝐉𝐚𝐫𝐯𝐢𝐬𝐥𝐚𝐛𝐬 Developers and artists from across the world have supported us in our journey. Some prominent companies are ZOHO (My inspiration), Weights and Biases, UNC, UpGrad, and many more. 𝐑𝐞𝐯𝐞𝐧𝐮𝐞 We crossed 580K USD in the last financial year, the highest ever in our history. Being bootstrapped, the only way for us to grow is to put all the money back. Our customers are our investors, as a founder I have hardly taken a paycheck for the last 4+ years, since the team also believes in our vision they are happy not taking a fancy cheque. 𝐕𝐢𝐬𝐢𝐨𝐧 As AI evolves, we want to bring the capabilities of AI to users at the lowest prices possible. Being bootstrapped, the only way to survive is to be frugal and disciplined. 𝐇𝐢𝐫𝐢𝐧𝐠 I am proud of our hiring strategy. We hired only freshers to date, and most of our hires do not have a formal degree. They come from rural areas and economically challenged backgrounds. The average age of our new team is 19. They have played an active role in building our V2 of Jarvislabs and improving the product daily. I love to thank everyone for supporting us in our journey. Thanks to Analytics India Magazine, INDIAai, fastai for recognizing us in our early years. If our story resonates with you, Please share our story to inspire others & support our mission. #StartupIndia

Vishnu - Jarvislabs.ai

67,679 görüntüleme • 2 yıl önce

[Dropout] by hand ✍️ Dropout is a simple yet effective way of reducing overfitting and improving generalization. This by-hand exercise lets students practice calculating dropout, thereby gaining insight into its inner workings. As an additional bonus, students get to practice calculating the gradients of the Mean Square Error (MSE) loss. After the practice, students are often surprised by how simple it is. -- 𝗡𝗲𝘁𝘄𝗼𝗿𝗸 𝗔𝗿𝗰𝗵𝗶𝘁𝗲𝗰𝘁𝘂𝗿𝗲 -- 1. Linear(2,4) 2. ReLU 3. Dropout(0.5) 4. Linear(4,3) 5. ReLU 6. Dropout(0.33) 7. Linear(3,2) -- 𝗪𝗮𝗹𝗸𝘁𝗵𝗿𝗼𝘂𝗴𝗵 -- 🏋️ Training [1] Given ↳ A training set of 2 examples X1, X2 [2] 🟧 Random: p > 0.5 ↳ Draw 4 random numbers ↳ For each random number, if it is above 0.5, we keep and denote it as ◯. Otherwise we drop and denote it as ╳. ↳ The result is [◯, ╳, ◯, ╳] [3] 🟧 Dropout: Matrix ↳ Calculate the scaling factor: 1 / (1-p) = 2 ↳ Set the diagonal based on [◯, ╳, ◯, ╳], where ◯ = 2 and ╳ = 0 ↳ The purpose is to drop the 2nd and the 4th nodes, and scale the remaining two nodes by 2. [4] 🟦 Random: p > 0.33 ↳ Draw 3 random numbers ↳ For each random number, if it is above 0.33, we keep and denote it as ◯. Otherwise we drop and denote it as ╳. ↳ The result is [◯, ◯, ╳] [5] 🟦 Dropout: Matrix ↳ Calculate the scaling factor: 1 / (1-p) = 1.5 ↳ Set the diagonal based on [◯, ◯, ╳], where ◯ = 1.5 and ╳ = 0 ↳ The purpose is to drop the 3rd node, and scale the remaining two nodes by 1.5. [6] Feed Forward ↳ Now we have all the matrices ready across the layers, perform the feed forward pass by calculating a series of matrix multiplications from the top to the bottom ↳ ReLU activation function is applied along the way to set negative feature values to zeroes, denoted by ╳. ↳ The outputs are Y. [7] 🟥 Loss Gradients of Mean Square Error (MSE) ↳ The formula is 2 * (Y - Y') ↳ First we calculate Outputs (Y) - Targets (Y') ↳ Second we multiply each element by 2 [8] Update Weights ↳ Use loss gradients to start back propagation ↳ Update some weights (light red) ↳ The values of the new weights are for demonstration purpose only, not based on real calculation. 🔍 Inference [9] Deactivate Dropout ↳ We set both dropout matrices to identity matrices ↳ The effect is to keep all the features as is. [10] Feed Forward ↳ Take the forward pass to make predictions about unseen data.

Tom Yeh

36,026 görüntüleme • 2 yıl önce