Tech with Mak's banner
Tech with Mak's profile picture

Tech with Mak

@techNmak • 48,037 subscribers

AI, coding, software, and whatever’s on my mind.

Shorts

Any finite sequence of numbers can be represented exactly as a combination of complex oscillations, even when the original data looks nothing like a wave. This is the mathematics behind the Discrete Fourier Transform (DFT), one of the fundamental tools of digital signal processing. When engineers record sound, measure vibrations, or collect data from sensors, they obtain a sequence of samples. The DFT provides a different way to represent those same measurements, using complex exponentials at discrete frequencies. The mathematics relies on a remarkable property called orthogonality. For N samples, the DFT uses N orthogonal complex exponentials. Each frequency component has a corresponding complex coefficient, and the complete set of coefficients contains enough information to reconstruct every original sample. When a complex sinusoid aligns exactly with a DFT frequency bin, its contributions reinforce at that bin and cancel at the others. When its frequency falls between bins, the finite observation window causes its contribution to spread across multiple frequencies. This is known as spectral leakage. These properties have enormous practical value in engineering. In audio processing, Fourier analysis helps identify frequency components and design digital filters. In mechanical engineering, it helps detect periodic vibrations in rotating machinery. In wireless communications, OFDM systems use Fourier transforms to generate and recover signals carried by orthogonal subcarriers. The mathematics also simplifies certain computations. Circular convolution in the sample domain becomes pointwise multiplication in the frequency domain. The Fast Fourier Transform (FFT) makes these calculations efficient by exploiting mathematical symmetries, reducing the computational work from the quadratic cost of direct evaluation to roughly N log N for standard FFT algorithms. There is one important distinction. The DFT preserves all the information in the original sequence of samples, but it cannot recover information already lost during sampling. It is an exact change of mathematical representation, not a way to recover frequencies that sampling has made indistinguishable.

Any finite sequence of numbers can be represented exactly as a combination of complex oscillations, even when the original data looks nothing like a wave. This is the mathematics behind the Discrete Fourier Transform (DFT), one of the fundamental tools of digital signal processing. When engineers record sound, measure vibrations, or collect data from sensors, they obtain a sequence of samples. The DFT provides a different way to represent those same measurements, using complex exponentials at discrete frequencies. The mathematics relies on a remarkable property called orthogonality. For N samples, the DFT uses N orthogonal complex exponentials. Each frequency component has a corresponding complex coefficient, and the complete set of coefficients contains enough information to reconstruct every original sample. When a complex sinusoid aligns exactly with a DFT frequency bin, its contributions reinforce at that bin and cancel at the others. When its frequency falls between bins, the finite observation window causes its contribution to spread across multiple frequencies. This is known as spectral leakage. These properties have enormous practical value in engineering. In audio processing, Fourier analysis helps identify frequency components and design digital filters. In mechanical engineering, it helps detect periodic vibrations in rotating machinery. In wireless communications, OFDM systems use Fourier transforms to generate and recover signals carried by orthogonal subcarriers. The mathematics also simplifies certain computations. Circular convolution in the sample domain becomes pointwise multiplication in the frequency domain. The Fast Fourier Transform (FFT) makes these calculations efficient by exploiting mathematical symmetries, reducing the computational work from the quadratic cost of direct evaluation to roughly N log N for standard FFT algorithms. There is one important distinction. The DFT preserves all the information in the original sequence of samples, but it cannot recover information already lost during sampling. It is an exact change of mathematical representation, not a way to recover frequencies that sampling has made indistinguishable.

20,324 просмотров

Richard Feynman called Euler's formula "the most remarkable formula in mathematics." And there's a fascinating engineering reason why this equation is so important. e^(iθ) = cos(θ) + i·sin(θ) It connects exponential functions, imaginary numbers, trigonometry, and rotation through a single mathematical relationship. But its usefulness goes much further. Consider a sinusoidal signal: x(t) = A cos(ωt + φ) Using Euler's formula, we can represent it as the real part of a complex exponential: A e^(i(ωt + φ)) Why would engineers use this representation? Because differentiating e^(iωt) simply multiplies it by iω. Phase shifts can be represented through complex multiplication. For linear, time-invariant electrical circuits in sinusoidal steady state, this allows differential equations to be reduced to algebraic equations. Engineers use this mathematics in phasor analysis, AC circuit calculations, impedance, and frequency-response analysis. The same complex-exponential representation is fundamental to Fourier analysis, signal processing, communication systems, and the study of mechanical vibrations. Feynman also described the complex-plane representation of numbers as the unification of algebra and geometry. And that's what makes Euler's formula so remarkable. An equation connecting imaginary numbers and circular motion provides a practical way to analyze real electrical signals, oscillations, and physical systems.

Richard Feynman called Euler's formula "the most remarkable formula in mathematics." And there's a fascinating engineering reason why this equation is so important. e^(iθ) = cos(θ) + i·sin(θ) It connects exponential functions, imaginary numbers, trigonometry, and rotation through a single mathematical relationship. But its usefulness goes much further. Consider a sinusoidal signal: x(t) = A cos(ωt + φ) Using Euler's formula, we can represent it as the real part of a complex exponential: A e^(i(ωt + φ)) Why would engineers use this representation? Because differentiating e^(iωt) simply multiplies it by iω. Phase shifts can be represented through complex multiplication. For linear, time-invariant electrical circuits in sinusoidal steady state, this allows differential equations to be reduced to algebraic equations. Engineers use this mathematics in phasor analysis, AC circuit calculations, impedance, and frequency-response analysis. The same complex-exponential representation is fundamental to Fourier analysis, signal processing, communication systems, and the study of mechanical vibrations. Feynman also described the complex-plane representation of numbers as the unification of algebra and geometry. And that's what makes Euler's formula so remarkable. An equation connecting imaginary numbers and circular motion provides a practical way to analyze real electrical signals, oscillations, and physical systems.

42,676 просмотров

The π in the normal distribution has a geometric origin. It comes from a famous mathematical result known as the Gaussian integral. ∫₋∞^∞ e^(−x²) dx = √π The interesting part is that e^(−x²) has no elementary antiderivative. Yet its integral over the entire real line can be calculated exactly. The solution involves turning a one-dimensional integration problem into a two-dimensional one. The resulting function, e^(−(x²+y²)), has circular symmetry, and polar coordinates reveal where π enters the calculation. This result is fundamental to probability theory. The familiar factor 1/(σ√(2π)) in the normal distribution comes from the same integral. It ensures that the total probability across all possible values is exactly 1. But its applications extend well beyond statistics. In engineering, Gaussian distributions are commonly used to model measurement errors and noise. With independent, zero-mean Gaussian errors of equal variance in an additive-error model, maximum-likelihood estimation is equivalent to minimizing the sum of squared residuals. Gaussian functions also appear in heat diffusion, where they describe how an initially concentrated point source spreads through an unbounded, homogeneous medium. In Fourier analysis, the transform of a Gaussian is another Gaussian, up to scaling. This property is useful in signal processing and image filtering. The same mathematical function connects probability distributions, statistical estimation, heat transfer, and frequency analysis. And the √π in its integral ultimately comes from the geometry of the plane.

The π in the normal distribution has a geometric origin. It comes from a famous mathematical result known as the Gaussian integral. ∫₋∞^∞ e^(−x²) dx = √π The interesting part is that e^(−x²) has no elementary antiderivative. Yet its integral over the entire real line can be calculated exactly. The solution involves turning a one-dimensional integration problem into a two-dimensional one. The resulting function, e^(−(x²+y²)), has circular symmetry, and polar coordinates reveal where π enters the calculation. This result is fundamental to probability theory. The familiar factor 1/(σ√(2π)) in the normal distribution comes from the same integral. It ensures that the total probability across all possible values is exactly 1. But its applications extend well beyond statistics. In engineering, Gaussian distributions are commonly used to model measurement errors and noise. With independent, zero-mean Gaussian errors of equal variance in an additive-error model, maximum-likelihood estimation is equivalent to minimizing the sum of squared residuals. Gaussian functions also appear in heat diffusion, where they describe how an initially concentrated point source spreads through an unbounded, homogeneous medium. In Fourier analysis, the transform of a Gaussian is another Gaussian, up to scaling. This property is useful in signal processing and image filtering. The same mathematical function connects probability distributions, statistical estimation, heat transfer, and frequency analysis. And the √π in its integral ultimately comes from the geometry of the plane.

21,780 просмотров

KV Caching explained!!

KV Caching explained!!

40,454 просмотров

One of the odd things about learning mathematics is how long you can manipulate an equation without having much of a picture of it in your head. You learn the notation, follow the derivation, get the answer, and move on. Then somebody shows you the same idea visually and you realize there was a geometric intuition hiding underneath the symbols the whole time. A Fourier sum becomes rotating vectors added head-to-tail. Attention becomes a weighted combination you can watch being formed. SVD becomes a sequence of transformations you can follow one step at a time. Even the equation behind a diffusion process feels different when you can see a point following a changing vector field. None of this replaces the mathematics. The equations are still where the precision lives. But once you have a picture in your head, the notation has something to attach itself to. I think this is why some mathematical ideas suddenly become much easier years after we first learned them. We finally stop seeing the equation as a line of symbols to manipulate and start seeing the process it describes. I wish more mathematics had been taught that way.

One of the odd things about learning mathematics is how long you can manipulate an equation without having much of a picture of it in your head. You learn the notation, follow the derivation, get the answer, and move on. Then somebody shows you the same idea visually and you realize there was a geometric intuition hiding underneath the symbols the whole time. A Fourier sum becomes rotating vectors added head-to-tail. Attention becomes a weighted combination you can watch being formed. SVD becomes a sequence of transformations you can follow one step at a time. Even the equation behind a diffusion process feels different when you can see a point following a changing vector field. None of this replaces the mathematics. The equations are still where the precision lives. But once you have a picture in your head, the notation has something to attach itself to. I think this is why some mathematical ideas suddenly become much easier years after we first learned them. We finally stop seeing the equation as a line of symbols to manipulate and start seeing the process it describes. I wish more mathematics had been taught that way.

57,471 просмотров

If you are trying to understand where AI agents are going, learn harness engineering. A capable model is only one part of an agent system. Once the model begins reading files, calling tools, modifying state and working across many steps, the quality of the system depends increasingly on the software around it. Consider a coding agent working through a large repository. The model can decide that it needs to inspect a file, search for a symbol, make an edit or run a test, but those decisions do not execute themselves. The surrounding runtime has to decide which resources are available, whether the requested action is permitted, how the operation should be performed, what result should be retained, and what information should be presented to the model on the next step. This becomes harder as the run gets longer. As history accumulates, replaying everything can become costly and less effective. The harness has to decide what should remain in context, what should be summarized or retrieved later, and what belongs in persistent state outside the context window. Execution has similar problems. A long-running agent may need to survive an interruption, avoid repeating completed work, enforce permissions around consequential actions, and preserve enough history to reconstruct what happened when the final result is wrong. These are harness problems. The harness is the layer that manages context, tools, execution, state, checkpoints, limits and traces around the model. Harness engineering is the work of designing and improving that layer. Engineers inspect execution traces, evaluate agents on representative tasks, look for recurring failure modes, and then change things such as context selection, tool interfaces, state handling or execution controls. That last part matters because agent failures are often not fixed by changing the model. Sometimes the useful change is in what the model sees, how a tool is exposed, what state is preserved, or what the runtime does after a failed step. As agents take on longer tasks, the demands on this surrounding software grow. Model capability remains essential, but harness engineering is what turns that capability into an execution process that can be controlled, inspected, tested and improved.

If you are trying to understand where AI agents are going, learn harness engineering. A capable model is only one part of an agent system. Once the model begins reading files, calling tools, modifying state and working across many steps, the quality of the system depends increasingly on the software around it. Consider a coding agent working through a large repository. The model can decide that it needs to inspect a file, search for a symbol, make an edit or run a test, but those decisions do not execute themselves. The surrounding runtime has to decide which resources are available, whether the requested action is permitted, how the operation should be performed, what result should be retained, and what information should be presented to the model on the next step. This becomes harder as the run gets longer. As history accumulates, replaying everything can become costly and less effective. The harness has to decide what should remain in context, what should be summarized or retrieved later, and what belongs in persistent state outside the context window. Execution has similar problems. A long-running agent may need to survive an interruption, avoid repeating completed work, enforce permissions around consequential actions, and preserve enough history to reconstruct what happened when the final result is wrong. These are harness problems. The harness is the layer that manages context, tools, execution, state, checkpoints, limits and traces around the model. Harness engineering is the work of designing and improving that layer. Engineers inspect execution traces, evaluate agents on representative tasks, look for recurring failure modes, and then change things such as context selection, tool interfaces, state handling or execution controls. That last part matters because agent failures are often not fixed by changing the model. Sometimes the useful change is in what the model sees, how a tool is exposed, what state is preserved, or what the runtime does after a failed step. As agents take on longer tasks, the demands on this surrounding software grow. Model capability remains essential, but harness engineering is what turns that capability into an execution process that can be controlled, inspected, tested and improved.

49,903 просмотров

Software Engineers when they hear "you will be replaced by AI"

Software Engineers when they hear "you will be replaced by AI"

167,821 просмотров

you don’t hate calculus. you just never saw what the gradient was doing.

you don’t hate calculus. you just never saw what the gradient was doing.

27,826 просмотров

Web Scraping is dead. Web Agenting is here. Writing selectors (div > .class > span) breaks every time a site updates. Building custom bots for every new target is a waste of engineering hours. TinyFish turns the entire real-time web into a single API. Input: Natural Language ("Find availability for X"). Target: 1 or 100 URLs. Output: Structured JSON. This isn't a simulation. It visits the Real-Time Web. 1./ One API, Many Sites - Same contract whether you hit 1 URL or 50. You focus on the Goal (Business Logic). TinyFish handles the How (Navigation, Clicks, Inputs). 2./ Real Automation - It doesn't just "read." It interacts. It fills forms, navigates multi-step flows, and handles dynamic JS content. 3./ Production Ready - This is the same infrastructure used by large enterprises, now exposed as a clean developer primitive. Logs, error handling, and structured data are built-in. The web is finally a proper API.

Web Scraping is dead. Web Agenting is here. Writing selectors (div > .class > span) breaks every time a site updates. Building custom bots for every new target is a waste of engineering hours. TinyFish turns the entire real-time web into a single API. Input: Natural Language ("Find availability for X"). Target: 1 or 100 URLs. Output: Structured JSON. This isn't a simulation. It visits the Real-Time Web. 1./ One API, Many Sites - Same contract whether you hit 1 URL or 50. You focus on the Goal (Business Logic). TinyFish handles the How (Navigation, Clicks, Inputs). 2./ Real Automation - It doesn't just "read." It interacts. It fills forms, navigates multi-step flows, and handles dynamic JS content. 3./ Production Ready - This is the same infrastructure used by large enterprises, now exposed as a clean developer primitive. Logs, error handling, and structured data are built-in. The web is finally a proper API.

119,865 просмотров

🚨 This is the best way to learn how LLMs work. Interactive. 3D. Step-by-step. Covers: → Embedding → Layer Norm → Self-Attention → MLP → Transformer layers → Softmax → Output Stop reading papers. Start seeing. Link in comments. Save this immediately.

🚨 This is the best way to learn how LLMs work. Interactive. 3D. Step-by-step. Covers: → Embedding → Layer Norm → Self-Attention → MLP → Transformer layers → Softmax → Output Stop reading papers. Start seeing. Link in comments. Save this immediately.

82,419 просмотров

Microsoft made 100B parameter models run on a single CPU. bitnet.cpp: The official inference framework for 1-bit LLMs. The math behind 1-bit LLMs is what makes them revolutionary. Traditional LLMs use 16-bit floating point weights. Every parameter is a number like 0.0023847 or -1.4729. When you run inference, you multiply these floats together. Billions of times. That's why you need GPUs, they're optimized for floating point matrix multiplication. BitNet b1.58 uses ternary weights: {-1, 0, 1}. That's not a simplification. That's a fundamental change in the math. When your weights are only -1, 0, or 1: → Multiply by 1 = keep the value → Multiply by -1 = flip the sign → Multiply by 0 = skip entirely Matrix multiplication becomes addition and subtraction. No floating point operations. No GPU required. This is why bitnet.cpp achieves: → 2.37x to 6.17x speedup on x86 CPUs → 1.37x to 5.07x speedup on ARM CPUs → 71.9% to 82.2% energy reduction on x86 → 55.4% to 70.0% energy reduction on ARM The speedups scale with model size. Larger models see bigger gains because there are more operations to simplify. A 100B parameter model running at human reading speed (5-7 tokens/second) on a single CPU. That's not optimization. That's a different paradigm. Why 1.58 bits? Because log₂(3) ≈ 1.58. Three possible values = 1.58 bits of information per weight. The key insight: These models aren't quantized after training. They're trained from scratch with ternary weights. The model learns to work within the constraint. No precision loss. No quality tradeoff.

Microsoft made 100B parameter models run on a single CPU. bitnet.cpp: The official inference framework for 1-bit LLMs. The math behind 1-bit LLMs is what makes them revolutionary. Traditional LLMs use 16-bit floating point weights. Every parameter is a number like 0.0023847 or -1.4729. When you run inference, you multiply these floats together. Billions of times. That's why you need GPUs, they're optimized for floating point matrix multiplication. BitNet b1.58 uses ternary weights: {-1, 0, 1}. That's not a simplification. That's a fundamental change in the math. When your weights are only -1, 0, or 1: → Multiply by 1 = keep the value → Multiply by -1 = flip the sign → Multiply by 0 = skip entirely Matrix multiplication becomes addition and subtraction. No floating point operations. No GPU required. This is why bitnet.cpp achieves: → 2.37x to 6.17x speedup on x86 CPUs → 1.37x to 5.07x speedup on ARM CPUs → 71.9% to 82.2% energy reduction on x86 → 55.4% to 70.0% energy reduction on ARM The speedups scale with model size. Larger models see bigger gains because there are more operations to simplify. A 100B parameter model running at human reading speed (5-7 tokens/second) on a single CPU. That's not optimization. That's a different paradigm. Why 1.58 bits? Because log₂(3) ≈ 1.58. Three possible values = 1.58 bits of information per weight. The key insight: These models aren't quantized after training. They're trained from scratch with ternary weights. The model learns to work within the constraint. No precision loss. No quality tradeoff.

23,202 просмотров

Videos

techNmak's profile picture

Most people first see Euler’s Formula as a strange equation in a textbook. Then years later, they realize it quietly powers the modern world. Leonhard Euler discovered that these seemingly unrelated mathematical ideas: → exponential growth → imaginary numbers → sine waves → cosine waves → rotation … are all deeply connected through one identity: e^(iθ) = cos(θ) + i·sin(θ) At first glance it looks impossible. How can an exponential function suddenly produce circles and waves? The key insight is that multiplying by a complex exponential creates rotation. As the angle θ changes: - the cosine term tracks horizontal motion - the sine term tracks vertical motion - together they trace a perfect circle in the complex plane Euler showed that waves and rotation are mathematically the same phenomenon viewed differently. That single idea changed science and engineering forever. Today, Euler’s Formula sits underneath: → Fourier Transforms → signal processing → wireless communication → MRI scanners → quantum mechanics → electrical engineering → audio compression → radar systems → GPS → neural network frequency analysis Even modern AI systems indirectly rely on mathematics built on top of these foundations. The famous special case is Euler’s Identity: e^(iπ) + 1 = 0 Richard Feynman reportedly called it: "our jewel." Because it revealed that mathematics is not a collection of separate topics. It is one connected language describing reality itself.

Tech with Mak

25,539 просмотров • 5 месяцев назад

Больше нет контента для загрузки