Загрузка видео...

Не удалось загрузить видео

На главную

How have the fundamentals of building large, distributed software systems changed the last decade? A conversation with Martin Kleppmann (author of Designing Data-Intensive Applications) - given that the second, updated edition of the book was just released. Timestamps: 00:00 Early career 05:46 Building Rapportive 10:47 Working at LinkedIn 14:09...

GergelyOrosz's profile picture

Gergely Orosz

343,420 subscribers

79,352 просмотров • 2 месяцев назад •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

Комментарии: 0

Нет доступных комментариев

Здесь появятся комментарии из оригинального поста

Похожие видео

OpenClaw - the agentic software spreading like wildfire - was built on top of Pi, a minimalist, self-modifying agent. I sat down with Pi's creator, Mario Zechner and longtime Pi user (+ the creator of Flask) Armin Ronacher ⇌ to talk Pi, and their (very grounded!) takes on building with AI. Timestamps: 00:00 Intro 07:30 How Mario, Armin, and Peter Steinberger met 15:15 How 30 dev teams use AI agents: learnings 21:50 The importance of judgment 24:26 Challenges when non-engineers write code 28:30 Downsides of over-automation 32:18 Pi 48:09 OpenClaw + Pi 50:54 “Clankers” 57:32 Open source and AI 1:00:22 Complexity as the enemy 1:02:50 Building an AI-native startup 1:11:52 “Slow the F down” 1:16:40 MCPs vs. CLI 1:25:03 Predictions and staying up to date • YouTube: • Spotify: • Apple: Brought to you by: • Statsig – ⁠ The unified platform for flags, analytics, experiments, and more. • Sonar — The makers of SonarQube, the industry standard for code verification and automated code review. Try it out for yourself. • WorkOS – WorkOS gives you APIs to ship enterprise features – SSO, directory sync, RBAC, audit logs – in days, not months. Visit learn more. --- Three parts I found especially interesting in this discussion: 1. New trend: AI makes it harder for senior engineers to reject pointless complexity. Historically, senior engineers kept software complexity at bay simply by saying “no” a lot. But Armin observes that these days, more junior engineers and product managers deploy agent-scripted counterarguments when a senior colleague kicks an idea to the curb. This makes decision-making exhausting, and more bad ideas make it into production as a result. 2. It should be MUCH easier to build specialized tools for specific tasks. Different projects need different harness types because, as Mario points out, the same hammer is not ideal for every single construction job. As such, Pi is built with the goal of allowing the creation of specialized harnesses. It can modify itself so that a user can create the bespoke harness needed for any task. Mario believes it’s a preview of how self-modifiable software might look in the future. 3. Automation bias is one of the biggest risks of working with AI agents. Once devs confirm that an AI agent can produce acceptable code, they start to review its output less often, even though agents can – and do! – produce slop. Mario advises being far more sceptical with agents, and cautions that the quality of their output isn’t guaranteed, however well they performed previously.
1:33:55

OpenClaw - the agentic software spreading like wildfire - was built on top of Pi, a minimalist, self-modifying agent. I sat down with Pi's creator, Mario Zechner and longtime Pi user (+ the creator of Flask) Armin Ronacher ⇌ to talk Pi, and their (very grounded!) takes on building with AI. Timestamps: 00:00 Intro 07:30 How Mario, Armin, and Peter Steinberger met 15:15 How 30 dev teams use AI agents: learnings 21:50 The importance of judgment 24:26 Challenges when non-engineers write code 28:30 Downsides of over-automation 32:18 Pi 48:09 OpenClaw + Pi 50:54 “Clankers” 57:32 Open source and AI 1:00:22 Complexity as the enemy 1:02:50 Building an AI-native startup 1:11:52 “Slow the F down” 1:16:40 MCPs vs. CLI 1:25:03 Predictions and staying up to date • YouTube: • Spotify: • Apple: Brought to you by: • Statsig – ⁠ The unified platform for flags, analytics, experiments, and more. • Sonar — The makers of SonarQube, the industry standard for code verification and automated code review. Try it out for yourself. • WorkOS – WorkOS gives you APIs to ship enterprise features – SSO, directory sync, RBAC, audit logs – in days, not months. Visit learn more. --- Three parts I found especially interesting in this discussion: 1. New trend: AI makes it harder for senior engineers to reject pointless complexity. Historically, senior engineers kept software complexity at bay simply by saying “no” a lot. But Armin observes that these days, more junior engineers and product managers deploy agent-scripted counterarguments when a senior colleague kicks an idea to the curb. This makes decision-making exhausting, and more bad ideas make it into production as a result. 2. It should be MUCH easier to build specialized tools for specific tasks. Different projects need different harness types because, as Mario points out, the same hammer is not ideal for every single construction job. As such, Pi is built with the goal of allowing the creation of specialized harnesses. It can modify itself so that a user can create the bespoke harness needed for any task. Mario believes it’s a preview of how self-modifiable software might look in the future. 3. Automation bias is one of the biggest risks of working with AI agents. Once devs confirm that an AI agent can produce acceptable code, they start to review its output less often, even though agents can – and do! – produce slop. Mario advises being far more sceptical with agents, and cautions that the quality of their output isn’t guaranteed, however well they performed previously.

Gergely Orosz

172,263 просмотров • 2 месяцев назад

What if we're actually in the middle of the third golden age of software engineering? This is what Grady Booch sees happening. If you are anxious about the state of the industry, you want to watch/listen to Grady's longer-term perspective and stories. Watch the full episode here: 00:00 Intro 01:58 The first golden age of software engineering 18:59 The software crisis 33:01 The second golden age of software engineering 42:21 Y2K and the Dotcom crash 45:47 Early AI 47:34 The third golden age of software engineering 51:48 Why software engineers will very much be needed 58:46 Grady responds to Dario Amodei 1:06:54 New skills engineers will need to succeed 1:10:04 Resources for studying complex systems 1:14:33 How to thrive during periods of change Brought to you by: • Statsig — ⁠ The unified platform for flags, analytics, experiments, and more. • Sonar – The makers of SonarQube, the industry standard for automated code review. Join me online at the Sonar Summit on March 3rd, where I talk about practical tactics for the AI era. • WorkOS – Everything you need to make your app enterprise-ready.
1:17:58

What if we're actually in the middle of the third golden age of software engineering? This is what Grady Booch sees happening. If you are anxious about the state of the industry, you want to watch/listen to Grady's longer-term perspective and stories. Watch the full episode here: 00:00 Intro 01:58 The first golden age of software engineering 18:59 The software crisis 33:01 The second golden age of software engineering 42:21 Y2K and the Dotcom crash 45:47 Early AI 47:34 The third golden age of software engineering 51:48 Why software engineers will very much be needed 58:46 Grady responds to Dario Amodei 1:06:54 New skills engineers will need to succeed 1:10:04 Resources for studying complex systems 1:14:33 How to thrive during periods of change Brought to you by: • Statsig — ⁠ The unified platform for flags, analytics, experiments, and more. • Sonar – The makers of SonarQube, the industry standard for automated code review. Join me online at the Sonar Summit on March 3rd, where I talk about practical tactics for the AI era. • WorkOS – Everything you need to make your app enterprise-ready.

Gergely Orosz

86,896 просмотров • 5 месяцев назад

Author of Designing Data-Intensive Applications, Martin Kleppmann, on why engineers don't get to ignore ethics just because the work is technical — a theme he felt strongly enough about to address in both editions: “I felt that ethics had been quite ignored as a concern during my time in industry. Especially in startups, people were very focused on building a product that their customers would love and deprioritized ethical questions in the process. For example, with consumer facing products, it might be that the products are very much geared towards essentially data harvesting, collecting behavioural data, because that's what can be monetized in the form of advertising. There seemed to be very little reflection on what was good and bad about these sorts of things. I really wanted to encourage a bit of thinking there. I didn’t want to prescribe a particular approach, but to point out there is this thing such as data protection legislation now, which we do have to think about in the architecture of our data systems and there is an ethical responsibility.”
1:55

Author of Designing Data-Intensive Applications, Martin Kleppmann, on why engineers don't get to ignore ethics just because the work is technical — a theme he felt strongly enough about to address in both editions: “I felt that ethics had been quite ignored as a concern during my time in industry. Especially in startups, people were very focused on building a product that their customers would love and deprioritized ethical questions in the process. For example, with consumer facing products, it might be that the products are very much geared towards essentially data harvesting, collecting behavioural data, because that's what can be monetized in the form of advertising. There seemed to be very little reflection on what was good and bad about these sorts of things. I really wanted to encourage a bit of thinking there. I didn’t want to prescribe a particular approach, but to point out there is this thing such as data protection legislation now, which we do have to think about in the architecture of our data systems and there is an ethical responsibility.”

The Pragmatic Engineer

783,831 просмотров • 2 месяцев назад

Sharding is: 1) The foundation of database scalability 2) Super interesting architecturally If you want to Design Data Intensive Applications (at scale) you must learn it, and learn it well.
6:33

Sharding is: 1) The foundation of database scalability 2) Super interesting architecturally If you want to Design Data Intensive Applications (at scale) you must learn it, and learn it well.

Ben Dicken

83,195 просмотров • 2 месяцев назад

It's always energizing to do a podcast with Steve Yegge (Steve Yegge, engineer+author, formerly at Amazon+Google, creator of Gas Town). Timestamps: 00:00 Intro 01:43 Steve’s latest projects 02:27 Important blog posts 04:48 Shifts in what engineers need to know 10:46 Steve’s current AI stance 13:23 Steve’s book Vibe Coding 18:25 Layoffs and disruption in tech 31:13 Gas Town 40:10 New ways of working 51:08 The problem of too many people 54:45 Why AI results lag in business 59:57 Gamification and product stickiness 1:04:54 The ‘Bitter Lesson’ explained 1:07:14 The future of software development 1:23:06 Where languages stand 1:24:47 Adapting to change 1:27:32 Steve’s predictions Brought to you by: • Statsig – ⁠ The unified platform for flags, analytics, experiments, and more. • Sonar – The makers of SonarQube, the industry standard for automated code review. • WorkOS – Everything you need to make your app enterprise ready. Three interesting thoughts from Steve that we talked about in this conversation: 1. Reading ability is becoming a blocker for wider AI adoption. Some struggle with walls of text that current AI tools produce, and Steve predicts that in the very near future, most people will program by talking to a visual avatar, not reading terminal output because he observes that five paragraphs is already a lot to read for many devs. 2. What software engineers need to know keeps changing. In the 1990s, any decent software engineer knew Assembly, and today almost no decent developer knows it because Assembly has long been superseded by technical progress. What engineers “need” to know these days is different from the ‘90s and that process continues with AI, changing the parts of the craft that are essential for devs. We grumble about this but that won’t change anything by itself. 3. There’s a “Dracula Effect” where AI-augmented work drains engineers faster than traditional work. This is because AI automates the easy tasks, meaning that engineers are stuck doing high-intensity thinking all day. Steve says you may only get three daily productive hours at max speed, but during that time, you could produce 100x more output than before.
1:31:59

It's always energizing to do a podcast with Steve Yegge (Steve Yegge, engineer+author, formerly at Amazon+Google, creator of Gas Town). Timestamps: 00:00 Intro 01:43 Steve’s latest projects 02:27 Important blog posts 04:48 Shifts in what engineers need to know 10:46 Steve’s current AI stance 13:23 Steve’s book Vibe Coding 18:25 Layoffs and disruption in tech 31:13 Gas Town 40:10 New ways of working 51:08 The problem of too many people 54:45 Why AI results lag in business 59:57 Gamification and product stickiness 1:04:54 The ‘Bitter Lesson’ explained 1:07:14 The future of software development 1:23:06 Where languages stand 1:24:47 Adapting to change 1:27:32 Steve’s predictions Brought to you by: • Statsig – ⁠ The unified platform for flags, analytics, experiments, and more. • Sonar – The makers of SonarQube, the industry standard for automated code review. • WorkOS – Everything you need to make your app enterprise ready. Three interesting thoughts from Steve that we talked about in this conversation: 1. Reading ability is becoming a blocker for wider AI adoption. Some struggle with walls of text that current AI tools produce, and Steve predicts that in the very near future, most people will program by talking to a visual avatar, not reading terminal output because he observes that five paragraphs is already a lot to read for many devs. 2. What software engineers need to know keeps changing. In the 1990s, any decent software engineer knew Assembly, and today almost no decent developer knows it because Assembly has long been superseded by technical progress. What engineers “need” to know these days is different from the ‘90s and that process continues with AI, changing the parts of the craft that are essential for devs. We grumble about this but that won’t change anything by itself. 3. There’s a “Dracula Effect” where AI-augmented work drains engineers faster than traditional work. This is because AI automates the easy tasks, meaning that engineers are stuck doing high-intensity thinking all day. Steve says you may only get three daily productive hours at max speed, but during that time, you could produce 100x more output than before.

Gergely Orosz

41,987 просмотров • 4 месяцев назад

Author of Designing Data-Intensive Applications, Martin Kleppmann, on why scaling down is just as challenging as scaling up: “Part of what is interesting about modern cloud services, and backend services in general, is how they've introduced the idea of horizontal scalability and shared nothing systems. We can build systems that are able to cope with very high load, even if the individual components are just fairly cheap commodity machines. The scalability story is not just scaling up, but scaling down as well. How do you run a service in such a way that if it has a very small amount of load, it's really cheap to run? That's sort of the same question as ‘how do you continue running a service if it has a very high load?’ Generally, you want the cost and the computing capacity to be roughly proportional to the load that you have and at the low end, that means actually being able to scale down to something that is extremely cheap to run. That's not necessarily a given. It's something that is hard with on-premises software because if you've got a physical machine, that is a unit of deployment. Yes, you could carve it up into two dozen virtual machines, but it still requires some sort of resource allocation. And so, part of what's interesting about some serverless systems, is their ability to scale down and say ‘okay, if you're going to handle just three requests per day, that's just fine as well’”.
2:10

Author of Designing Data-Intensive Applications, Martin Kleppmann, on why scaling down is just as challenging as scaling up: “Part of what is interesting about modern cloud services, and backend services in general, is how they've introduced the idea of horizontal scalability and shared nothing systems. We can build systems that are able to cope with very high load, even if the individual components are just fairly cheap commodity machines. The scalability story is not just scaling up, but scaling down as well. How do you run a service in such a way that if it has a very small amount of load, it's really cheap to run? That's sort of the same question as ‘how do you continue running a service if it has a very high load?’ Generally, you want the cost and the computing capacity to be roughly proportional to the load that you have and at the low end, that means actually being able to scale down to something that is extremely cheap to run. That's not necessarily a given. It's something that is hard with on-premises software because if you've got a physical machine, that is a unit of deployment. Yes, you could carve it up into two dozen virtual machines, but it still requires some sort of resource allocation. And so, part of what's interesting about some serverless systems, is their ability to scale down and say ‘okay, if you're going to handle just three requests per day, that's just fine as well’”.

The Pragmatic Engineer

21,239 просмотров • 2 месяцев назад

Dear ICP community, the Internet Computer has now been running strong for 5 years 👏👏👏 Here is a celebratory preview of ICP "cloud engines," the sovereign frontier cloud technology the network shall soon provide from Main points: — Cloud engines enable anyone to spin up their own sovereign frontier cloud. The technology involves an extraordinary inventive step, in which cloud is created from a mathematically secure network of nodes. The nodes run as part of the Internet Computer network ( but are selected and configured by the cloud engine's owner. — The frontier cloud provided by engines is strongly focused on enabling AI agents to build and update online applications and services for us. The world is changing fast, and nearly all new online apps and services are already being built with the help of AI, and thus cloud engines target the future of cloud. — Software hosted on cloud engines is tamperproof, which means that it is immune to infrastructure hacks, because it runs inside a mathematically secure network protocol, rather than on computers directly. This means that AI agents, and those building with them, don't need to have a security team in the loop, or to trust someone else's security team. This is crucial, because in the future, non technical people will demand the freedom to build with full automation — where they just need to issue instructions to AI about what to build, and don't need to worry about anything or anyone else. Of course, apps and services running on engines are also vastly safer from the new breed of hacker being enabled by frontier AI. (The cloud engines themselves are also "tamperproof." Even if a hacker gains physical access to some portion of a cloud engine's nodes, and can make arbitrary changes, the computations and data of the hosted apps and services cannot be corrupted or interrupted so long as the network's fault bounds aren't exceeded. The recent hack of Vercel, a major cloud platform, which gave hackers access to the apps it hosted, provides additional perspective on the importance of this advantage.) — Software hosted on cloud engines is guaranteed to run, so long as a sufficient number of the engine's nodes are running. This means that AI can build applications and services without the need to have a human systems admin team constantly tinkering with the underlying platform to keep it running, which is again crucial, because in the future, non technical people will expect the freedom to use AI to build without the support of others. — New frontier programming language technology, in the form of the Motoko language developed by Caffeine Labs, leverages seminal "orthogonal persistence" technology that unifies program logic and data to deliver further unlocks for AI (Motoko is the first computer language being developed that targets agents that are writing software rather than humans engineers per se). Nowadays, AI can build and update production apps at a prodigious rate, even at the speed of conversation. But it can also make mistakes, and there's a risk that an update it creates might be "lossy" in the sense it causes some transformed data to be lost. Again, in this new world, it's both undesirable and impractical for everyone to have to have a systems admin team on-hand to detect lossy updates and roll them back, but Motoko provides a solution: it can detect new software updates are lossy before they are applied, reducing potentially catastrophic errors by AI to harmless coding retries. — Software hosted on cloud engines is "serverless" but unlike traditional serverless software, directly it directly incorporates data through "orthogonal persistence." Another key purpose is simplify backend software logic and fuel the modeling power of AI by increasing abstraction (sorry for the technical language!!!). Put simply, this enables AI to produce more sophisticated backends, faster, and at dramatically lower costs, as measured by the number AI API tokens consumed during coding. (Tip for the technical: orthogonal persistence is a new paradigm where "the program is the database," and data lives inside program variables, which is possible because it's as if hosted software runs forever in persistent memory). — An expanding database of skills at shall make it possible to develop and directly deploy apps and services to your cloud engines directly from Claude Code, Perplexity, Codex and other AI platforms. Further, your account on can be connected, so that new apps and updates created through conversation automatically appear hosted from your cloud engine. In the future, R&D is going to be very seamless. You converse with AI, and your secure and unstoppable apps or services are created or updated. Cloud engines are designed to directly support this "self-writing cloud" future where we can work hands-free. — Tech sovereignty is becoming a huge issue worldwide, with governments and corporations seeking to create sovereign tech stacks owing to geopolitical tensions. Increasingly, people are realizing that tech provided by foreign nations can come with hidden backdoors and kills switches, from the base platform, right up through hosted apps and services. ICP technology is open source, and those building on ICP using AI own their own source code. When you have the source code, you can verify that there are no backdoors, and when you own the source code thanks to AI, you can update it at will, freeing you from vendor lock-in. But cloud engines take sovereignty much further... — You create a cloud engine by selecting the nodes that will be combined. You can choose the class of nodes used, and their number, but more importantly, you can choose who operates the nodes, and where they are located. Almost any configuration is possible, because the Internet Computer scales the security privileges afforded to hosted software within the network according to configuration (software hosted on cloud engines can directly interoperate with software on other engines and traditional subnets, but base restrictions are applied according to security rules). A cloud engine can be created within a region such as Europe, to comply with regs such as GDPR, or completely within a sovereign state like Switzerland or Pakistan. But cloud engines go further still... — Sovereignty is also about freedom from vendor lock-in. Cloud engines are essentially ICP (Internet Computer Protocol) network configurations, and this means the underlying compute nodes they combine can be swapped out without interrupting their hosted apps and services. This is a big deal. In addition, cloud engines now support nodes that are instances running on Big Tech's clouds, in addition to nodes that are dedicated specialized hardware, as per the Gen I and Gen II nodes that dominate the Internet Computer today. For example, it is possible to have an engine running across different AWS data centers, say, and then reconfigure the engine to run across a mixture of AWS, Google, Azure and Hetzner for even more resilience, without the users of hosted apps and services noticing a thing. That's true freedom. — Sovereign AI is becoming increasingly important too, and cloud engines allow special "AI nodes" to be added to them, so that hosted software can perform inference on hardware provisioned by the owner from a location the owner has selected. Even though the AI nodes are only accessible within the cloud engine, they can still benefit from the forthcoming Internet Intelligence Gateway (IG), which will make it possible to validate inference performed on key frontier open weights LLMs, even when the inference is performed on completely independent AI clouds. When the results of inference are received, this technology can verify that neither the prompt+context (input) nor the inference result (output) have been modified, and that the results were produced by the precise LLM expected. This ensures that AI clouds don't cheat by running inference on cheaper models than are being paid for, and bad actors aren't modifying the inputs or outputs to surreptitiously insert advertising into results, say, or change facts, or insert malware when code is being generated. What's super cool about this technology is the cost of the verification is scalable. A very valuable additional security can be achieved with only 1-2% of extra cost. — Scaling apps and services when they hit capacity limits is another thorny problem that cloud engines help the world address. Engines make scaling possible without rewriting or reconfiguring software. The query workload capacity of hosted software can be horizontally scaled simply by adding new nodes to an engine, and nodes can also be added in geographical proximity to demand. Meanwhile, update workload capacity can first be scaled-up by swapping an engine's nodes out for the next class up, and then when no larger class of node is available, horizontally scaled-out by "splitting" the engine into two, which doubles available capacity. (Technical tip: horizontally scaling update capacity by splitting engines requires multi-canister architectures). — For those who have been following how Caffeine builds apps that can efficiently store large numbers of files, I should mention that apps built on cloud engines will also support the new ICP Blob Storage cloud network (since cloud engines currently have up to about 3 TB of memory, which apps storing large amounts of files can easily exceed). We are also working on allowing blob storage nodes to be added to cloud engines, to enable sovereign mass blob storage within an engine, similarly to how AI nodes can be added currently. — Lastly, but certainly not least, I should mention that cloud engines are multi-blockchain capable, and ready for digital assets, thanks to the clever math at their core. For example, an e-commerce service built on a cloud engine can securely accept and custody stablecoin payments, or a multi-chain DEX could be hosted. Further, engines can support software autonomy (software orchestrated and controlled by other autonomous software, in a decentralized way) and can themselves be orchestrated by SNS technology, and thus run autonomously too. Today, though, the focus is on *mainstream* cloud. This year, the cloud industry will generate approximately one trillion dollars in revenue. That number is already huge, but is expected to grow to two trillion dollars by 2030. After years of continuous development, which have seen more than $500m spent on R&D, the Internet Computer network is now tacking directly toward this mainstream cloud market with cloud engine technology. In their first version, cloud engines are not meant to be a cloud panacea. For example, currently they are not ideal for working with big data. You should use something like DataBricks for that. Cloud engines are carefully targeted at enabling AI to produce traditional online applications and services, including SaaS, in a safer and more productive way, which represents a new market segment with tremendous potential. Of course, DFINITY will continue to work relentlessly to push forward ICP's capabilities, so expect further developments. It's worth mentioning that this cloud segment isn't just about creating new apps and services using AI, it's also about replacing legacy systems and apps built on super expensive SaaS services. Caffeine Labs is working to produce technology (Caffeine Snorkel) that can study an enterprise's legacy systems and app built on SaaS, create replacement systems and apps, and migrate the data, while supporting key stakeholders through the process over email and chat, with full automation. Thus the legacy systems and SaaS markets shall also be addressed by cloud engines. Zooming out, and reasoning in a more metaphysical way, we believe, as we always have, that there is room for a new kind of cloud created by mathematical networks, that provides seminal advances in the fields of security and resilience, as well as true sovereignty and freedom from lock-in. That this same technology, with the help of additional technologies like orthogonal persistence and Motoko, enables AI to build for us without the need for so much oversight, and to create more backend sophistication while consuming fewer AI API tokens, enables ICP to bring game-changing advances to the world. Cloud engines will work synergistically with the Intelligence Gateway, which will enable apps and services running on engines to seamlessly leverage AI, wherever that AI is running, while providing verifiability at extremely low cost for open weights frontier models. We believe that cloud engines represent an inflection point in the storied history of the Internet Computer project, and I'm very proud to be sharing the details with you on the network's fifth birthday 💪 I'll be back with more news soon!!
19:06

Dear ICP community, the Internet Computer has now been running strong for 5 years 👏👏👏 Here is a celebratory preview of ICP "cloud engines," the sovereign frontier cloud technology the network shall soon provide from Main points: — Cloud engines enable anyone to spin up their own sovereign frontier cloud. The technology involves an extraordinary inventive step, in which cloud is created from a mathematically secure network of nodes. The nodes run as part of the Internet Computer network ( but are selected and configured by the cloud engine's owner. — The frontier cloud provided by engines is strongly focused on enabling AI agents to build and update online applications and services for us. The world is changing fast, and nearly all new online apps and services are already being built with the help of AI, and thus cloud engines target the future of cloud. — Software hosted on cloud engines is tamperproof, which means that it is immune to infrastructure hacks, because it runs inside a mathematically secure network protocol, rather than on computers directly. This means that AI agents, and those building with them, don't need to have a security team in the loop, or to trust someone else's security team. This is crucial, because in the future, non technical people will demand the freedom to build with full automation — where they just need to issue instructions to AI about what to build, and don't need to worry about anything or anyone else. Of course, apps and services running on engines are also vastly safer from the new breed of hacker being enabled by frontier AI. (The cloud engines themselves are also "tamperproof." Even if a hacker gains physical access to some portion of a cloud engine's nodes, and can make arbitrary changes, the computations and data of the hosted apps and services cannot be corrupted or interrupted so long as the network's fault bounds aren't exceeded. The recent hack of Vercel, a major cloud platform, which gave hackers access to the apps it hosted, provides additional perspective on the importance of this advantage.) — Software hosted on cloud engines is guaranteed to run, so long as a sufficient number of the engine's nodes are running. This means that AI can build applications and services without the need to have a human systems admin team constantly tinkering with the underlying platform to keep it running, which is again crucial, because in the future, non technical people will expect the freedom to use AI to build without the support of others. — New frontier programming language technology, in the form of the Motoko language developed by Caffeine Labs, leverages seminal "orthogonal persistence" technology that unifies program logic and data to deliver further unlocks for AI (Motoko is the first computer language being developed that targets agents that are writing software rather than humans engineers per se). Nowadays, AI can build and update production apps at a prodigious rate, even at the speed of conversation. But it can also make mistakes, and there's a risk that an update it creates might be "lossy" in the sense it causes some transformed data to be lost. Again, in this new world, it's both undesirable and impractical for everyone to have to have a systems admin team on-hand to detect lossy updates and roll them back, but Motoko provides a solution: it can detect new software updates are lossy before they are applied, reducing potentially catastrophic errors by AI to harmless coding retries. — Software hosted on cloud engines is "serverless" but unlike traditional serverless software, directly it directly incorporates data through "orthogonal persistence." Another key purpose is simplify backend software logic and fuel the modeling power of AI by increasing abstraction (sorry for the technical language!!!). Put simply, this enables AI to produce more sophisticated backends, faster, and at dramatically lower costs, as measured by the number AI API tokens consumed during coding. (Tip for the technical: orthogonal persistence is a new paradigm where "the program is the database," and data lives inside program variables, which is possible because it's as if hosted software runs forever in persistent memory). — An expanding database of skills at shall make it possible to develop and directly deploy apps and services to your cloud engines directly from Claude Code, Perplexity, Codex and other AI platforms. Further, your account on can be connected, so that new apps and updates created through conversation automatically appear hosted from your cloud engine. In the future, R&D is going to be very seamless. You converse with AI, and your secure and unstoppable apps or services are created or updated. Cloud engines are designed to directly support this "self-writing cloud" future where we can work hands-free. — Tech sovereignty is becoming a huge issue worldwide, with governments and corporations seeking to create sovereign tech stacks owing to geopolitical tensions. Increasingly, people are realizing that tech provided by foreign nations can come with hidden backdoors and kills switches, from the base platform, right up through hosted apps and services. ICP technology is open source, and those building on ICP using AI own their own source code. When you have the source code, you can verify that there are no backdoors, and when you own the source code thanks to AI, you can update it at will, freeing you from vendor lock-in. But cloud engines take sovereignty much further... — You create a cloud engine by selecting the nodes that will be combined. You can choose the class of nodes used, and their number, but more importantly, you can choose who operates the nodes, and where they are located. Almost any configuration is possible, because the Internet Computer scales the security privileges afforded to hosted software within the network according to configuration (software hosted on cloud engines can directly interoperate with software on other engines and traditional subnets, but base restrictions are applied according to security rules). A cloud engine can be created within a region such as Europe, to comply with regs such as GDPR, or completely within a sovereign state like Switzerland or Pakistan. But cloud engines go further still... — Sovereignty is also about freedom from vendor lock-in. Cloud engines are essentially ICP (Internet Computer Protocol) network configurations, and this means the underlying compute nodes they combine can be swapped out without interrupting their hosted apps and services. This is a big deal. In addition, cloud engines now support nodes that are instances running on Big Tech's clouds, in addition to nodes that are dedicated specialized hardware, as per the Gen I and Gen II nodes that dominate the Internet Computer today. For example, it is possible to have an engine running across different AWS data centers, say, and then reconfigure the engine to run across a mixture of AWS, Google, Azure and Hetzner for even more resilience, without the users of hosted apps and services noticing a thing. That's true freedom. — Sovereign AI is becoming increasingly important too, and cloud engines allow special "AI nodes" to be added to them, so that hosted software can perform inference on hardware provisioned by the owner from a location the owner has selected. Even though the AI nodes are only accessible within the cloud engine, they can still benefit from the forthcoming Internet Intelligence Gateway (IG), which will make it possible to validate inference performed on key frontier open weights LLMs, even when the inference is performed on completely independent AI clouds. When the results of inference are received, this technology can verify that neither the prompt+context (input) nor the inference result (output) have been modified, and that the results were produced by the precise LLM expected. This ensures that AI clouds don't cheat by running inference on cheaper models than are being paid for, and bad actors aren't modifying the inputs or outputs to surreptitiously insert advertising into results, say, or change facts, or insert malware when code is being generated. What's super cool about this technology is the cost of the verification is scalable. A very valuable additional security can be achieved with only 1-2% of extra cost. — Scaling apps and services when they hit capacity limits is another thorny problem that cloud engines help the world address. Engines make scaling possible without rewriting or reconfiguring software. The query workload capacity of hosted software can be horizontally scaled simply by adding new nodes to an engine, and nodes can also be added in geographical proximity to demand. Meanwhile, update workload capacity can first be scaled-up by swapping an engine's nodes out for the next class up, and then when no larger class of node is available, horizontally scaled-out by "splitting" the engine into two, which doubles available capacity. (Technical tip: horizontally scaling update capacity by splitting engines requires multi-canister architectures). — For those who have been following how Caffeine builds apps that can efficiently store large numbers of files, I should mention that apps built on cloud engines will also support the new ICP Blob Storage cloud network (since cloud engines currently have up to about 3 TB of memory, which apps storing large amounts of files can easily exceed). We are also working on allowing blob storage nodes to be added to cloud engines, to enable sovereign mass blob storage within an engine, similarly to how AI nodes can be added currently. — Lastly, but certainly not least, I should mention that cloud engines are multi-blockchain capable, and ready for digital assets, thanks to the clever math at their core. For example, an e-commerce service built on a cloud engine can securely accept and custody stablecoin payments, or a multi-chain DEX could be hosted. Further, engines can support software autonomy (software orchestrated and controlled by other autonomous software, in a decentralized way) and can themselves be orchestrated by SNS technology, and thus run autonomously too. Today, though, the focus is on *mainstream* cloud. This year, the cloud industry will generate approximately one trillion dollars in revenue. That number is already huge, but is expected to grow to two trillion dollars by 2030. After years of continuous development, which have seen more than $500m spent on R&D, the Internet Computer network is now tacking directly toward this mainstream cloud market with cloud engine technology. In their first version, cloud engines are not meant to be a cloud panacea. For example, currently they are not ideal for working with big data. You should use something like DataBricks for that. Cloud engines are carefully targeted at enabling AI to produce traditional online applications and services, including SaaS, in a safer and more productive way, which represents a new market segment with tremendous potential. Of course, DFINITY will continue to work relentlessly to push forward ICP's capabilities, so expect further developments. It's worth mentioning that this cloud segment isn't just about creating new apps and services using AI, it's also about replacing legacy systems and apps built on super expensive SaaS services. Caffeine Labs is working to produce technology (Caffeine Snorkel) that can study an enterprise's legacy systems and app built on SaaS, create replacement systems and apps, and migrate the data, while supporting key stakeholders through the process over email and chat, with full automation. Thus the legacy systems and SaaS markets shall also be addressed by cloud engines. Zooming out, and reasoning in a more metaphysical way, we believe, as we always have, that there is room for a new kind of cloud created by mathematical networks, that provides seminal advances in the fields of security and resilience, as well as true sovereignty and freedom from lock-in. That this same technology, with the help of additional technologies like orthogonal persistence and Motoko, enables AI to build for us without the need for so much oversight, and to create more backend sophistication while consuming fewer AI API tokens, enables ICP to bring game-changing advances to the world. Cloud engines will work synergistically with the Intelligence Gateway, which will enable apps and services running on engines to seamlessly leverage AI, wherever that AI is running, while providing verifiability at extremely low cost for open weights frontier models. We believe that cloud engines represent an inflection point in the storied history of the Internet Computer project, and I'm very proud to be sharing the details with you on the network's fifth birthday 💪 I'll be back with more news soon!!

dom | icp

258,251 просмотров • 2 месяцев назад

In the future, you’ll be able to accomplish a goal by just giving Claude an outcome and a budget. That’s the direction Anthropic is building in with its new Managed Agents features, announced at this week’s Code with Claude developer event. The basic idea: Claude, wrapped in a computer in the cloud, that you can spin up, scale, and manage as needed. Anthropic is taking on the infrastructure that kills most agent products, and making sure that it scales to meet the needs of agents running 24/7. On this week’s AI & I from Every 📧, I talk with Angela Jiang (Angela Jiang), head of product for the Claude platform, and Katelyn Lesse (Katelyn Lesse), head of engineering for the Claude platform, about what Anthropic is building and what it takes to make agents reliable in production. We get into: - Why the "build a generic harness, hot-swap any model behind it" playbook is already outdated. Angela points to eval data on Memory where the same task across different harnesses performed drastically differently. - The infrastructure wall every team hits in production—and why Katelyn thinks “my sandbox died and took the agent with it” is the real reason internal agents don't ship. - Why Anthropic is so bullish on using file systems and skills within Claude, including Angela's argument that those early design choices can compound for years. This is a must-watch for anyone trying to take an agent past the demo and into production. Watch below! Timestamps: How the Claude platform evolved from API to agents: 00:01:48 The primitives that make up Claude Managed Agents: 00:04:09 Why the harness and the model are becoming a single unit: 00:10:37 The infrastructure wall that kills most agent projects in production: 00:18:49 Why team agents need a different shape than individual productivity tools: 00:24:49 How Anthropic's legal team uses an agent to review marketing copy: 00:26:36 Using multi-agent orchestration for advisor strategies, adversarial pairs, and swarms: 00:34:24 How to measure agent success with outcome and budget as the end state: 00:35:50 What the platform looks like a year from now, when Claude writes its own harness: 00:39:11
43:20

In the future, you’ll be able to accomplish a goal by just giving Claude an outcome and a budget. That’s the direction Anthropic is building in with its new Managed Agents features, announced at this week’s Code with Claude developer event. The basic idea: Claude, wrapped in a computer in the cloud, that you can spin up, scale, and manage as needed. Anthropic is taking on the infrastructure that kills most agent products, and making sure that it scales to meet the needs of agents running 24/7. On this week’s AI & I from Every 📧, I talk with Angela Jiang (Angela Jiang), head of product for the Claude platform, and Katelyn Lesse (Katelyn Lesse), head of engineering for the Claude platform, about what Anthropic is building and what it takes to make agents reliable in production. We get into: - Why the "build a generic harness, hot-swap any model behind it" playbook is already outdated. Angela points to eval data on Memory where the same task across different harnesses performed drastically differently. - The infrastructure wall every team hits in production—and why Katelyn thinks “my sandbox died and took the agent with it” is the real reason internal agents don't ship. - Why Anthropic is so bullish on using file systems and skills within Claude, including Angela's argument that those early design choices can compound for years. This is a must-watch for anyone trying to take an agent past the demo and into production. Watch below! Timestamps: How the Claude platform evolved from API to agents: 00:01:48 The primitives that make up Claude Managed Agents: 00:04:09 Why the harness and the model are becoming a single unit: 00:10:37 The infrastructure wall that kills most agent projects in production: 00:18:49 Why team agents need a different shape than individual productivity tools: 00:24:49 How Anthropic's legal team uses an agent to review marketing copy: 00:26:36 Using multi-agent orchestration for advisor strategies, adversarial pairs, and swarms: 00:34:24 How to measure agent success with outcome and budget as the end state: 00:35:50 What the platform looks like a year from now, when Claude writes its own harness: 00:39:11

Dan Shipper 📧

66,339 просмотров • 2 месяцев назад

Just six months ago, DHH (creator of Ruby on Rails and Omarchy) said how he doesn’t really use AI tools to write code, because they are not good enough. Things have changed, a lot. Timestamps: 00:00 Intro 02:11 Omarchy and Ruby on Rails 08:25 37signals overview 10:12 Launching HEY 18:38 Building HEY 22:47 Designers at 37signals 28:08 The craft of design 31:52 Why DHH now embraces AI workflows 39:45 The AI inflection point 44:23 DHH’s agent-first workflow 55:09 AI’s impact on junior developers 1:03:08 Developer experience with AI 1:16:43 What does AI mean for developers? 1:23:33 37signals teams and hiring 1:38:20 Work-life balance with AI 1:41:41 Why DHH keeps building 1:45:24 Closing Brought to you by: • Statsig – ⁠ The unified platform for flags, analytics, experiments, and more. Stop switching between different tools, and have them all in one place. • WorkOS – Everything you need to make your app enterprise ready. WorkOS gives you APIs to ship enterprise features in days. Check out • Sonar – The makers of SonarQube, the industry standard for automated code review. See how SonarQube Advanced Security is empowering the Agent Centric Development Cycle (AC/DC) with new capabilities. Three interesting observations from this conversation: #1 DHH's philosophy on AI has not changed, but the available tools very much have. Autocomplete-style coding assistants were genuinely annoying for experienced developers six months ago. Things changed with the shift from tab-completion to agent harnesses, plus the emergence of powerful models like Opus 4.5 – when agents started producing code which DHH does want to merge with little to no alteration. #2 Beautiful code and products aren’t matters of vanity; they’re signals of correctness. Dipping into philosophy, DHH says: “When something is beautiful, it’s likely to be correct.” He argues that Steve Jobs wanted the inside of a computer to be beautiful because people who care about circuit board layout are also those who sweat on the details of the UI. #3 DHH’s development workflow, today: He runs tmux to have two models running, and neovim in the center. Specifics: - One fast LLM running (typically Gemini 2.5) in one split terminal - A slow but more powerful model in another terminal (usually Opus) - NeoVim for reviewing diffs via Lazygit
1:47:20

Just six months ago, DHH (creator of Ruby on Rails and Omarchy) said how he doesn’t really use AI tools to write code, because they are not good enough. Things have changed, a lot. Timestamps: 00:00 Intro 02:11 Omarchy and Ruby on Rails 08:25 37signals overview 10:12 Launching HEY 18:38 Building HEY 22:47 Designers at 37signals 28:08 The craft of design 31:52 Why DHH now embraces AI workflows 39:45 The AI inflection point 44:23 DHH’s agent-first workflow 55:09 AI’s impact on junior developers 1:03:08 Developer experience with AI 1:16:43 What does AI mean for developers? 1:23:33 37signals teams and hiring 1:38:20 Work-life balance with AI 1:41:41 Why DHH keeps building 1:45:24 Closing Brought to you by: • Statsig – ⁠ The unified platform for flags, analytics, experiments, and more. Stop switching between different tools, and have them all in one place. • WorkOS – Everything you need to make your app enterprise ready. WorkOS gives you APIs to ship enterprise features in days. Check out • Sonar – The makers of SonarQube, the industry standard for automated code review. See how SonarQube Advanced Security is empowering the Agent Centric Development Cycle (AC/DC) with new capabilities. Three interesting observations from this conversation: #1 DHH's philosophy on AI has not changed, but the available tools very much have. Autocomplete-style coding assistants were genuinely annoying for experienced developers six months ago. Things changed with the shift from tab-completion to agent harnesses, plus the emergence of powerful models like Opus 4.5 – when agents started producing code which DHH does want to merge with little to no alteration. #2 Beautiful code and products aren’t matters of vanity; they’re signals of correctness. Dipping into philosophy, DHH says: “When something is beautiful, it’s likely to be correct.” He argues that Steve Jobs wanted the inside of a computer to be beautiful because people who care about circuit board layout are also those who sweat on the details of the UI. #3 DHH’s development workflow, today: He runs tmux to have two models running, and neovim in the center. Specifics: - One fast LLM running (typically Gemini 2.5) in one split terminal - A slow but more powerful model in another terminal (usually Opus) - NeoVim for reviewing diffs via Lazygit

Gergely Orosz

328,899 просмотров • 3 месяцев назад

What does it mean for software engineering when we no longer write the code? Here's the take from Boris Cherny (Boris Cherny), the creator of Claude Code. Timestamps: 00:00 Intro 11:15 Lessons from Meta 19:46 Joining Anthropic 23:08 The origins of Claude Code 32:55 Boris's Claude Code workflow 36:27 Parallel agents 40:25 Code reviews 47:18 Claude Code's architecture 52:38 Permissions and sandboxing 55:05 Engineering culture at Anthropic 1:05:15 Claude Cowork 1:12:48 Observability and privacy 1:14:45 Agent swarms 1:21:16 LLMs and the printing press analogy 1:30:16 Standout engineer archetypes 1:32:12 What skills still matter for engineers 1:35:24 Book recommendations Brought to you by: • Statsig — ⁠ The unified platform for flags, analytics, experiments, and more. • Sonar – The makers of SonarQube, the industry standard for automated code review. Proactively find and fix issues in real-time with the SonarQube MCP Server: • WorkOS – Everything you need to make your app enterprise ready. Three interesting things from this conversation: 1. Boris automated himself out of code review well before AI. Boris was one of the most prolific code reviewers at Meta company. And he worked hard to minimize time spent on code review. His system::every time he left the same kind of review comment, he logged it in a spreadsheet. Once a pattern hit 3-4 occurrences, he’d write a lint rule to automate it away! 2. PRDs are dead on the Claude Code team: prototypes replaced them. Instead of writing Product Requirement Documents (specs), they build hundreds of working prototypes before shipping a feature. Boris: “There’s just no way we could have shipped this if we started with static mocks and Figma or if we started with a PRD.” 3. This is the year of the generalist (and maybe the year of those with ADHD) Boris’s work has shifted from deep-focus single-threaded coding to managing multiple parallel agents and context-switching rapidly. As Boris put it: “It’s not so much about deep work, it’s about how good I am at context switching and jumping across multiple different contexts very quickly.”
1:37:58

What does it mean for software engineering when we no longer write the code? Here's the take from Boris Cherny (Boris Cherny), the creator of Claude Code. Timestamps: 00:00 Intro 11:15 Lessons from Meta 19:46 Joining Anthropic 23:08 The origins of Claude Code 32:55 Boris's Claude Code workflow 36:27 Parallel agents 40:25 Code reviews 47:18 Claude Code's architecture 52:38 Permissions and sandboxing 55:05 Engineering culture at Anthropic 1:05:15 Claude Cowork 1:12:48 Observability and privacy 1:14:45 Agent swarms 1:21:16 LLMs and the printing press analogy 1:30:16 Standout engineer archetypes 1:32:12 What skills still matter for engineers 1:35:24 Book recommendations Brought to you by: • Statsig — ⁠ The unified platform for flags, analytics, experiments, and more. • Sonar – The makers of SonarQube, the industry standard for automated code review. Proactively find and fix issues in real-time with the SonarQube MCP Server: • WorkOS – Everything you need to make your app enterprise ready. Three interesting things from this conversation: 1. Boris automated himself out of code review well before AI. Boris was one of the most prolific code reviewers at Meta company. And he worked hard to minimize time spent on code review. His system::every time he left the same kind of review comment, he logged it in a spreadsheet. Once a pattern hit 3-4 occurrences, he’d write a lint rule to automate it away! 2. PRDs are dead on the Claude Code team: prototypes replaced them. Instead of writing Product Requirement Documents (specs), they build hundreds of working prototypes before shipping a feature. Boris: “There’s just no way we could have shipped this if we started with static mocks and Figma or if we started with a PRD.” 3. This is the year of the generalist (and maybe the year of those with ADHD) Boris’s work has shifted from deep-focus single-threaded coding to managing multiple parallel agents and context-switching rapidly. As Boris put it: “It’s not so much about deep work, it’s about how good I am at context switching and jumping across multiple different contexts very quickly.”

Gergely Orosz

489,599 просмотров • 4 месяцев назад

Within software architecture, few people shaped the industry as much as Grady Booch. Safe to say he's a true legend. In today's The Pragmatic Engineer Podcast episode, he shares fascinating stories, insights, observations. Listen or watch the full episode: - YouTube: - Spotify, Apple, or web: Thank you to our wonderful sponsors: 🌟 WorkOS — The modern identity platform for B2B SaaS. Learn more at 🌟Sevalla — Deploy anything from preview environments to Docker images. Check it out at 🌟Chronosphere — The observability platform built for control. Get details at --- 3 of my takeaways from this fascinating conversation: 1. Surprising: The US Department of Defense and the military built some of the most complicated software systems in the 70s and 80s. In the 70s, these organizations probably had the most code to deal with - globally! - and things like distributed computing were pioneered thanks to these use cases. 2. The three axes of software architecture Grady argues that when talking about software architecture, we should look at these three dimensions: ceremony, risk and complexity. 3. The economics of software and software architecture are always connected Machine time was very expensive in the 1960s and 1970s, and software had to be written from scratch. Good architecture meant writing highly performant code to utilize these rather limited machines. However, these days, machine time has gotten very cheap, and there are also plenty of “building blocks” at our disposal: from frameworks to cloud services. Software architecture is frequently still connected with cost: to decide on what services and technologies to use, taking the cost aspect into account! I hope you enjoy this conversation - together with a history overview of the last 50 years in software - as much as I did! Thanks a lot Grady Booch
0:43

Within software architecture, few people shaped the industry as much as Grady Booch. Safe to say he's a true legend. In today's The Pragmatic Engineer Podcast episode, he shares fascinating stories, insights, observations. Listen or watch the full episode: - YouTube: - Spotify, Apple, or web: Thank you to our wonderful sponsors: 🌟 WorkOS — The modern identity platform for B2B SaaS. Learn more at 🌟Sevalla — Deploy anything from preview environments to Docker images. Check it out at 🌟Chronosphere — The observability platform built for control. Get details at --- 3 of my takeaways from this fascinating conversation: 1. Surprising: The US Department of Defense and the military built some of the most complicated software systems in the 70s and 80s. In the 70s, these organizations probably had the most code to deal with - globally! - and things like distributed computing were pioneered thanks to these use cases. 2. The three axes of software architecture Grady argues that when talking about software architecture, we should look at these three dimensions: ceremony, risk and complexity. 3. The economics of software and software architecture are always connected Machine time was very expensive in the 1960s and 1970s, and software had to be written from scratch. Good architecture meant writing highly performant code to utilize these rather limited machines. However, these days, machine time has gotten very cheap, and there are also plenty of “building blocks” at our disposal: from frameworks to cloud services. Software architecture is frequently still connected with cost: to decide on what services and technologies to use, taking the cost aspect into account! I hope you enjoy this conversation - together with a history overview of the last 50 years in software - as much as I did! Thanks a lot Grady Booch

Gergely Orosz

55,806 просмотров • 1 год назад

Enrollment is now open for the Data Engineering Professional Certificate! Data engineers are the architects of modern organizations, ensuring data is reliable, accessible, and ready for analytics and machine learning. This professional certificate is tailored to equip you with the critical skills, through frameworks and hands-on practice, to excel in this role. Taught by industry expert Joe Reis, co-author of the best-selling book "Fundamentals of Data Engineering," along with 17 guest instructors from the data field, you will gain expertise to start and further your career in the high-demand field of data engineering. Key focus areas: 🗂️ Data Engineering Lifecycle: Learn the important stages of building an efficient data pipeline that creates business value. 📥 Data Ingestion: Learn how to efficiently gather data from various sources. 💾 Data Storage: Master the techniques for storing data securely and cost-effectively. 🔄 Data Transformation: Understand how to clean, organize, and prepare data for analysis and machine learning. 🏗️ Data Architecture Design: Build robust architectures that support scalable, efficient data workflows. 📊 Serving Data: Ensure that data is available to stakeholders when and where they need it to drive business decisions. Enroll now!
1:33

Enrollment is now open for the Data Engineering Professional Certificate! Data engineers are the architects of modern organizations, ensuring data is reliable, accessible, and ready for analytics and machine learning. This professional certificate is tailored to equip you with the critical skills, through frameworks and hands-on practice, to excel in this role. Taught by industry expert Joe Reis, co-author of the best-selling book "Fundamentals of Data Engineering," along with 17 guest instructors from the data field, you will gain expertise to start and further your career in the high-demand field of data engineering. Key focus areas: 🗂️ Data Engineering Lifecycle: Learn the important stages of building an efficient data pipeline that creates business value. 📥 Data Ingestion: Learn how to efficiently gather data from various sources. 💾 Data Storage: Master the techniques for storing data securely and cost-effectively. 🔄 Data Transformation: Understand how to clean, organize, and prepare data for analysis and machine learning. 🏗️ Data Architecture Design: Build robust architectures that support scalable, efficient data workflows. 📊 Serving Data: Ensure that data is available to stakeholders when and where they need it to drive business decisions. Enroll now!

DeepLearning.AI

20,833 просмотров • 1 год назад

We The Builders EPISODE 1 is LIVE with Shahin Farshchi at Lux Capital Shahin has invested in companies manufacturing in space (Varda Space Industries), building autonomous transportation systems (Zoox), providing robots as a service (Formic), automating industries (Covariant), using space to revolutionize the Earth observation industry (Planet), 3D printing rockets (Relativity Space) and many more. The Jetsons Future I thought I would be living in 2025 is partly being built. We have a long way to go but this is exactly how we inspire the next generation. If someone from 1900 woke up and read this, they would call it science fiction. Thank you Shahin, for believing in and being a partner to these incredible companies and founders. Shahin is an incredibly kind soul, helpful, accessible and a role model for younger aspiring venture capitalists. From what I have personally observed and heard from friends building companies, he will cheer you on and have incredible empathy for the fact that you had the courage to build. In many cases, more than capital, that is what you need most. Highlights & Timestamps: 00:00 - Introduction 02:13 - Berkeley to Iran in teenage 05:01 - Adapting to different education systems 09:05 - Early transformative experiences 14:49 - Mentors and role models 18:20 - On breaking into research labs and flipping rejections 25:01 - Story of getting into grad school, applying for funding, and early research 32:41 - Going from research to venture capital 34:50 - Commercializing his research and starting a company 38:04 - Thoughts on ingredients of a good startup 45:11 - Where do the best founders come from 50:34 - Why PhDs are not equipped to commercialize 1:02:00 - Breakthrough innovation timeline 1:05:00 - Solving for shareholder demand 1:10:22 - How to think about what problem to solve 1:14:10 State of the education system
1:21:47

We The Builders EPISODE 1 is LIVE with Shahin Farshchi at Lux Capital Shahin has invested in companies manufacturing in space (Varda Space Industries), building autonomous transportation systems (Zoox), providing robots as a service (Formic), automating industries (Covariant), using space to revolutionize the Earth observation industry (Planet), 3D printing rockets (Relativity Space) and many more. The Jetsons Future I thought I would be living in 2025 is partly being built. We have a long way to go but this is exactly how we inspire the next generation. If someone from 1900 woke up and read this, they would call it science fiction. Thank you Shahin, for believing in and being a partner to these incredible companies and founders. Shahin is an incredibly kind soul, helpful, accessible and a role model for younger aspiring venture capitalists. From what I have personally observed and heard from friends building companies, he will cheer you on and have incredible empathy for the fact that you had the courage to build. In many cases, more than capital, that is what you need most. Highlights & Timestamps: 00:00 - Introduction 02:13 - Berkeley to Iran in teenage 05:01 - Adapting to different education systems 09:05 - Early transformative experiences 14:49 - Mentors and role models 18:20 - On breaking into research labs and flipping rejections 25:01 - Story of getting into grad school, applying for funding, and early research 32:41 - Going from research to venture capital 34:50 - Commercializing his research and starting a company 38:04 - Thoughts on ingredients of a good startup 45:11 - Where do the best founders come from 50:34 - Why PhDs are not equipped to commercialize 1:02:00 - Breakthrough innovation timeline 1:05:00 - Solving for shareholder demand 1:10:22 - How to think about what problem to solve 1:14:10 State of the education system

Suffiyan Malik

23,844 просмотров • 11 месяцев назад

Today, Sierra is releasing Ghostwriter, our agent for building agents. With Ghostwriter, you can create an AI agent for your customer experience — one that can chat, pick up the phone, speak dozens of languages, take action on your systems of record, and be protected with industry-leading guardrails — simply by having a conversation. No clicking, no forms, no menus. Codex and Claude Code have transformed how we build software, making it possible for software engineers to orchestrate and review the work rather than doing all the work themselves. We think the same transformation will happen for all software. Rather than every enterprise app having a web app for humans and an API for automation, every software platform’s UI will be an agent that can do the work on your behalf. I recorded a demo of my building and optimizing an agent with Ghostwriter so you can see how powerful and easy it is to use. It’s completely changed the way our early adopters build agents, and it’s changed the way I think about the software industry. Let me know what you think, and, if you’re interested in trying it out at your business, please reach out directly.
10:43

Today, Sierra is releasing Ghostwriter, our agent for building agents. With Ghostwriter, you can create an AI agent for your customer experience — one that can chat, pick up the phone, speak dozens of languages, take action on your systems of record, and be protected with industry-leading guardrails — simply by having a conversation. No clicking, no forms, no menus. Codex and Claude Code have transformed how we build software, making it possible for software engineers to orchestrate and review the work rather than doing all the work themselves. We think the same transformation will happen for all software. Rather than every enterprise app having a web app for humans and an API for automation, every software platform’s UI will be an agent that can do the work on your behalf. I recorded a demo of my building and optimizing an agent with Ghostwriter so you can see how powerful and easy it is to use. It’s completely changed the way our early adopters build agents, and it’s changed the way I think about the software industry. Let me know what you think, and, if you’re interested in trying it out at your business, please reach out directly.

Bret Taylor

1,060,194 просмотров • 3 месяцев назад

I sat down with Nicolas Sharp, founder of Attio, to talk in depth about how his company is disrupting the $80 billion CRM market. Attio has raised $116m, is 4x'ing ARR and is one of the fastest growing companies in Europe. We talk about: - Why Attio went against all conventional wisdom and spent years 3 years building the product before launching - Why Attio doesn’t hire ‘Software Engineers’ and who they hire instead. - How Attio chose investors who would back a long-term bet against multi-billion $ incumbents - How Attio is building CRM from first principles for the AI era - Who should you avoid hiring at all costs, and who should you hire for your startup And so much more. If you’re interested in learning about the story of how Nick built Attio into the incredible company it is today and is disrupting one of the most important software categories, you’re going to want to see this. Enjoy // Timestamps 00:00 Intro 00:39 Why Attio spent 3 years building their product before launch 5:05 The Power of Building Systems, Not a Box of Features 9:05 How to know when it’s time to launch 13:30 Nick’s Playbook For a Killer Product Launch 18:09 How To Go From an Investor to a Founder 23:05 How Failure Led to Attio's Big Break 28:07 Why startups need to hire "Hidden Gems," 34:05 Fundraising 49:36 Why Attio Invests In Inexperienced Talent 55:07 The Case For Not Hiring Software Engineers (& Who You Should Hire Instead) 1:02:46 The 8 Persona Hiring Framework 1:09:05 How Attio doesn’t use OKR's 1:27:11 Where does Nick's Ambition and Grit come from? 1:36:59 How Attio is Building CRM from First Principles for the AI era 1:45:29 How to Successfully Market In A Crowded Industry 1:51:10 Breaking Down Attio’s Viral Marketing Strategies 2:02:15 The Change That Had The Biggest Impact on Customer Conversion 2:05:04 Why Attio created A “Reverse Trial” 2:10:16 Why Nick is building from London, not Silicon Valley 2:22:48 The 10-Year Vision for Attio
2:26:34

I sat down with Nicolas Sharp, founder of Attio, to talk in depth about how his company is disrupting the $80 billion CRM market. Attio has raised $116m, is 4x'ing ARR and is one of the fastest growing companies in Europe. We talk about: - Why Attio went against all conventional wisdom and spent years 3 years building the product before launching - Why Attio doesn’t hire ‘Software Engineers’ and who they hire instead. - How Attio chose investors who would back a long-term bet against multi-billion $ incumbents - How Attio is building CRM from first principles for the AI era - Who should you avoid hiring at all costs, and who should you hire for your startup And so much more. If you’re interested in learning about the story of how Nick built Attio into the incredible company it is today and is disrupting one of the most important software categories, you’re going to want to see this. Enjoy // Timestamps 00:00 Intro 00:39 Why Attio spent 3 years building their product before launch 5:05 The Power of Building Systems, Not a Box of Features 9:05 How to know when it’s time to launch 13:30 Nick’s Playbook For a Killer Product Launch 18:09 How To Go From an Investor to a Founder 23:05 How Failure Led to Attio's Big Break 28:07 Why startups need to hire "Hidden Gems," 34:05 Fundraising 49:36 Why Attio Invests In Inexperienced Talent 55:07 The Case For Not Hiring Software Engineers (& Who You Should Hire Instead) 1:02:46 The 8 Persona Hiring Framework 1:09:05 How Attio doesn’t use OKR's 1:27:11 Where does Nick's Ambition and Grit come from? 1:36:59 How Attio is Building CRM from First Principles for the AI era 1:45:29 How to Successfully Market In A Crowded Industry 1:51:10 Breaking Down Attio’s Viral Marketing Strategies 2:02:15 The Change That Had The Biggest Impact on Customer Conversion 2:05:04 Why Attio created A “Reverse Trial” 2:10:16 Why Nick is building from London, not Silicon Valley 2:22:48 The 10-Year Vision for Attio

Wouter Teunissen

26,369 просмотров • 9 месяцев назад

AI has changed software engineering more in the last 3 years than it has changed in the previous 30. What’s needed is not a debate about whether it’s going away—instead it’s a serious discussion about its future: What are the new primitives, techniques, and best practices for software engineering in the age of AI. That’s why I brought Scott Wu (Scott Wu) on AI & I. He’s the founder of Cognition, the company behind the world’s first autonomous AI coding agent, Devin. Cognition got to $73M ARR in less than 2 years—and they just acquired Windsurf to accelerate their growth. I had Scott on the show to talk about where the programming goes from here. We get into: - What the new tools and workflows are for AI engineers. In the near term, Scott sees software engineering defined by a spectrum of tools. At one end are AI features that speed up coding, like tab complete; at the other are agentic systems, like Devin, that can take on tasks independently. Until engineers can operate entirely at the higher layer of abstraction, he argues, both are essential. - Why Scott thinks AGI is already here. By the benchmarks of a decade ago—passing the Turing test, solving hard math problems, and operating agentically—AGI is already here. The line keeps moving, he argues, because humans constantly redefine work around what machines can’t yet do. - Why developers will turn into product architects. Scott sees the long-term future of software engineering as a steady climb up the ladder of abstraction. Just as programming went from assembly to languages like Python and JavaScript, he thinks the future is humans focusing on the product, while AI agents execute. - How Devin stacks up against Anthropic’s Claude Code. Scott credits Claude Code’s success to great product design and the models becoming capable enough to support autonomous workflows. But according to him, the CLI itself isn’t the breakthrough, it’s how a tool fits into a developer’s workflow. Claude Code’s paradigm is that the AI is you, taking the wheel of your computer, he says, while Devin is like the engineer sitting beside you: it runs in its own cloud environment, manages the repo, and improves over time at testing and refining code. This episode of Every 📧’s AI & I is a must-watch for anyone interested in the brass tacks of how AI changes the future of programming. Watch below! Timestamps: Introduction: 00:02:02 Why Scott thinks AGI is here: 00:02:32 Scott’s personal journey as a founder: 00:09:27 Why the fundamentals of computer science still matter: 00:16:55 How the future of programming will evolve: 00:22:30 A new workflow for the AI-first software engineer: 00:26:50 How Devin stacks up against Claude Code: 00:29:33 Reinforcement learning to build better coding agents: 00:40:05 What excites Scott about AI beyond Cognition: 00:50:05
53:21

AI has changed software engineering more in the last 3 years than it has changed in the previous 30. What’s needed is not a debate about whether it’s going away—instead it’s a serious discussion about its future: What are the new primitives, techniques, and best practices for software engineering in the age of AI. That’s why I brought Scott Wu (Scott Wu) on AI & I. He’s the founder of Cognition, the company behind the world’s first autonomous AI coding agent, Devin. Cognition got to $73M ARR in less than 2 years—and they just acquired Windsurf to accelerate their growth. I had Scott on the show to talk about where the programming goes from here. We get into: - What the new tools and workflows are for AI engineers. In the near term, Scott sees software engineering defined by a spectrum of tools. At one end are AI features that speed up coding, like tab complete; at the other are agentic systems, like Devin, that can take on tasks independently. Until engineers can operate entirely at the higher layer of abstraction, he argues, both are essential. - Why Scott thinks AGI is already here. By the benchmarks of a decade ago—passing the Turing test, solving hard math problems, and operating agentically—AGI is already here. The line keeps moving, he argues, because humans constantly redefine work around what machines can’t yet do. - Why developers will turn into product architects. Scott sees the long-term future of software engineering as a steady climb up the ladder of abstraction. Just as programming went from assembly to languages like Python and JavaScript, he thinks the future is humans focusing on the product, while AI agents execute. - How Devin stacks up against Anthropic’s Claude Code. Scott credits Claude Code’s success to great product design and the models becoming capable enough to support autonomous workflows. But according to him, the CLI itself isn’t the breakthrough, it’s how a tool fits into a developer’s workflow. Claude Code’s paradigm is that the AI is you, taking the wheel of your computer, he says, while Devin is like the engineer sitting beside you: it runs in its own cloud environment, manages the repo, and improves over time at testing and refining code. This episode of Every 📧’s AI & I is a must-watch for anyone interested in the brass tacks of how AI changes the future of programming. Watch below! Timestamps: Introduction: 00:02:02 Why Scott thinks AGI is here: 00:02:32 Scott’s personal journey as a founder: 00:09:27 Why the fundamentals of computer science still matter: 00:16:55 How the future of programming will evolve: 00:22:30 A new workflow for the AI-first software engineer: 00:26:50 How Devin stacks up against Claude Code: 00:29:33 Reinforcement learning to build better coding agents: 00:40:05 What excites Scott about AI beyond Cognition: 00:50:05

Dan Shipper 📧

34,753 просмотров • 9 месяцев назад

Sharding is the best way to scale a database. But how does it work? And what's the best way to ensure data and queries are evenly distributed? Our interactive article on database sharding teaches you how.
0:25

Sharding is the best way to scale a database. But how does it work? And what's the best way to ensure data and queries are evenly distributed? Our interactive article on database sharding teaches you how.

PlanetScale

28,639 просмотров • 2 месяцев назад

99% of AI applications are cool-looking demos. Impressive, but don't get fooled by the hype. It takes a lot to build enterprise-grade products that deliver real value. I have at least three weekly conversations with companies that want to use a Large Language Model with their data. The demand is huge! Here is one idea about what you can do to help. The use cases that most of these companies want to solve are similar: They have an extensive knowledge base and want to build a simple application that uses that information to answer questions. In other words, they need help building Retrieval Augmented Generation (RAG) applications they can use in many different scenarios: 1. To train new employees 2. To help their support team 3. To search old meetings and documents 4. To help with their research However, building these systems is not straightforward. Yes, there's a lot of information online, but there aren't enough people who know how to create solutions that work. Here is the idea: Today, you can build an enterprise-grade RAG application without writing code. A couple of MIT PhDs with 10+ years of experience building AI applications created . It's a no-code platform for building applications using Large Language Models. They are partnering with me on this post. You can use Stack AI to create, test, and deploy an end-to-end production-ready AI system. It's SOC-2, HIPAA, and GDPR compliant and offers SSO, role management, access control, and on-premise deployments. Of course, you can use the platform with any LLM on the market now. It's the whole nine yards for building AI applications. Check them out here: 2023 was about models. 2024 is about the tools using these models to build production-ready applications. That's where I'd start.
0:31

99% of AI applications are cool-looking demos. Impressive, but don't get fooled by the hype. It takes a lot to build enterprise-grade products that deliver real value. I have at least three weekly conversations with companies that want to use a Large Language Model with their data. The demand is huge! Here is one idea about what you can do to help. The use cases that most of these companies want to solve are similar: They have an extensive knowledge base and want to build a simple application that uses that information to answer questions. In other words, they need help building Retrieval Augmented Generation (RAG) applications they can use in many different scenarios: 1. To train new employees 2. To help their support team 3. To search old meetings and documents 4. To help with their research However, building these systems is not straightforward. Yes, there's a lot of information online, but there aren't enough people who know how to create solutions that work. Here is the idea: Today, you can build an enterprise-grade RAG application without writing code. A couple of MIT PhDs with 10+ years of experience building AI applications created . It's a no-code platform for building applications using Large Language Models. They are partnering with me on this post. You can use Stack AI to create, test, and deploy an end-to-end production-ready AI system. It's SOC-2, HIPAA, and GDPR compliant and offers SSO, role management, access control, and on-premise deployments. Of course, you can use the platform with any LLM on the market now. It's the whole nine yards for building AI applications. Check them out here: 2023 was about models. 2024 is about the tools using these models to build production-ready applications. That's where I'd start.

Santiago

197,675 просмотров • 2 лет назад

Why did Uber build thousands of microservices? No better person to answer than Uber's first CTO, Thuan Pham. Timestamps: 00:00 Intro 05:32 Getting into tech 16:09 The dot-com bust 20:42 VMware 26:29 Getting hired by Travis at Uber 33:22 Early days at Uber and scaling challenges 40:57 Uber’s China launch 47:12 The platform and program split 50:26 From monolith to microservices 53:38 Internal tools at Uber 57:05 Helix: Uber’s mobile app rewrite 59:55 Thuan’s email about naming 1:02:03 Org structure changes under 1:06:34 Thuan’s work philosophy 1:12:23 The “three tours of duty” at Uber 1:15:37 Why Thuan left Uber 1:17:34 Coupang and Nubank 1:21:59 Faire 1:25:31 How Faire uses AI 1:28:24 AI’s impact on software engineering 1:31:09 The role of the CTO 1:35:13 Career advice Brought to you by: • Statsig – ⁠ The unified platform for flags, analytics, experiments, and more. • WorkOS – Everything you need to make your app enterprise ready. • Sonar – The makers of SonarQube, the industry standard for automated code review. Check out SonarQube Advanced Security: Three interesting parts from this conversation: 1. The program/platform split came before microservices. The concept of cross-functional “program” teams and dedicated “platform” teams became necessary because an org split across backend, frontend and mobile engineers slowed down in execution speed when Uber grew to around 100 engineers. Every feature required negotiating bandwidth across the mobile, backend, and dispatch teams. Thuan, Travis Kalanick, and Jeff Holden literally used color-coded sticky notes with people’s names to reorganize into self-sufficient teams. We cover more about this split in this The Pragmatic Engineer deepdive, The Platform and Program split at Uber: 2. Expect multiple rewrites during hypergrowth. The right architecture depends on how fast a product and company are growing. At Uber, repeated rewrites were common because each one “bought” another window of survival for the company. Thuan’s recommendation is to understand that a rewrite simply means a company is outrunning its existing architecture: this is not necessarily a bad thing! 3. Uber is the only major company that had a “Senior 1” and “Senior 2” level – and Thuan is unapologetic. Thuan introduced the Senior 1 (L5A) and Senior 2 (L5B) levels because the jump from senior (L5) to Staff (L6) became very big, and larger than between previous levels. One problem this split level created was that Uber’s L5B was akin to Google’s and Facebook’s L6/E6. Thuan resisted the title inflation of just renaming L5B to ‘Staff’.
1:39:34

Why did Uber build thousands of microservices? No better person to answer than Uber's first CTO, Thuan Pham. Timestamps: 00:00 Intro 05:32 Getting into tech 16:09 The dot-com bust 20:42 VMware 26:29 Getting hired by Travis at Uber 33:22 Early days at Uber and scaling challenges 40:57 Uber’s China launch 47:12 The platform and program split 50:26 From monolith to microservices 53:38 Internal tools at Uber 57:05 Helix: Uber’s mobile app rewrite 59:55 Thuan’s email about naming 1:02:03 Org structure changes under 1:06:34 Thuan’s work philosophy 1:12:23 The “three tours of duty” at Uber 1:15:37 Why Thuan left Uber 1:17:34 Coupang and Nubank 1:21:59 Faire 1:25:31 How Faire uses AI 1:28:24 AI’s impact on software engineering 1:31:09 The role of the CTO 1:35:13 Career advice Brought to you by: • Statsig – ⁠ The unified platform for flags, analytics, experiments, and more. • WorkOS – Everything you need to make your app enterprise ready. • Sonar – The makers of SonarQube, the industry standard for automated code review. Check out SonarQube Advanced Security: Three interesting parts from this conversation: 1. The program/platform split came before microservices. The concept of cross-functional “program” teams and dedicated “platform” teams became necessary because an org split across backend, frontend and mobile engineers slowed down in execution speed when Uber grew to around 100 engineers. Every feature required negotiating bandwidth across the mobile, backend, and dispatch teams. Thuan, Travis Kalanick, and Jeff Holden literally used color-coded sticky notes with people’s names to reorganize into self-sufficient teams. We cover more about this split in this The Pragmatic Engineer deepdive, The Platform and Program split at Uber: 2. Expect multiple rewrites during hypergrowth. The right architecture depends on how fast a product and company are growing. At Uber, repeated rewrites were common because each one “bought” another window of survival for the company. Thuan’s recommendation is to understand that a rewrite simply means a company is outrunning its existing architecture: this is not necessarily a bad thing! 3. Uber is the only major company that had a “Senior 1” and “Senior 2” level – and Thuan is unapologetic. Thuan introduced the Senior 1 (L5A) and Senior 2 (L5B) levels because the jump from senior (L5) to Staff (L6) became very big, and larger than between previous levels. One problem this split level created was that Uber’s L5B was akin to Google’s and Facebook’s L6/E6. Thuan resisted the title inflation of just renaming L5B to ‘Staff’.

Gergely Orosz

69,369 просмотров • 3 месяцев назад

What is it like to be a founding engineer at a startup, and how do you become one? Michelle Lim joined Warp as the founding engineer, and she's now a founder at Flint AI where she is hiring founding engineers. We talk through her journey and tactics helpful for anyone wanting to make an impact at a startup. Watch or listen: • YouTube: • Spotify: • Apple: Brought to you by: •⁠ Statsig ⁠ — ⁠ The unified platform for flags, analytics, experiments, and more. •⁠ Linear ⁠ — ⁠ The system for modern product development. I really liked how Michelle was realistic how, as a founding engineer, you have two jobs at once: #1: Software engineering. Build the stuff you need to do #2: Everything else to help the business succeed, including lots of things that are not to do with software engineering! Do all of these once you did job #1 well. In return, you get to learn a lot on how a business is built - which will help you if and when you become a founder yourself! Thanks for the great convo, Michelle, and keep it up!
0:58

What is it like to be a founding engineer at a startup, and how do you become one? Michelle Lim joined Warp as the founding engineer, and she's now a founder at Flint AI where she is hiring founding engineers. We talk through her journey and tactics helpful for anyone wanting to make an impact at a startup. Watch or listen: • YouTube: • Spotify: • Apple: Brought to you by: •⁠ Statsig ⁠ — ⁠ The unified platform for flags, analytics, experiments, and more. •⁠ Linear ⁠ — ⁠ The system for modern product development. I really liked how Michelle was realistic how, as a founding engineer, you have two jobs at once: #1: Software engineering. Build the stuff you need to do #2: Everything else to help the business succeed, including lots of things that are not to do with software engineering! Do all of these once you did job #1 well. In return, you get to learn a lot on how a business is built - which will help you if and when you become a founder yourself! Thanks for the great convo, Michelle, and keep it up!

Gergely Orosz

17,126 просмотров • 7 месяцев назад