AI is an amplifier. It allows you to move faster. And anytime you take take a task and you compress it, it puts pressure on all the others. So if you have really slow CI, well, guess what? That’s going to be a big problem. So you need fast CI if you don’t have test cases, that’s going to be a big problem. And so but but fast CI, test cases like these are all actually best practice anyways. And so again, what AI is doing is forcing forcing a conversation. Well, everybody is now a manager.
You have to think about things with manager thinking. Even if you’re super proud of never being a manager or something, well, you have to bring and learn the thinking. Even if you’re fresh out of school. And again, I think some of these folks learn way faster than others. You have to bring that thinking because it’s not just about just about the outcome, but it’s about what am I trying to achieve? How am I? What’s the best way to do it? How what’s it going to be like to maintain this over time?
Hey, Luca here. Welcome to a new episode of the Refactoring Podcast, where every two weeks we interview a world-class tech leader. Today’s guest is Chris Lattner, one of the most important engineers for modern computing. Chris invented key compiler infrastructure tech like LLVM and MLIR, but he’s also the inventor of the Swift language, a key contributor in bringing Google TPUs to market, and a lot more. Today he’s the founder and CEO of Modular, where he’s reinventing AI infrastructure
to make software portable across GPUs and platforms. So with Chris, we talked about his vision for AI and computing. We talked about how Modular’s mission makes us closer to a future where AI is open and portable. And we talked about open source, how AI is both empowering and straining contributors, how it’s changing intellectual property, and what workflows we need to change as engineers. And finally, we discussed how AI is changing software craftsmanship, how he’s bullish about junior engineers, and a lot more.
This was truly an incredible chat, so let’s dive in. Welcome Chris and thank you so much for being here with us today. Hey Luca, I’m very excited to be here, so thank you for having me. So Chris, you really need no introduction, but I’ll try anyway. These days you’re a founder and CEO of Modular, where you build AI infrastructure to make software portable across platforms and GPUs. That builds on your lifetime experience with the compiler infrastructure projects you created like LLVM, MLIR.
But you also created the Swift language, your pro to market, Google TPUs, and contributed to many, many projects over your time in companies like Apple, Tesla, Google, and more. You’re also, and that does not escape me, you pretty damn fine woodworker. Would you agree with that? Well, I try. I don’t know if I’m particularly elite, but I do make some furniture and some treehouses and a variety of other things. So I like to build things. Yeah. While I was prepping for this chat, I saw the pictures on your website in
complete awe of the things you built, like the treehouse, the dining table. So I think I know what’s going on here. I think woodworking is like your hedge against this AI future, in which only the practical skills survive. Am I right? Well, so I think that what you can read into that is that I enjoy building things, and I enjoy discovery. I like to be proud of the things that I build. I don’t think that’s completely unusual, but maybe I’m a little bit more willing to throw myself into it
and make a few mistakes along the way. Yeah. Love it. So jokes aside, I’d love to talk about your work at Modular, which as the name suggests, it is made of many pieces, many interconnected parts like the Max Framework, Mojo, Mammoth, and a lot more. So first of all, how do you even define what Modular is? What’s your elevator pitch as a founder raising money left and right? Sure. Well, so first of all, let’s take a step back and look at where we are in the world. You may have noticed the AI is happening.
Everybody’s trying to figure all this stuff out. People are buying billions of dollars of GPUs and putting them in gigantic data centers. Everybody’s in a race to build into and explore all this technology. Meanwhile, it turns out that it’s really hard. There’s a lot of technologies that exist that were built over the last 10, 20 years. They sort of kind of work, but it’s all duct-taping baling wire. And so what’s ending up happening is that we have this huge gap between what can be
achieved by the most elite teams, the biggest companies, or the most well-funded research labs, and then what everybody else can do. Yeah. And so you see this great consolidation, like everybody’s running on the open AI endpoint or something like this, but it doesn’t square with, like, AI is not new. Everybody coming out of a computer science program has been taught AI for the last five, 10 years, but we can’t actually map that. And so what Modular is trying to do is it’s trying to crack this open.
The root cause of this is a consolidation in the industry with many powerful forces all coming together, but then also a lack of modern technology. And so our hypothesis, our approach is to say, “Look, let’s bring the best-in-class compilers and languages and ML frameworks and this technology cloud serving. Let’s bring this all together in a best-in-class way that was actually designed for the current world rather than trying to take something that’s 10 years old, which, by the way, is
ancient in AI terms, or 20 years old, like CUDA, which is just completely a dinosaur. Instead of taking these technologies and trying to get them to work, let’s actually build something designed for the modern era.“ Yeah. The Modular approach then is say, “Okay, let’s go do this fundamental research. Let’s go make big investments. Let’s really crack this open.” And that’s what brings us to the outcome, which is things like Max and Mojo, which I’m really happy to talk about.
Yeah. So what do you think was sub-bar in the solutions that we had before? What’s about performance, interoperability? What is the gap that you’ve seen exactly? Yeah. Well, so there’s a couple of different ways you can look at it depending on what part of the problem you’re studying. So if you look at software today, you look at software engineers today, vastly all of them, like, I don’t know, 99%, probably, 98%, are all programming CPUs. Yeah. Right. And so people are fighting over,
“Is Rust going to be better than something else?” Well, Rust is a CPU thing. It’s 15 years old. It’s cool. But that’s not actually what’s going on right now. What’s going on right now is AI and GPUs and accelerators and new chips and, like, this is a huge explosion of compute, and people are vastly all focused on the wrong thing, in my opinion. Now, CPUs are important, and CPU software is well understood, and I get it, but what I want is I want to crack that open, like, get those people so they have
access to this new technology. Modern computers are massively accelerated already, and the CPU is a tiny part of that. Yeah. So that’s one and a half of this. So it says accessibility, democratizing it for people that otherwise wouldn’t even touch the stuff that’s scary and complicated. The other side of it is the people doing GPUs, there are these battle-hardened veterans that can actually, you know, they’re super awesome, super smart, amazing people. They don’t really want new technologies
because they’re experts in the old stuff, and so they’re the wizards that then get pulled out of the closet to go solve these crazy problems. And they can be effective, and amazing things are happening, like look at what AI is doing all around us. But the problem is there’s so few of them. We’re being held back. And the technologies they’re using, again, they’re, I mean, they can work if you hold them the right way. But it’s actually so difficult to do that, so expensive and slow,
that even the biggest labs are having big problems. And finally, what’s ending up happening, the biggest problem of all, is that there’s so many new chips coming from startups and hyperscalers and things like this, and there’s no consistent software platform that can scale across all of them. And so what we’re doing is we’re saying, again, build a modern software platform that can scale, that can be usable, so that even Python programmers can get into GPU programming, that can scale across different chips.
So yes, it can work on NVIDIA, which is really important, because NVIDIA is an amazing platform, and there’s so many amazing GPUs out there, but also on AMD and Apple, and we have upcoming new ASICs and specialized accelerators that we’ll be launching this year. And when you do this, like what we really want to do is we want to pull forward innovation. We want to pull forward the community. We want to pull more people into this modern world so that we can get more innovation and more cool things happening,
not just from the biggest guys, but every tiny research group that has five people in it should be able to do this. Yeah, so I’m thinking back at what you said before that one problem is that there are generally few people who can work on this today, and I’m wondering if the problem you think it is… I mean, on one end, it’s probably that it’s generally some kind of more new stuff that is growing today, but maybe it’s also because, as you said, tools and infrastructure is still less
mature than what we have had with CPUs for a long time, and so we work at the level of abstraction that is maybe too low. It’s too hard versus how hard it should be, am I right? Well, let me give you an analogy. So years ago, I was at Apple, and Objective-C had made the iPhone possible. It was this amazingly powerful technology. It was efficient when you needed it, but it was also high-level for frameworks. But it was also built on C, and it was also decades old, and it was also pointers and square brackets and scary,
and it was actually really difficult to use, but if you were an expert, it was very powerful. And so I built and brought Swift into the world, and one of the things that people saw is that, “Okay, well, it’s prettier, it has modern features, it’s a cool thing, I’m very happy and proud of Swift.” But what practically happened is it massively multiplied the number of people that could build iPhone apps. And so it was very funny because some of the Objective-C people were very grumpy about this.
“Well, I’m the next expert already, I don’t see the problem.” But what was happening is Objective-C was gatekeeping app development and preventing people from getting into the ecosystem. And so today, you have exactly the same thing happening. You have CUDA, which is venerable and very important, but it’s also based on C++, which is 40 years old. CUDA itself is 20 years old. It wasn’t designed for modern GPUs at all, right? And so it’s possible to use it, and if you’re an expert, cool,
you can be effective, but I look at all the people that should be using it, and there’s 10 times, 100 times more people out there that should be programming GPUs that are. And I think that as we crack that open, it’s just a huge opportunity for the world. It’s a huge opportunity for programmers to upskill. There’s a lot of great things that you can do with this. Yeah, I think the analogy between Swift and Objective-C is perfect. And it also speaks to me because I was a startup founder by the time,
I mean, Swift came out, and I remember we were working on our mobile app in Objective-C, and we were coming from Ruby on Rails. And I remember thinking, “Wow, Objective-C.” It’s really something when it comes to the different altitude that you have to work with compared to a web app. So I can totally see that. And I think if we keep the analogy for your work on GPUs now, one valuable position is that we are going to raise the level of obstruction without being any penalty in terms of performance,
in terms of how the thing works, just like with Objective-C are compatible with the underlying infrastructure. Yeah, and so if you dive into Mojo and say, “Well, what are we doing about it?” So there’s a big problem out there. There’s a big opportunity, in my opinion. What we did was we said, “Okay, let’s go build a new programming language.” Yeah. But in the case of Swift, Swift had to go solve a syntax problem. Like Objective-C syntax was kind of a big part of the problem.
And so build a modern language with new syntax. Today, we don’t have that problem. Today, we have good syntax, Python. But what we have a problem with is we have missing compilers. We don’t have a good way to be able to target heterogeneous accelerated compute from different vendors, right? And so what we did was we said, “Okay, let’s go and build a fancy new compiler stack.” We used the MLIR compiler framework that I built a few years ago for accelerators. Let’s build a completely novel pipeline
that allows very powerful metaprogramming, full control over the hardware. You can go all the way down to assembly if you need to, but you can actually build much more powerful high-level abstractions. And as you say, abstractions give you power because you can reason about things more declaratively and at a higher level and get away from the details. But we also need to be able to expose all the wacky AI stuff. And so there’s these 4-bit floating point formats. They’re super fancy and very important.
And let’s lean into this and prove that we can build a stack that can be as fast or better performance than CUDA on NVIDIA, but also portable. And unlike any of these things, also run on CPUs. And so one of the really cool things about Mojo is you can go pip install it. It’s free, by the way. Just go nuts. You can pip install it and then you can run on CPU. And so you can say, hey, a plot C compiler or a plot, a plot, go, please, migrate my Python code over to Mojo. And often what you’ll find is it will
grind away and burn some tokens and then you’ll get something that runs 700 times faster. Yeah, yeah. And the resulting code is Pythonic. You can read it. It actually still looks reasonable. It’s not like it switched to a completely different universe. And when you do this, suddenly you say, OK, well, that’s that’s cool. How about we get on a GPU? Now you can get another thousand times faster. And then, you know, you can keep taking these steps. And what you can do is you can learn each step along the way.
Yeah, it’s amazing. I mean, I remember while I was doing some research for this interview and got more into Modular, what struck me was the scale of this work and the fact that it seems to me and then you’ll tell me if it feels that way to you, too, that it builds on everything you have deeply cared about in your career. Like open source, interoperability, compilers, programming languages. It’s all in there. And and basically, I mean, I think some of the best founder stories, you look at them backwards and it seems
like the founder was destined to do that. Right. Because you can connect it up backwards and it feels like everything you’ve done in life has led to that. That does you feel like that to you? Yeah, absolutely. I mean, the way the way I explain it is that Modular is kind of my life’s work. This is not something that woke up one Thursday and decided, let’s go build a startup and see what happens. It’s been very carefully planned. A lot of the work has been done across the course of many years.
A lot of the and a lot of the work was learning. And so part of the learning that went into this was building and scaling the Google TPU software stack. Yeah. So for years, I was working on TensorFlow and TPUs and XLA and JAX and all building compilers and runtimes and understanding the scope of the problem. And yeah, without that experience, I could not never have done this because, you know, you need to see things at scale, you need to be able to work with the researchers. They’re pushing the boundaries.
You need to understand what they really care about instead of just like building an abstract thing that you hope is useful. Yeah, absolutely. And I mean, you say with your story, you say, this is my life’s work. It’s really, it’s really something right. So it’s really fantastic work. And and say, well, and look, I’ll admit, like, I actually love what I do. Right. So this is that is also part of the secret. And I’m very happy that it’s valuable and it’s solving real problems
for very, very important people. So so in speaking about the fact you love what you do for someone like you who has seen like everything in engineering, what makes you the most excited about your work right now? That is, what is the challenge that gets you up in the morning and you’re excited to solve about this? So I’m a very strange person because you talk about woodworking. Yeah, I love building. I love building all the things. And so I love whether it be, you know, last night working on a compiler thing and the mojo
compiler fixing the variatics representation is like this tiny obscure thing that nobody should have to care about. But but when it clicks and it gets right, you know that it’s right. Now you can continue to build the world on top of it. And just like that happiness from one small thing that is right. It gives you a lot of pleasure. You tell me you’re still coding as a CEO of a billion dollar company. You can you can look at my GitHub. I probably have a few thousand contributions this year.
So yes, I’m I’m actively helping the team. I’m not a lead programmer or anything, but I’m a good obvious. But I also love building teams. So building like building and getting amazing people together, inspiring them, giving them ways that they can actually grow in their careers and actually they can contribute and they can do really cool things and they can point to it too and they can take great pride in making the world a better place. I love the business side and working with customers.
And when when they’re so excited because they can get better performance or latency or they get you know, we have people that love our stuff because they just can move faster. Right. Yeah. Or they like it because they’re trying to hire new people. And it’s really difficult to get people that can work on these legacy technologies. Yeah. Right. But it’s very easy to learn the Modular tech and upscale and grow. And this is fantastic for people who want more AI in their products. And that’s amazing. It makes me so happy.
I do like talking to people about this and educating them. And so some of the blog posts when I get a chance to write, I enjoy that piece as well. There’s there’s a ton of different aspects of my work. And so I love all the different pieces in different ways. Yeah. Love that. So I wanted to ask you also about about the vision, because I think AI companies are about the future. Right. And it feels like especially the best ones, they’re bringing some kind of point of view and something that they stand for.
And it feels to me like looking at your work, the Modular that thinks you stand for portable infrastructure, open source interoperability and so on. And we know the AI stack is made of many parts and some of them are more vertically integrated. Right now, different tier others, we have open source that is lagging behind. Sometimes slightly, sometimes more. So do you think you have a vision of the future which AI is eventually more like a commodity, more like electricity, like some people say, which opens to the
top to bottom, or it’s always going to be a race against Frontier Lab and centralization and so on. What do you think? Well, so I agree with you completely. So I do have several strong points of view. And if you push me on them, I’m very happy to share them. Yeah, absolutely. I’m here for it. Yeah, so my view is that, again, we get out of the bottleneck we have right now where all the systems and processes and things like this force a consolidation on a few pretty legit amazing teams,
building the most world leading models and then everybody tries to build on top of those and follows their lead. What should happen when we solve that, when we crack that is the AI becomes just another programming paradigm. Hmm. That’s all it is. It’s a great way to solve a class of problems in software. And usually those problems are at the boundary between human perception and human interaction and the real physical world and computers. And AI is the thing that really changes the narrative because before AI, you
know, you always had to work with the computer. Like you have to figure out how to type on a pretty keyboard of all things. Like what the heck? Like that’s not a human thing. That’s a forced computer thing that we’re having to meet the computer. But now AI is allowing the computer to meet us and work with us in the modalities and the world that we live in naturally. Right. And so that’s fantastic. And when you look at a lot of the reasoning capabilities, like this allows us to build into new kinds of
capabilities that we otherwise couldn’t do. But at the same time, you still have traditional software for loops are still a thing. Like they’re definitely not going away. And so replacing all software with just an AI model doesn’t really make sense for any economic or efficiency or latency or whatever reasons. Right. And so I’m a huge believer in AI, but I also want people to be a little bit more balanced than the hyperbolic, like we should just replace all software with end-to-end models, which
doesn’t make any sense to me. So, but when you, when you look at that and you say, what is AI as a programming paradigm? You know, it can be something like object oriented programming back in the day or structured programming when, you know, people figured out the go tos were a bad idea. And what they can do is they can open up more complicated, higher abstraction problems that then allow software to grow and solve bigger problems and have bigger impact. And that’s really what I want. And yes, LLM chat bots are a cool thing
and agentic things, like all that stuff is super cool and that definitely should not go away. But I’d be way more excited to see, you know, your next watch app having an innovative model that does some triangulation on real world data that, you know, only it can detect and triangulate it with, I don’t know, something in your bike tracker or something. Yeah, absolutely. And speaking of AI and programming paradigms, as you’ve said, one topic that I wanted to discuss with you is about open source.
Because of course, you’ve done a massive amount of open source work in your life and AI is changing a lot how people and maintainers are thinking about open source. And I know you’ve written recently about the Cloud C compiler and there are many instances of other projects being rewritten or created with AI. And there are so many angles to discuss about this. But first of all, at a glance, are you more excited by what AI makes it possible for open source or are you more worried? I’m a little bit worried, honestly.
And so I think that I’m worried about it for a couple of different reasons. Again, I’m very pro AI and so I’m very excited about the tools and capabilities. But what it means for open source, I think is a really big open question. And so I’ve seen so among other things, I’m on the board, the board of directors for the LVM Foundation, right? That oversees the LVM community and things like this. I’m involved with many other different open source projects in different ways.
It’s very open source is very important. Modular is open sourcing like tons of stuff like companies join our community. It’s super important. And I think that what’s happening is that AI slop is really overrunning a lot of the maintainers. And the trade off with the well run open source project has always been that you have maintainers that are long term community members that are responsible for certain areas that are growing their responsibility in their career. They’re investing in the project.
And as you have new contributors coming in, the maintainers want to invest in those people so that they can grow into being contributors and an important part of the project some day. And for a project like LVM and many others, this dynamic has always been a very healthy co-investment like between long term maintainers and new people coming in. But with AI tools, what’s happening is that a lot of maintainers are getting overrun. They’re just like all this people are slop coding things. And because the contributor doesn’t have
to do nearly as much work, but the reviewer has to do the same, but at a bigger scale. Like what’s happening is that I think it’s going to lead to new contributors not getting the attention that they deserve. And I think that could have a long term bad impact on the health and vibrancy of communities if you cut off that inflow of new contributors coming into projects. Now, it could be that we just need better, better tools. Some people think that, OK, well, you know, someday in the next future model
that will come out, the models will actually be perfect and all the contributions just work. And if that’s the case, then I’d say, OK, well, do you need open source anymore if you can just theoretically vibe code entire kernels from scratch? I mean, if if you think that’s a good outcome, then that also isn’t really great for open source projects and open source communities either. And I’m personally very skeptical of that. Yeah, I’m skeptical too for what I mean, for what matters.
Just like before you said, for as good as AI can be, it’s not like we’re going to replace old software with AI because it doesn’t make sense for an efficiency performance. Latency and da da da. At the same time, I think that composability and using stuff that you’ve already done, it’s something that I find a hard time thinking that it’s going to go away because we can just recreate stuff for scratch. I mean, I agree. Even if we are if we were able to write it, it would feel like completely
inefficient and would feel weird that we haven’t found a better way to just reuse stuff. That’s right. Well, so it doesn’t even honestly make sense like so. AI can generate large scale software. That’s proven. So I wrote a blog post I know you’ve seen talking about the cloud C compiler. And it’s very impressive what a team of agents with effectively zero human input can go and make a C compiler that it’s not production quality, but it’s like in it’s interesting.
Like I find it very interesting, let’s just say. Yeah, but there’s no novelty to it. And the reason it worked is because it has a huge amount of C compilers in its training set. Yeah. And so it’s basically transcoding LVM and GCC compilers that exist into a new form. Yeah. Well, so that’s cool, but that’s that’s not cool for creating new things or driving the world forward or building consolidated communities of people working together that I think is going to be most useful for people that want to
like strip licenses. And so you say, okay, well, I don’t like the GP. I don’t like the GPL license. Hey, Claude, go rewrite this from C into Rust. Yeah, now it’s not. I have questions about that too. In my notes. So here’s here’s a proprietary program. There’s been a problem in the LVM community where somebody took some proprietary IDE tools and then used used agents to basically clone the behavior. Yeah, actually disassembled the tools and reverse engineered them, which is against
the license, by the way, to generate an outcome that then cloned a lot of behavior of this proprietary tool. And so I don’t I was like, it’s just open source is kind of figure this out. I think all software has to figure this out. But but I do think it means we should change how we think about the value of the software that we’re creating and the people that maintain it and the communities that drive it. And I think that’s actually the key thing that a lot of folks aren’t even talking about.
Yeah. This is so interesting. I have like 1000 questions about this topic. So about mantainers and open source autos being overrun by this. I mean, there are many prominent people that are coming out saying these from Mitchell Hashimoto to actually closing external contributions. And as Mitchell said, like throwing the baby with the bathwater while doing this. Do you think this is a matter of figuring out a better workflow? Because maybe, you know, the way we have been working so far with PRs and stuff,
it’s it was optimized for humans and had some implicit assumptions about humans that just don’t stand anymore. But we will settle on something that works. Or you think we are kind of in trouble because it’s not clear where we should go from here. No, I think it’s like your intuition is exactly right. I agree with your first part, which is. We just we just need to learn new processes. And I think that again, I have the infinite belief in the human ingenuity and ability to solve problems.
And this is definitely a rising problem. Like we need to get on this. We need to figure this out. But we will. And the same thing that’s happening with, you know, open source projects is happening inside of companies, right? Where certain companies are incentivizing people to write the most amount of code and they’re telling them you have to use agents for things. And, you know, the CEO is saying, like, I want to measure productivity of my people and we need to use these tools and blah,
blah, blah, blah, blah, blah. And they may be well intended. But then what you get is you get a lot of bugs. You get a scope creep. You get lots of code that doesn’t work very well. You get huge maintenance problems and you get like toxic toxic code bases that nobody wants to work with. And so I think that, again, different teams are better and well, better or worse managed. It’s not consistent for everybody. But I do think the way is an industry need to figure this out. And I think we will.
It’s just a matter of these tools are moving really rapidly. And kind of as we build perhaps new processes like you say, or maybe the models get slightly better or other things evolve, we’ll definitely find a new stable point. Yeah, I believe that too. And you mentioned licenses before. And there have been instances, as you said, of I mean, many different cases. Either some library gets rewritten with AI and with this different license from GPS to MIT claiming there was a case where the maintainer claimed it was a
cleaver implementation, quote unquote. But actually, because it was done by AI, so it’s all I mean, new territory. So nobody knows what they’re doing basically with this. Do you think that we need to think about how licenses work or do you have a vision about what’s going to happen? Yeah, I think that absolutely like copyright needs to evolve. I’m saying copyright currently protects the kind of the form of the output. And I think that’s going to be obsolete here. I think it is already obsolete.
And so it’ll take some time for the legal processes to figure this out. And I have no idea when that will be. It probably take way longer than it would be desirable for. And so I think that more of these examples will keep happening and we can expect that. Yeah, I think it’s very hard to predict some. Some are saying we may became copyright like the interfaces and they use those as anchors. I mean, I don’t know. Yeah, but look again, I’ll just give you my my take on it. And again, I’m one
person. I’m very strange. I have my own ideas. Right. I think that anybody who’s trying to build a business on like copyrighted headers or something is just not trying hard. It’s really crazy. Right. And so I from my approach, I just look at this as just such an amazing time. Right. We’re building an amazing open community. We’re open sourcing all the stuff because open source is like the better way to do software. And and you know, if you’re still running a business where you’re trying to keep
things proprietary and that’s your secret sauce, which there’s a lot of companies, particularly hardware companies that do that. Yeah. Well, you’re already 20 years out of date. Like this is really a new thing. And so, yeah, you need to adapt and you need to figure it out. And they’re going to be kind of having some challenges. But but maybe it’s just a forcing function to get them to up level their approach in general. And that’s also good for the world. So yeah, I’m generally the optimist.
Right. And so I look at things from a what can we do with the perspective? But but that’s the other way to think about it. Also, because I mean, we have we have basically covered all the doom and gloom. But there is also like the bullish case in which AI allows for an explosion of more open source software. And it makes it easier for people to pick up open source projects because now they’re, you know, the ramp up to get to learn a new project is easier. I mean, there are a ton of positives that
we can find if we look for that. Well, I’ll tell you the positive I see. But let me let me let me just caution you on that. Right. I I agree with you. We’re going to see a lot more software. That is. Yeah, definitely true. The problem I have with that, though, is that we don’t want more software. We want more high quality software. Having more left pads or other clones of other things is not actually going to make the world better. That that is not actually what we want. We want high quality new
ideas that move the world forward. That’s actually value. And so for me, so building Mojo and building Max and building new communities and building new tools. Yeah, it’s fantastic because what we’re seeing is so Mojo, because it’s open source and because we have like hundreds of thousands of lines of GPU kernels and tons of example code, you can just say, hey, take this CUDA kernel, move to Mojo and Claude will just do it. It will. And it’s verifiable. It’s a perfect use case for AI tools.
And that’s an amazing way to get started because now you get off of a proprietary ecosystem onto an open one really fast. And you can do this without being the super expert. And one of the other amazing things is because the way we built Mojo and because we have these high level abstractions that cover a lot of hardware details. It’s really amazing because you can say, hey, I have this like legacy GPU kernel over here. Yeah, move to Mojo and then adopt the best practices and often it runs faster.
And so that’s that’s actually fire. That’s really cool. Yes. And the ability to create communities and catalyze new technologies faster than ever before is just like amazing. It’s just such a powerful time in the industry. Yeah, absolutely. And now I’m happy that we’ve given our own balanced take that with the with the bads and the goods above this. I also wanted to ask you about AI and software specifically, because I saw that in I think in November you went to a podcast and said you used AI for
coding and you were probably getting 10 to 20 percent faster. Something like that. How is it going today? Three to four months later? Yeah, well, so it’s a great question. So it the tools gotten way better since November. Yeah, a pretty significant step change, but it hasn’t directly made my productivity any higher. And so it’s probably 10 to 15 percent. It’s way more reliable. And so it’s way more pleasurable. And I think that don’t underestimate programmer happiness as
a actually a good thing. And so productivity isn’t the only measurement, but I don’t think the aggregate has been that big of a difference. And so, yeah, here’s the thing that I think that a lot of people gloss over is that software production and creation is only a tiny part of that is about writing the code. Most of it is deciding what to build, working with other humans, working with a whole bunch of other issues. And so AI definitely takes the friction and the microaggressions out of large
scale refactoring and things like this. It’s fantastic for these automatable and verifiable tasks, but you still have to make decisions. You still have to engage with things. And so what I found is I spend most of my time on the thorniest problems. So this is actually the frustrating part is that kind of the easy, it feels good part has been automated away and you get left with some of the harder pieces and so different lived experience from that perspective. Yeah, I think you said something very
interesting before you said that tooling got better. AI makes you happier as a programmer, even on things that don’t move. Let’s say the bottom line of your productivity. What are these things that make you happier as a programmer, even if they don’t count to the to the world productivity? Oh, yeah. So for me, updating test cases. So you make it you make a change and then you it’s an intentional behavior change. And so you have to go up to all the stuff. It’s like, hey, just go do it for me.
It’s so makes me so happy and I have to do that because I can start writing the PR summary or whatever and do the stuff while this is just being done for me. I don’t have to worry about the mechanical piece. Doing the rename X to Y, right? And but do it contextually, sensitively. And you have to move an argument around or just like very mechanical stuff going away. It’s just it’s just wonderful. And so there’s a lot of things like this. Also, the tools are so great within context learning.
They can even handle obscure things like MLR syntax and stuff like this. And so it’s just amazing to me just you have the magic moments where even tab, tab, tab is still still very valuable and still quite powerful. And so there’s there’s a lot of different aspects of that. But again, you still have to you know, I haven’t seen it where it’s like, OK, well, here’s a generalized idea of which direction I want you to go. And it can actually do that. I don’t see that working.
Yeah, but I think you mentioned very valuable things and it stays within a pattern that I’ve seen also with many teams that are using AI successfully. And they all mentioned rather than, I don’t know, some big moonshot that they were able to do with the AI. It’s more about enforcing better the basics and the basic hygiene like testing dogs and and basic code health and good quality. All things now you can enforce in an easier way than before, because AI can actually take care of a lot of that.
Yeah, so I’ll give you the caveat here. So again, I’m a pretty pretty experienced programmer. And so lifting me 10 percent is actually a big deal. Yeah. And so what I’ve seen is that people that are earlier in their journey and they’re still learning things get way more than 10 percent. So I’m definitely an outlier in that way. But I’ll tell you an example of something that I want to see. This is something I would love even Modular to do. We haven’t had time to to build this yet is say, hey AI, I’m going to
have a read me MD in every directory of our technology that we’re building. And I want an architecture doc in the repo that describes how that each module works and what it does and what the responsibilities are, etc. So that are just good, really, really amazing design and say, well, first of all, you need a little bit of co design with the AI to make sure that the architecture doc is right. And then it’s structured the right way. And you need review of what’s going on. But AI can generate 80 percent of that
almost one shot. Right. And so you can iterate and then you can get that in. And then you do upfront work to build out this architecture doc. But then I want a weekly cron job that says, go scan all these docs and identify likely out of date. things. And I think that then what you could do with you have to do both. You can’t just do one. You have to do both. But once you do that, you could have really amazing design docs that make it way easier for people to onboard into the project, make it so that the AIs
themselves can understand the design of this desire, not just what exists. Right. And I think that in the process of creating these architecture docs, you get a lot of these like, OK, well, that is how it works. But it shouldn’t work that way. And it could actually drive some of the architecture forward. And I think that an approach like that would be very powerful, both for projects like ours internally, but also for open source projects as well. Because again, what you want is you want people to get into the project, be able
to understand things and learn and upskill very quickly. And I think this is one amazing opportunity that I haven’t seen people tackle yet. Yeah, it’s a fantastic use case and I agree and it also saves your precious time and brain for the things that actually deserve it as opposed to updating the docs or these or these other tasks. Well, and this is something where we do actually have internal docs, but they get out of date. And so what’s worse, not having docs or having docs that are out of date?
Yeah, it’s almost worse to have them out of date, if you ask me, almost worse. I don’t know. That’s right. Yeah. You also said something interesting before when you said you’re a very experienced programmer. So for you even getting 15 percent better, it’s something. And while if you look at younger folks, they might get way more than that out of AI, right? Absolutely. So has the latest models and tooling with AI? Is shifting what kind of qualities you’re looking for in engineers that you’re
hiring or even how junior engineers are working? And are you being intentional about this in a way? I understand. So again, I’ll give you these weird ideas about the world where most people are running around saying, yeah, we’re not hiring junior programmers or we’re laying off all of our people. We are actively hiring. So we have way more interesting problems to solve than we have amazing people to do it. And we do have a lot of amazing people. But we’re also hiring new college grads
and other people that are junior interns. It’s amazing because a lot of these folks bring a completely different expertise, like they’re AI tool native. Some of the more experienced people, you know, they’re using the tools, but they’re not using them the best way. The tools change very rapidly. And so like this willingness to adapt new technologies and things is very powerful. But also, again, what I think is true for all teams, and again, I like to build things, including teams, is that you want
people at all levels within a team. If you have all super experienced engineers and no juniors or you have all juniors and those experienced people, you get into problems on either side of this. Good teams have people at multiple different levels and they’re learning from each other. They’re working well. They can be effective. They can be taking on different kinds of problems and things like this. And so what I think that agents and new coding tools have really shifted is it means that new college grads or, you
know, the power I see that it’s very proud of never being a manager. Well, everybody is now a manager. Well, everybody is now a manager. You have to think about things with manager thinking. Even if you’re super proud of never being a manager or something, well, you have to bring and learn the thinking. Even if you’re fresh out of school. And again, I think some of these folks learn way faster than others. You have to bring that thinking because it’s not just about just about the
outcome, but it’s about what am I trying to achieve? How am I? What’s the best way to do it? How what’s it going to be like to maintain this over time? Right. These second order of questions end up mattering way more than how quickly can I pound out the code? Yeah, yeah, absolutely. And I think I mean, it feels natural to think that junior engineers might be in trouble because they are exposed to a lot of these things from the get go that we’re not exposed to before. I like, you know, high
level thinking code reviews. But I mean, I have this feeling this is part of my own weird ideas that we are making this judgment based on kind of our outdated intuition about what engineers need to be able to do versus what they need. Not are not needed to do. Well, instead, when the tools evolve, people learn based on on the tooling they have at disposal. So it’s like, you know, a liquid that fills the container, the shape of the container they’re in. So I think that junior engineers are just
growing at will grow as fast as they’ve always grown in ways we probably don’t expect or can’t predict. But they were probably, as you said, they’re an AI native. They’re probably going to be more effective than many or the most seasoned engineers who may have trouble adapting after many, many years of doing things in a way. Well, the way I look at it is the AI amplifies things. And so it amplifies your ability to write code, it amplifies your ability to learn, it amplifies good or bad architecture.
Like if you scale it rapidly with AI, you can get a mess if it’s already a mess. But within teams, I think that what’s going to happen is that the people, as you say, that refuse to use new tools or refuse to evolve, those just kind of get left behind. And the people that lean into this, the people that are so motivated and hungry and curious and want to learn and grow, they’ll grow way faster. And so, again, you can choose to do what they want to do. I think that I’ve seen lots of people say
AI is ridiculous and none of this stuff is working and it’s all BS. And that’s not true. I’ve also seen people say like, oh, my God, like coders are out of a job. Like all AI will just be generated or all code will just be generated. It’s like, well, I don’t believe that either. Right. So but you do have to lean in and have figured it out. Yeah, I think you got a great analogy before you said basically all engineers and managers now and AI rewards manager behavior. But I mean, my provocative take is also
that engineers notoriously, most of them, they don’t want to become managers. Right. I mean, they like the fact that is required. Yeah. So are you finding like easy to for your people, people on your team, this kind of transition psychologically in terms of skills that need to apply? How is it? Yeah, well, so I would I can’t give you an answer to that because we have a large team and so different people are adapting at different speeds. And so some people are struggling more than others.
And I don’t think there’s a good indicator. I don’t think the Modular is unique in that way. I think that we probably have a representative slice of many very high tech teams. So I don’t think there were unique or not. I think we have a magic solution. What I do think is important is I do think that it’s important to encourage some of the people that are leaning in, even if they feel crazy and their token budgets are expensive and enable them to pathfind and figure out, OK, well, what
actually works in practice? And what I always challenge people to do is say, look, vibe coding out some amazing demo and like having some crazy thing that’s really exciting and very cool is actually not success. Yeah. What is success is when code gets in the product and ships. Yeah. And so there’s sometimes a gap there. And I’ve seen some danger cases where somebody got very excited and some some person who maybe isn’t even a professional software engineer is like, oh, wow, look what I can.
It’s like 80 percent of a cool thing. It’s like, OK, that’s that is really cool, but that’s not actually a success for us building production software. Like it needs to be developed the right way. Good incremental development. Land with test cases. Like you have to actually do the whole thing and landing and integrating and all that kind of work is also super, super important. And without it, you don’t ship or if you ship, you have a product that you can’t support. And that. Yeah. Those are some dangers.
Yeah. Do you think before you mentioned that that we probably need to figure out better models for for open source projects, for example, for PRs and for development process of that kind of software. Do you think we need to figure out something similar in a way that changes how we we’ve been thinking about the software development lifecycle and all diverse steps? Or it’s just that AI is accelerating every single of these steps, but the pipeline stays the same anyway. I think that it’s the same.
The same systems are useful. There could well be big breakthroughs in that. I’m not exactly sure what they are. I can’t see anything obvious. But what I see is that AI is an amplifier. that AI is an amplifier. It allows you to move faster. And anytime you take take a task and you compress it, it puts pressure on all the others. So if you have really slow CI, well, guess what? That’s going to be a big problem. So you need fast CI if you don’t have test cases, that’s going to be a big problem.
And so but but fast CI, test cases like these are all actually best practice anyways. And so again, what AI is doing is forcing forcing a conversation. And maybe it’s gigantic amounts of tech debt. The teams have accumulated that have never been a problem. But now those tech debt that tech debt ends up really holding them back. Well, maybe it’s time to invest. And maybe that’s that’s what it forces. And I think that’s actually probably pretty good for the health of various projects.
So basically, we’re saying that best practices today have even a higher return on investment than they ever had because of AI. Yep. Well, AI is an amplifier. And so if you if you have a mess, you’ll get bigger mess fast. Yes, I don’t think that’s what anyone wants to do. But I think it waits for what many teams are doing, actually. Well, and also, I think that many management teams or non-technical CEOs or things like this, they’re really encouraging that because they don’t
understand what’s really happening. And this is something where it’s actually very challenging because I talked to lots of lots of really smart people and many of them are seeing this firsthand because they’re the engineering mindset and yet they’re getting pressured like you need to go faster. You need to do this. You need to do whatever. And the people that are putting that pressure, they’re well meaning, of course, right? They want people to be productive and they want their products to move fast,
but they don’t fundamentally understand the thing that’s holding them back. And if you don’t make those investments and you’re just saying go faster and I’m going to measure you based on the lines of code you create, right, you’re going to get you’re going to get a bad outcome. Yeah, I think it’s complicated because as you say, of course, there are very bad CEO stories and so on, but there are also people that were, as you said, well meaning because they are meeting with
systems from engineers, as we said before. So I think it’s tricky to find the right balance where you want to be intentional and try to drive something top down and support, right? Not just leave it bottom up without being imposing and constructive. Well, on balance, I think that it’s good to encourage people. Like if you push the issue, if you encourage people to adopt new tools, if you do it with a balanced set of expectations, I think you get the right outcome because you don’t want to be left behind.
Like you don’t want to not fix your CI or something, right? But you have to have reasonable expectations. And I think this is where the whole world is calibrating and there’s a lot of a lot of experiments being run. Different people are taking different takes. Maybe my take isn’t aggressive enough. I don’t know. And we’ll see what happens in a year or two. And what matters is, again, do these companies ship more good products? Like does it make their customers happier? Does it actually work?
Do they have service outages? Like there’s like these things that are measurable outcomes. And so it’s a little bit too early to tell in many ways. Yeah. How is it that Modular MQs you have like shared artifacts like Cloud MD files with the team or you have ceremonies to come where people talk about practices to share knowledge, how it is on the ground? Yeah, so Modular is really different than most companies for a bunch of different reasons. So first of all, we do follow best practices, I believe,
in software engineering. We have a mono repo. We have fast CI. That seems like an obvious thing. But for a company working on AI infrastructure, that’s almost novel as far as I’m aware. Like TensorFlow CI used to take days to run, for example, because you’re testing all these exotic GPU things, etc. And they’re all in tests and it’s just a gigantic mess. And so Modular is very different from that perspective. The other major difference is that we have open source and we’re going to open
source a heck of a lot more of our code. And so Modular to higher standards months ago will look very much like an open source project from our software perspective, which is extremely different than most companies. And I think it’s the modern way to run a company, but it’s something that I think is very different. And so as a consequence of that, first of all, yes, we have shared Claude skill files and things like this and we’re publishing them already in our in our community. We’re using them internally. And so
again, this is part of the tooling. This is like get from 10 or 20 years ago, right? You need to figure out how to use the tools and you need to work with them as they evolve. And so, yeah, you need to invest in this kind of stuff. But we have certain advantages because, first of all, the architecture side of building our software development ecosystem and our software development lifecycle practices and things like this were built for scale and built for with the best knowledge of all the open source.
Projects and how to keep high velocity software engineering. But then also as we continue to open code, right, it really pushes us to like invest in building scalable communities. And that’s actually what you want is you want to be able to scale in scale with a number of contributors and number of people working with your technology. And that’s something I think that most software teams have never even considered. Most people are just trying to get the last bug fix out. It’s just a very different
way of looking at things. Yeah, I think it’s one of the benefits of running successful big open source projects. It raises your standards and keeps you more accountable. Right. That’s right. Yeah. Here’s one of the bigger problems I have as a leader at Modular. So it was just very different than I think many companies. On average, most Modular and software engineers are so principled and they like push back so hard on tech debt. They’re like, no, no, no, we have to do it the right way, which is a
core part of our culture. Right. Yeah. We’re building things that the last 10, 15, 20 years. Like you have to build the right way. But but you also have to be balanced and you have to think about this. And so many companies I’ve looked at most people wouldn’t even have a conversation. They’ll just you know, the engineer will just do whatever to get the bug out of their queue and just hack something and nobody cares. And so we have we have so many people that want to build things the right way
and invest in building and being proud of what they’re doing. That sometimes it’s like, OK, well, actually, tech debt can be a good thing. Yeah. Yeah. So funny. Tech debt is only bad if it doesn’t get paid down, but it can be the right way to unblock progress. And it’s a very funny dynamic I haven’t seen before. Do you find this tension yourself? I mean, as a as a principled engineer, who has created a lot of fantastic software now, you’re also a founder, CEO. So you need to take care of time to
market and investors and funding rounds. So is that in your head as well? Well, I want us to be productive, right? But but also our entire approach with Modular has been to win for the long term. And so internally we we talk about winning big in the future is more important than just winning tomorrow. Right. And so yes, the long term thinking is very important for building the kinds of architectures and technology and making the investment that we’re we’re doing. And so for me as a founder and as a CEO,
that means that I look for investors who are aligned with that approach, aligned with that vision that understand the stakes. We’re building a platform for heterogeneous compute. We’re building effectively. It’ll take a little time, but it will feel like a hypervisor for compute. This will give people choice. This allows people to scale across lots of different chips. This allows hardware makers to be able to innovate and get their chips adopted and used by software enterprises. The stakes are extremely high.
Like we’re we’re the only ones even remotely trying to solve this problem. We’re not just a layer on top of PyTorch like roughly everybody else is. We’re making fundamental investments. And by the way, it works, which is one of the things that makes it a very it’s a very big bet. It’s a very big investment. Our team’s almost 200 people at this point. But it’s it’s really aligned with what compute is doing, what the world needs. And it’s very much a compliment to people
that are just buying and installing GPUs and reselling them. And so this is one of the things that makes Modular so fun is that we’re able to make the deep tech, high tech investments. We’re able to make a ton of open source code. We’re able to invest in communities and build new programming languages, all the fun, shiny objects. We have an amazing cloud platform. And so if you’re if you’re into cloud, Modular is an amazing place because we’re not only building, you know, the Kubernetes and SAS wrapper
on open source thing, we’re building that entire stack and vertically integrating the whole thing together. And so being able to work in that kind of environment is really pretty unusual and particularly for startups. And so for me, it’s my generalized approach to solving and cracking large scale problems. This is why I’ve been able to do certain things over the years. But I think it’s very unusual. And a lot of people haven’t seen that before. Yeah, but that’s a fantastic vision and mission.
I think it’s a fantastic closing note for this chat. Thank you so much, Chris, for for coming on the show and for this amazing chat. Yeah, well, thank you for having me, Luca. And if anybody out there is interested in Modular, we have a really cool web page, of course, and we have a lot of open positions. And so please check out our page. We link everything in the show notes. So thank you again. Awesome.