Pybites Podcast

#143 - Mastering Python with Łukasz Langa: The Story Behind Core Development and Black

Julian Sequeira & Bob Belderbos

Join us in this intriguing podcast episode where we delve into the world of Python with Łukasz Langa, a key figure in the Python community.

This episode, packed with technical insights and personal anecdotes, covers a range of topics from Łukasz's groundbreaking work with Python to his musical journey. Enjoy!

Chapters:
00:00 Intro
00:53 Guest intro + win of the week
05:27 Łukasz' new podcast core.py
06:22 Black - what inspired you + how does it work
12:30 Is its success thanks to a lack of config?
17:00 About code line length
18:07 Challenge of getting default options right
22:00 Comparing to JS Prettier
23:44 Using project.toml and the day Black broke pip
24:30 PDM ad segment
25:36 How to say No when people request features
27:50 Using AST to double-check accuracy
29:20 Indenting docstrings
31:19 Challenges maintaining Black and noticeable open source contributions
40:00 Some features you're particularly proud of
41:10 Maintaining a single file for a long time and emoji (unicode) support
45:20 Your music background + influence on your development work
48:04 What are you reading: Hypermedia Systems + The Guitar Circle
52:00 Where to reach out to Łukasz + final shoutout
54:48 Outro music

Links:
- Łukasz website
- Connect with Łukasz on X
- core.py - Pablo Galindo and Łukasz Langa's new Python podcast

Books:
- Hypermedia Systems
- The Guitar Circle

Ad segment:
- The PDM program

---
Connect with us on LinkedIn: Julian | Bob

And to get our weekly developer / mindset emails, sign up here.

There was a fun day when actually Pip started reading that file, and it couldn't find a section that it considered was, well, mandatory by Project Toml. So it was refusing to install any packages in your virtual environment. So any black user for a day or two was unable to use Pip. Hello, and welcome to the PY Bytes podcast, where we talk about Python career and mindset. We're your hosts. I'm Julian Sequeira. And I am Bob Eldebos. If you're looking to improve your python, your career, and learn the mindset for success, this is the podcast for you. Let's get started. Hello and welcome back, everybody, to the Pibytes podcast. This is Bob Eldibos. I'm here with Wukash Langa and Will Frei. And we're back with an exciting episode to learn all about Bukai's background, what he's doing in the Python space. So, yeah, let's get started. Welcome. How are you doing? Yeah, glad to be here. I'm doing very well, so, yeah, very happy to have this invitation and talk a bit about what we're doing at the moment. It's an exciting time for Python. Indeed, indeed. So, to kick it off, I think most people are familiar with your name and what you do, but just in case, maybe you want to introduce you a bit to the audience. And we always start with a win of the week, if you have one. Could also be a previous week, but something cool that you achieved. Cool. So, I'm Luka Shlanga. I started contributing to Python in 2010, and then I contributed for eight years. Nobody knew who I was, even though I had a few peps under my name. And then I, out of frustration, wrote small order format that I wanted for Facebook, where I worked at the time, to use, and that suddenly started being extremely popular. So suddenly everybody knew me as the guy that created black. And around that same time, I got this invitation to be a release manager of Python. So I started doing that and being kind of, you know, public on Twitter about how the risk process works and so on and so on. So now, yeah, like, I guess there might be more people knowing my name, but, like, yeah, until pretty recently, that wasn't the case. Like, back in 2018, I remember talking to Hinek Stavak about, like, hey, how do you get 1000 followers on Twitter? Like, a thousand followers. That would be something, you know, that means you're notable. Yeah. So it wasn't that long ago. These days, I'm working as the C Python developer residence for the Python Software foundation. We just managed to hire another one. So there's going to be two of us starting January, which is very exciting. It transforms my kind of loan gig into a team, so it's going to be pretty cool. That's awesome. Is that also your win? Oh, yeah. So, like, that win, like, would be like, you know, a month ago or so, like, you know, it took a while to get right. Like, the win of this week would probably be actually getting the coverage that I talked about, like in my own podcast about, you know, like, okay, like a little context. It is very hard to cover, you know, well, to gather test coverage for the sound library because coverage libraries themselves use the standard libraries, so they import the things that you want to measure coverage off, which means the coverage is not going to be correct. The numbers are going to be skewed. There used to be a hack that Brett cannon devised a long time ago that allowed us to use coverage Py for this purpose, but that hack stopped working. Instead of coming up with another one for 313, I'm like, well, let's just make a proper way of hooking into the interpreter very early on so we can start gathering coverage for stuff that includes the standard library and that thing. I actually did the hook I did at the sprint, but this week I actually made the coverage part of this work, which was a little of an unknown because what we are also doing with the tests now with the standard library is to encourage everybody to use worker processes to gather coverage. First of all, because most of people even using containers have more than one core available, meaning it's going to be way faster to run tests if you run them with multiple workers. But more importantly, the tests are just more predictable and more stable if there are worker processes dedicated for running tests. And those processes are separate from the main test runner process that gathers all the results. We never had support for test coverage in that scenario where you had worker processes do their own thing, and then we didn't have anything that would summarize all those partial results and gather them in a way where we can present them. So now we do. So, yeah, that would be the win of the week, I guess, for me. Nice win. And I think you speak about it on your podcast, right? Is it cordo type? Yeah. Yes, exactly right. So new podcast is one episode. I hope there's going to be more. Yeah. Like, I'm going to. I'm going to claim a win when we get, like, ten episodes, like, so far. Like, Pablo Galindo, the release manager of Python 310 and 311, had this idea, like, well, we should talk more about what the internals look like and whatnot. So he invited me to co host with him, and I was very excited because I was thinking it would be nice to have something like this, but I didn't really have an idea who I would team up with. So when he reached out, I was like, well, perfect. That is flattering. And also, I was actually excited to do this. So we used the coresprint that happened in Brignaud at the beginning of the month to just kick start it. And so far, so good. We'll see. Cool. Bill, your turn. Sure. So, for those unfamiliar black's, as you said, a very well known auto formatter for Python. Now, I'd say it's really become kind of the de facto standard. So you mentioned before that you've started writing it out of frustration. So I'd be curious to know, like, what inspired you. Like, there were specific things that frustrated you and then how bit. How about how it operates to get the job done? Well, it is not something you should be doing in 2018, let alone 2023. Charlie, to start a new other formatter. But, you know, kind of people need all the formatting, like, for kind of consistency when they're working in large organizations. Right? And open source is a kind of weird organization at the same time. You don't really think about this, but when you have maintainers and you have a ton of drive by contributors who only want to fix their own thing, some of the processes that are very valued at large companies are even more important. The attempt at Facebook to kind of remove the bike shedding from code reviews was done, like, way before black. We tried to adopt Olapep eight, and then we turned. We understood that this kind of partial approach of only fixing what is raising lint errors isn't really particularly good because it produces files that have super inconsistent style, like when you're reading the file afterwards and you read code more. The new ride. So we actually saw this as a flaw where the file, in time just got less and less consistent. So perfection less important than consistency. So we tried to adopt the app, which was already unestablished Python formatter at the time under the Google banner. So it had some form of validation through that as well. I even contributed to this, since some of the code style that Facebook wanted was different from what Yapp allowed us to do. So I had some changes to it that were approved. They were part of Yap. But Yap is using dynamic programming and this approach, where it will calculate a number, which this number is your dissatisfaction with how a line looks like. And you can influence how that number is calculated by just saying, oh, if I have this closing parenthesis hanging on there like that is worth 50 points of dissatisfaction, and so on and so on. So you have all sorts of parameters. It calculates the dissatisfaction with the line, and it tries to reformat it so that the number is as slow as possible using dynamic programming that actually finishes in finite time. So it's a brilliant algorithm, pretty impressive. It works, and it actually produces code that looks sensible. Other problem is, obviously you have this large configuration file where you can configure all those toggles, and those are very kind of granular, so you can influence a lot of the formatting. And what we discovered very quickly was that the bike shedding moved from the reviews to the configuration file. And the reason why was that there were always some edge cases that were very suboptimal. So when we changed the toggles just a bit to fix some suboptimal case, very often that would work. It would fix that particular case, but it would break other formattings that were perfectly fine until now. Suddenly they stopped being produced in a way that people liked. So after some point, we just, you know, claimed, well, defeat. We just said, well, there's no way we can roll this out across the entire company. So the rollout was partial and slow and painful and whatnot. So at some point I said, like, you know, I don't really need this to be beautiful. Like, most of python code at Facebook at the time was actually configuration. Like, there was an entire separate repository that stored configuration not in JSON, not in some other format, but in Python. And that Python was kind of declarative, and that was later compiled to JSON files and everything, including your fridge, reads JSon. So everybody's happy with that, but you don't want to actually write JSon as a human. So Python was used for that, and it was like over 20 million lines of code on that alone. So I said, we just need something that is very reproducible, like very kind of JSON. Like, honestly, because those are just nested data structures. We just want them to just look sensible. So how hard can it be? Like, obviously it is very hard. I had, like, first kind of, you know, well, test alpha version that I wanted to release for my birthday. But obviously, like, every deadline, like, you're gonna miss, like in software. So I missed it by a week, but it was PI day, so that was also perfect. I released it and I just, you know, thought maybe a few people just say, like, you know, okay, nice. And that's gonna be it. But by next day, that was 500 stars on GitHub. Twitter was talking about it. Kenneth Wright's at the time was like, oh, yeah, the new request version is going to be reformatted with that. And I was like, day one of this alpha thing that was quite different from what, you know, currently as black. So for whatever reason, it struck a nerve. Like, the readme was really about, like, hey, this is not going to be configurable. There's just one style. Like, you know, kind of just run it and forget it. It's going to be consistent. It's going to do a bunch of things that are different from other formatters, but at least you can explain how the tool arrived at those decisions. And through that, you can just forget about the bike shedding. Amazing. So you think that, yeah, that lack of confirmation contributed to its initial success, that people just had a tool they hadn't, they were not really having to worry about too much config. It was just run it and it worked. Is that, you think, part of success or something else as well? Yeah, well, I mean, like, there were, I think, a few things that contributed to the early success. Like the lack of configurability was one of those because it was really like, you know, kind of. The tool is called black. The cold style is called black. There's, like, very little that you can do about, you know, changing how it configures, how it formats things. Like, the one thing that I had there from day one, because I knew it would be lost battle to try to figure any number would be the line length. What is the line length to which black is going to configure things? And I just arrived at 80 plus 10%. And the way I arrived at it was back before auto formatting on Facebook, we had this rule that we're going to just format to 80 characters. We would obviously had a linter rule that would say, oh, you're at 81, so sorry, you have to go back to your div and you have to change your patch because, you know, it's 81 characters per line. Like, terrible crime. You should rethink your career at the company. 81 characters is obviously, obviously not allowed. Like, you know, what were you thinking? So, you know, this was annoying a lot of people, me included, because very often, like, this really didn't matter. Right. You know, can, especially in configuration whatnot. Like, obviously we didn't want to have lines that are 150, but if it's 80 or 81, especially that, you know, kind of. Raymond Hettinger had this talk about, like, beyond pep eight, where he says, like, well, honestly, with Python, where a lot of code ends up with classes and inside methods, and then you have an if and then you have a try accept. So you lost so many levels of indentation that you don't really have any space to write any code anymore. So 80 or really 79, as it says, like, is really, like, a little too narrow. Just. Just 90 ish would already be much better. So what I thought about was how you're driving on a highway, right? And then the speed limit is 65, but, like, you're driving 72 next to a police car. Like, and everybody's happy. They're not going to stop you. They're not going to just waste anybody's time. Like, you know, just because you're doing, like, a few miles over the limit. Like, yeah, try 88, and then people are going to be like, no, no, no, that is too late. But, you know, like, if you're just going just a few miles above the speed limit, that is considered reasonable. So I thought, well, so let's do a lint rule where it's going to still tell you, okay, the limit is 80 and you went over, but only if you go over 10%, right? So if. If you went, like, 83, it won't trigger, right? It won't tell you, like, oh, like, you terrible percent. Like, it's. It's actually 80 because it's a waste of time. So only after you actually made a more visible, well, kind of, you extended the line too, visibly too long, then it will tell you that the actual rule is 80, and it's been all along, like, go back to the drawing board. So once we started auto formatting, I was like, well, so what is the number I should be order formatting to? Obviously, ad, like, we already know, is a little too narrow, and it's going to just produce, like, those annoying shorter lines that have a lot of nesting in them. So I actually did a run over the entire codebase of Facebook with, like, 808-182-8384. I tried all of this to see how many lines will this produce? And by, like, 79 and 80, like, the number was like, x. And obviously you see, like, the lines shrink. The number of lines shrink as you allow for longer lines. But what I noticed is, like, around, like, 89 ish, right? Like, it was like, well, at that point, the difference is still, like, you know, shrinking, but it's no longer such a cliff. It's essentially, you know, it doesn't matter, because if your data structure is very nested, you would have to have 200 for this line to be just one line. So obviously it's going to still be, you know, split into multiple lines. But it actually turned out that Raymond's intuition that comes from decades of experience was correct, that 90 ish is actually fine. So I just used what people already were using at the company, which is this 80 plus 10% as the cut off line that we're going to be by default formatting to. Also, I thought this is such a ridiculous number that if people actually want to change it, they will, because obviously the default is crazy, so people are going to be changing it. But actually most people didn't, including established projects like Django, which was formatting to 120 before when they adopted black, they were like, meh, let's just use the default. And they switched to 88. And, you know, that was, that was very surprising for me to see that a lot of people really just value the fact that, okay, the tool tells you what to do. Now we don't have to argue about it, and let's just let it do its thing. Cool. Yeah. So we had a question about primary technical hurdles, but from the sounds of it, obviously making an auto formatter is a complex job, but it seems that definitely one of the challenges was to really get the defaults or the default config options figured out then. Right, right. So, like, to be clear, you know, kind of, sometimes people think like, oh, like, the style of black is like the favorite style that I personally have as the original author. Like, so that's not true. Right? Like, you know, a lot of the cases where, you know, the formatting does something or something else, like just stem from what we've seen, like at a big corporation, you know, like, what are those things that otherwise cause people to make mistakes? So for example, like the sad face, right? So like the closing parenthesis with the column that, you know, kind of delimits the signature from the body of the function that was made so that you don't waste any additional indentation levels, but also for it to still be visible that, okay, there is separation between the block. And if you're using type annotations, the return type annotation will make that closing parenthesis and colon look way less weird because there's going to be a type annotation right there with the arrow and everything. But some of those things do look unfamiliar when you're still starting out. And last I've heard, Guido still doesn't like the sad face. He still thinks that should probably just be explicitly disallowed in pep eight, which it isn't at the time when I created this. So as far as I could tell, and with my reading of Pep eight, we were perfectly compatible with the wording of that Pep. Obviously some of those things are really subjective. You have things that you like, you have things that you dislike. Some of the defaults, like standardizing on double quotes, are still pretty controversial. There's a lot of heated arguments about that versus the other. And the fun thing about this is my original intuition was to use single quotes because it just seemed like those are a little easier to type on an us english keyboard layout. So yeah, why not? And then we had an open issue on the black tracker and there it turned out that there were people with sensible arguments for actually keeping double quotes as the default, since it creates more consistency when you're using apostrophes inside the string, which is way more common, it turns out, than using double quotes inside a string and so on. So that can still be dug out. Like, you can still see the issue on the tracker. I was convinced to flip the default because it just turned out that, okay, I have good arguments now to use the double quotes versus single quotes, whereas the only thing that I can say about the single quotes, and this is something that people still sometimes point at me, is like, that's what the default wrapper is using. So if Python is choosing single quotes, so should we. And I'm like, well, it's just some default that was actually chosen by a person who implemented this 30 years back, like probably Guido, and it stayed away. It's not like Python, the interpreter is conscious, so it chooses this because it likes this better or this particular sequence of bytes is better than another one. So yeah, that wasn't really a strong argument for me. But sure, some of the defaults and some of the formatting had to be chosen how we should do those things. I was fortunate enough to work on the same floor as the maintainer of the prettier or the formatter for JavaScript. So we had quite a few conversations about how he allows things, how he disallows things like things that he actually broke and allowed people to configure. And now he was unhappy about it because that essentially revived the bike, shedding about some particular thing that wasn't configurable originally in prettier and in fact originally black didn't even support any configuration in files. You had those two or three command line options and then if you wanted to have configuration you had to use toxini or whatever else that you had in your CI or pre commit to put something non standard there. That was because there wasn't really a good common place to put configuration in at the time. People were already set up. CFG is outdated. We shouldn't be using this. Those are any files, those are ugly and those should be deprecated. And then I talked with Ian Cordasco, who was at the time the maintainer of Flake eight, and he told me, well, we agreed to support a few configuration files and that was a mistake because now the precedence rules are unclear and they keep getting error reports about weird behavior of flake eight. But that's just a consequence of the precedence rules for the configuration file being really complex. So don't do this mistake Wokash don't have any configuration, and if you do, choose one file. So I chose pyproject toml when that arrived and I got told that, you know, don't worry, this is going to be the default configuration file for everything soon enough. Like now, five years later, maybe it's starting to be true. It certainly wasn't at the time. And there was a fun day when actually Pip started reading that file and it couldn't find a section that it considered was well mandatory by project toml. So it was refusing to install any packages in your virtual environment. So any black user for a day or two was unable to use PIP and people were really upset. Black overstepped, but I was literally invited to use the configuration file when it was first introduced and I feel like did help its adoption to some extent as well. Ever bought a course and saw zero results? I've definitely been there. There are very great courses and materials out there, but if you don't implement, nothing will change. But with Pirates developer mindset coaching, it's a whole new game. In just twelve weeks, you transform from a Python intermediate to a pro. How? Through one on one coaching, building real applications, and mastering advanced python techniques. Our hands on approach isn't just about learning, it's about doing and achieving. Get certified, push your career and join empowering communities. Apply for the Pibytes PDM program now. Check out the link in the description below. That's awesome. Yeah, it seems like you can't get away from bike shedding because the bike shedding went to the discussion of quotes. So no matter what, there will be bike shedding and I will never be able to unsee the sad face now. But I'm curious. So obviously you wanted to have minimal configurations but, and I know I've seen issues opened on the black repository where people want a certain thing, or they think it should be formatted a different way. How have you handled essentially saying no to people as ideas have been thrown in? Because that's a hard thing to do for a lot of folks, right? Yes. Well, initially, the scope of the project was narrow enough that I was rather comfortable saying no to feature requests because simply they didn't play well with other things that we needed to support. So, for example, we talked about the reasons why Black was initially adopted by users who already had access to other auto formatters. Why was it suddenly popular? Like, one reason was that I very initially just decided, like, a thing that we need to have is for black to run twice every time on every file that you have. And the reason why is that if it changes its mind to format a file another time, a different way, that means there's a bug somewhere there. And in this case, as the Zen of Python tells us, we should refuse the temptation to guess. We should just, you know, kind of just throw an error and saying, like, you're trying to use a tool that sucks. Like, this file is showing you an error. Please report this error. So we had this kind of check for consistency in formatting that it actually arrives at a stable formatting and it doesn't change its mind, which Yap did. Yap did change its mind. You could format a file, it would be happy with it, and they would format it again and reformatted to something else. So obviously, like, you know, kind of, there were some stability problems there, but worse yet, you could arrive at a formatting that changed the meaning of the program. And that is terrible. That is not what you want to have. Like, you. You care about some comments there and some white space there, but you want the code to mean the same thing. So from day one, black also had this ASD check where before and after were actually the ASD trees were built, and we were checking that they are exactly the same. So as a tool, you could just run it on a million lines of code. And now, okay, this is still going to behave the same. You don't have to actually look at this gigantic diff. Like, you can just assume there's not going to be any problems. And this is how it operates to this day. We planned to maybe flipped the default at some point, but then we had my PI c, which made black, like, over twice as fast compared to just pure Python. So we just kept it. And, you know, people do value this sort of safety of the tool where they know, like, it's not going to screw up your code. So, like, very many changes that people wanted for black to make, which it doesn't, were changing this contract. If you changed, for example, how the insides of a string looks like that actually changes the est. So now we'd have to have some sort of special handling for this. For everything else, we're keeping things really simple to allow for this check to continue. But there was one, one place where we finally kind of decided, okay, that we need to change because it's just too suboptimal. And that was, you know, indentation of doc string contents. So for doc strings alone, if you are first adopting black, and you are adopting it, coming from, say, the Google code style that uses two space indents, what you're going to see, like, or what you were seeing in the past, was that it would nicely reformat everything and leave your doc strings unaligned. Now ugly. They would be visually aligned to indentation levels that black is not using, which is, you know, two characters versus four. So now you'll have to go and by hand, reformat every docstring to actually fit the thing. So what we did instead is like, okay, we can actually do only the left indentation in docstrings so that it matches what we expect. And then in the ast check, we would have this transformation done on the left and on the right side. So the ast check is still effective. But for docstrings alone, which we judged, it's a special case that is kind of pragmatic to break the rule there. We changed the default because there were very many kind of bug reports about this, like, hey, we're trying to adopt black, but it's not doing this re indentation in a place that is very painful to do manually now, because there's very many doc strings in our code base. So it's always a balance. You cannot easily give in to feature request, because now, not only it's the maintenance burden for everybody else, but more importantly, you are now betraying everybody else who trusted you with the defaults. You're now second guessing the default and so on and so on. So you cannot easily give in to this. But also, it's a balance. You cannot be super stubborn on things that are obviously needed and obviously are still controversial. That sounds challenging indeed. And that also leads to the next question. So now, with black having cemented its place in the python ecosystem, widely used, a lot of eyeballs on it and usage apart from the balance you just mentioned, what are some other challenges you have had maintaining this project, including maybe moments of burnout as that happened? Because I guess a lot of that maintenance fell on you. And then moving forward, have you been able to. I mean, are you still actively maintaining it, or have you delegated parts of it? And if so, how has that worked? Right. So initially in 2018, I was pushing it very hard, obviously not in terms of marketing or whatever, but simply the initial release got way more popular than I expected. There were a lot of expectations from people who are close to me. You know, kind of people wanted the auto formatter to succeed, but they had real problems with it, so they reported issues, and I wanted to solve them. So there were very many releases. I pushed them out, like, you know, sometimes three times per month. You know, those were alphas, so we could just change a lot of things. But by 2019, I quit Facebook. So I was here in Poland, you know, kind of not having any job at the time. You know, I had to kind of decompress after working for a big corporation for over five years. And I kind of started, you know, some of the deeper changes in black that were needed to solve some more fundamental limitations. And we arrived at a release made in October. There were already very, like, way fewer releases there since I had to be more careful with adoption being wider. Like, you kind of just willy nilly, like, change things now because people would be complaining every time they upgrade. So we arrived at a release in October that turned out to be very successful. Like, people are very happy with the 1910 release. And, you know, and then I had, like, this kind of huge block where I wanted the tool to be marked as stable. But what that would mean is we would no longer make, like, massive changes to its formatting style since, like, not only the tool, but the code style is called black. Like, it's supposed to essentially be predictable. That was the entire point. So if there are any things that we want to change, we need to change them now. And there were a few things like, you know, kind of that were listed as issues that we were like, oh, yeah, we need to handle this before we mark this as a stable tool. So I kind of had a few unsuccessful attempts on dealing with those since, you know, kind of, that's one of those cases where you're fixing something for somebody and somebody else will be unhappy, right. So, you know, kind of, there were no longer any low hanging fruits there. And at some point, I just got kind of quite paralyzed at this. Like, you know, what is. What does it mean for us to release a stable version. Like, I'm not really able to reach the point that I wanted for the stable version, but at the same time, there's a lot of pushback for this to be still beta since the versioning I used, like, used the b instead of a final dot in the version, like, just to mark that this is a pre release product, and that was breaking some tools, like some kind of virtual management tools. I think PiPNF was a law fender, where you actually had to use some additional command line option for it to even install black. So people were upset about the b and the name again, we even had this internal joke where b stands for black in the version number. So it just has to be there forever. But at some point, I was like, we need the stable version, but I'm unable to really just provide it. And then COVID hit. So obviously, it was a big distraction for everybody for a year, and nobody was really kind of pushing hard on us to finish the tool. We did, I think only one release over 2020, and that fixed a bunch of issues that people had. So a lot of people migrated, but some still stayed on the 2019 version. And to this day, there is a holdout going group that is just like, yeah, that's the vintage version. We were gonna use this for as long as we can. Good luck. Because it doesn't support the match statement. So, you know, might not really have, like, you know, a long shelf life left in it, but, you know, kind of the 2020 version was, I think, was August that we released it. But, like, at that point, it wasn't so much as I like that I was burnt out from, like, maintenance kind of as a, you know, I think, influx of issues or whatever. It's simply, I couldn't overcome my own kind of expectation of what the stable version means versus what I could deliver there. And what I did right, though, was very early on when I saw a few people, you know, just being active on the tracker and whatnot. And they were helpful, and I trusted them. I just gave them the commit bit. They could just make changes to. To the project. And by 2020, we actually had, like, a really nice group of maintainers, and to this day, like, they're actually doing the day to day maintenance of the project. Like, I'm very grateful to have them. There's Yele, there's Cooper. Like, you know, there's Schultz. Like, awesome people. I'm very happy that, you know, kind of, they joined the project, and by early 2021, like, they had the Zoom call with me saying, like, hey, yeah, we have to just, you know, kind of drop the bullshit, like drop the dreams of a perfect, stable release and just, you know, kind of do what we can to fix the rest of the problems that we have, but actually release something that we're going to mark as stable. So, you know, kind of, we managed to reach that point. I was very happy with this because, you know, kind of some, some of those things that we needed to arrive at was like, there's going to be always changes in formatting. That's what changing the tool means. Even if you're bug fixing. What are you bug fixing? You're bug fixing some defect in formatting. So there's always going to be some changes. So how do we protect people from like just having changes every release? And we kind of arrived at this idea of additions where every year we would have some maybe larger changes that were anticipated by the community that people wanted to have through bug reports. But within a year you're only going to have minimal changes there introduced with bug fixes and whatnot. If there's a new feature introduced, is going to be introduced a year from now, you can test it today by just running black with Dash dash preview so that you're going to see the code style from next year because it needs to kind of, you know, evolve. It doesn't evolve very rapidly. So don't get me wrong, it's not going to be a tool that formats things like more or less the same, but there might be changes, you know, and that unblocked my kind of mental thinking about like what it means for us to be stable but still be able to maintain the tool. So, yeah, like, so the team I kind of unconsciously gathered helped me a lot with that. And to this day I mostly appear to anti gordon knots and stuff like that. So most of the actual work is made by the team and some kind of contributors who come and go for just a smaller number of changes, which I'm also very happy about. Batuhan, one of the core devs of Python, contributed this unholy hack that allowed us to use our ll one parser to still successfully format files with the match statement, which is a piece of grammar that famously cannot be parsed with ll one grammars. And yet we do this. There's an approach there that unless you're ready for some, well, let's call it a typical uses of parsers, you might not want to look inside the code that does this, but it's been very successful. It lets people to actually format the code just fine. So yeah, some of those contributions by external people are very valuable to us. That's great. It sounds like the 2019 version might become the Python 2.7 of Black if people just keep holding onto it until the better end. Yeah. Do you have a feature real quick that you're just most proud of? Like one thing that really stands out for you in black, and even if it's something that isn't user facing, just something that you were just like, yeah, this is awesome. Well, so this is no longer true in 2023, but originally when I created black, one thing that wasn't user facing, but was very important to me was a solution to a question I was getting very often, which was like, can you just show me some small program that has everything done right? So I'm going to. It has proper tests, it is properly documented, it has functions split in sensible ways. It uses generators, it uses async IO, it uses a bunch of other modern things in python type annotations, and so on and so on. Every time somebody asked me this question, I was like, I don't really have a great example for you. Once I started black, I'm like, people are going to look at it anyway. I might just as well make it an example. Project the initial versions. Like, I really kind of spent additional time to make sure that it was just one file. That was the kind of point. And for a long while, I stuck to this gun. The other maintainers were like, come on, we really have to split it. It's too big now. But it was initially very important to me for this to be simple enough that somebody could open it in the editor and just follow what is going on. And see, this is how we are splitting functions so that there are generators and now we can consume them here and so on and so on. And this is how you type annotate generators, which was also like a popular question back in 2018, and so on and so on. So, you know, kind of one weird thing that black is doing was that it is using asynchro for multi processing. So, you know, to just allow you to format an entire folder of files, like, you know, like to the extent of your allowed, you know, course on your cpu locally and, you know, kind of. I still consider the subprocess API of async IO to be kind of the most elegant that we have in python, since it allows you to actually just run more than one at the same time, like in a. In a very visible and predictable way. So I like that. Yeah, like another thing that is a little user facing and I was proud because it was very selfish. It was extremely selfish. Was that black from day one was using a bunch of cute emojis, like, okay, it would just break your heart when it kind of couldn't reformat something because, know, kind of was broken. So like report an error or it would just like, you know, like show you sparkles because they reformatted the file successfully. So the reason I did that was because the first character of my first name is also a non standard unicode character. So if your terminal cannot, you know, show you an emoji, it probably cannot show you my name as well. So like, that was, that was a way for me to kind of, you know, kind of maybe force people a little to consider like Unicode support in their terminals. And we got some very motivated people to report to us. They're like, no, you should drop the emojis. First of all, they're not serious. This is a serious tool. And second of all, this breaks on my old windows terminal or whatever. And yeah, I pushed back on this because I really needed this to be part of the tool. I really needed people to move to this world where Unicode is expected and it's widely supported. And another example of exactly that is that inside the source code of black, and that's still the case, there is an empty set that I used for comparisons, and I just used a mathematical empty set symbol to name the variable. And that was a deliberate choice. I needed to have a Unicode identifier inside the source code because first of all, black from day one was using itself to test some things. That was the simplest thing I could do. Does it reformat its own source code successfully? And so on and so on. It was important there, but also I wanted to show people, look, the antenna virus might be a little more complicated than you think. In fact, very early on, when Charlie was doing the rough formatter, which redoes what black is doing only 30 times faster, I'm hearing like he discovered this because like that was like a, you know, thing that actually didn't initially work for him. So, you know, kind of, haha, good job. Like this actually found a deficiency that is obviously now handled. Everything is good. But yeah, like some of those things like, you know, left that don't seem deliberate. They sometimes are. That's funny. And thank you for doing what you did as far as making like black a really great example, because I personally, I didn't know you intended this, but I used it as a great reference to seeing how to break things up. And just like, as just reading great code, because you can learn so much from that. So thank you for doing that. I'm happy it worked out for you, for sure. So changing gears a little bit. There's the guitar in the background there. For anyone who's watching the video, and you are a pianist, has your musical training and musical background influenced your journey through programming or complemented it, or how has that interaction worked for you? Well, I believe that everybody should have. Should stand on two legs, so you should have something else outside of your one fascination with programming or python in particular and whatnot, since there needs to be some balance. Often, for me, the moments where I'm away from, like, my particular task are the moments where I would arrive at a solution to something, but also, like, you know, kind of, truth be told, like, 80% of programming is just frustration. Right? You know, like. And sometimes, like, that 80% is not evenly spread, so you might have, like, weeks on end. That is just frustration. And obviously the remaining. Why do we even do this? Right? Yeah. So, like, the 20% is what actually convinces us to keep doing it because it's extremely rewarding. Right. You know, kind of it. This kind of something that you feel like you have control over, you have positive impact on the world, which, like, isn't an easy thing to do these days, it turns out. So, you know, kind of, yeah, obviously we still do this, but, you know, kind of when you're in this ditch of this 80% that is just unrelenting, it is nice to have somewhere to just, you know, step away and just do something else for a while. So, you know, kind of. There are very many, uh, python people that I know who have some sort of hobby like this that they are serious about, like, whether that's fitness or, you know, woodworking or, you know, having a farm or whatever else. For me, that thing is music. Um, you know, kind of. I do produce some music. You know, it's not, you know, something that I want to do a career out of, but, you know, kind of. It's something that is, like, constantly on my mind to the extent that, like, I, in fact, have quite a extensive library of musical things that bit I write in python to help me with that kind of, you know, relaxing task, which sometimes finds things that I should fix or change in python as well. So those things kind of, you know, loop back to each other. But, you know, kind of, in the end, I do feel it's mostly there for me to just have something, somewhere to go, and I'm like, you know, kind of overwhelmed with the other part of my life. Yeah, it sounds great for balance. I have the same with fitness. If I don't do the gym, then it becomes much harder. Exactly right? Yeah, drums. Been learning drums the past two years, so nice. Hitting things with sticks is always a great outlet. Very cool. Like, yeah, my son is also playing drums. So yeah, fantastic. Family band. Very cool. Awesome. Well, thanks for sharing all this insight. There are some really nice details, even easter eggs of the source code. Last question we have is about reading. What are you currently reading? What book do you want to share? Right, so I actually am currently reading two books, and it's funny because one of them is music, one of them is programming. So like, the programming one is hyper media systems. So it's from the authors of HTMX, if I'm pronouncing it correctly, because it turns out, like, maybe it's another case of GiF. GIF and it's actually pronounced something else, but this very small library that you put in your HTML. So now any HTML element can issue HTTP requests and they don't have to be just get in post, which is a way for traditional apps, like a traditional Django app, to appear as if it was a single page application. You can do a lot of the dynamic things that otherwise you would use react for, but in a way that is more hyper media first and more natural to an old timer like me who started with Django. And now it turns out you can make applications that are usably indistinguishable from an spa, but you write them in a more familiar style where you're generating HTML on the server side and transferring that instead of JSON. So, yeah, so that's on the programming side. It's a very interesting book. I'm kind of like two thirds through that now, I think. And the second one is by Robert Fripp, the leader of King Crimson. Like a progressive rock band from the seventies, they're kind of on and off across the year, so I'm not sure if it's still going or not at the moment. It's hard to tell honesty. And he wrote this thick book is called the guitar circle, and it's about guitar practice, but really it just feels to me like, you know, I'm like third into the book that it's more a philosophical book about life and discipline. So, so that's interesting. And, you know, kind of, I'm reading the two kind of interchangeably since like hyper media systems is more like, you know, kind of straightforward programming where you have code examples in code and whatnot. And the guitar circle is not even straight about like, oh, this is the exercise you should be doing. Like, it's more about like, this is what you should be thinking when you're exercising and so on and so on. And it has like a lot of kind of really, you know, kind of bite sized quotes that you can just, you know, put somewhere like inspirationally or whatever. So, you know, kind of. It is a way different kind of book, but I enjoy both. Cool. Sorry I didn't pick up the title of the first one. Hyper media systems. Okay. Yeah, yeah. It is actually available for free if you don't mind reading from a web browser on just hypermedia systems. Okay, awesome. Yeah, I need to read that because we definitely use HTMX in PDM, in the coaching myself for my web apps, and it's amazing. Right. You don't have to write JavaScript, which usually is a joy for pythonistas. Exactly right. Bill, are you reading anything? Not currently. Just getting prepared for the new baby. So actually, I lied, we're reading a lot of baby books, also important. Yeah, indeed. And I'm actually on the good Inside book still, which is one step further when they're getting to adolescence age. So handling struggles and emotions. Yeah. Cool. Well, ukesh, any final shout out or links where people can connect with you, preferably? We can link that in the show notes. Right. Well, I think I'm relatively easy to find. My email address is kind of public, my twitter handle is public, and I respond there. So. Yeah, like, if you need to reach me for whatever reason, I'm available. Yeah. Like any other shout outs, pep 703 has been just accepted officially by the steering console. So Python 313 will have an option to be built without the global interpreter lock, which is going to be a new future for us. We'll see if that works out in long term. But, you know, the only way to find out whether this is something that we actually want and can handle is to allow experimentation with it, which is exactly what the stream console intends to now do. So now we only need to make it work. But, you know, obviously the hard part is going to be when 313 is out and the users will be able to pick up the experimental build and actually try it out, whether it fixes anything for them, whether they can make things faster and so on and so on. So, yeah, like, you know, kind of if you're interested in not maybe Python performance, but Python scalability, this is, you know, like definitely something that you should or could look into, you know, like you can look forward to it you can help, because there's plenty of things we're going to have to be doing with the interpreter. And if you're like, no, but like, the c code that you're going to be touching, like, that is way too scary. But also, the standard library will have to be adopted to a free threading environment. So there's plenty of opportunities for people to contribute to python these days. In fact, it's a new kind of renaissance of larger changes in the interpreter, where between 3536 and 310, the interpreter was quite stable. There were changes on the library and some more pointed changes, like moving coroutines from being regular generators to their own object, and so on and so on. But since Python 310, 311, 312, now we see way deeper changes in the interpreter. So if you're curious about contribution, it's the kind of best time we've had in a decade to start being really impactful in the project. Wow. Amazing. Cool. Bill, anything else from you? I'm good. Thank you so much for your time. This was great. Yeah. Thanks so much for joining us and sharing, and this has been amazing. So, yeah, it was fun. Thank you. Yeah. Bye. Bye. All right, thank you, Lukash. We hope you enjoyed this episode. To hear more from us, go to Pybite, France. That is Pibit es friends, and receive a free gift just for being a friend of the show. And to join our thriving community of Python programmers, go to Pibytes community. That's Pibit es forward slash community. We hope to see you there and catch you in the next episode.