Pybites Podcast

#108 - Teaching packaging by building a Python package

Julian Sequeira & Bob Belderbos

Welcome back to our podcast. In this week's episode we look at Python packaging.

---
As stated in the intro, it's recommended to watch this episode on YouTube.
---

I was teaching this on our weekly PDM Code Clinic call and we ended up building quite a useful Pybites Open Source tool. 

Introducing pybites-search, a command line tool to search our content (articles, Bite exercises, podcast episodes, youtube videos and tips). 

We look at how to build a package and some of the code + design that went into pybites-search and how open sourcing this is a double win: our PDM bot project can leverage it and people can now contribute to this project. 

Hope you enjoy this episode and comment your thoughts below as well as preferences for more Python / Developer / Mindset content. Thanks for watching. 

Links / resources:

I decided to build something really useful. On one hand, we have a slack bot in PDM that's doing all the searching for us. I basically abstracted all that behavior into its own package now, and I can strip out a whole bunch of code in that tool. And that code is now open source. I can receive contributions, people that want to make it better add functionality. That's all possible now. Hello, and welcome to the Pibytes podcast, where we talk about Python career and mindset. We're your hosts. I'm Julian Sequeira. And I am Bob Valdebos. If you're looking to improve your python, your career, and learn the mindset for success, this is the podcast for you. Let's get started. Welcome back, everybody, to the PY Bytes podcast. This is Bob Aldebos, and today, an episode about packaging and our pibytes search tool that came out of teaching that one heads up, one warning. There will be a bit of code in this episode, and I will walk through it, and I do my best to make it understandable for audio only. But if you want to watch along and see the code, then I highly recommend consuming this episode on our YouTube channel. So I hope to see you there and enjoy the episode. Thanks. Hello, and welcome back to the Pyewites podcast. This week, a quick episode. It's just me, not Julian, but he will be back in the next one or the one after. Anyways, this week I wanted to talk a little bit about packaging and a little story experience I had this week. And yeah, I want to show a package we built and also how it's a good example of scratching your own itch before diving in, though quickly. A win. And at the end, I will do a book as well. The win is that thanks to the packaging learning, this week we have a new package called Pivots Search, and it's a tool to search through our articles, our bytes, our podcast, our YouTube videos, and our tips. And I'm pretty excited about it. You can just pip install it, and then from the command line, you can search our content. And that's actually what I'm going to show you as well in this podcast or talk about. But that's also a win for us as a team. So let's dive straight in. This week I had the task to demo packaging on the PDM code clinic, and it's a massive topic. It's something people struggle with quite a bit. However, I did it mostly from the packaging Python projects webpage, so let me pull that up for the viewers on YouTube. This is the guide. So there's a whole section Python packaging user guide and this is under tutorials packaging Python projects. What you basically have to do is create the right structure. So you make a source directory unique package name in init Py to make the folder a package and then one or more modules. Then you create a Toml file. So before it was a setup Py, but now the Pyproject Tumbl is the way to go and you add a license file and a readme. And starting off with Piproject tumbl you set up a build system so you can use hatchling setup tools flit PDM not to be confused by our coaching PDM program. And I just used the hatchling, the default one that comes here in the documentation. I didn't have to install anything, it just worked. Then in the same pipe project Tumblr you start to add the metadata. Here you give your package a name, version, authors, et cetera, the Python version and URL's to your project. This is a quick overview of what I had to do to make a package basically beyond the code. Then to build the package distribution you can pip install build run Python mbuilt and then in a dist folder it will create your package with the version you set in your Pyproject TaMl. And then to distribute the archive or package you can install twine pip install twine and then you can use Twine to upload it to PyPi and or test PyPi. It's probably good to first try it on a test pipe PI to see if it's all working. And you can do that with repository test Py PI. If it's all working out then you can do the real upload with twine upload and then just dist star which by default it takes the live pyPi. For that you will need to request an API token, make an account on PyPi. What I built, what we started building on the code clinic was pibyte search. You recognize the same package layout source pibyte search with in there and init Py and a bunch of modules. I want to go into the code a little bit as well because I used a kind of interesting design. And also notice this dunder main py and I did a video on YouTube what that means, it's basically the entry point. Enter your app in this case that just runs app from CLI. And in CLi Py there's the typer command line stuff typer by the way, awesome library, almost effortlessly lets you make command line apps here have a bunch of commands and each of those commands will be a switch to the command line program. So I can do search article search, byte search, podcast search tip search video and I will demo the tool in a little bit as well. Now regarding the code, I could totally have done this with just functions and that's usually my first go, but I thought it was fun also for the sake of teaching to use an abstract method or an ABC abstract based class. So here I made a base class called pibytes search with metaclass ABC meta and I defined one method in it with the decorator add abstract method and the method is called matchcontent takes a search string and it returns a list of content pieces, and content pieces are name tuples. So list of objects. The cool thing about this is that now we have a base class that defines the interface. So any class that now inherits from Pibyte's search from the base class needs to implement this abstract method called match content. So I had this design in mind because I thought ok, we're going to build a search engine that's going to search a lot of different channels like YouTube, articles, bytes, et cetera. So I might as well enforce this interface from the start and every subclass needs to follow this pattern. You also see a show matches method here, which is not an abstract method. That was basically to show the results and I had that in the subclass, but then I quickly recognized that it was going to be generic. So I pulled that here into the parent class so that every child can use this method. So you can look at this code. I will link it below, especially if you're listening. This might be harder to follow, but it's under our Pibet's open source and it's the project search. Now the cool thing is then with that base class in place, all the child classes, for example article Py. Article Py has a subclass called article search and it inherits from Pibat search. I can just implement this method match content and every channel will have a different implementation to search the content. So here it's using an endpoint on our code challenges platform. So it does a request get to an articles endpoint and filters that output and builds off a list of matching articles. But if you then look at podcast for example, that's a different implementation because we have feeds bussprout.com RSS, so we're using RSS feeds. The implementation does not use request, but it uses feed parser. This extra code already had elsewhere. So I could just pull this in, but it's doing some more complex stuff to get that working. But it's the same principle. It's a subclass called podcast search inherits from tie byte search. It has to implement the match content method because that's decorated with the abstract method decorator in the parent class. So the only requirement is that I implement this match contents method, but I am free to do however I want. Hence here's a different implementation that fits the podcast search requirement. Yeah, and then it all comes together, as I showed before in the CLI pyramid where we hook up all these classes. So the article search, the byte search, the podcast search, the tip search, the YouTube search that comes all together with typer and that then makes it a command line app. So that's the implementation. Why did I build this? Just a quick break for our sponsor, which is pybytes. Just a quick note about our PDM program. It's growing, it's highly successful. People get amazing results and packaging people shipping their first package to PyPi is a common theme. And what people always do there, and they get their software out there, they build amazing things. It's all stuff their ideas that we help them implement. And it leads to great stuff they can show on their portfolio. And they have it on PYPI so they can get contributions from other people in the open source space. So if you're eager to take the next step in your Python career, check out the PDM program linked below and hit us up to have a chat to see where you are in your career, what is holding you back, and see how as Py bytes, we can best help you. All right, back to the show. Well, twofold, I wanted to have an interesting example when I was teaching, not just a calculator or something that was less relevant. So although it could have been really good to show packaging, it would probably not really have been that engaging. So I decided to build something really useful. On one hand, we have a bot, a slack bot in PDM that's doing all the searching for us. I basically abstracted all that behavior into its own package now, and I can strip out a whole bunch of code in that tool. And that code is now open source so it can receive contributions. People that want to make it better add functionality. That's all possible now. And yeah, that's it. It's PIV installable and everybody can use it. It's not a closed tool anymore because the PDM bot code base and is private. So I could never really do much with that code or not get other people to contribute to it. So whole part is now broken out of that and open source. So that's exciting and contributions are welcome, of course. The other thing is, yeah, scratching your own itch. I was always wanting to do this, making a search engine for content, because we have a lot of content and we have a lot of channels. I think if you add up articles, bytes, tips and all that, we surely have over a thousand pieces and they're now all searchable. It's still a primitive search, but it's something working. And that's the other point, right? Ship something fast and make improvements as you go, as users are going to try it out. Then you will learn a lot of what needs to be improved. But first you need to just ship a basic version. And the other thing I already mentioned is scratching your own it. So this is something the business can benefit from our community, and that will also again make the examples and when you're teaching more interesting. So lastly, I will quickly demo the tool how it works and I encourage you to try it out yourself as well. And then lastly I'll share what I'm reading right now and then we call it a wrap. So I made a new directory and I'm making a new virtual environment. I'm using an alias for that because I do that a lot. So I have pvem that's alias to Python, MVM and then source vamp bin activate. So that will make a virtual environment and it will activate it as well. To get this tool running, you can now just do pip install PI bytes search and that will install it in the virtual environment. It will also make a shell alias because I didn't show that in the ToML file. There's a section in there called project script. It has an alias called Search and it points to pibytes search which is the package CLI which is the module colon app which is the typer object. When you pip install this package, you get this alias for free. If I do which search, you see that there's a search script in my virtual environment. So if I do search help, that's the typer command line interface. So I can give it five commands article by podcast tip video and if I give one of those so search article, I get an error missing argument search because I have to give it a search string. So let's look for Django articles and here we get a nice rich table of two columns where the first column is title and the second column is the URL. So these are all the articles we have that match Django. But I can also look for tips. For example tip unpack and now we get all the PI bytes tips that match unpack. So tuple unpacking, dictionary unpacking and a few more. I can search for podcasts. For example we had Antonio Melayon and that was episode 89. We spoke about fast API on 87 and 80. What else we can look for bytes, maybe decorator bytes and a whole lot of bytes. But again, all these tables are the same, all the results are title and URL. And lastly we can look at their YouTube videos as well. So I can do search and let's do video and then main that contains the video that we uploaded yesterday, which was an explanation of if name equals main. This is pretty powerful and it will be a nice tool to quickly find Pibyte's content. It was an example of scratching our own niche and it was really a success demoing this on the code clinic because it was a really engaging example. If you go to pipe Bi and you search for pibyte search, you can see that the latest version is zero dot zero dot seven and it's about 7 hours ago. I hope this is helpful, that it gave you a bit of insight into packaging. Better to watch it on YouTube. But if you have listened to the audio, I hope that this was helpful as well. I hope it inspires you to rethink where you have a need for automation or something, a process that might be painful that could be enhanced with Python and just go build it. Ship a package, ship soon and just keep iterating, get real people to look at it. And you can always ship a new version, so there's no problem with that. And yeah, also a reminder, if you teach Python, try to use interesting examples. It's more engaging and also live coded that whole ABC on the code clinic and that's how people learn most. Because of course I made some mistakes, things went wrong and it was a really great session. So there are a couple of lessons. Lastly, book, what I'm reading. Oh, I'm still reading fluent Python because that's a very thick book, but it's epic. Every time I pick it up I learn new things. And I'm also reading a bit more on mindset and that's from Gap to gain by Ben Hardy. And that's a great book. In a nutshell, it's kind of that whole analogy with class half anti class, half full. And it's kind of surprising how often we go into gap thinking like this is lost or basically limited thinking. But if you look at your gains, for example, if we think about how much is there still left to do and it's progressing slowly, et cetera, if you look at rather how far you've come and what you've gained, what you've learned, that's way more motivating. And that's also the reason, as Piebite said, we're big on the Winsfell, both ourselves as well as with our clients. And looking at what you've gained and how far you've come is way more positive and way more motivating. So that's my book of the week, and next week we'll share some more reading. So yeah, I hope this was helpful. And yeah, see you in the next episode. Bye. We hope you enjoyed this episode. To hear more from us, go to Pybite, France, that is Pibit es friends, and receive a free gift just for being a friend of the show. And to join our thriving slack community of Python programmers, go to Pibytes community. That's Pibit es forward slash community. We hope to see you there and catch you in the next episode.