Future-proof your skills with Linux, Python, vim & git as I share with you the most timeless and love-worthy tools in tech through my two great projects that work great together.

Concurrent Web Crawling in Python

Today, I'm celebrating President's Day by accomplishing a mission: creating an async Python function to return zipped input/output promises. I'm also working on handling a list of tuples as input and building a package called mlseo on PyPI. To top it off, I've completed a concurrent site crawler in Python that uses SQLite. Join me in celebrating President's Day with this amazing progress!

Celebrating President's Day with Amazing Progress: Concurrent Web Crawling in Python!

By Michael Levin

Monday, February 21, 2022

Happy President’s Day.

Mission accomplished on returning zipped input / output promises on an async Python function. This makes using it for key/value database recording very easy. Next step: handling a list of tuples as input instead of a list of URLs.

We’re going to make a package on PyPI (Python Package Index) where things get pip installed from. The package will be called mlseo, and it will have lots of great things for doing generic broad SEO tasks.

We finished a site crawler. It’s concurrent. It’s Python. It uses SQLite.