This time of the year has come! Pycon 2018 just took place at Cleaveland, Ohio at 9-17 May. It was an amazing conference with a lot of new and revised ideas! Recently I’ve noticed a tweet from Python Software with all the videos from PyCon, thanks to my friend @mpetyx who also wrote an amazing article, Third Week of May in #Python News: @PyCon
[tweet https://twitter.com/ThePSF/status/997254157252808704]
I am going to dive into some of the videos concerning data science and back-end development. So let’s start.
The Big-O, algorithmic complexity
In this video Net Batchelder is describing a term called Big-O. This term determines how scalable is your code. For example, given a list of 10 elements, to find one of them it will take at most 10 iterations. This term is also called complexity and it is fairly used in Data Science too! Every time you write a piece of code you always think of the worst-case scenario, i.e., in a search, you are looking for an element which it might be the last one.
There are several techniques out there that are going to reduce that Big-O in order to make your code faster when you have a lot of data.
For example
# given a list a = [1, 2, 3, 4, 5, 6] # find if number m is in list a for i, x in enumerate(a): if x == m: return i return None
This piece of code will run at worst-case scenario 6 times. Given a list of N items it will run at worst case N times. This means my algorithm has O(N) complexity. Can I do better? Of course, you can. As you can see the list is ordered, that means I can use binary search. Binary search is a recursive algorithm that takes O(logN) time to complete the task. This is done by going to the middle of the list and compare the given number if it is smaller then it goes to the left half of the list and find again the middle. And so on, until it lands between a number that is lower and a number that is bigger.
Another problem might be something like this.
# given a list a = [ ['acb', 1], ['def', 10], ... ] # find that number has 'def' for x, y in a: if x == 'def': reuturn y
This again has O(N) complexity, but if you use dictionaries then you can retrieve ‘def’ value in O(1). That means, for whatever key I look for in the dictionary it will always take the same amount of time. This is a dictionary’s property.
Pipenv, the replacement of pip?
Kenneth Reitz (@kennethreitz), the creator of the well knows requests package is talking about his new creation Pipenv.
Well, this looks AMAZING! Pipenv is using lock files to install packages, auto creates virtual environments and you can keep your development and production dependencies in one place!
I was really startled when I watched this video and I am going to share Pipenv with the rest of my co-workers. It really worth your time to take a look at it.
A few words about it that make it stand out. Pipenv offers a command that can show you a tree-like structure of your packages.
pipenv graph
It also provides a command that checks the vulnerability of the code.
pipenv check
You can also change the version of your python to 3 (or vice versa) by just typing
pipenv --three
Or choose a specific version, for example, 2.7.14 you can type
pipenv --python 2.7.14
Last but not least there is also a check for the lock file. When you are about to deploy your code you can type
pipenv --deploy
and it will check the hash of your lock file and validate that everything is there.
Introduction to Python for Data Science
This video of the workshop was really amazing! I love it! I will not say much because we are going to stay here for a long time, so I am just going to place the key sections of this video and I urge you to go and watch it all (already watched it twice)!
This workshop is being hosted by @jseabold and he goes through the following topics.
- An amazing introduction to data manipulation with Pandas
- You learn to visualize your data with plotting with python
- Introduces Scikit-learn, a powerful data science package for python
It is very clean, understandable and used Jupyter where all of his code is in his repository.
That’s it for today! You can find me at Twitter @siaterliskonsta, I would love to hear your thoughts, or you can post a comment in the comment section below! Till next time, take care and bye bye!
