Understanding Generators in Python.

Generators is a function in which objects are created at once but not all code is executed at once as done in normal function. In normal function execution from top to the return statement. A function that consists of a yield statement is called a generators function. The execution of the generator function happens differently, in which the code execution stops at the yield statement rather than a return statement, to move to the next statement next() method is called which will start the execution of the code from where it is left over. If no yield statement is found a StopIteration exception is raised.

So lets see how to create, execute a Generators in python.

def fib(n):
    a, b = 0, 1
    while a <= n:
        yield a   # yield statement.
        a, b = b, a + b

Now let execute the method fib().

fib_fun = fib(10)
next(fib_fun) # 0
next(fib_fun) # 1
next(fib_fun) # 1
.
.
.
next(fib_fun) # 8
next(fib_fun) # reached the end will raise StopIteration Error.
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
StopIteration
 

# Else you can use for loop which call next() in the background.

for fib_value in fib(10):
    print(fib)

# Output
0
1
1
2
3
5
8

So here we today understand the Generators concept in python. Now you would be thinking where we can use this, let me state some use cases.

  • Can be used for memory management, where we pass the whole list as once, we can use Generator to pass data one by one so that less load comes on memory.
  • Generator can be used to define infinite streams.

If you know any more use case, please do share in the comments and if want to share something else or talk about Generators feel free to ping me on twitter

Till then Cheers 🙂
Happy Digging.

What is Closures ?

Today we will talk about Closure a functional object in Python.

Closure is a function object which has access to the local variables/free variables of the enclosing scope and can be executed outside of its scope. Nested function is a Closure function if

  • It can access variables that are local to enclosing scope.
  • It can be executed outside for its scope.

Closure can used for one of these

  • Replacing the hard coded constants.
  • Eliminating global.
  • Can be used for data hiding and many other things.

So lets see with a code snippet to see how closure and nested function are different from each other.

# nested functions 
def inc_x(x):  
    def add_10(x=x):
        print("{0} is increased by 10 = {1}".format(x, x+10)) 
    return add_10()  # remember about the parenthesis.
  
inc_value = inc_x(10)
inc_value  #Output: 10 is increased by 10 = 20

So above function will be called a Nested function, not a Closure because

  • Inner function [add_10] doesn’t access the local variable of enclosing function inc_x. It used the value of X rather than using a reference.
  • Inner function [add_10] cannot be executed outside the scope of inc_x.

Now let see the Closure function example.

# closure functions 
def inc_x(x):  
    def add_10(): 
        print("{0} is increased by 10 = {1}".format(x, x+10)) 
    return add_10 # returning function without parenthesis, passing only references.
  
inc_value = inc_x(10)
# We are able to execute the inner function outside of its scope.
inc_value() 
#Output: 10 is increased by 10 = 20

So above code will be called as Closure function rather than Nested function because

  • add_10 function is accessing the local variable of the inc_x function.Here a reference to the local variable of inc_x is maintained in the add_10.
  • add_10 can even be executed outside the body/scope of inc_x function.

Closure in python is created by a function call, here every time inc_x is called a new instance of this function is created. So whenever you call inc_x a binding reference is made to x which is used in add_10 function.

So let see how under the hood these variable reference are maintained

  • Function attributes func_closure in python < 3.X or closure in python > 3.X save the these references to these variable or also called as free variable. Let see how to access these values.
# Taking same example for the above code
def inc_x(x):  
    def add_10(): 
        print("{0} is increased by 10 = {1}".format(x, x+10)) 
    return add_10

add_10 = inc_x(30)

add_10()
# Output: 30 is increased by 10 = 40

# Checking whether it is Closure or not.

'__closure__'  in  dir(add_10)

# Output: True

# Getting the free variable value from closure.

add_10.__closure__[0].cell_contents 

# Output: 30

While talking about the closure we also heard the term free variables which is also an interesting topic to discuss, which I will cover in the next blog post, till then

Cheers !! 🙂
Happy Learning

Image by JacLou DL from Pixabay

Chanakya Neeti by B.K. Chaturvedi

So another book read, this book is different from all other books I have read before which are not many though 😛 . B.K. Chaturvedi has beautifully explained Chanakya teaching in this book ranging from The Individual, Society, Good Company, General Observation. Chanakya is one of the greatest figures of wisdom and knowledge in Indian history. Chanakya’s all great wisdom is well stated in this book in the form of shlokas/saying in both English and Sanskrit language with explanation. Chanakya with his knowledge and teaching turn Chandragupta Maurya into emperor of one of the largest-ever empires on India. Let me share some of that teaching with you.

No disease is more deadly than desire, no enemy is more dangerous than infatuation, no fire is hotter than the fire of wrath and no happiness is better than self-knowledge.

If one craves of comfort, then he should drop the idea of studying and if one wants to study sincerely then he should stop craving for comfort. One cannot get comfort and education simultaneously.

Knowledge and guess, both, must be used while examining the possible consequences of a job to be undertaken.

Book is filled with these type of Shlokas/Saying which gives quality knowledge about different perspective about life. A must read for the one who want to know Chanakya’s Neeti.

Happy Reading 🙂

How to merge specific files from another branch/repository

Let says you are working on a project and you have dependencies of code from some another repository, which you need in your project code.

  • One way is to copy the code from another repository manually to yours whenever it gets update not so good way 😦
  • Another way is to use git version control system to do that for you and its super easy to do 🙂

Let me show you how we can do it.

Fetching from Another Repository

git add remote other <repository_link>
git fetch other
git checkout <your_target_branch>
git checkout -p other/target-branch file_path

If you have multiple files you have to just change the last checkout statement to

git checkout other/target-branch file_path1 file_path2

But wait there is one catch here that is the path of the file in your repository should be a mirror image (same as) of the path of the file in another repository you are fetching from.

Fetching from Same Repository

Now If you want to fetch files from another branch of the same repository you have to just do

git checkout other_branch_name file_path1 file_path2

I have to admit that it has been three years now working with git, but it still excites me that there is lot of things that I do not know about git, If you also have some important time-saving git commands which you feel can save someone else time do share in comment because sharing is caring :sunglasses: .

Cheers!

Happy Coding


Photo by Yancy Min on Unsplash

Scrapy The Tool

scrapy logo

As part of my job, I have to scrape some website to help our sales team with data on the market, as of now they were doing it manually which is a bit of tedious job to do and consumes lot of their productive time. So on bit searching and going through different tools and framework came across a framework named Scrapy. So here I am going to share how to set up and use Scrapy.

Scrapy is a free and open source web-crawling framework written in python which is used to extract data from a website without much of hassle. They have a very nice documentation you can check out here.

Steps to Install Scrapy

sudo apt-get install python-dev python-pip libxml2-dev libxslt1-dev zlib1g-dev libffi-dev libssl-dev
pip install Scrapy

Steps to Create New Project

To create a Scrapy project type this command in your terminal.scrapy startproject <project name>. Project structure will look like this

now go ahead and create a python file at path /spiders and paste below code.

#!/usr/bin/env python3
import scrapy

class RedditSpider(scrapy.Spider):
    # name of the scrapper, it should be unique.
    name = "reddit"
    # list of the URL need to be iterated.
    start_urls = ['https://www.reddit.com/']

    # Called to do any operation on the response of the above URL.
    def parse(self, response):
       # css selector of the anchor tag which contains the headers
       top_post = response.css("a.SQnoC3ObvgnGjWt90zD9Z")
       for post in top_post:
           self.log(post.css('::text').extract_first())

To start scrapping, type

scrapy crawl reddit

Here we are scrapping the Reddit website for the latest post and getting the header of all the post. The output of the above code will look like this.

  • Trump Organization ‘Sold Property to Shell Company Linked to Maduro Regime,’ Says Report
  • Blind people of Reddit, what do you find sexually attractive?
  • A “caravan” of Americans is crossing the Canadian border to get affordable medical care
  • A “caravan” of Americans is crossing the Canadian border to get affordable medical care
  • [Post Game Thread] The Houston Rockets defeat the Golden State Warriors, 112-108, behind Harden’s 38 points to level the series 2-2, despite the continued brilliance of Kevin Durant 18, my friend here is failing biology and thinks she’s unroastable. Go for it guys, and go hard If you strike me down, I shall become more powerful than you can possibly imagine. [BOTW]
  • ELI5: Why are all economies expected to “grow”? Why is an equilibrium bad?
    ….

Now the best part of Scrapy is if you want to experiment around any website before creating any project you can easily do that.

scrapy shell 'https://www.reddit.com/'

and then can try different CSS selector on the response . Though there is a lot more you can do with Scrapy like saving the result in JSON, CSV format and even integrate with Django project might show that in next post, till then good bye.

Cheers

Music for productivity

Past few days back I have finished my Learning How to Learn course from Coursera recommended by Jason Braganza.

So here sharing a piece of knowledge from the course, that can really help you to increase productivity. We all love to hear music while working, but did you ever give a thought about which music genre you should here while working? So, here is straight advice from people who have tried this out.

Classical Music : If your work involves numbers or attention to detail

Pop music : If your work involves data entry or working to deadlines

Ambient music : If your work involves solving equations

Dance music : If your work involves proof-reading and problem solving

 


Love to hear from you if you know more music genre, or have playlist of your own productive music. Thanks to  Mark Solarski for the cover image.

ILUG-BOM Meet-Up

This Saturday, I went to ILUG-BOM meet-up which was on Image processing by Raghavendra Kamath.

It was great meet-up where we discuss about image processing using the GIMP tool which is an open source tool.

Learn about Mask, Layer, Color curve using GIMP tool.

After the meet-up floor was open for discussion where the discussion is about how to encourage schools to use open source tool and aware them about its benefits. I really enjoyed being part of the discussion as being my first day with the ILUG -BOM they don’t let me feel that and appreciate my participation in the discussion.Soon i will share image edit with GIMP tool.

Python Tutorial -1

As wiki state

Python is a widely used high-level, general-purpose, interpreted, dynamic programming language.Its design philosophy emphasizes code readability, and its syntax allows programmers to express concepts in fewer lines of code than possible in languages such as C++ or Java.

Setting your PC to run Python

  • Window user can download Python from here
  • GNU/Linux can download with simple command :
    dnf install python
    Or you can download straight from here
  • Mac user head over straight to this Link

Now as you all are ready with python now let’s have some fun
I am using on Fedora 23, but it same for other OS
So, GNU/Linux user head straight to terminal and type python3 and python interpreter will open.

To declare a variable and print it’s value

This will declare variable a with value 10
a=10
To print the value type a and press enter
In real world(when we write our program in file) we use print(a) to print the value of variable
now further you can play around python and learn more from resources on internet.

Let me show you some Pythonic way of coding.

  • Pythonic way to swap two number’s
    C++ way to swap two numbers
    int a=10;
    int b=20;
    int temp=a;
    a=b;
    b=temp;

    But in python you can do this in one line
    a=10
    b=20
    a,b=b,a

    Yea this way only one line can do this
    Python is amazingly great soon will share more pythonic way of coding.

Till than cheers!
Happy Coding