Understanding Generators in Python.

Generators is a function in which objects are created at once but not all code is executed at once as done in normal function. In normal function execution from top to the return statement. A function that consists of a yield statement is called a generators function. The execution of the generator function happens differently, in which the code execution stops at the yield statement rather than a return statement, to move to the next statement next() method is called which will start the execution of the code from where it is left over. If no yield statement is found a StopIteration exception is raised.

So lets see how to create, execute a Generators in python.

def fib(n):
    a, b = 0, 1
    while a <= n:
        yield a   # yield statement.
        a, b = b, a + b

Now let execute the method fib().

fib_fun = fib(10)
next(fib_fun) # 0
next(fib_fun) # 1
next(fib_fun) # 1
next(fib_fun) # 8
next(fib_fun) # reached the end will raise StopIteration Error.
Traceback (most recent call last):
File "<stdin>", line 1, in <module>

# Else you can use for loop which call next() in the background.

for fib_value in fib(10):

# Output

So here we today understand the Generators concept in python. Now you would be thinking where we can use this, let me state some use cases.

  • Can be used for memory management, where we pass the whole list as once, we can use Generator to pass data one by one so that less load comes on memory.
  • Generator can be used to define infinite streams.

If you know any more use case, please do share in the comments and if want to share something else or talk about Generators feel free to ping me on twitter

Till then Cheers 🙂
Happy Digging.

What is Closures ?

Today we will talk about Closure a functional object in Python.

Closure is a function object which has access to the local variables/free variables of the enclosing scope and can be executed outside of its scope. Nested function is a Closure function if

  • It can access variables that are local to enclosing scope.
  • It can be executed outside for its scope.

Closure can used for one of these

  • Replacing the hard coded constants.
  • Eliminating global.
  • Can be used for data hiding and many other things.

So lets see with a code snippet to see how closure and nested function are different from each other.

# nested functions 
def inc_x(x):  
    def add_10(x=x):
        print("{0} is increased by 10 = {1}".format(x, x+10)) 
    return add_10()  # remember about the parenthesis.
inc_value = inc_x(10)
inc_value  #Output: 10 is increased by 10 = 20

So above function will be called a Nested function, not a Closure because

  • Inner function [add_10] doesn’t access the local variable of enclosing function inc_x. It used the value of X rather than using a reference.
  • Inner function [add_10] cannot be executed outside the scope of inc_x.

Now let see the Closure function example.

# closure functions 
def inc_x(x):  
    def add_10(): 
        print("{0} is increased by 10 = {1}".format(x, x+10)) 
    return add_10 # returning function without parenthesis, passing only references.
inc_value = inc_x(10)
# We are able to execute the inner function outside of its scope.
#Output: 10 is increased by 10 = 20

So above code will be called as Closure function rather than Nested function because

  • add_10 function is accessing the local variable of the inc_x function.Here a reference to the local variable of inc_x is maintained in the add_10.
  • add_10 can even be executed outside the body/scope of inc_x function.

Closure in python is created by a function call, here every time inc_x is called a new instance of this function is created. So whenever you call inc_x a binding reference is made to x which is used in add_10 function.

So let see how under the hood these variable reference are maintained

  • Function attributes func_closure in python < 3.X or closure in python > 3.X save the these references to these variable or also called as free variable. Let see how to access these values.
# Taking same example for the above code
def inc_x(x):  
    def add_10(): 
        print("{0} is increased by 10 = {1}".format(x, x+10)) 
    return add_10

add_10 = inc_x(30)

# Output: 30 is increased by 10 = 40

# Checking whether it is Closure or not.

'__closure__'  in  dir(add_10)

# Output: True

# Getting the free variable value from closure.


# Output: 30

While talking about the closure we also heard the term free variables which is also an interesting topic to discuss, which I will cover in the next blog post, till then

Cheers !! 🙂
Happy Learning

Image by JacLou DL from Pixabay

Chanakya Neeti by B.K. Chaturvedi

So another book read, this book is different from all other books I have read before which are not many though 😛 . B.K. Chaturvedi has beautifully explained Chanakya teaching in this book ranging from The Individual, Society, Good Company, General Observation. Chanakya is one of the greatest figures of wisdom and knowledge in Indian history. Chanakya’s all great wisdom is well stated in this book in the form of shlokas/saying in both English and Sanskrit language with explanation. Chanakya with his knowledge and teaching turn Chandragupta Maurya into emperor of one of the largest-ever empires on India. Let me share some of that teaching with you.

No disease is more deadly than desire, no enemy is more dangerous than infatuation, no fire is hotter than the fire of wrath and no happiness is better than self-knowledge.

If one craves of comfort, then he should drop the idea of studying and if one wants to study sincerely then he should stop craving for comfort. One cannot get comfort and education simultaneously.

Knowledge and guess, both, must be used while examining the possible consequences of a job to be undertaken.

Book is filled with these type of Shlokas/Saying which gives quality knowledge about different perspective about life. A must read for the one who want to know Chanakya’s Neeti.

Happy Reading 🙂

Books I’ve Read



  • Don’t You Have Time to Think? — by Richard P. Feynman
  • The Richest Man in Babylon — by George S. Clason
  • The Subtle Art of Not Giving a F*ck: A Counterintuitive Approach to Living a Good Life — by Mark Manson
  • Atomic Habits: An Easy & Proven Way to Build Good Habits & Break Bad Ones — by James Clear
  • Unposted Letter — by Mahatria Ra
  • Chanakya Neeti — by B.K. Chaturvedi
  • I want 2 do project tell me what 2 do — by Shakti Khannan
  • Sapiens A Brief History of Humankind — by Yuval Noah Harari

I want 2 do project. tell me wat 2 do.

What a great book, packed with a lot of advice, guidelines, and methodologies which will be of great help to all people working in the technology world or want to start contributing to the F/OSS world.

Shakthi has made all the point so precise and meaningful that I have a sweet smile on my face(you can relate to lot of things from the book) while reading the book. Shakthi has covered very important topics ranging from mailing list guidelines, project guidelines, and communication which one might realize late that how helpful these things can be in their profession.

Here is example from the book where Shakthi explaining how Experience is the the best teacher with the help of (a + b)² proof and show that rote learning does not last long.

Figure: Illustrative proof of
(a + b)² = a² + b² + 2ab

Proof: The area of the square is (a + b)², which is same as the same as the sum of the areas comprising the square, which is, a² + ab + ab + b² = a² + b² + 2ab.

The best part I like about the book, the quotes that open each section and example that makes reading the book more enjoyable. I enjoy every line of the book and love the way how Shakthi has encouraged the entrepreneur mindset with all other things one should think of. Its great book that one should give a read once.

You can check about the book over here

Google Cloud IAM: Identity and Access Management

Identity & Access Management let the user manage access control/policies to the resources by defining who(identity) and what they can access(roles). Today we will talk about the Google Cloud Identity & Access Management and understand what it is and How to use it.

Policy in IAM is composed of the binding list which binds the Member Identity and Roles together to limit the access on google cloud resources.

Member can be of the following type

  • Google Account: This can be any valid Google account with gmail.com or with any other domain name.
  • Service Account: Account related to the application rather than an individual, you can have as many numbers of service account for the logical components of your application.
  • Google Group: Google group are the collection of the different Google account and service account. Every group has a unique email id which can be used to identify members in the IAM policy. The benefit of group account is that if you want to change the permission of user you can simply move the user from one group to another group rather than changing the permission of the users.
  • G Suited domain: Is the virtual group of all the account created in the organization Suite.

Roles on other hand is collection of permission which is mainly represented as <service>.<resource>.<verb>, for example pubsub.subscriptions.consume. Permission determines what type of operation can be performed on resources. Permission cannot be directly applied to resources instead you can assign roles which are a group of different permission.

In Google Cloud Platform Roles are of three kinds

  • Primitive Roles
  • Predefined Roles
  • Custom Roles

Primitive Roles

These are of three types Owner, Editor and Viewer as the name suggest.

  • Viewer has only access to view the resources and data.
  • Editor has Viewer permission + permission to change/edit the resources.
  • Owner has the permission of editor + permission to manage all resources and user.

Predefined Roles

These are the roles provides by Cloud IAM in addition to primitive roles which provide more granular level access to the resources and these primitive roles differ based on different resources in the cloud, you can check these roles over here

Custom Roles

Cloud IAM let the user define different custom roles if primitive and predefined roles do not fulfill their requirements. Though there is some pointer to remember while creating the custom roles. Custom roles can be defined on Organization and Project level but not on Folder level and custom roles should have an iam.roles.creator

So now the question is how these rules actually work as we know that policy is the binding list which binds the member and roles. These policies are connected to resources and are enforce access control when these resources are accessed.

Google Cloud Policy have a hierarchy Organization > Folder > Project > Resources, every resource has exactly one parent and inherit the policy from its parent. Any policy assign on the parent is applied to all its child’s. Here is the diagram from Google Cloud IAM docs which show how this hierarchy looks.

Here an example from official docs how permission hierarchy works,

In the diagram above, topic_a is a Cloud Pub/Sub resource that lives under the project example-prod. If you grant the Editor role to micah@gmail.com for example-prod, and grant the Publisher role to song@gmail.com for topic_a, you effectively grant the Editor role for topic_a to micah@gmail.com and the Publisher role to song@gmail.com.

So here my effort to explain Google IAM policy in simple words, Hope you find it usefully. Please do share any feedback or any topic you think I should cover in this post. Till then Happy Clouding 🙂

Sapiens A Brief History of Humankind

I came across this book on the Knowledge Project blog post where naval mentioned this book. It’s an enjoyable ride which begins from 13.5 billion years back when Matter and energy appear and where the beginning of physics and chemistry happens to today’s world where we are building Super Humans and Cyborg and many more things…

Long long back we have more than one species of human, Homo rudlofensis, Homo erectus and Homo neanderthalenis are the one which we are aware off, but now we the Homo sapiens have only survived and is at the top of the food chain, because we Sapiens have the capability of imagination and thinking which make us different from others species. This unique quality makes us able to survive for such a long time, though the survival come with the extinction of other species. Way back we Sapiens used to live in the small social group mainly hundred of people even less, which was a common way of surviving and alike these groups there were many groups spread vastly over the earth. These groups have there own set of rules and regulation which were followed by all in which powerful men hunt to feed all other people of the group which was the main source of food apart from wild grasses or fruits.

Then comes the Agriculture Revolution which shifted us from hunting to growing crops(mainly wheat) and even started accumulating the food for the bad time. This was a big change as before we were not bound to one place we move from one place to another and depends on the group to survive but because of crop cultivation people start living in one place and starting spending more and more time on fields looking after the crops all day long and building houses around the fields so that they can take care of the crops days and night. Sapiens become good at this as they were able to estimate the rain or drought effectively and able to survive on the stored food in bad time. This makes people stick at one place which in turn raise many physical problems as Sapiens body was not used to it.

We did not domesticate wheat.It domesticated us.The word domesticate come from Latin 'domus' which mean house.Who's the one living in a house? Not the wheat. It's the Sapiens

Sapiens started acquiring more and more land for crop cultivation which lead to dispute with each other, to handle these dispute we start laying imaginative laws and belief so that people follow these rules and live happily with each other but this eventually leads to boundaries, city, state, country and that happy is lost. The people start trading with each other with resources and then money came into existence which basically solves a lot of problems and standardize the exchange of the things. Money brings more scope of business and standard in trading which ultimately made people explore more and more which leads us to Scientific Revolution.

Scientific Revolution opens all new path of exploration and discoveries of medicine, god particle, different planets, and artificial intelligence which eventually lead to an easy complex life. Now humans are surrounded by things which might not even be imagined back before. Humans are also able to build cyborg and are in the process of creating a Superhuman and are ready to colonize planet around the earth(Mars we are coming :P). These also have a varying degree of effect on human life, like average life of human has increased greatly due to medicines, transportation has become easy people can compute from one corner of the earth to others even outside :).

Scientific Revolution is still in the continuous phase , what it can bring next is unknown, but this is Our journey from that Animal to God. Will end the post with a quote quoted from Bhagavad Gita by Robert Oppenheimer after the first atomic bomb was detonated.

Now I am become Death, the destroyer of worlds

Photo by Hammad Siddiqui on Unsplash

How to merge specific files from another branch/repository

Let says you are working on a project and you have dependencies of code from some another repository, which you need in your project code.

  • One way is to copy the code from another repository manually to yours whenever it gets update not so good way 😦
  • Another way is to use git version control system to do that for you and its super easy to do 🙂

Let me show you how we can do it.

Fetching from Another Repository

git add remote other <repository_link>
git fetch other
git checkout <your_target_branch>
git checkout -p other/target-branch file_path

If you have multiple files you have to just change the last checkout statement to

git checkout other/target-branch file_path1 file_path2

But wait there is one catch here that is the path of the file in your repository should be a mirror image (same as) of the path of the file in another repository you are fetching from.

Fetching from Same Repository

Now If you want to fetch files from another branch of the same repository you have to just do

git checkout other_branch_name file_path1 file_path2

I have to admit that it has been three years now working with git, but it still excites me that there is lot of things that I do not know about git, If you also have some important time-saving git commands which you feel can save someone else time do share in comment because sharing is caring :sunglasses: .


Happy Coding

Photo by Yancy Min on Unsplash

Scrapy The Tool

scrapy logo

As part of my job, I have to scrape some website to help our sales team with data on the market, as of now they were doing it manually which is a bit of tedious job to do and consumes lot of their productive time. So on bit searching and going through different tools and framework came across a framework named Scrapy. So here I am going to share how to set up and use Scrapy.

Scrapy is a free and open source web-crawling framework written in python which is used to extract data from a website without much of hassle. They have a very nice documentation you can check out here.

Steps to Install Scrapy

sudo apt-get install python-dev python-pip libxml2-dev libxslt1-dev zlib1g-dev libffi-dev libssl-dev
pip install Scrapy

Steps to Create New Project

To create a Scrapy project type this command in your terminal.scrapy startproject <project name>. Project structure will look like this

now go ahead and create a python file at path /spiders and paste below code.

#!/usr/bin/env python3
import scrapy

class RedditSpider(scrapy.Spider):
    # name of the scrapper, it should be unique.
    name = "reddit"
    # list of the URL need to be iterated.
    start_urls = ['https://www.reddit.com/']

    # Called to do any operation on the response of the above URL.
    def parse(self, response):
       # css selector of the anchor tag which contains the headers
       top_post = response.css("a.SQnoC3ObvgnGjWt90zD9Z")
       for post in top_post:

To start scrapping, type

scrapy crawl reddit

Here we are scrapping the Reddit website for the latest post and getting the header of all the post. The output of the above code will look like this.

  • Trump Organization ‘Sold Property to Shell Company Linked to Maduro Regime,’ Says Report
  • Blind people of Reddit, what do you find sexually attractive?
  • A “caravan” of Americans is crossing the Canadian border to get affordable medical care
  • A “caravan” of Americans is crossing the Canadian border to get affordable medical care
  • [Post Game Thread] The Houston Rockets defeat the Golden State Warriors, 112-108, behind Harden’s 38 points to level the series 2-2, despite the continued brilliance of Kevin Durant 18, my friend here is failing biology and thinks she’s unroastable. Go for it guys, and go hard If you strike me down, I shall become more powerful than you can possibly imagine. [BOTW]
  • ELI5: Why are all economies expected to “grow”? Why is an equilibrium bad?

Now the best part of Scrapy is if you want to experiment around any website before creating any project you can easily do that.

scrapy shell 'https://www.reddit.com/'

and then can try different CSS selector on the response . Though there is a lot more you can do with Scrapy like saving the result in JSON, CSV format and even integrate with Django project might show that in next post, till then good bye.


Music for productivity

Past few days back I have finished my Learning How to Learn course from Coursera recommended by Jason Braganza.

So here sharing a piece of knowledge from the course, that can really help you to increase productivity. We all love to hear music while working, but did you ever give a thought about which music genre you should here while working? So, here is straight advice from people who have tried this out.

Classical Music : If your work involves numbers or attention to detail

Pop music : If your work involves data entry or working to deadlines

Ambient music : If your work involves solving equations

Dance music : If your work involves proof-reading and problem solving


Love to hear from you if you know more music genre, or have playlist of your own productive music. Thanks to  Mark Solarski for the cover image.