Up next in 10
In this tutorial we will learn what python iterators and generators are and how to use them.
-----
Install Python:
- MacOS: https://www.jcchouinard.com/install-python-on-macos/
- Windows: https://www.jcchouinard.com/install-python-on-windows/
- Anaconda: https://www.jcchouinard.com/install-python-with-anaconda-on-windows/
Subscribe to my Python Course waitlist: https://ba995fe7.sibforms.com/serve/MUIEAPv50TFkNuoknCNvzRUKLLhvvZd5jzCwEZvi9BNjtkhVuEtLpEG58-Khdquyg9v5V-1qeGDwIXy739I4eFIVcCqeqqV3UUisW_-hAd5ljC1dGMrQvXHC7JvORh9TbnLA1CHqWro4N7YVZ4730-D5dXGxqd3CbaVHSJpS5fyylPMVzCe1_y9xOTl2-CsvEuhO01E0Ytv59HEJ
Support my work: https://www.buymeacoffee.com/jcchouinard
Subscribe to this channel:
https://www.youtube.com/channel/UCthJJqyUGdI5EA0YjnjXthw?sub_confirmation=1
Follow me: https://twitter.com/ChouinardJC
https://www.linkedin.com/in/jeanchristophechouinard/
https://www.buymeacoffee.com/jcchouinard
https://www.youtube.com/channel/UCthJJqyUGdI5EA0YjnjXthw?sub_confirmation=1
Show More Show Less View Video Transcript
0:00
hello everyone welcome to this
0:02
fascinating world of python
0:06
iterators uh they are super useful
0:10
because python iterators allow you to
0:12
iterate between
0:14
iterable iterables are objects and
0:16
python that you can iterate over such as
0:19
list strings or D dictionaries for
0:23
example uh if you want to go into a
0:26
simple example of Pon iterator is that
0:29
you can take list let's I have a list of
0:31
one two three for example and I can say
0:34
for I in each item and I want to print
0:40
I so if I run this you can see that it
0:43
goes through the list it prints one two
0:46
and then three uh iterables are
0:51
essentially objects that have an
0:53
Associated either method so we can take
0:56
that same list that we had earlier and
1:00
we can look at the Ider
1:03
method and you'll see that we indeed
1:07
have a list iterator in this
1:10
case so we can have a built-in function
1:13
in order to obtain that uh iterator from
1:16
an object and this is the Ider function
1:19
one two one two three and this is how uh
1:23
you obtain the iterator from an object
1:26
under the hood the for Loop what it does
1:29
is it applies the eror function that
1:32
you're seeing here to create an iterator
1:34
and then it iterates over that
1:38
object okay so that's the complex and
1:42
too too hard to understand way of using
1:45
iterator but is the iterator in Python
1:48
is just an object that you can iterate
1:50
over a collection of
1:53
data and it allows you to return the
1:56
data line by line okay so for example
1:59
what is the difference between an
2:01
iterable and in iterator so we have my
2:03
string this is an iterable so we can it
2:08
iterate over each letter of the string
2:12
the iterator
2:15
instead what it is is that it is iner of
2:18
that y
2:20
string so now I have the erable and the
2:26
iterator I can then take that iterator
2:31
and itate over one line at the time so
2:35
we can use the next function to just
2:37
itate and then I run it again and then
2:40
it's e and then it's l and then it's l
2:43
and then it's O and then it's over stop
2:47
iteration um this is super useful to
2:51
save uh resources uh Computing resources
2:54
because you just don't load everything
2:57
you just load one line at a time uh so
3:00
then we have this iteration error that
3:03
we have just
3:05
seen we can also iterate over all the
3:10
values of an iterator using the star
3:13
operator the star operator is often
3:15
called the Splat operators but what it
3:18
does is that it unpacks all the elements
3:22
of an iterator or it rivales so let's
3:25
take again my string and my iterator and
3:29
we want to just go through each of them
3:31
we can say print Splat operator and new
3:38
iterator and as you see we just unpacked
3:41
it all at
3:43
once but one thing to consider is that
3:46
you cannot do it multiple
3:49
time so here nothing happens because
3:53
youve already unpacked it so if you want
3:55
to do it again you would need to
3:57
redefine these
4:00
if you want to unpack this into a list
4:04
then we can take our earlier example and
4:07
just do
4:09
list
4:13
iterator and then it just impact
4:16
everything and returns it as a list you
4:19
can also unpack it as a tupo for example
4:22
so you can say
4:25
tupo iterator and you impact it into two
4:29
polls instead so it's super useful here
4:33
uh to to learn about that then you can
4:37
also
4:39
uh just open a file with
4:43
open and you would say my
4:47
file. text in this case uh let me copy
4:53
the full
4:56
pad to the file but I can have my file.
4:59
text in this case let's make this uh as
5:03
read and then as F and then I have this
5:08
um this unpacking and then I can do
5:12
iterator equals iter F and then it's
5:17
interesting because what I can do here
5:19
is I can just read the first line of my
5:23
object saving me a ton of bandwidth and
5:27
here I'm just printing LOL world
5:33
there are building operators in Python
5:36
that can help you work with iterators
5:38
you have uh animate and zip uh uh
5:44
iterators so what animates does is that
5:47
it will yield an index in a value pair
5:50
so it will put an index for each value
5:52
of a item and uh ZIP will combine to
5:57
itable so in order to use anyer right
6:01
you just have your list and let's say I
6:03
have a list it's called ABC and what I
6:07
want to do is I want to return for I and
6:09
B in
6:12
enumerate so enumerate list then I have
6:17
print IV so as you see uh enumerate just
6:22
is an iterator that allows you to create
6:25
an index 01 two for each of the values
6:28
of my list
6:30
I could have used next as well with
6:33
enumerate
6:35
list and you'll see that it will just
6:38
put zero then uh yeah it will just put z
6:44
a uh there's a useful function too that
6:47
you can do is that uh you can use the
6:51
list operator on the enumerate
6:55
object wee
6:59
can see it returns a list of two pole
7:02
but what I want to do is I want to start
7:04
at three
7:06
instead uh and where's my error I have a
7:11
parentheses
7:13
missing
7:15
uh oh no okay there it is I have the
7:18
parentheses missing and then you see I
7:20
have started at 34 five instead of
7:23
starting at zero the zip operator is a
7:27
little bit more interesting as L I have
7:29
a tuple and a list both of them are
7:33
iterables and I can I want to join them
7:36
together so I can do zip equal zip and
7:38
do list T and then I return my uh list
7:44
zi so here let's say I don't return my
7:48
list zip you'll see uh what it returns
7:51
is a zip object so you want to do that
7:54
list zipped so you can unpack all the
7:57
values into the list and you can see
7:59
that it has matched a to dog B to cat
8:03
and C to
8:04
bird you can also use that as the star
8:08
operator uh so you can use the zip and
8:13
print unpa
8:15
zi and that essentially uh prints it uh
8:19
similar it prints the unpacked
8:25
list now let's go into python generators
8:30
python generators they are a way to
8:33
create iterators in a more concise and
8:36
memory efficient
8:38
matter so let's introduce the yield
8:41
keyword in Python the Y yield keyword is
8:44
used inside a function to produce values
8:47
and and pause the
8:50
execution so what it does is that show
8:53
the value it pauses it iterates over the
8:56
next it pauses again and again
9:00
in order to create a generator function
9:02
we would just do Def and simple
9:08
generator and we would say yield one
9:14
yield to
9:18
yield three so there are no limit as to
9:22
uh the number of uh of yields that you
9:26
want to do but you can just say okay I
9:28
have my simple gener generator and it's
9:30
here so now I've created a generator
9:34
object if I want to uh do a generator
9:38
object I would just use it example
9:43
similar as what I just did with uh the
9:46
generator object so I can say okay if I
9:50
just print next gen then I run this one
9:53
and then two and then again three and
9:57
then overall it would stop iteration so
10:01
we have created our own generator object
10:04
to to iterate
10:10
over it is very interesting because then
10:13
you can build your infinite sequence so
10:15
we could say
10:17
infinite
10:19
sequence where generally we would say
10:22
while true num equals zero and we would
10:25
say while true
10:30
num plus equal one if I run this I will
10:35
it will always be true and it will or
10:38
iterate over and just print one to three
10:41
until until my program
10:44
crash uh but if I use a generat object I
10:47
can use yield and return num and this is
10:52
where it's really interesting because
10:54
printing infinite value has potentially
10:58
infinite computing
10:59
resources required but if I just use
11:03
this to as a generator then I can just
11:08
print
11:10
next uh gen and in this case I go from
11:14
printing infinite uh amount I go from uh
11:19
infinite amount of um of computing power
11:24
to uh to know potentially at all problem
11:30
because I can print each item one at the
11:34
time I could add as many as I want but
11:37
it's it the Computing resources are very
11:41
low because uh This truly shows you uh
11:44
the potential and power of a generator
11:50
object if you want to create a generator
11:53
reder object you can do it without heeld
11:57
uh and then what you would do is is you
11:59
just use the parentheses if you use the
12:02
the parenthesis you can save X for X in
12:07
range five so it really looks like a
12:10
list comprehension but in this case it
12:14
it is Tru a generator
12:18
object a list comprehension would be
12:21
like this right for those who didn't
12:24
know this is a list comprehension it
12:26
just returns it and uh parenthesis
12:30
instead of uh bracket is a generator
12:34
object so then you can just do over the
12:37
generator and say for X in
12:41
generator print
12:46
X so why should you use
12:49
generator uh generators in Python it is
12:53
super useful when you work with
12:55
extremely large data set if I have a
12:58
file with 10 billion rows you might not
13:02
want to load those 10 billion rows
13:04
straight up in your computer maybe you
13:05
want to run a load just 10 rows instead
13:10
um so let's see how uh how uh
13:15
generators are efficient let's take time
13:19
and I I'm going to time the execution
13:22
here I'm going to do a list
13:24
comprehension this X for X in range
13:32
10 n okay that's a very large operation
13:36
and I just want to pick up the first
13:39
item of a very large list if I run
13:53
this okay I want to time it instead of
13:57
time so you can see it takes a super
14:01
long time to
14:03
run if instead of doing this uh I want
14:07
to execute and I use a generator I have
14:10
time to uh to write a function see it
14:14
just took 16 seconds to run sorry for uh
14:19
not having this but if I
14:21
take my generator for X in
14:25
range then
14:29
nine you can see uh it's a similar thing
14:32
but in this case I'm doing a generator
14:34
instead of just a list and I just want
14:37
to pick up the next
14:42
gen around this you can see 0.0 second
14:46
it loaded instantly because I'm just
14:48
trying to pick the first object of the
14:51
generator so it's super useful to use in
14:54
context of large
14:57
scale if you want want to create your
15:00
own iterator it can be complex uh I will
15:04
not go into the depth of P classes but
15:08
I'm just going to show a an example if
15:11
you don't know how this work I strongly
15:14
urage you to take a class on uh uh
15:16
object oriented programming but in this
15:19
case I'm going to create a class of
15:21
positive
15:22
integers
15:24
integers and I will initialize this as
15:30
this in
15:33
it and self and I will just take the
15:39
self.
15:41
current equal one so I want to have only
15:45
positive numbers with this class I will
15:48
create the iterate uh iter method here
15:52
for my
15:53
class
15:54
um death ER self and return self and and
15:59
then I want to create the next so if you
16:01
create your own iterator you need to be
16:03
able to uh Define the next function so
16:07
remember I told you right at the
16:09
beginning that an iterator had the iter
16:12
method so you have to create that it
16:14
method and it you need to define the
16:17
next function so that we can use it
16:19
later so we could say self and we say
16:23
okay what happens if uh if we do self we
16:27
say if
16:29
self but
16:31
current is less equal to five
16:36
result equals self.
16:39
current self
16:42
do current and then I will increment by
16:46
one so to return the
16:51
result else remember when we got to the
16:55
end of our um end our our Loop we need
17:00
to raise uh stop iteration so in this
17:04
case I if you don't know what this mean
17:07
uh you can look at our example on try
17:10
and except on my tutorial and try and
17:12
except uh I teach you how to uh raise
17:15
your own exception but that's
17:17
essentially what it does so when you
17:19
reach the end of the loop then you will
17:21
stop uration so essentially I'm saying
17:24
print everything until it's five
17:29
uh so then
17:31
uh those are comments and then if you
17:34
want to use this you can uh create your
17:37
numbers iterator and then you can say
17:41
positive
17:43
integers so here you have your objects
17:47
uh I've created that class I've created
17:49
initialized my class I've created the
17:52
iter method I've created the next method
17:55
and I've raised the exception if I reach
17:58
the end so then I can just do
18:04
next I can just
18:07
print
18:09
next no so let's let's just run this in
18:13
a separate section uh so it doesn't
18:17
instantiate every time but you can see
18:19
that every time I run this I get to the
18:21
end or and yes I've just stopped the
18:25
iteration if I wanted to come back I
18:28
would need to
18:30
uh um instantiate my class again and
18:34
instead of doing next I can just say for
18:37
Num in number iterator and then I would
18:41
just print
18:43
num and you can see that it works as an
18:46
iterator so this was the end of our
18:48
tutorials on python iterator make sure
18:51
that you like and subscribe this Channel
18:53
or follow me on on my blog where there's
18:55
a lot of resources for python for
18:57
beginners than thank you very much see
18:59
you next time