This notebook was put together by Jake Vanderplas for UW's Astro 599 course. Source and license info is on GitHub.
%run talktools.py
We have spent much of our time so far taking a look at scientific tools in Python. But a big part of using Python is an in-depth knowledge of the language itself. The topics here may not have direct science applications, but you'd be surprised when and where they can pop up as you use and write scientific Python code.
We'll dive a bit deeper into a few of these topics here.
Here is what we plan to cover in this section:
Python can be used as an object-oriented language. An object is an entity that encapsulates data, called attributes, and functionality, called methods.
Everything in Python is an object. Take for example the complex object:
z = 1 + 2j
# The type function allows us to inspect the object type
type(z)
complex
# "calling" an object type is akin to constructing an object
z = complex(1, 2)
# z has real and imaginary attributes
print(z.real)
print(z.imag)
1.0 2.0
# z has methods to operate on these attributes
z.conjugate()
(1-2j)
Every data container you see in Python is an object, from integers and floats to lists to numpy arrays.
Here we'll show a quick example of spinning our own complex-like object, using a class.
Class definitions look like this:
class MyClass(object):
# attributes and methods are defined here
pass
# create a MyClass "instance" named m
m = MyClass()
type(m)
__main__.MyClass
Things get a bit more interesting when we define the __init__ method:
class MyClass(object):
def __init__(self):
print(self)
print("initialization called")
pass
m = MyClass()
<__main__.MyClass object at 0x10370fc90> initialization called
m
<__main__.MyClass at 0x10370fc90>
The first argument of __init__() points to the object itself, and is usually called self by convention.
Note above that when we print self and when we print m, we see that they point to the same thing. self is m
We can use the __init__() method to accept initialization keyword arguments. Here we'll allow the user to pass a value to the initialization, which is saved in the class:
class MyClass(object):
def __init__(self, value):
self.value = value
m = MyClass(5.0) # note: the self argument is always implied
m.value
5.0
Now let's add a squared() method, which returns the square of the value:
class MyClass(object):
def __init__(self, value):
self.value = value
def squared(self):
return self.value ** 2
m = MyClass(5)
m.squared()
25
Methods act just like functions: they can have any number of arguments or keyword arguments, they can accept *args and **kwargs arguments, and can call other methods or functions.
There are numerous special methods, indicated by double underscores. One important one is the __repr__ method, which controls how an instance of the class is represented when it is output:
class MyClass(object):
def __init__(self, value):
self.value = value
def squared(self):
return self.value ** 2
def __repr__(self):
return "MyClass(value=" + str(self.value) + ")"
m = MyClass(10)
print(m)
print(type(m))
MyClass(value=10) <class '__main__.MyClass'>
Other special methods to be aware of:
__str__, __repr__, __hash__, etc.__add__, __sub__, __mul__, __div__, etc.__getitem__, __setitem__, etc.__getattr__, __setattr__, etc.__eq__, __lt__, __gt__, etc.__new__, __init__, __del__, etc.__int__, __long__, __float__, etc.For a nice discussion and explanation of these and many other special double-underscore methods, see http://www.rafekettler.com/magicmethods.html
Create a class MyComplex which behaves like the built-in complex numbers. You should be able to execute the following code and see these results:
>>> z = MyComplex(2, 3)
>>> print z
(2, 3j)
>>> print z.real
2
>>> print z.imag
3
>>> print z.conjugate()
(2, -3j)
>>> print type(z.conjugate())
<class '__main__.MyComplex'>
Note that the conjugate() method should return a new object of type MyComplex.
If you finish this quickly, search online for help on defining the __add__ method such that you can compute:
>>> z + z.conjugate()
(4, 0j)
0/0
--------------------------------------------------------------------------- ZeroDivisionError Traceback (most recent call last) <ipython-input-22-6549dea6d1ae> in <module>() ----> 1 0/0 ZeroDivisionError: division by zero
Or you may call a function with the wrong number of arguments:
from math import sqrt
sqrt(2, 3)
--------------------------------------------------------------------------- TypeError Traceback (most recent call last) <ipython-input-23-80e335c13035> in <module>() 1 from math import sqrt ----> 2 sqrt(2, 3) TypeError: sqrt() takes exactly one argument (2 given)
Or you may choose an index that is out of range:
L = [4, 5, 6]
L[100]
--------------------------------------------------------------------------- IndexError Traceback (most recent call last) <ipython-input-24-25e9eefaf101> in <module>() 1 L = [4, 5, 6] ----> 2 L[100] IndexError: list index out of range
Or a dictionary key that doesn't exist:
D = {'a':2, 'b':300}
print D['Q']
File "<ipython-input-25-3e70854ae47f>", line 2 print D['Q'] ^ SyntaxError: invalid syntax
Or the wrong value for a conversion function:
x = int('ABC')
--------------------------------------------------------------------------- ValueError Traceback (most recent call last) <ipython-input-26-3cb6d4b5dc55> in <module>() ----> 1 x = int('ABC') ValueError: invalid literal for int() with base 10: 'ABC'
These are known as Exceptions, and handling them appropriately is a big part of writing usable code.
try...except¶Exceptions can be handled using the try and except statements:
def try_division(value):
try:
x = value / value
return x
except ZeroDivisionError:
return 'Not A Number'
print(try_division(1))
print(try_division(0))
1.0 Not A Number
def get_an_int():
while True:
try:
# change to raw_input for Python 2
x = int(input("Enter an integer: "))
print(" >> Thank you!")
break
except ValueError:
print(" >> Boo. That's not an integer.")
return x
get_an_int()
Enter an integer: e >> Boo. That's not an integer. Enter an integer: rew >> Boo. That's not an integer. Enter an integer: 3.4 >> Boo. That's not an integer. Enter an integer: 4 >> Thank you!
4
Other things to be aware of:
except statements for different exception typeselse and finally statements can fine-tune the exception handlingMore information is available in the Python documentation and in the scipy lectures
In addition to handling exceptions, you can also raise your own exceptions using the raise keyword:
def laugh(N):
if N < 0:
raise ValueError("N must be positive")
return N * "ha! "
laugh(10)
'ha! ha! ha! ha! ha! ha! ha! ha! ha! ha! '
laugh(-4)
--------------------------------------------------------------------------- ValueError Traceback (most recent call last) <ipython-input-34-8f794d3f8feb> in <module>() ----> 1 laugh(-4) <ipython-input-32-11268c7129b2> in laugh(N) 1 def laugh(N): 2 if N < 0: ----> 3 raise ValueError("N must be positive") 4 return N * "ha! " ValueError: N must be positive
For your own projects, you may desire to define custom exception types, which is done through class inheritance.
The important point to note here is that exceptions themselves are classes:
v = ValueError("message")
type(v)
ValueError
When you raise an exception, you are creating an instance of the exception type, and passing it to the raise keyword:
raise ValueError("error message")
--------------------------------------------------------------------------- ValueError Traceback (most recent call last) <ipython-input-37-c623d04e0b6c> in <module>() ----> 1 raise ValueError("error message") ValueError: error message
In the seminar later this quarter we'll dive into object-oriented programming, but here's a quick preview of the principle of inheritance: new objects derived from existing objects:
# define a custom exception, inheriting from the base class Exception
class CustomException(Exception):
# can define custom behavior here
pass
raise CustomException("error message")
--------------------------------------------------------------------------- CustomException Traceback (most recent call last) <ipython-input-38-06445ec64aea> in <module>() 4 pass 5 ----> 6 raise CustomException("error message") CustomException: error message
Iterators are a high-level concept in Python that allow a sequence of objects to be examined in sequence.
We've seen a basic example of this in the for-loop:
for i in range(10):
print(i)
0 1 2 3 4 5 6 7 8 9
In Python 3.x, range does not actually construct a list of numbers, but just an object which acts like a list (in Python 2.x, range does actually create a list)
range(10)
range(0, 10)
list(range(10))
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
import sys
print(sys.getsizeof(list(range(1000))))
print(sys.getsizeof(range(1000)))
9120 48
D = {'a':0, 'b':1, 'c':2}
D.keys()
dict_keys(['a', 'b', 'c'])
D.values()
dict_values([0, 1, 2])
D.items()
dict_items([('a', 0), ('b', 1), ('c', 2)])
for key in D.keys():
print(key)
a b c
for key in D:
print(key)
a b c
for item in D.items():
print(item)
('a', 0)
('b', 1)
('c', 2)
itertools: more sophisticated iterations¶import itertools
dir(itertools)
['__doc__', '__loader__', '__name__', '__package__', '_grouper', '_tee', '_tee_dataobject', 'accumulate', 'chain', 'combinations', 'combinations_with_replacement', 'compress', 'count', 'cycle', 'dropwhile', 'filterfalse', 'groupby', 'islice', 'permutations', 'product', 'repeat', 'starmap', 'takewhile', 'tee', 'zip_longest']
for c in itertools.combinations([1, 2, 3, 4], 2):
print(c)
(1, 2) (1, 3) (1, 4) (2, 3) (2, 4) (3, 4)
for p in itertools.permutations([1, 2, 3]):
print(p)
(1, 2, 3) (1, 3, 2) (2, 1, 3) (2, 3, 1) (3, 1, 2) (3, 2, 1)
for val in itertools.chain(range(0, 4), range(-4, 0)):
print(val, end=' ')
0 1 2 3 -4 -3 -2 -1
# zip: itertools.izip is an iterator equivalent
for val in zip([1, 2, 3], ['a', 'b', 'c']):
print(val)
(1, 'a') (2, 'b') (3, 'c')
Write a function count_pairs(N, m) which returns the number of pairs of numbers in the sequence $0 ... N-1$ whose sum is divisible by m.
For example, if N = 3 and m = 2, the pairs are
[(0, 1), (0, 2), (1, 2)]
The sum of each pair respectively is [1, 2, 3], and there is a single pair whose sum is divisible by 2, so the result is 1.
yield statement¶Python provides a yield statement that allows you to make your own iterators. Technically the result is called a "generator object".
For example, here's one way you can create an generator that returns all even numbers in a sequence:
def select_evens(L):
for value in L:
if value % 2 == 0:
yield value
for val in select_evens([1,2,5,3,6,4]):
print(val)
2 6 4
The yield statement is like a return statement, but the iterator remembers where it is in the execution, and comes back to that point on the next pass.
Here is a loop which prints the first 10 Fibonacci numbers:
a, b = 0, 1
for i in range(10):
print(b)
a, b = b, a + b
1 1 2 3 5 8 13 21 34 55
Using a similar strategy, write a generator expression which generates the first $N$ Fibonacci numbers
def fib(N):
# your code here
for num in fib(N):
print(num)
Here is a function which uses the Sieve of Eratosthenes to generate the first $N$ prime numbers. Rewrite this using the yield statement as a generator over the first $N$ primes:
def list_primes(Nprimes):
N = 2
found_primes = []
while True:
if all([N % p != 0 for p in found_primes]):
found_primes.append(N)
if len(found_primes) >= Nprimes:
break
N += 1
return found_primes
top_primes(10)
[2, 3, 5, 7, 11, 13, 17, 19, 23, 29]
def iter_primes(Nprimes):
# your code here
# Find the first twenty primes
for N in iter_primes(20):
print(N, end=' ')
2 3 5 7 11 13 17 19 23 29 31 37 41 43 47 53 59 61 67 71
If you finish the above examples, try wrapping your head around generator expressions.
We previously saw examples of list comprehensions which can create lists succinctly in one line:
L = []
for i in range(20):
if i % 3 > 0:
L.append(i)
L
[1, 2, 4, 5, 7, 8, 10, 11, 13, 14, 16, 17, 19]
# or, as a list comprehension
[i for i in range(20) if i % 3 > 0]
[1, 2, 4, 5, 7, 8, 10, 11, 13, 14, 16, 17, 19]
The corresponding construction of an iterator is known as a "generator expression":
def genfunc():
for i in range(20):
if i % 3 > 0:
yield i
print(genfunc())
print(list(genfunc())) # convert iterator to list
<generator object genfunc at 0x104537b40> [1, 2, 4, 5, 7, 8, 10, 11, 13, 14, 16, 17, 19]
# or, equivalently, as a "generator expression"
g = (i for i in range(20) if i % 3 > 0)
print(g)
print(list(g)) # convert generator expression to list
<generator object <genexpr> at 0x104537a00> [1, 2, 4, 5, 7, 8, 10, 11, 13, 14, 16, 17, 19]
The syntax is identical to that of list comprehensions, except we surround the expression with () rather than with []. Again, this may seem a bit specialized, but it allows some extremely powerful constructions in Python, and it's one of the features of Python that some people get very excited about.