Python - Magic Methods

It is often said that everything in Python is an object of one or the other class. As per the data model of Python, every class - whether built-in or user-defined - is inherited from object class. The object class itself is defined in built-ins module which is automatically inherited whenever Python interpreter starts.

Python class has a __bases__ attribute which returns tuple of its base classes.

>>> int.__bases__
(<class 'object'>,)

Likewise base class for any built-in class will be found to be object class. This is the case for any user-defined class too.

>>> class A:
pass
>>> A.__bases__
(<class 'object'>,)

If we inherit another class from A, the class hierarchy of method resolution order will show up as follows:

>>> class B(A):
pass
>>> B.__bases__
(<class '__main__.A'>,)
>>> x=B()
>>> x.__class__.__mro__
(<class '__main__.B'>, <class '__main__.A'>, <class 'object'>)

Here <class '__main__.B' and class '__main__.A' indicate that they are defined in an interactive session. But object class remains at the top of hierarchy.

The object class has a number of properties whose name is preceded and followed by double underscores (__). Each of these properties is a wrapper around a method of same name. Such methods are called special or magic methods. One such method is __dir__() which is internally called by dir() built-in function.

>>> dir(object)
['__class__', '__delattr__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__le__', '__lt__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__']

Naturally, these attributes are inherited by all Python classes. We define a dummy class and check the output of dir() function. 

>>>class xyz:
pass
>>>xyz.__bases__
(<class 'object'>,)
>>>dir(xyz)
['__class__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__le__', '__lt__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__']

For a built-in class, the dir() method shows few more additional special attributes having prefix and postfix double underscore.

>>> dir(int)
['__abs__', '__add__', '__and__', '__bool__', '__ceil__', '__class__', '__delattr__', '__dir__', '__divmod__', '__doc__', '__eq__', '__float__', '__floor__', '__floordiv__', '__format__', '__ge__', '__getattribute__', '__getnewargs__', '__gt__', '__hash__', '__index__', '__init__', '__init_subclass__', '__int__', '__invert__', '__le__', '__lshift__', '__lt__', '__mod__', '__mul__', '__ne__', '__neg__', '__new__', '__or__', '__pos__', '__pow__', '__radd__', '__rand__', '__rdivmod__', '__reduce__', '__reduce_ex__', '__repr__', '__rfloordiv__', '__rlshift__', '__rmod__', '__rmul__', '__ror__', '__round__', '__rpow__', '__rrshift__', '__rshift__', '__rsub__', '__rtruediv__', '__rxor__', '__setattr__', '__sizeof__', '__str__', '__sub__', '__subclasshook__', '__truediv__', '__trunc__', '__xor__', 'bit_length', 'conjugate', 'denominator', 'from_bytes', 'imag', 'numerator', 'real', 'to_bytes']

The int class is also a subclass of an abstract class Integral defined in numbers module. The int class inherits magic methods like __add__, __sub__ etc from Integral class. These are called when addition or subtraction operation is performed by + or – respectively.

>>> a=10
>>> b=20
>>> a+b
30

This is equivalent to

>>> a.__add__(b)
30

Such magic methods are overridden in user-defined class to overload respective operators mapped to them. They will be described later in this chapter.

Object customization

The __init__() method is quite familiar. It acts as a constructor and is automatically called to initialize instance variables of an object. But how is the object itself created?

__new__() method

The __init__() method first internally calls the __new__() magic method. The __new__() method actually returns a new object which is then initialized. You have to provide overridden implementation of __new__() method to customize creation of object in a user-defined class. It is a static method that requires first argument as reference to the class whose object is to be created.

class User:

       def __new__(cls, *args):
               print ("calling __new__ magic method.")
               inst = object.__new__(cls)
               return inst
       def __init__(self, *args):
               print ("calling __init__ magic method now")
               self.name=args[0]
               self.email=args[1]
               self.role=args[2]

Output:

>>> from tmp import User
>>> x=User('ram', 'ram@abc.com', 'manager')
calling __new__ magic method.
calling __init__ magic method now

__repr__() and __str__() methods

These methods are internally called by built-in functions repr() and str() respectively. The repr() function returns 'official' string representation of the object while str() is the informal or printable verion.

Any valid Python expression value may be return value of __repr__() method. However if it returns a non string value, call to repr() function results in TypeError.

In above User class we add following __repr__() method:

def __repr__(self):
             d=dict(name=self.name, email=self.email,role=self.role)
             return d

Note that it returns a dictionary object. Hence repr() raises TypeError although call to __repr__() is executed.

>>> x.__repr__()
{'name': 'ram', 'email': 'ram@abc.com', 'role': 'manager'}
>>> repr(x)
Traceback (most recent call last):
 File "<pyshell#6>", line 1, in <module>
   repr(x)
TypeError: __repr__ returned non-string (type dict)

The magic method is __str__() is overridden to return a printable string representation of any user-defined class. Let us now override __str__() method in User class to return string representation of its object.

 def __str__(self):
             return 'name={} email={} role={}'.format(self.name, self.email,self.role)

Both str() function and __str__() method can be called equivalently.

>>> x.__str__()
'name=ram email=ram@abc.com role=manager'
>>> str(x)
'name=ram email=ram@abc.com role=manager'

Operator overloading 

An operator is usually a symbol that is predefined to a certain operation. For instance arithmetic operators by default operate upon numeric operands. Hence only numeric objects are allowed to be used along with operators like +, -, *, / etc.

In Python, each operator actually invokes a corresponding method from object class that has two underscores in beginning and end. For example the '+' operator actually calls __add__() method as shown below:

>>> a=10
>>> b=20
>>> a+b
30

This is equivalent to:

>>> a.__add__(b)
30

The + operator is also defined as concatenation operator in string, list and tuple classes.  So along with numbers '+' performs addition, and with sequences it performs concatenation. We say that + operator is overloaded.In general, operator overloading is the mechanism of defining alternate meaning of an operator.

In order to make overloaded behaviour available in custom class, corresponding magic method should be overridden. For example, in order to use + operator with objects of a user-defined class, it should have __add__() method.

Following table shows magic methods to overload different operators.

__add__() method

In following example, a class named MyTime is defined with two instance attributes hour and min. Addition of two MyTime objects is desired to be performed by overloading + operator.

To achieve this, __add__() magic method is overridden which performs addition of hour and min attributes of two objects. The __str__() method returns object’s string representation.

class MyTime:

       def __init__(self, h=None, m=None):
               self.hour=h
               self.min=m
       def __add__(self,other):
               temptime=MyTime()
               temptime.hour=self.hour+other.hour
               temptime.min=self.min+other.min
               if temptime.min>=60:
                       temptime.hour=temptime.hour+1
                       temptime.min=temptime.min-60
               return temptime
       def __str__(self):
               return str(self.hour)+" Hr."+str(self.min)+" min."

Let us set up two objects of above class and use + operator with them.

>>> a=MyTime(4,30)
>>> b=MyTime(2, 50)
>>> print (c)
7 Hr.20 min.

__sub__() method

Other operators can be similarly overloaded. If __sub__() method is provided in MyTime class, subtraction of two objects can be done.

Add following method to above code.

def __sub__(self,other):
               temptime=MyTime()
               if self.min<other.min:
                       self.min=self.min+60
                       self.hour=self.hour-1
               temptime.hour=self.hour-other.hour
               temptime.min=self.min-other.min
               return temptime

Result of subtraction of two MyTime objects will be:

>>> a=MyTime(4,30)
>>> b=MyTime(2, 50)
>>> c=a-b
>>> print (c)
1 Hr.40 min.

__ge__() method

The logical operator '>' is mapped to __gt__() magic method. Let us add its overridden implementation in MyTime class to overload '>' operator. We will have to convert hours of each MyTime object to minutes to compare them.

   def __gt__(self,other):
               min1=self.hour*60+self.min
               min2=other.hour+other.min
               if min1>min2:
                       print ('First time is greater than second')
               else:
                       print ('First time is not greater than second')

This method gets invoked when > operator is used and returns appropriate .

>>> a=MyTime(4,30)
>>> b=MyTime(2, 50)
>>> a>b
First time is greater than second

You can try overloading other operators in the above table too.