The __new__() Method in Python

In Python, the `__new__()` method is automatically invoked whenever you create a new instance of a class. It's also referred to as the constructor.

This method is rarely overridden, but it can be extremely useful in specific cases, such as when you need to control the process of object creation.

When a new object is created, Python calls the methods in the following order:

1. the `__call__()` method
2. the `__new__()` method
3. the `__init__()` method

Thus, the `__new__()` method is the first critical step in object creation because it is executed before `__init__()`.

What are the differences? The `__new__()` method is responsible for actually creating the object. It takes the class itself as the first argument, typically named `cls`, and returns a new instance of the class. The `__init__()` method, by contrast, is called immediately after and handles initializing the newly created object. It takes the instance of the object (usually named `self`) as its first argument.

Let's look at a practical example to make this clearer.

Define a class called "MyClass"

class MyClass:
    def __new__(cls, *args, **kwargs):
      print("Called __new__")
      instance = super(MyClass, cls).__new__(cls)
      return instance

    def __init__(self, value):
      print("Called __init__")
      self.value = value

This class defines both the __new__() and __init__() methods.

Now, create an instance of the class "MyClass"

obj = MyClass(10)

During the instance creation process, Python first executes the `__new__()` method to create a new object, and then the `__init__()` method to initialize it.

Here’s what the output will look like:

Called __new__
Called __init__

Notice that within `__new__()`, we used `super()` to call the `__new__()` method of the base class, which is essential for actually creating the object.

If we didn't delegate the creation of the new instance to the base class, it would lead to infinite recursion.

What happens if a class doesn't have the `__new__()` method?

If a Python class doesn't define a `__new__()` method, Python automatically uses the `__new__()` method from the base class that the current class inherits from.

For most classes, this base class is the object class, the fundamental superclass in Python from which all other classes derive.

The `__new__()` method of the object class is responsible for creating and returning a new instance of the class.

In other words, when you define the `__new__()` method in a class, you're overriding a method that is already inherited from the base class. In Python, all classes derive from the object class where the constructor method `__new__()` is defined. As a result, all classes in Python automatically implement the `__new__()` method.

This is the default behavior and is sufficient for most use cases.

That's why, in the vast majority of Python classes, it's not necessary to explicitly define a `__new__()` method.

When is it useful to override `__new__()`?

In general, it's uncommon to override the `__new__()` method. However, there are scenarios where doing so can be beneficial.

Overriding the __new__ method gives you the ability to create a new data type while inheriting the methods and properties of an existing one.

For instance, a tuple is an immutable type, meaning once it's created, its contents can't be changed.

Now, imagine you want to create a tuple that only accepts positive values. You can do this by creating a new tuple type that inherits all the behavior of a standard tuple.

class PositiveTuple(tuple):
    def __new__(cls, seq):
        # Check if all elements in the sequence are positive
        if not all(x > 0 for x in seq):
            raise ValueError("All elements must be positive numbers.")
        
        # Create the tuple using only positive values
        return super().__new__(cls, seq)

By overriding the __new__ method, you enforce the condition that all elements in the tuple must be positive.

For example, to create a tuple of this new type, you would write:

pt = PositiveTuple((1, 2, 3, 4))

Since the class inherits all the methods of a regular tuple, you can still use those methods with your new PositiveTuple type.

However, if you try to include a zero or a negative value, the class will raise an exception and prevent the tuple from being created.

pt = PositiveTuple((1, -2, 3, 4))

ValueError: All elements must be positive numbers.

This approach allows you to create custom types that retain the properties and methods of predefined ones.

Keep in mind that immutable types like tuples cannot be altered once they're created, so overriding the __new__ method is sufficient. For mutable types like lists, you would also need to override the __init__ method and other methods that modify the data, such as __append__, __insert__, and __extend__.

So, while you don't usually need to override the __new__ method, there are advanced scenarios where it can be particularly useful.

Understanding how this works gives you deeper insight into how Python handles object creation.

I hope this explanation was helpful.