13. Advanced Python¶

There are several advanced Python notions that you need to be familiar with.

13.1. Decorators¶

13.1.1. What is a Decorator?¶

A decorator is a function that takes another function as input, adds some functionality to it, and returns it. You can apply decorators using the @decorator_name syntax.

13.1.2. Basic Example of a Decorator¶

[3]:

# Define a simple decorator
def my_decorator(func):
    def wrapper():
        print("Something is happening before the function is called.")
        func()
        print("Something is happening after the function is called.")
    return wrapper

# Define a function and decorate it
@my_decorator
def say_hello():
    print("Hello!")

# Call the decorated function
say_hello()

Something is happening before the function is called.
Hello!
Something is happening after the function is called.

@my_decorator is syntactic sugar for: python say_hello = my_decorator(say_hello)

13.1.3. Decorator with Arguments¶

[6]:

def repeat_decorator(times):
    def decorator(func):
        def wrapper(*args, **kwargs):
            for _ in range(times):
                func(*args, **kwargs)
        return wrapper
    return decorator

@repeat_decorator(times=3)
def greet(name):
    print(f"Hello, {name}!")

greet("Boris")

Hello, Boris!
Hello, Boris!
Hello, Boris!

Here:

The repeat_decorator takes an argument (times) and returns the actual decorator.
The decorator wraps the greet function to execute it multiple times.

13.1.4. Using `functools.wraps` to Preserve Metadata¶

When you wrap a function using a decorator, the original function’s metadata (like its name and docstring) can be lost.

Use functools.wraps to preserve it.

[7]:

import functools

def my_decorator(func):
    @functools.wraps(func)
    def wrapper(*args, **kwargs):
        """Wrapper function."""
        print("Calling decorated function...")
        return func(*args, **kwargs)
    return wrapper

@my_decorator
def example_function():
    """This is an example function."""
    print("Example function is running!")

example_function()
print(example_function.__name__)  # Output: example_function
print(example_function.__doc__)  # Output: This is an example function.

Calling decorated function...
Example function is running!
example_function
This is an example function.

In this example, we see that the metadata of the decorated function is preserved, namely the name (__name__) and the docstring (__doc__).

13.1.5. Class-Based Decorators¶

Decorators can also be implemented as classes.

[9]:

class MyDecorator:
    def __init__(self, func):
        self.func = func

    def __call__(self, *args, **kwargs):
        print("Before the function call.")
        result = self.func(*args, **kwargs)
        print("After the function call.")
        return result

@MyDecorator
def say_hello():
    print("Hello!")

say_hello()

Before the function call.
Hello!
After the function call.

Here, the __call__ method makes the class instance callable, so it acts like a decorator. In this case, you don’t need to define a wrapper function inside the decorator.

13.1.6. Chaining Multiple Decorators¶

You can apply multiple decorators to a function.

[10]:

def uppercase(func):
    def wrapper(*args, **kwargs):
        result = func(*args, **kwargs)
        return result.upper()
    return wrapper

def exclaim(func):
    def wrapper(*args, **kwargs):
        result = func(*args, **kwargs)
        return result + "!"
    return wrapper

@exclaim
@uppercase
def greet(name):
    return f"hello, {name}"

print(greet("Boris"))

HELLO, BORIS!

@uppercase is applied first, transforming the string to uppercase.
@exclaim is applied next, adding an exclamation mark.

13.1.7. Real-World Example:¶

13.1.7.1. Timing a Function¶

[11]:

import time

def timing_decorator(func):
    def wrapper(*args, **kwargs):
        start_time = time.time()
        result = func(*args, **kwargs)
        end_time = time.time()
        print(f"Execution time: {end_time - start_time:.4f} seconds")
        return result
    return wrapper

@timing_decorator
def slow_function():
    time.sleep(2)
    print("Finished!")



@timing_decorator
def fast_function():
    time.sleep(0.1)
    print("Finished!")

slow_function()
fast_function()

Finished!
Execution time: 2.0050 seconds
Finished!
Execution time: 0.1015 seconds

In this example we don’t need to re-implement the timing logic, thanks to the decorator.

13.1.7.2. Vectorizing a Function¶

If you have a regular function and want to apply np.vectorize, you can explicitly wrap the function with np.vectorize.

[15]:

import numpy as np

# Define a regular function
def my_function(x):
    return x ** 2 if x > 0 else 0

# Vectorize it
vectorized_function = np.vectorize(my_function)

# Apply it to a NumPy array
data = np.array([-2, -1, 0, 1, 2])
result = vectorized_function(data)
print(result)  # Output: [0 0 0 1 4]

[0 0 0 1 4]

[14]:

# my_function(data)

Using np.vectorize as a decorator simplifies the process and automatically transforms the function for vectorized operations.

[16]:

import numpy as np

# Use np.vectorize as a decorator
@np.vectorize
def my_function(x):
    return x ** 2 if x > 0 else 0

# Apply it to a NumPy array
data = np.array([-2, -1, 0, 1, 2])
result = my_function(data)
print(result)  # Output: [0 0 0 1 4]

[0 0 0 1 4]

Here, the @np.vectorize decorator is applied directly to my_function, so you don’t need to explicitly wrap it later.

To recap:

Without Decorator:
- You have to manually apply np.vectorize to the function.
- More verbose.
With Decorator:
- Cleaner and more readable.
- Automatically makes the function vectorized when defined.

np.vectorize is useful for extending scalar functions to arrays when you don’t want to manually iterate over elements.

However, it doesn’t provide true performance benefits like NumPy’s ufuncs (universal functions), as it’s just a convenience for element-wise operations. If possible, prefer writing functions that natively work with arrays for better performance.

[19]:

np.sqrt(np.abs(data))

[19]:

array([1.41421356, 1.        , 0.        , 1.        , 1.41421356])

[20]:

np.abs(data)**0.5

[20]:

array([1.41421356, 1.        , 0.        , 1.        , 1.41421356])

13.2. Universal Functions (ufuncs)¶

A universal function (or ufunc for short) is a function that operates on ndarrays in an element-by-element fashion, supporting array broadcasting, type casting, and several other standard features. That is, a ufunc is a “vectorized” wrapper for a function that takes a fixed number of specific inputs and produces a fixed number of specific outputs. (see here)

For example:

[21]:

import numpy as np

# Element-wise addition
arr1 = np.array([1, 2, 3])
arr2 = np.array([4, 5, 6])

result = arr1 + arr2  # Uses ufunc np.add
print(result)  # Output: [5 7 9]

[5 7 9]

Square-root, and trigonometric functions in numpy are also ufuncs:

[22]:

arr = np.array([1, 4, 9, 16])

# Square root
sqrt_result = np.sqrt(arr)
print(sqrt_result)  # Output: [1. 2. 3. 4.]

# Trigonometric functions
angles = np.array([0, np.pi / 2, np.pi])
sin_result = np.sin(angles)
print(sin_result)  # Output: [0. 1. 0.]

[1. 2. 3. 4.]
[0.0000000e+00 1.0000000e+00 1.2246468e-16]

You can create a custom ufunc from a Python function as well.

[23]:

def custom_add(x, y):
    return x + y

# Create ufunc
custom_add_ufunc = np.frompyfunc(custom_add, 2, 1)

result = custom_add_ufunc([1, 2, 3], [4, 5, 6])
print(result)  # Output: [5 7 9]

[5 7 9]

Here is an example of the huge performance benefits of ufuncs:

[24]:

import numpy as np
import time

# Data
arr = np.arange(1e6)

# Using ufunc
start = time.time()
result_ufunc = np.sqrt(arr)
end = time.time()
print(f"ufunc time: {end - start:.6f} seconds")

# Using np.vectorize
def scalar_sqrt(x):
    return x ** 0.5

vectorized_sqrt = np.vectorize(scalar_sqrt)
start = time.time()
result_vectorized = vectorized_sqrt(arr)
end = time.time()
print(f"np.vectorize time: {end - start:.6f} seconds")

ufunc time: 0.007639 seconds
np.vectorize time: 0.117718 seconds

In this example, the ufunc version is 100 times faster than the vectorized version. The vectorized version actually iterates over the elements of the array, while the ufunc version is implemented in C under the hood and is highly optimized.

13.3. Logging¶

Logging is used to track events in a program and helps debug or monitor it without relying on print statements.

It’s better because it categorizes messages by severity levels (e.g., INFO, WARNING, ERROR) and can output logs to different destinations (e.g., files).

13.3.1. Basic Setup¶

[25]:

import logging

# Configure basic logging
logging.basicConfig(level=logging.INFO, format='%(levelname)s: %(message)s')

# Example: A simple computation
def compute_square(x):
    if x < 0:
        logging.warning("Received a negative number. Returning zero.")
        return 0
    result = x ** 2
    logging.info(f"Computed square of {x}: {result}")
    return result

# Call the function
compute_square(5)
compute_square(-3)

INFO: Computed square of 5: 25
WARNING: Received a negative number. Returning zero.

[25]:

Log Levels Used:
- INFO: Logs useful computation results.
- WARNING: Logs when unexpected input is received.
Why Use Logging?
- It separates debugging messages from your main program output.
- It’s easy to expand (e.g., write to files, add timestamps).

13.3.2. Activating and Deactivating Logging¶

Sometimes, you may want to turn logging on or off depending on the situation (e.g., during debugging or production).

How to Deactivate Logging?

To deactivate logging, set the logging level to logging.CRITICAL. Since this is the highest level, only critical errors will be logged, effectively “silencing” other logs.

[26]:

import logging

# Configure basic logging
logging.basicConfig(level=logging.INFO, format='%(levelname)s: %(message)s')

# Example function with logging
def compute_square(x):
    if x < 0:
        logging.warning("Received a negative number. Returning zero.")
        return 0
    result = x ** 2
    logging.info(f"Computed square of {x}: {result}")
    return result

# Activate logging (default level INFO)
compute_square(5)  # Logs: INFO: Computed square of 5: 25
compute_square(-3)  # Logs: WARNING: Received a negative number. Returning zero.

INFO: Computed square of 5: 25
WARNING: Received a negative number. Returning zero.

[26]:

[27]:

# Deactivate logging
logging.getLogger().setLevel(logging.CRITICAL)

compute_square(5)  # No logs
compute_square(-3)  # No logs

[27]:

How to Reactivate Logging?

[28]:

logging.getLogger().setLevel(logging.INFO)

compute_square(5)  # Logs: INFO: Computed square of 5: 25
compute_square(-3)  # Logs: WARNING: Received a negative number. Returning zero.

INFO: Computed square of 5: 25
WARNING: Received a negative number. Returning zero.

[28]:

This allows you to easily toggle logging without removing or modifying your logging statements.

13.4. Global and Local Variables¶

Variables in python can have global or local scope.

Global variables are accessible everywhere.

Local variables are only accessible in the function they are defined in.

Variables created in for, while and if statements are global, so available outside the loop (as are the iterators, e.g. i in for i in range(10)).

[31]:

for i in range(3):
    j=100
print(j)
print(i)
i=0

while i<2:
    j=101
    i=5
print(j)

if (True):
    j=102
print(j)