Profiling before rewriting

Beginner Accelerating numerics & developer hygiene

Created by Pavel · 29.04.2026 at 19:11 UTC

cProfile, py-spy, and IDE profilers show where time goes—function call counts and cumulative time. Blindly rewriting “slow Python” without a profile often optimises cold modules while the real cost is a hidden merge, repeated I/O, or an accidental Cartesian join.

In teams, profiling evidence also communicates priority: stakeholders understand “40% in this encoder” better than “code feels slow.”

Pair profiles with line profilers (line_profiler) for hot functions once candidates are known.

The profile and cProfile modules: [1].

Sources

[1]https://docs.python.org/3/library/profile.html Return to text

University approvals: 0

Tasks

Question 1

First sensible step when a training script is “too slow”:

Hint

Evidence before rewrites.

Rewrite everything in C++ immediately

Measure/profile to locate hotspots before changing algorithms

Disable logging

Increase batch size without measuring memory

Question 2

cProfile reports high cumulative time in pandas.core.reshape.merge. What does that suggest?

Hint

Follow the heavy function names.

You should vectorise a tiny pure-Python helper

A join/merge step may dominate—inspect shapes and keys before micro-optimising

NumPy is broken

Profiling is misleading for pandas

Question 3

CounterBox with .hits starting at 0; record() increments and returns new count.

Hint

Mirror profiler hit-count idea.

class CounterBox:
    def __init__(self):
        pass

    def record(self) -> int:
        pass

Starter code is prefilled; replace TODO blocks with your solution.

Runtime output (stdout/stderr)

2 test cases will be used for grading

Run checks runtime behavior only. Final correctness is evaluated when you submit.

Card Info

Topic: Accelerating numerics & developer hygiene
Difficulty: Beginner
Completed: 0 users

Creator

Pavel