Memory, generators & performance

How does Python manage memory?

Two mechanisms. Reference counting frees an object the moment its count hits zero (deterministic, immediate). A generational cycle collector then handles reference cycles that refcounting alone can never free — three generations, scanned with decreasing frequency, based on the observation that most objects die young.

import sys, gc
a = []; sys.getrefcount(a)      # 2 — one is the temporary argument reference
a.append(a)                     # a cycle; refcount never reaches 0
del a                           # only gc.collect() can reclaim it

Things worth adding: gc.freeze() before forking keeps long-lived objects out of the scanned set (a real win for pre-fork servers like Gunicorn, since the collector's writes would otherwise unshare copy-on-write pages). Objects with __del__ in cycles were uncollectable before 3.4; they are collectable now, but __del__ is still a bad place for cleanup — use a context manager.

Generator vs list: when does it actually matter?

A list materialises everything; a generator yields one item at a time in O(1) memory, and is lazy so you can pipeline over infinite streams.

sum(x * x for x in range(10_000_000))     # constant memory
sum([x * x for x in range(10_000_000)])   # ~400 MB list first

def read_rows(path):                      # streaming pipeline
    with open(path) as f:
        for line in f:                    # file objects are already lazy
            yield json.loads(line)

rows   = read_rows("events.ndjson")
active = (r for r in rows if r["active"])
emails = (r["email"] for r in active)     # nothing has been read from disk yet

Costs to acknowledge: a generator is single-pass, has no len(), cannot be indexed, and if an exception occurs mid-iteration the pipeline's state is gone. Do not return a generator from a public API where callers reasonably expect a list — or document it loudly.

Explain yield from, and write a generator that consumes values.

yield from sub() delegates iteration and forwards send/throw/return — it is not sugar for a loop.

def batched(iterable, n):
    batch = []
    for item in iterable:
        batch.append(item)
        if len(batch) == n:
            yield batch
            batch = []
    if batch:
        yield batch          # don't drop the partial final batch — classic off-by-one bug

Generators are also consumers via send(), which is the basis of coroutines:

def averager():
    total = count = 0
    while True:
        x = yield (total / count if count else None)
        total += x; count += 1

avg = averager(); next(avg)      # prime it
avg.send(10); avg.send(20)       # 15.0

(itertools.batched exists in 3.12+; knowing the manual version still matters.)

How do you find why Python code is slow?

Measure before optimising, and use the right granularity:

1. time.perf_counter for a rough answer; timeit for micro-benchmarks (it disables GC and repeats).
2. cProfile + pstats or python -m cProfile -s cumtime app.py for function-level cost. Sort by cumulative first to find the expensive subtree, then tottime for the hot function.
3. py-spy for production — it samples a running process with zero code changes and no restart (py-spy top --pid 1234, py-spy dump for stuck processes). Naming py-spy is a strong signal.
4. tracemalloc or memray for memory: tracemalloc.take_snapshot().compare_to(earlier, "lineno") finds leaks by line.
5. line_profiler (@profile) when you have narrowed it to one function.

Then the fixes, in the order you should try them: fix the algorithm (O(n²) → O(n) with a dict/set), remove I/O from the loop (batch queries — this is usually an N+1), use built-ins and str.join/comprehensions instead of manual loops, cache with functools.lru_cache, vectorise with NumPy/Polars, and only then reach for C, Cython or Rust (PyO3).

Why is += on strings in a loop bad, and how fast is in on a list vs a set?

Strings are immutable, so s += x allocates and copies each time → O(n²). Use "".join(parts) or io.StringIO. (CPython has an in-place optimisation that sometimes hides this — do not rely on it.)

x in list is O(n); x in set/dict is O(1) average. Turning a repeated membership test against a list into a set lookup is the single most common real speed-up in review:

banned = set(load_banned())        # once
[u for u in users if u.email not in banned]     # O(n), not O(n*m)

Know the table: list append O(1) amortised, list insert(0)/pop(0) O(n) — use collections.deque for a queue; dict/set lookup O(1); heapq push/pop O(log n); sort O(n log n) with Timsort, which is stable and near-linear on partly-sorted data.

What does functools.lru_cache do and when is it dangerous?

It memoises on the arguments (which must be hashable) with an LRU eviction bound. @cache (3.9+) is lru_cache(maxsize=None) — unbounded, so a genuine leak if the key space is large.

Dangerous when: the function is not pure (caches stale DB reads), arguments are mutable or huge, or you decorate a method — the cache holds self, so instances are never freed. For methods, use cached_property, or cache a module-level function taking only the id.

How do you handle data too large for memory?

Stream it. Read files line by line or in fixed chunks; use pandas.read_csv(chunksize=...) or Polars' lazy frames / DuckDB, which push work to a query engine; use server-side cursors for databases (yield_per in SQLAlchemy, .iterator() in Django) so the driver does not buffer the whole result; use mmap for random access into large binaries; and use array/NumPy dtypes instead of lists of Python ints (a Python int is ~28 bytes plus a pointer; a NumPy int64 is 8).

What is the difference between __iter__ and __getitem__ for iteration?

iter(obj) first tries __iter__. Failing that it falls back to the old protocol: call __getitem__(0), (1), … until IndexError. So a class with only __getitem__ is still iterable — a small piece of trivia that occasionally explains surprising behaviour in legacy code. Implement __iter__ (usually as a generator) in new code.

Name three quick wins you would apply to a slow Python web endpoint.

1. Fix the N+1 query — almost always the real cause. Confirm with query logging or APM, fix with select_related/joinedload/a batched IN query.
2. Add a cache with a sane key and TTL — response cache or per-object cache in Redis, plus Cache-Control so the CDN can help.
3. Move non-essential work off the request path — emails, webhooks, thumbnails, analytics go to Celery/a queue; the endpoint returns as soon as the durable write commits.

Then: pagination with keyset instead of OFFSET, gzip/brotli, and connection pooling. Notice that none of these are "rewrite it in Rust" — that ordering is what the interviewer is checking.