inlined comprehension and closed over variables have buggy interactions

[cfbolz]

Bug report

Bug description:

We are a bit late to report this, but the following of PyPy's unit tests got broken by inlined comprehensions in CPython 3.12:

def test_inlined_comprehension_private_name_reuse():
    x = 3

    def f():
        [x for x in [1]]
        return [x for _ in [1]]

    assert f() == [3]

test_inlined_comprehension_private_name_reuse()

this passes fine in CPython 3.11 but breaks like this in 3.12 onwards, including on main:

$ ./python /tmp/bug.py
Traceback (most recent call last):
  File "/tmp/bug.py", line 10, in <module>
    test_inlined_comprehension_private_name_reuse()
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^
  File "/tmp/bug.py", line 8, in test_inlined_comprehension_private_name_reuse
    assert f() == [3]
           ~^^
  File "/tmp/bug.py", line 6, in f
    return [x for _ in [1]]
            ^
UnboundLocalError: cannot access local variable 'x' where it is not associated with a value

I wonder whether that is the kind of bug that would be easier to fix when doing #124697

CPython versions tested on:

CPython main branch

Operating systems tested on:

No response

Linked PRs

• gh-156664: Fix interaction between free variables and comprehensions #156691

[iritkatriel]

CC @carljm

[JelleZijlstra]

Surprised this has never been caught before! I even ran a fuzzer when we were implementing PEP 709 to find things like this. I'm attempting a fix now.

[JelleZijlstra]

An analogous case that is also broken in 3.12+:

def test_inlined_comprehension_then_nested_function():
    x = 3

    def f():
        [x for x in [1]]

        def g():
            return x

        return g()

    assert f() == 3


test_inlined_comprehension_then_nested_function()

And a more complicated version:

def test_captured_target_then_sibling_comprehension():
    x = 3

    def f():
        funcs = [lambda: x for x in [1]]
        return funcs[0](), [x for _ in [1]]

    assert f() == (1, [3])


test_captured_target_then_sibling_comprehension()