Python closures and collections are broken

It is too bad that “lambda”, in Python, is so misleadingly named. It is even more regrettable that the scoping semantics of Python’s attempted closure mechanism are equally broken. Python’s “lambda” is a very thin syntactic veneer on top of Python’s local function behavior.

The specific problem is that the scope of a local function does not nest inside its containing scope(s). Here is an example:

class FooClass:
    def inject_into_(self, aValue, aTwoArgBlock):
        """This is a common idiom for enumerating members of a collection."""
        anInjectValue = aValue
        def localFunctionWithArg_(each):
            """This needs access to the locally-assigned anInjectValue."""
            anInjectValue = aTwoArgBlock.value_value_(anInjectValue, each)
        map(localFunctionWithArg_, self.tests())

The above code fails with an “unbound local” exception, because anInjectValue is not in scope of localFunctionWithArgument. The desired behavior is to iterate across the collection, inserting the results in the variable whose initial value is supplied by the caller.

This is an elegant and pervasive idiom that works in Lisp, Smalltalk, Ruby, Perl, and others. The fact that it doesn’t work in Python highlights the failing of Python’s broken closure scoping mechanism.

As a result, the Python programmer is forced to confront the bewildering (and therefore bug-generating) Python zoo of “map”, “filter”, list comprehensions, and so on. Note that “map” and “filter” are functions, not methods. The python “list”, “dictionary”, “set” and similar container classes provide different protocols for shared operations. A specific consequence is that applications of the “Composite” pattern become unnecessarily (Python) language-specific.

Python developers have been debating this failure for years — see

A Zeetix work-around will likely end up in Kernel.Core.Block, as well as in the clazzes of the Kernel.CLDT subassembly (the Zeetix collection clazzes).

I am writing at least stubs for tests in KernelTest.CoreTest and KernelTest.CLDTTest that exercise these failures and therefore document the desired Zeetix kernel behavior.


2 Responses to “Python closures and collections are broken”

  1. Mike Burns Says:

    List comprehensions and the built-in generators solve 80% of the most common problems. They may be less than ideal (though list comprehensions are powerful syntactic sugar over filter and map), but they get the job done much of the time.

    … and that’s why we put up with it.

  2. TestSuiteTestCase added to ZeeTestTest « ZeeForge Says:

    […] ZeeForge Stuff that works « Python closures and collections are broken […]

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: