Closures And Objects Are Equivalent

The venerable master Qc Na was walking with his student, Anton. Hoping to prompt the master into a discussion, Anton said "Master, I have heard that objects are a very good thing - is this true?" Qc Na looked pityingly at his student and replied, "Foolish pupil - objects are merely a poor man's closures."

Chastised, Anton took his leave from his master and returned to his cell, intent on studying closures. He carefully read the entire "Lambda: The Ultimate..." series of papers and its cousins, and implemented a small Scheme interpreter with a closure-based object system. He learned much, and looked forward to informing his master of his progress.

On his next walk with Qc Na, Anton attempted to impress his master by saying "Master, I have diligently studied the matter, and now understand that objects are truly a poor man's closures." Qc Na responded by hitting Anton with his stick, saying "When will you learn? Closures are a poor man's object." At that moment, Anton became enlightened.

From: people.csail.mit.edu

Poll: Are objects and closures equivalent?

Aye's:

(vote here)

Nays:

Winheim Raulsh - closures are immutable, have no methods, receive no messages, and don't provide the equivalent of inheritance and polymorphism on their own.

[Eric Tetz - a closure *is* mutable (it closes over mutable state, so it's mutable to the same extent that a class instance is) and it's callable (it can receive an "execute" message) so it's like a class instance with one public method]

Mark Janssen - they are equivalent as things, static in themselves, but not in actual usage in anyway whatsoever. See Computer Science Version Two.

(vote here)

It is well-known that objects and closures over lexical scopes (Lexical Closures) are equivalent things; which one you prefer is a matter of taste and background. (Actors are also equivalent to both; see Actors Model.) Those with a strong functional background (in particular, Lisp/Scheme) tend to prefer closures; and many object systems in Lisp Language and Scheme Language (but not including Common Lisp Object System, which uses method polymorphism on pure data objects) use closures to implement objects.

Structure And Interpretation Of Computer Programs gives a [mitpress.mit.edu nice sample for this type of object system].

It is not well-known that objects and closures are equivalent. On the contrary, what is "well-known" is what Luca Cardelli presents in the 18th chapter of Theory Of Objects, precisely that encoding object oriented features in lambda calculi is unwieldly and cumbersome, especially when it comes to typed version of those calculi. This page is further evidence as it presents only objects with one method and no inheritance.

Using Pattern Matching, supporting more than one method is no more difficult than supporting a single method. Delegation Is Inheritance, and delegation can be supported straightforwardly by adding a self parameter to calls.

In general, emulating object features in a statically typed functional language with closures (and even with more advanced features like modules), for example Standard Ml, is at least cumbersome if not impossible. There are object systems written for Lisp and Scheme, some of them using macros (CLOS), some in functional style without macros, however I don't think there's any particular object system with all the major features (polymorphism, inheritance, etc) written without macros. Nor there should be since object systems with macros look easier to write and comprehend.

What exactly is the problem with macros? They are used to give you a nicer syntax, which is a Good Idea IMHO.

[Well it is well known that an object is one particular form of a closure, and a closure can be used as an object, which they often are, and is what the above paragraph is trying to convey. Sure, some languages give us a nice clean little package that wraps up polymorphism and inheritance into a thing called class, but one doesn't need either polymorphism or inheritance to have an object. Polymorphism and inheritance are used to specialize objects, closures are used to create anonymous one-shot objects that don't need specialization, they are complimentary abstractions. You are setting up a straw man here, polymorphism and inheritance are not required to build objects, polymorphism and inheritance may be required to call it object oriented, but not object based. No claim was made that Closures = OO programming, just that objects and closures are equivalent, which is true.]

Please make up your mind: either closures and objects complement each other, or they are equivalent. You can't have them both. If objects can be perfectly emulated by closures, than objects would be unnecessary entities and unnecessary entities do not complement anything. To complement means to add something essential in order to make up a whole, bring to perfection, supplant, etc. With regards to polymorphism and inheritance, they are essential to objects insofar that high level programming languages are essential to programming (of course we can program in assembly and that would still be called programming).

[You can have both, don't be so closed minded. Objects wrap up polymorphism and inheritance in an easy to use fashion that would be more verbose with closures, however they don't give you anything special, just a shorter syntax. Objects are necessary for domain entities and such, well-defined objects so to say. Closures are necessary for one-shot objects that aren't well defined, and are different just about every time, but essential none the less. Polymorphism and inheritance are not essential to build an object, that is if you define an object as a thing with state and identity, which closures have.]

I don't think equivalence and complementariness (better word for that?) are mutually exclusive. You can use one to implement the other - that is sufficient to demonstrate equivalence.

More precisely, equivalence means that two concepts can simulate each other without requiring a global transformation on programs. Otherwise we get caught in the Turing Trap.

But that doesn't make one or the other redundant; having both is nice as each can be used to solve different problems. Furthermore, I can simulate either with a rats-nest of pointers and macros, but that doesn'e mean we should abandon both high-level concepts and go back to programming everything in Cee Language.

The fact that you can use one to implement another do not an equivalence make. The same good old argument that says that Java and assembly are not equivalent programming languages, in spite of the fact all computations that can be implemented in one can be implemented in the other, and you can use Java to implement an assembly interpreter or you can use assembly to implement a Java (bytecode) interpreter.

The page as it stands now gives a false advice in the sense that if the claim was true, than it would be easier to program in Object Oriented style in a language like Standard Ml, but it isn't. Neither functional programming subsumes object oriented programming, nor the other way around. It is beneficial for most programmers to know both styles.

Agreed

Many modern OO languages, however, don't have full closures. Some languages have something similar (Java Inner Classes); others don't bother at all. It has been argued (Global Variables Considered Harmful) that full lexical closures aren't always a good thing - they dramatically increase coupling between modules (as the closure has hooks into the guts of the enclosing scope), and they complicate a language's implementation. (See also What Is Closure for more on that).

For many of the purposes (not all) that a Lisp user might use a closure - an OO programmer would use an object - and an object is often a perfectly good substitute.

Wait. Hold on. What exactly here makes a "closure" have to be functional? You can have a "closure" also be an object. Java's anonymous inner classes close over the final variables in scope. [[Ee Language's object definitions are closures, which allows it to avoid the concept of a "class" without using prototypes and yet still having a central module used to spawn new objects of a given type. Yet, neither of those are exactly "lambdas". Yes, they could arguably be defined as Hash Maps of lambdas, but the point still stands: closures can be defined as anything which can capture lexical variables and take them into other stack frames. They don't actually have to be lambdas.

Of course, most people think of them as lambdas. I know what a "closure" implies. Still, Food For Thought.]]

[Moved from Lexical Scoping]

In many cases, a bunch of related functions defined within a single outer function (and accessing - incestuously - it's variables) are parts of a single concept - the functions are all tightly coupled and using explicit parameters would become a nightmare (giving you a bunch of functions with dozens of parameters to pass around). Whenever I see this, I see an object waiting to be born. Refactor the whole mess into a class. If you are in a non-OO language; find the union of all the shared state and make it a struct/record - and pass that around (by reference). In both cases, you get the ability to make lots of instances of your abstraction (and to access the inner methods without calling the outermost method first). In the OO case, you get the additional benefits of OO - Poly Morphism and such.

This is (or is very similar to) a Method Object.

Umm, this is not really the common case for closures. You see examples like that, but they're largely academic. Indeed in 90% of the cases where you'd see that pattern, it'd be wiser to use an Object in Lisp. Especially since CLOS provides such an incredible system. Most of the time we're using closures in lambda-passing functions. It is far superior to Java inner classes or C++ functors for this. As an example:

While trivial and academic, it demonstrates why lisp and scheme programmers like function-passing style with closures so much. The standard combinators like the mapping functions and sorting functions can take the surrounding context into account. Also, closures allow for awesome control-flow constructs like the lisp condition handling system. Heck, people have made a limited kind of continuation just with "goto" and closures (but I am not saying that I don't envy Scheme's real continuations).

Closures and Objects are closely related, but closures are more general. You can't really use objects to implement closures unless your object system is actually doing closures under the covers. You can implement objects with closures by linking methods across a set of variables (note in lisp this practice is considered archaic and generic functions are instead used to handle this).

Maybe I just need another cup of coffee, but my brain is boggling on this phrase: "linking methods across a set of variables", could you help me out on its meaning?

I was unclear. You can make objects (without any kind of dispatch) by creating a closure over a set of variables (data portion) and defining a list of lambdas (code portion) to create rudimentary objects. Adding inheritance and polymorphism from there really isn't that hard. As I said though, these kinds of object systems are unpopular in Lisp and Scheme today, because of the prevailing style in Lisp. Also, some people claim the message-passing system (which people also note is "specialization on only one argument") which most systems base themselves off of is inherently inferior compared to the pure multimethods approach that CLOS favors. I personally am on the fence on this issue.

My point though, is that closures and objects aren't really equivalent. Objects are a subset of closures, so they carry some similar traits.

Note: objects and closures over lexical scopes are equivalent. Idiomatic style of programming in a language that allows nested function definitions takes advantage of this fact, rather than being confounded by it. E.g. a function in the Scheme Language can return one or more local functions that can use variables in the defining scope, and that are therefore fully encapsulated.

I'm not sure if I'm grokking the level of equivalence of the above example. It shows me how a single method object is equivalent to an closure, but I'm not seeing how this extends to a multiple-method object (does it?). Could somebody provide a translation of the following into closures:

My scheme skills are a bit weak, but try this:

make-multiplier returns a function that takes a method selector, and then returns the appropriate method. The method can then be called normally.

This is probably not idiomatic Scheme though, which would look more like this:

This time, make-multiplier returns a list of 3 functions: the methods in the class. A production scheme system would probably use a hashtable, alist, or record, some data structure where the methods are not positionally dependent. In any case, there're accessor functions that return the correct lambda, so that the client programmer doesn't need to worry about the actual data structure returned.

You can get polymorphism too, by replacing the functions within the data structure returned by the constructor, but subclass methods won't be able to access superclass data unless the subclass is lexically enclosed by the superclass. This is annoying, because any moderately deep inheritance hierarchy becomes virtually impossible to read. Granted, most moderately deep inheritance hierarchies in traditional languages aren't very easy to follow either, if the subclasses use large amounts of data in the superclass.

An approach which might match the original request better:

This code is a variant of the closure-object idiom demonstrated in Structure And Interpretation Of Computer Programs, chapter 3, which is widely known among Schemers; it has been tested under Dr Scheme and works as designed. While objects could be built manually this way, it would make more sense to define a set of forms to automatically generate the class form, which IIUC is how SOS works. HTH. - Jay Osako

It's been claimed, on Smalltalk Instead Of Python, that smalltalk is preferable because it has real closures, where Python only has one-shot lambda functions. But if you want to encapsulate a structured state, why not use an object? Perl gives you closures ... but the only use I've seen for them is to hide the data members of an object ... what else can you do with 'em?

Well, for one you can write an object system with them. This is a popular exercise in Scheme and at least one of the object systems in SLIB requires no macros.

A class is simply one form of a more general concept, a closure. In other words, classes are closures... with names, but not all closures need names. The use of closures goes beyond the class concept, and is more general and more powerful. Object systems are written with closures. Closures allow one shot specialized objects, and are far more general than classes.

In Smalltalk they are an enormous convenience. You could create a class with a value method that does just that one computation, but this is a lot of work for little or no gain, and you can't do ifTrue: and while: and such very easily at all. In Python they are unnecessary but still an enormous convenience, and their lack is unfortunate.

Can you give an ur-python snippet that would illustrate what could be done with them if they were in the language?

If closures were added to Python tomorrow, I could do this:'

This is a very easy xrange equivalent, if Iterator is a simple class that just handles __getitem__ and the usual Python iteration protocol. It is harder to do in stock Python and as a result people don't do it; I know of no equivalent in the standard libraries save xrange.

Maybe the above message is old, but closures exist in Python today. Only the example you given above doesn't work because Python doesn't distinguish between "let" and "setf"; It creates a new "let" block for i within temp, making it different from the i in iterator. The way around this is to uglify the code slightly with an array, which changes "i = i + 1" from assignment to a modification of an object.

You don't need closures for this:

Locking can also be implemented without closures:

>>> def func():
print "... Now in func!"

Though this still doesn't seem like something I'd use, I see no difficulty. It can be done readily with python's builtin compile and eval:

You might leave locals() out for the most part. I haven't tried to make this work, but it seems straightforward. Do Smalltalk closures do something more magical than this?

Yes. For one thing, they don't make deep copies of the variables referred to:

or, in the hypothetical Python-with-closures:

actually works. For another thing they're much more efficient and transparent; they only refer to some locals, and you needn't do the full setup for so called downward-funargs, etc. The current Python Way of doing this is to create an object:

While certainly feasible this is by no stretch of the imagination convenient, particularly for short iteration methods.

Okay, I tried the Closure class above and had to ditch the copying because modules wouldn't let themselves be copied. No matter, any relevant state can be stuck inside the statements string.

So then you can do all this with map:

The trick is that arg and kword get stuck into locals(). It seems possible to go into all kinds of conniptions over this - see starship.python.net - but I'm still not getting what smalltalk does better than the little idiom here.

Well, Smalltalk works for one :) But try this:

Although the original now works, you're right, this foo(list) doesn't work. sum and prod were going into globals, not locals. Unfortunately (?) locals() returns a read-only dictionary, which means Closure can't affect it. You can still do it in a horrible way with globals ...

Don't lose hope entirely, though. locals() is mutable in Java Python, so foo(list) should work fine there.

The problems with Closure:

for one:
whitespace is important in Python, and sticking it in weird places and trying to exec the results has problems.

That one's not a big deal once the idiom is established.

For two:
syntax errors will not be discovered until run time. Umm, python being a language which typically compiles just-in-time, would that really make much difference?

So what? Code Unit Test First.

For three there is no way I can recall to get `what are the local variables in my caller's frame?' without going down to the bytecode level (bytecodehacks), so it can't actually work.

Actually, there is. locals(). To make this look cute we should capture locals() without explicitly passing it in ... maybe we need to choose between the lesser of two evals? Ugh.

Bytecodehacks attempts closures, but has problems; you have to wrap all the local variables in lists if you expect to change them, it's very low level and in fact contradicts the reference, if not the reference implementation, and probably won't work at all for JPython.

Yes, agreed, the bytecodehacks stuff is more a suggestion to Guido Van Rossum and his crew, not intended useful to regular python programmers except under very special circumstances.

Since you can dodge the issue by using an auxiliary object, this is a flaw, not an immense gaping hole, but it's still irritating.

Just curious: using this idiom, can you do a non-local return?

Can you be a little more explicit?

Sure. Non-local return is basically the equivalent of longjump in C. Take for example, this Smalltalk block:

The semantics of non-local return (the ^ symbol) is that instead of returning from the method the block is evaluated in, it returns from the method the block is defined in.

Here's an example:

The implementation is Dictionary at:ifAbsent: looks like this:

at:
key ifAbsent: aBlock

^self contents lookUpValue:
key for: self ifAbsent: aBlock

the contents of the dictionary is actually a hash table, and its implementation of #lookUpValue:for:ifAbsent: looks like this:

lookUpValue:
key for: client ifAbsent: aBlock

The bold line will evaluate the block from waaaay back up in Block Example>foo and cause control to return to the sender of #foo, not the sender of #lookUpValue:for:ifAbsent:.

Can you do this in Python with closures?

The trick above is merely an attempt to do closures by glomming the local and global dictionaries. Stack frames don't seem so easy to pick up ... though maybe rooting around in the IDE traceback could show how to grab them. I seem to recall, however, that reusing them is very unsafe.

So instead you really ought to check out the new Stackless Python with its continuations, which I think are exactly what you're looking for. These should be standard pretty soon (1.6?). Or use python's try/throw exception handling. Does smalltalk have try/throw?

Smalltalk does have exceptions. Smalltalk also has non-local returns. They are useful for different things, and in fact quite a few languages have both: Common Lisp has RETURN-FROM, which uses lexical scoping to decide where to return from, as well as the condition system. It is arguable that non-local returns are easier to understand than using exceptions for non-exceptional purposes, just because both sides must necessarily be in the same textual location.

Combined with map, closures would provide a powerful tool that augments but doesn't compete with objects. The Python Closure class we got above is syntactically and semantically crippled by the current python implementation. The Continuations in Stackless Python, however, seem like they could provide the equivalent of Smalltalk closures. They already do Co Routines and much else.

Pardon a doubting Thomas, but how does Stackless Python help the scope issues?

Good question. Dang, now I gotta go actually understand these continuation things ...

I wrote what I understand of Continuations And Coroutines on the Stackless Python page -- Shae Erisson

Good job, Shae. Now to use a continuation as a closure we need a way to get data into it ... if those Co Routines could message each other maybe ... ?

TCL is actually quite suitable for smalltalk-style blocks; uplevel and upvar can be used to get the scoping right, and non-local returns work as well. (This makes a certain amount of sense, tcl and smalltalk both use unevaluated blocks to generate control structures) I don't believe python is as plastic, but some of the examples above are just plain silly. [I am only an egg. -- PM] Here are some rewrites in a style that doesn't go against the Grain Of The Language:

Note that the above code can be more simply written as:

or, ideally, you would go for absolute simplicity:

Opinion: Closure ws. Class is not an issue for large code blocks; the size/effort difference in defining them pales in comparison to what you're really trying to accomplish. If you need a process/alogorithm object for some nontrivial purpose, it really won't hurt to go ahead and write it. It goes with the grain of, eg., Python Language.

On the other hand, for trivial bits you have Lambda Forms, List Comprehensions, and functions/methods as first-class, accessable objects that you can pass around and call directly.

Lastly, you should implement the iterator protocol for interesting collections.

Here's the classic example of sum-and-product, with an internal iterator:

(sum, prod) = Internal Iterator((lambda (s, p), x:
(s+x, p*x)), (0, 1))

Tested on python 2.2 by Ian Kjos

Blocks In Java shows how one can fake closures in a language that doesn't have them. The idea is to construct an "expression object" that acts like the desired closure. Effectively, the constructors for expression objects form a small sub-language towrite closures in. In principle, a pre-processor to the real compiler could translate "real" closures into such an expression sub-language. This shows how closures can be implemented without back-end support.

-- Stephan Houben

The Pizza Language extends Java Language by adding First Class functions, and so encourages a functional style of programming. Much more so than the anonymous classes of Java.

I just don't understand why first class functions have to be unrelated to objects. A function is just a stateless Value Object, after all. Why would melding functions and objects be problematic? I don't see why they they could not participate in inheritance, etc.

-- Michael Feathers (or am I missing something?)

You're absolutely right. Furthermore, if a function is a closure, it may have a state. Closures thus completely implement objects, things with an identity and state that can reply to messages. In 1992 Ken Dickey wrote a fascinating paper Scheming With Objects, which starts as follows:

There is a saying - attributed to Norman Adams - that 'Objects are a poor man's closures.' In this article we discuss what closures are and how objects and closures are related, show code samples to make these abstract ideas concrete, and implement a Scheme Object System which solves the problems we uncover along the way.

One might think that Referential Transparency (e.g., lack of mutation) may preclude objects with a mutable state. Fortunately, this is not the case. For one thing, we can use general, proven techniques - monads and unique types. There are simpler approaches, for example Oleg Kiselyov's purely functional object-oriented system (pobox.com ). It shows a fully referentially transparent OO system: objects with a distinct state and identity, inheritance and polymorphism -- and not a single assignment. This system shares the drawbacks of OOP as well.

I have used objects to implement closures in languages where they were unavailable. I have use closures to implement objects in languages where they were unavailable. QED. -- Bottom Mind

Category Closure

See original on c2.com