Duck Typing and Protocols vs. Inheritance

Posted by Werner Schuster on Nov 30, 2007

Sections

Development,

Architecture & Design

Topics

Language ,

Object Oriented Design ,

Architecture ,

Ruby ,

Design ,

Programming

Tags

Languages ,

SmallTalk ,

Erlang

A question on the ruby-core list yielded a discussion about duck typing in Ruby :

According to irb,
>> StringIO.is_a?(IO)
>> => false

This seems illogical to me. Is this intentional? If so, why?

One answer to this:

Since StringIO doesn't use IO to base its implementation, it is not an IO.
Why does it matter? If you use standard duck typing, it shouldn't. Class hierarchy doesn't matter with duck typing. Classes are just an implementation detail with duck typing. You are best off not looking #is_a?, #respond_to?, etc. Just assume the object can do the methods you'll be using and use.

The original question (and some posts in the mailing list debate) seems to come from the confusion about the role of inheritance in OOP. StringIO and IO don't need to go back to the same superclass, because they don't share any commonalities except one: a set of methods they support.

This sounds like the basic idea of duck typing: instead of requiring an object to be of a certain type, it suffices that the object responds to a set of methods.

None of this is new or specific to Ruby. By replacing the word "methods" with the OOP term "messages", the above statement reads: "[...] it suffices that the object responds to a set of messages". This does sound a lot like the idea of a (network) protocol. After all, a system that understands and responds to messages such as "PUT", "GET", "POST", etc. sent to it in a certain manner, can be said to understand the HTTP protocol.

Let's go further with this idea: a client, say a web browser, only cares that a server it communicates with understands HTTP; it does not require the server to be of a certain type or make. After all: the web would have had a harder time growing, if, say, the Mosaic browser had been hardcoded  to require the server on the other end of the line to be NSCA's or  Netscape's HTTP servers. By ignoring the other end of the communication, and only requiring that a "GET" is answered in a certain way, both ends of the webs development (client, server) were able to evolve independently.

The similarity of this approach to protocols was clear to users of OOP languages long ago. Smalltalk and ObjectiveC, both dynamic OOP languages, have long used the term Protocol to refer to this concept.

The Protocol concept is certainly useful, if only to give a name to a particular set of messages. This also helps with clearing up the questions raised in the above mailing list post. What the StringIO and IO share is not common ancestry but a common Protocol.

The concept is certainly not limited to dynamic languages. On the more static side, Java made a crucial step by introducing interfaces, as a way of:

• naming a set of messages
• statically checking that a class actually implements them

While this was certainly possible with, say, C++, using Abstract Base Classes and Multiple Inheritance, Abstract Base Classes mix the concepts of Inheritance and Protocol.

Of course, as is often the case with static guarantees, interfaces only check the presence of a set of methods with certain signatures - it doesn't guarantee the behavior or even the returned values of the methods. It's not even guaranteed that the method is invokable - it's still prudent to watch out for Java's java.lang.UnsupportedOperationException when working with, say, Java's Collection API.

Interfaces do have their own share of problems, such as evolvability. Changes to an interface are breaking changes for all classes implementing it. Big projects, such as Eclipse, have guidelines for API evolution, which rule that an interface can't be changed once it's been published. This sometimes yields the advice to favor Abstract classes over interfaces, since Abstract classes can add new functions, but define empty implementations, so subclasses can, but don't have to implement them. This problem could, of course, be solved by making fine grained interfaces, with one method each, and building the full interface up with new interfaces that extend the previous ones, each adding a single API call.

The problem of ensuring that an object supports a Protocol  could of course be solved in Ruby by delegating the interface check to a predicate function. An example:

def supports_api?(obj)
 obj.respond_to?(:length) && obj.respond_to?(:at)
end 

While this works, it's also fraught with subtle problems - respond_to? only works with methods defined in a class. If an object implements parts of its interface with method_missing (e.g. a Proxy), respond_to? would return false to these methods, although a call to these methods would work.  This means that respond_to? checking can work, but it's not a drop-in replacement for static checks (Rick de Natale refers to this coding style as "Chicken Typing", i.e. it's a kind of defensive programming).

Another, static, approach to fine grained interfaces are Scala's Structural Types:

def setElementText(element : {def setText(text : String)}, text : String) = 
{
  element.setText(text.trim()
   .replaceAll("\n","")
   .replaceAll("\t","")) 
} 

This function requires the argument element to have a setText(text:String) method, but does not care about the particular type of the object.

The focus on specific types or class hierachies of an object also limits the flexibility. An interface that requires an int parameter can only be called with an int or - through AutoBoxing - an Integer, but nothing else. A method that just requires the object to have an to_i method (that returns an integer number, either Fixnum or BigNum) is more flexible in the long run. Classes not under the developers control might not inherit from the needed classes, although they might still support a particular Protocol.

Somewhere between the dynamic (duck typing) and static (interfaces, Scala's Structural Types) are Erlang's Behaviors. Erlang's modules (which are comparable to Ruby's Modules) group functions together. A module can use -behaviour(gen_server). to state that it implements the Behavior gen_server - which means that it exports a certain set of functions. This is supported by the Erlang compiler in that it complains if the module doesn't export all of the Behaviors required functions.
This also shows that the general principle of Protocols is not limited to languages that define themselves as "OOP" languages by providing concepts of Class, Object, Inheritance, etc.

How do you document the use of Protocols in Ruby, i.e. how do you talk about the set of messages that interacting objects need to agree upon?

1. Back to top

   In Groovy

   by Steven Devijver

   I don't know about Ruby ...
   
   But in Groovy you can implement methodMissing() so that methods that are called are added to the type. Next time you call this method it will directly go to that method and no longer through methodMissing(). repondsTo() would also recognize this method after the first invocation.
   
   Grails uses this approach all over the place. See one of Graeme's talks on the Grails Exchange for more details.
   
   Also in Groovy, when I realize a method I expect to be there isn't I just add it using ExpandoMethodClass. In this way anything can be turned into a duck.

   Reply

2. Back to top

   Re: In Groovy

   by Michael Neale

   You could do the same in ruby, although the problem still remains, it has to be called once before the method is attached.

   Reply

3. Back to top

   Is it becuase I'm Canadian? or..

   by Deborah Hartmann

   ... or did other people also think that was Duct Taping, at first glance? :-D

   Reply

4. Back to top

   Re: In Groovy

   by Paul King

   Of course, the real question you should ask yourself is why you want additional checking of the protocol of your objects.
   
   If it is for greater clarity at the source code level, in Groovy you would mix and match Java's static checking techniques with Groovy's dynamic typing capabilities. You could statically define the type of a variable (as a fine-grained interface of course) or you could implement an interface in your class definition if you wanted. These techniques provide you with a more declarative specification of your objects protocol(s) than without. That doesn't mean you abandon dynamic typing where it makes sense and you should still use fine-grained (interface-oriented programming) when thinking about interfaces anyway. It certainly puts Groovy miles ahead of many other dynamic languages for those scenarios where you want to apply a declarative style.
   
   If it is for more robustness in your code, than you have various techniques available to you at the metaprogramming layer. In Groovy, I would possible hook into the beforeInvoke() interceptor or the methodMissing() level if needed. Groovy is a little bit more powerful (but mostly on par) with other dynamic languages in this regard.

   Reply

5. Back to top

   Python and Zope interfaces

   by Kevin Teague

   We have had the same problem with duck typing and implicit assumptions that can be made based on particular implementation details in Python and Zope. Python is tackling the problem by adding abstract base classes to Python 3:
   
   www.python.org/dev/peps/pep-3119/
   
   Zope tackled the problem by using metaprogramming to supply their own interface implementation:
   
   griddlenoise.blogspot.com/2005/12/zope-componen...
   
   
   

   
   "This problem could, of course, be solved by making fine grained interfaces, with one method each, and building the full interface up with new interfaces that extend the previous ones, each adding a single API call."
   

   
   
   Interfaces are just a formal and semi-formal way of describing a specification. Changing the specs means breaking existing code. Interfaces in Zope are usually inherited in a fine-grained manner. Not always as fine-grained as one method or attribute per interface, but it is good design have each interface as small as makes sense, and add new functionality by layering on more specific interfaces.
   
   The real joy though is when you can apply these same principles of interface specificity when doing data modelling and working with a persistence layer that allows complex model inheritance structures without messy ORM issues ...
   
   

   
   from zope.interface import interface
   from zope.schema import TextLine, SourceText
   
   class IBaseContent(Interface):
     "Shared attributes amongst all content objects"
     title = TextLine(title=u'Title')
     description = SourceText(title=u'Description')
   
   class IDocument(IBaseContent):
     "Plain document"
     body = SourceText(title=u'Body')
   
   class INewsItem(IDocument):
     "News Item"
     slug = SourceText(title=u'Slug for lead-in text')
   

   
   
   You can declare that an object provides and implementation "after the fact" with the Zope Component Architecture. That is, you can use the equivalent of method_missing, or open a Class and add the required methods, and then declare that your object or class now provides a certain interface.
   
   

   
   >> class Whatever(object):
   ...   "some random plain old object"
   ...
   >>> random_obj = Whatever()
   >>> random_obj.title = 'My Doc'
   >>> random_obj.description = 'A simple doc'
   >>> random_obj.body = 'simple body'
   >>> interface.directlyProvides(random_obj, IDocument)
   >>> IDocument.providedBy(random_obj)
   True
   

   
   
   Kind of a convoluted example, as normally you would have a concrete class that declared that it implement specific interfaces.
   
   This can be persisted in Zope as easy as working with any normal Python object that implements a mapping interface:
   
   

   
   >>> mydatacontainer['some-unique-name'] = random_obj
   

   Reply

6. Back to top

   Re: In Groovy

   by Rusty Wright

   What bothers me about protocols is that intent seems to be left aside. For example, in the following, what is intended by the call to draw() by DrawThing? With interfaces, at least you can see from the inheritence what is intended if you implement only one of the interfaces that specifies a draw method.
   
   

   public interface Graphics {
       void draw();
   }

   
   

   public interface DeckOfCards {
       void draw();
   }

   
   

   public final class DrawThing implements DeckOfCards, Graphics {
       public void draw() {
       }
   }

   Reply

7. Back to top

   Duck Typing.

   by Porter Woodward

   Ahh the old - it looks like a duck, sounds like a duck - it must be a duck. Except it's not.
   
   

   The original question (and some posts in the mailing list debate) seems to come from the confusion about the role of inheritance in OOP. StringIO and IO don't need to go back to the same superclass, because they don't share any commonalities except one: a set of methods they support.

   
   
   Well then why do they both have a lot of the same methods? Obviously they do have something in common - a common set of behaviors. Sometimes though it's not just the behavior that you're interested in. Sometimes it doesn't just suffice to know that an object can respond to a specific message or method call. Sometimes you have to know what class the object really is. It's got a beak, and webbed paddles - is it a duck, or is it a platypus?
   
   Chicken typing indeed. All the sudden you're sprinkling your code with a ton of respond_to? clauses trying to find out if a given object can respond to a particular call. On the one hand it's a workable solution - especially if you're used to programming against low-level hardware (better to ask the CPU, or GPU if it has a capability than ask what type it is and make assumptions about it's functionality). But sometimes you'd like to be able to ask - are you a chicken - or are you a duck? And get an honest answer.
   
   And while interfaces codify this very well - you rightly point out that there is no guarantee that a given interface is even implemented correctly. There's no way to determine that in languages that don't have them anyway - just because an object .respond_to? a given method is no guarantee that the internal implementation of that method is any good.
   
   Strongly recommend checking out Interface Oriented Programming from Pragmatic.

   Reply

8. Back to top

   Re: Duck Typing.

   by Kevin Teague

   
   Well then why do they both have a lot of the same methods? Obviously they do have something in common - a common set of behaviors.
   Sometimes though it's not just the behavior that you're interested in. Sometimes it doesn't just suffice to know that an object can
   respond to a specific message or method call. Sometimes you have to know what class the object really is.
   It's got a beak, and webbed paddles - is it a duck, or is it a platypus?
   

   
   
   You should not need to know what the implementation class is, you just need the ability to distinguish between two different interfaces that share the same method names. The authors of Design Patterns summed it up very well with the statement, "Program to an interface, not an implementation." As soon as you use is_a? or responds_to? you are asking objects about their implementation details and "programming to an implementation".
   
   Say you have defined your interfaces as:
   
   

   
   class IQuacker(Interface):
   	"It quacks!"
   	def quack(self):
   		"Make some noise"
   
   class IDuck(IQuacker):
   	"Definition of a duck"
   
   class IPlatypus(IQuacker):
    	"Definition of a platypus"
   

   
   
   Then you have some basic imlementations:
   
   

   
   class Duck(object):
   	implements(IDuck)
   	def quack(self):
   		print "Duck noise is the only noise!"
   
   class Platypus(object):
   	implements(IPlatypus)
   	def quack(self):
   		print "Platypus noise is better than duck noise!"
   

   
   
   Then in your client code:
   
   

   
   >>> platypus =  Platypus()
   >>> IQuacker.providedBy(platypus)
   True
   >>> IPlatypus.providedBy(platypus)
   Ture
   >>> IDuck.providedBy(platypus)
   False
   

   
   
   Without interfaces, this distinction is possible but needs to deal with
   implementation specific details:
   
   

   
   >>> platypus = Platypus()
   >>> hasattr(platypus,'quack')
   True
   >>> isinstance(platypus,Platypus)
   True
   >>> isinstance(platypus,Duck)
   False
   

   
   
   In the second case, where you are programming to an implementation, your
   code would not work if you had a test suite that used a mock implementation:
   
   

   
   class MockPlatypus(object):
   	implements(IPlatypus)
   	def quack(self):
   		pass
   

   
   
   This mock object will still pass the method names check, but because it
   has a different implementation class the check for the Class type will not
   act as desired.

   Reply

9. Back to top

   Sounds familiar...

   by Rickard Öberg

   These ideas sound a lot like what we are trying to accomplish with Qi4j in Java:
   www.qi4j.org/160.html
   
   By focusing on interfaces, and allowing an object to easily implement as many interfaces as are necessary without having to use class inheritance or manual delegation to implement it, it becomes a lot easier to write and (re)use code.

   Reply

10. Back to top

    Re: Duck Typing.

    by Porter Woodward

    You should not need to know what the implementation class is, you just need the ability to distinguish between two different interfaces that share the same method names. The authors of Design Patterns summed it up very well with the statement, "Program to an interface, not an implementation." As soon as you use is_a? or responds_to? you are asking objects about their implementation details and "programming to an implementation".

    
    
    I agree to an extent. However there are times when the implementation becomes important. Hence my mention of low-level interfaces: CPU and/or GPU. Sometimes you really need to know whether something is implemented - and _how_ it's implemented. For a lot of high-level applications programming - absolutely the goal should be to program to the interface. Hence my cite of Interface Oriented Design at the end of my post.
    
    Even in higher level application programming sometimes you need to _know_; for example you might have abstracted away a lot of your persistence to a database. It can still be important to know some details about the underlying database implementation. Ideally those are encapsulated in the persistence layer - but the persistence layer will do those queries to determine the functionality of the underlying database, and specifically _how_ certain features are implemented in order to provide a common interface to users of the API regardless of the underlying implementation.
    
    So - while it's all very well and good to say the GOF say to program to interfaces - there also needs to be the realization that sometimes to produce an elegant interface the code underneath may have some very un-OOP properties. In order to make that process a little easier it can be handy to have features that allow a high degree of introspection (is_a, responds_to, etc.). Both from the perspective of building working code - and building tools to work with the code - such features make it a little easier to reverse engineer a working code base - and can make automated refactorings a little easier.

    Reply

11. Back to top

    Re: Duck Typing.

    by John DeHope

    First of all, let me say, the original article was great, and I really enjoyed it. Second, I'd like to weigh in on this conversation regarding if and when you should program to an implementation. It seems like you can divide up programming into three sorts of "layers". I don't mean layers of an application, but layers of code. You have the "system" layer, which is virtually defined by it's need to program to implementations. Nowadays we have runtimes like Java and the CLI to do that for us, so we rarely need to worry about this any more. A piece of "system" code serves no purpose other than to provide functionality to the higher levels. Then you have "libraries" which mix interface and implementation access. These are somewhat meant for higher-order code to consume, but also need to be very effective dealing with the specifics of the system layer. Finally you have the "application" layer. This layer should be 100% interface driven. I can't see any reason why implementation details should leak into this layer, they should instead be in the library layer if your system layer isn't providing them.
    
    I think if architects first demanded code be driven by the code layer (as I have defined it) and only then by the application layer, we'd get better enterprise code. My point is that I find it worse to see data-access layer code that has generic library and custom business logic mixed in together, than it is to have controllers with a bit of data-access and user-interface code mixed together.
    
    Let me say that another way... I'd rather see application code that communicates with two different app-layers, but both through interfaces rather than implementations, than I would to see code that communicates with low-level library or system implementations and higher-level interfaces.

    Reply

12. Back to top

    Regarding Chicken Typing

    by Craig McClanahan

    The concern was expressed that a Ruby class that implements part of its API dynamically via

    method_missing?()

    would give incorrect answers for the

    respond_to?()

    test. That is true if you rely on the default inherited implementation. However, an intelligently designed class that uses this sort of dynamic implementation is also free to override

    respond_to?()

    so that it works as expected.
    
    An example of this technique is the way that <code>ActiveRecord::Base</code> overrides

    respond_to?()

    to return true for all the attribute getters and setters inferred from the underlying table, even though these methods are not directly declared.

    Reply

13. Back to top

    commonalities in class hierarcy

    by Arash Bizhanzadeh

    StringIO and IO don't need to go back to the same superclass, because they don't share any commonalities except one: a set of methods they support.

    
    I am a little confused here. Isn't the methods most important characteristics of classes? If they share a set of similar methods, I assume that they have something in common.
    
    
    On the other hand if the concept of duck typing and protocols could be adopted, why should somebody bother with inheritance?
    

    Reply

14. Back to top

    Re: Duck Typing.

    by Kevin Teague

    
    Sometimes you really need to know whether something is implemented - and _how_ it's implemented.
    

    
    
    Using interfaces does not have to mean that you should or need be detached from implementation details. In the end, it's the implementations that do all the work.
    
    Going back to the Zope Component Architecture, it's possible, and common, for your implementations to be named, and then you can ask for a specific implementation by name. If you had an ORM Interface and two implementations:
    
    

    
    
    class ISimpleORM(Interface):
    
    	def do_orm_stuff(self, data):
    		"worlds worst ORM interface"
    
    class GenericPoorPerformingORM(object):
    	implements(ISimpleORM)
    	
    	def do_orm_stuff(self, data):
    		# bunch of implementation details
    
    class MySQLSpecificORM(object)
    	implements(ISimpleORM)
    	
    	def do_orm_stuff(self, data):
    		# bunch of MySQL-specific implementation details
    
    

    
    
    Then you might use specific implementations by registering those implementations and querying for them:
    
    

    
    
    # registration (usually done in XML or with 'convention over configuration')
    component.provideUtility(GenericPoorPerformingORM(), ISimpleORM, 'generic')
    component.provideUtility(MySQLSpecificORM(), ISimpleORM, 'mysql-tuned')
    
    # later on in your application code
    ORM_IMPLEMENTATION = 'mysql-tuned'
    orm = component.getUtility(ISimpleORM, ORM_IMPLEMENTATION)
    
    

    Reply

15. Back to top

    Traits

    by Alejandro Gonzalez

    I found traits to be a valuable concept in this matter.
    
    www.iam.unibe.ch/~scg/Archive/Papers/Scha03aTra...
    
    It helps to construct classes by composing them from behavioral building blocks (called traits), so a class can respond to complete set of protocols without having to use inheritance as the fundametal tool for it.
    It's not a concept to replace single inheritance but to complement it. Also helps a lot in code reuse.
    
    It was already implemented in Squeak (an open source Smalltalk dialect) and IMHO is a promising concept.

    Reply

16. Back to top

    Re: Traits

    by Kevin Teague

    Good paper.
    
    Smalltalk's Traits is very similar to Python's zope.interface and Perl 6 is also applying the same concept and calling it Roles. They all differ somewhat in the details, but they all aim to solve the problem as eloquently stated in the Smalltalk paper, "The purpose of traits is to decompose classes into reusable building blocks by providing first-class representations for the different aspects of the behaviour of a class."

    Reply

17. Back to top

    Re: Is it becuase I'm Canadian? or..

    by Keith Thomas

    Please don't post messages like this. My whole office is staring at me because I burst out laughing and disturbed them.

    Reply

• Older >