Joshua Bloch on Closures, Resource Management, Google

Interview with Joshua Bloch on Nov 07, 2006 11:50 AM

Community

Java

Topics

Programming

Tags

Closures,

Java SE,

Language Features

Summary
Core Java language designer Joshua Bloch talks about the need for Closures, Functional Programming, Java's need for better resource management, Java Puzzlers, and cool stuff Google is doing with Java. Recorded at www.javazone.no conference, Norway's best Java conference.

Bio
Joshua Bloch is a Principal Engineer at Google. He led the design and implementation of numerous Java platform features, including the Java Collections Framework. He is the author of the Jolt Award-winning book Effective Java. He holds a Ph.D. in Computer Science from Carnegie-Mellon University and a B.S. in Computer Science from Columbia.

Josh can you tell us a bit about yourself and what you are doing at Google these days?

As most of our listeners probably know I worked at SUN for a number of years doing Java Libraries architecture best know for doing the collections library, for writing Effective Java and more recently for writing Java Puzzlers. At Google I'm doing Java infrastructure and I'm also our liaison with the Java community, I'm the representative on the Java community process Executive Committee as well as serving on a number of expert groups.

Are you still heavily involved with Java's evolution even though you're at Google?

Indeed I am. As I say I'm our EC rep, I'm still on the JSR 166 expert group with Doug Lea, and I'm working on other ongoing efforts the Java platform both language and libraries. I'm as involved as ever.

What's going on in Java these days that excites you?

1.6 is a good release, arguably it's not a super-exciting release, it doesn't add a lot of features but on the other hand you don't want to do that every release because you end up with more than people can swallow. From our initial experiments it's a very fast virtual machine, very stable, and there are some nice things that have been added, for example the sorted collection, the sorted mapping were never where they should have been and this is my fault; there are certain asymmetries in those APIs that have bothered me since I wrote them. We've finally fixed that. We have a new Navigable map and Navigable Set which are in essence updated, so those interfaces and the key implementations of them which are the TreeSet and TreeMap have been retrofitted. Similarly Doug Lea and I have added a double-ended queue interface and a concurrent that is a blocking version of it - a deck and blocking deck and a very fast implementation called ArrayIndexer. Basically the collection interfaces continue to grow and to mature and I'm pretty excited about that.

Where do you stand on the closures debate?

I like closures. I think they work very well in languages that design them, like Smalltalk and Scheme and so forth. I think closures in java are a good thing. We've basically had them since 1.1 in the form of anonymous class instances and those are a bit clunky so I definitely see a place for improvement support for closures; on the other hand some of the proposals that I've seen floating around are overly complex; they involve massive changes to the type system, things like function types and I'm severely worried about pushing the complexity of the language beyond the point where Joe Java can't use the language anymore. If we add more support for closures I think it has to be in the spirit of the current support, which means that closures should take the form of instances, of types that have a single abstract method, whether they are interface types such as Runnable , or class types such as TimerTask. I think it would be nice if a better syntax in the current anonymous class syntax were adopted and if requirement with regards to Final variables were somehow made more tangible, which doesn't mean necessarily relaxed; I think it's actually good that you can not close over non Final variables, but it's a pain to actually mark them final, so if they automatically marked themselves final which would be nice.

So Java itself should not approach more functional programming style?

I love functional programming style. In one sense I think it should. I always advise people to maximize mutability, so functional mutability is good. But on the other hand, I don't think that a structural functional type system embedded within Java's current type system is necessarily a good idea. Java's type system is already very complex and it will be more so by addition of generics to the language in Java 5. I don't know if Java can withstand another blow to the type system of that magnitude.

What else do you think Java needs from a language perspective?

I think that one way or another Java needs some automatic resource management blocks, something akin to the using statement in C sharp or to destructors in C++. I think that could evolve out of the closures or more likely could be a purpose built control construct along the lines of the for-each loop which automatically does a before/after type construct; when you exit the the lexical scope of this block, you automatically call the close method on some resource so that people don't have to close their files anymore. I would like to see something like that in the language and infact I sent a proposal to Sun for that. There are few little things which I think should be added which are harmless. One if them is just fixing a flaw in one of my designs; when I did enums, I neglected to have them operate properly with the comparison operators less than or equal to, greater than or greater than or equal to. Given that enums implement Comparable because identity is isomorphic to equality. Those comparison operators should have worked with Comparble as well, and they don't so it's takes just a no-brainer to make them work right now it's illegal. You just make them legal. Also I see no reason why case statements don't work on strings. These are little, harmless things that would make life better for the programmer on the street or at his desk.

About the resource management idea: what are the use-cases for this, why do we need it?

Basically there are lots of resources in Java that have to be explicitly closed and people do it with a try/finally Whether is database connections in JDBC or files or even locks in java.util.concurrent.lock which are locked and unlocked and right now there's this little boiler plate code that you always do, where you allocate or you open the object and then you do a try/finally and in the finally you close it. There are about four or five lines of code and they are ugly. It would be much nicer and we could do this in other languages; you just have a block that has the resource at the top of the block somehow and then we exit the scope of the block it gets automatically terminated for you and leaks of this things can be devastating. They can cause the system to crash. Also we can learn from the experience of previous attempts at doing this. For example: the using statement only takes a single resource and I've talked to people who use it and they don't like that. You have to nest these things when you have multiple resources which you often do, so we can define a construct that actually allows you to say that you're using multiple resources with the scope of a given block and the variables are in scope only at the top of the block in that statement and throughout the block. When you exit the lexical scope of the block, not only did the variables go out of scope but they automatically terminated, that is the resources are allocated for you just by invoking a name method and this can done using an interface like Closable except the Closable is tide to IOException which is a little bit unpleasant. There are still some design problems to be solved, but that's what JCP expert groups are for.

As the Java platform becomes multi-lingual, how do you see the standards bodies functioning and the platform itself evolving?

I think that the JCP will continue to standardize the JVM itself. So for example there's a JSR that SUN has already launched to add the InvokeDynamice structures into the JVM and support of these various dynamic languages and it think that JCP will continue to do that. It will of course continue to standardize the core Java language. Sun has some challenges in that area as it open-sources Java, as they promised to do. But I suspect that JCP will continue to play a part in its evolution or at least in the evolution of the core Java platform. On the other hand, as more languages start to run on the JVM, clearly those won't all be subject to standardization by the JCP and shouldn't be because the resources of any standards body are limited. I think there will be experiments standardized by no one which I think it's great; there will be universities that will Try new things. I think there will be new standards bodies and new duties for old standards bodies.

Now that you're not at Sun anymore. Can you give us your honest opinion on how you think open-source java should be governed?

I haven't given the matter that much thought and I'm a little bit of a curmudgeon. I care a lot less than my friends about this. Because it think Sun has done a fairly good jobs stewarding the language up to now and there is a research license that allows organizations, in particular universities, to do anything they want to the language for research purposes and when they come up with good ideas, those good ideas get folded back into the mother ship; for example the generics stuff came out from the Pizza language, and after that generic Java GJ. I think that's a reasonable model for the evolution of the language going forward. On the other hand, there are some people who really care about actually open-sourcing it and find more power to them. Yes there still are problems to be solved but I don't actually like thinking about that stuff. I let other people think about it.

If you were all powerful and you could change two things about how Java evolved over the last 8-9 years what would you change?

First of all, Java should have had an unsighned int type from day one. Second of all, that type should have been used to index Arrays. Right now there's only 31 bits of re-index; we're running out of space. Sometimes you want an array that has more than 2 billion elements. I think the clonable interface should have had a clone method in it. I think the switch statement should have been structured, should not have had a fall through and should have been precisely one statement after each case label or sequence of case labels. Also languages don't just pop-out fully formed; the interaction between a Arrays and Generics is unfortunate and the ones using generics have fallen into this trap; you cannot allocate an array of T's, nor can you even have a vararg involving T's without at least getting a warning from the compiler. That's unfortunate; the fact that array types are covariant and generics aren't is confusing and unfortunate, but on the other hand I think that at the time they probably did the right thing with arrays given that they couldn't have put generics into the language back then. It's very hard to second guess people; when you think about the size of the team that they had and the time pressure under which Java was released, I think they did a spectacular job. They hit a real sweet spot and that's why it took off like a rocket. But there were a few problems and I think I've listed my main gotcha's.

What is Google doing with Java?

Google uses Java in many places. We use it a lot in the middle tier obviously on the client, we use Ajax, we use Javascript and typically a lot of the applications that we use everyday like GMail, Google calendar, talk directly to Java and then the data is often stored on a back-end that's written in C++; we also use Java for our ads frontend server which is a very important part of our infrastructure, as you know ads are an important part of the engine that makes Google go. We use them a lot for internal applications. To make all this work we have a fairly large piece of Java infrastructure and that's my job at Google, I maintain that infrastructure.

Can you tell us more about this awesome Java infrastructure at Google?

There's a lot of classical google infrastructure. If you were at the Trundheim open house at Oslo last night, but the director discussed three of the main pieces of infrastructure google has: google file system, map reduce and big table and all these things were written in C++. But my team has made them accessible from Java in some cases, by re-implementing parts of them and in some cases by writing Java layers that talk to them through native methods and all that has proven quite successful. So now one can write all this core Google infrastructure in Java and additionally we have a lot of JDK-like stuff of our own invention and some of it we intend to submit for possible inclusion in the JDK. We have already done a little bit of it. Earlier I was referring to a this ArrayDeck class and that came out of some code that I wrote at google. There's a guy Kevin Burley who has written a lot of good stuff that I hope to see in Java 7. For example he wrote a facility called TimeLimiter which is a really cool thing. You take an interface and an implementation of that interface and it returns another implementation that delegates to the written original implementation and it time limits the calls and if they time out an unchecked time-out exception is thrown in the calling thread, so it allows you without the hassle of managing your own executor and your own futures and it works out really nice.

What's new with Puzzlers?

Actually there's only one book so far and in fact there will probably be one book and I'm hoping there won't that many more puzzles in that platform if we keep it from getting too complex. The book has 95 puzzles in it and since it has come out we came up with eight more which have to do with Tiger (in fact I'll be presenting all of those tomorrow at the Java Zone) and Neil and I continued to travel around the world to just spreading the Puzzlers message. It's a lot of fun, an easy way for people to learn the traps and the pitfalls of the platform, so I think that's the message.

What's your favorite puzzle?

Perhaps it is the puzzle in which a multithreaded program deadlocks without any visible locks in the program. I don't remember what the puzzle was called (it's in the book; I think it's in the Advanced Library Puzzles chapter). This perfectly innocent looking program that provably deadlocks every time it runs having to do with the JLS definition of class initialization in a multithreading setting. You've heard of a lock free algorithm, this is a lock free deadlock.

 

RelatedVendorContent

SpringSource Launches New Application Server without Java EE

RESTful todo list sample tutorial with Groovy & Project Zero

Fighter Jets and Agile Development at Lockheed Martin (Case study)

Create a photo album application with Project Zero and REST design principles

Introducing the SpringSource Application Platform

Related Sponsor

Announcing the SpringSource Application Platform - an enterprise Java appserver that redefines the way in which Java applications are deployed and run. >> Download beta here

1. Back to top

   Using block for Java

   Nov 8, 2006 12:40 PM by Jonathan Allen

   I really like this idea, but as the Joshua says it needs to have more than one resouce on the same line. I don't think this would be hard, as VB does it with its Using block.

   Reply

2. Back to top

   The Reference Puzzle

   Nov 8, 2006 9:13 PM by Scott Gelb

   In case anyone is interested, in the last part of the interview Dr. Bloch makes a reference to Puzzle 85: Lazy Initialization (pg. 205 of Java Puzzlers).

   Reply

3. Back to top

   Thumbs up for the text !

   Nov 9, 2006 1:56 PM by ZedroS Schwartz

   Hi That's great you put a written translation of the interview. Good on you ! Cheers, ZedroS

   Reply

4. Back to top

   Listening to Josh

   Nov 11, 2006 6:59 AM by Alex Popescu

   Listening Josh speaking is always an amazing experience. The resource auto-manageable blocks sounds very well and I would definitely be happy to have something like this as soon as possible. Even if I haven't hit myself many of the "issues" Josh would have theoretically changed, still just thinking about them make you see things pretty different. Excellent interview! ./alex -- .w( the_mindstorm )p.

   Reply

5. Back to top

   Resourcing Problem.

   Nov 13, 2006 10:28 AM by Geoffrey Wiseman

   Completely agree that the resourcing problem is one that Java needs a better solution for, which is definitely something that Closures are used for in other languages. Ruby's full of idioms like this where you 'use' a connection with a block, rather than opening, doing stuff, then closing, and catching.

   Reply

6. Back to top

   a using()-like block for Java

   Nov 24, 2006 11:15 AM by Roger Voss

   The thing about C# using() is that it only works with objects that implement IDisposable interface. There is no similar universal interface in Java for standardizing the cleanup of non-memory resources. The various resources that one might want to subject to such a language construct in Java have varying conventions as to what to invoke. So in the end there might need to be a way to associate a closure in which one places the code that invokes whatever method does cleanup operation. Also if this new feature is made to work with multiple resources, then how does this new construct insure that each resource cleanup method gets invoked (i.e., how to automatically deal with the potential of a given cleanup method to throwing an exception)? If closures are introduced as part of the solution for this, there is possibility that a cleanup method throws a checked exception so the closure must deal with that. I'd prefer not to have to wrap calls in try/catch - hopefully a new construct for cleaning up deterministic resources could automate dealing with this concern.

   Reply

7. Back to top

   Re: The Reference Puzzle

   Apr 2, 2008 12:30 PM by seri katil

   Hepl Blog >>> evden eve nakliyat

   Reply

8. Back to top

   Re: The Reference Puzzle

   May 6, 2008 3:29 PM by taker yurtok

   havalandırma havalandirma izolasyon Sohbet iso 9001 iso 14001 Yangın Söndürme yangın söndürme cihazları yangın dolapları yangın tüpü izalasyon ısıtma soğutma isitma sogutma Aspirator Aspiratör Vantilatör sohbetim turizm işletme belgesi turizm belgesi turizm yatırım belgesi Chat sohbet odası sohbet sitesi türkiye sohbet tr sohbet tüm türkiye sohbet arkadaş sohbet türkiye sohpet kızlarla sohbet kızlarla sohpet muhabbet muhappet kızlarla çet çet türkiye çet çet sohpet mırç mirç türkiye mirc mirc muhabbet Sohbet Sitesi Chat Sohpet Yangın yangın güvenlik yangın söndürme sistemleri yangın tüpü dolum yangın merdiveni yangın çıkış kapısı  Hava Soğutma Hücreli Aspiratörler Fanlar Radyal Körükler Toz Toplama Soğutma Kulesi Klima Santraller Malzeme Nakil Vantilatörleri iso 14001 iso 14001 iso 22000 iso 22000 haccp belgesi haccp belgesi ikamet tezkeresi yabancı çalışma izni yabancı personel çalışma izni yabancı çalışma izni yabancı personel çalışma izni ohsas 18001 ohsas 18001 iso belgesi iso 9001 belgesi ohsas belgesi ISO 9001 Teşvik Belgesi Çocuk Bezi Hasta Bezi Makyaj Malzemeleri Makyaj Temizleme Mendili Kişisel Bakım kolonyalı mendil Islak mendil Dudak Koruyucu Temizlik Ürünleri Göz Kalemi Diyet Ürünleri Süper Site driver Güvenlik Kamerası Islak Mendil Kolonyalı Mendil Kolonyalı Mendil JoyTurk driver ara web tasarım Güvenlik Kamerası paketleme Kamera gizli kamera Kamera Kurulum Tatil Tatil Yerleri Tatil Beldeleri Perde Perde Modelleri Kamera Epilasyon Emlak Yaşam Tatil Video Cilt Bakımı video süper perde jaluzi perde stor perde dikey perde perde modelleri perde jaluzi perde stor perde dikey perde perde modelleri magazin haberler spor haberleri video eğitim Giyim

   Reply

9. Back to top

   Re: The Reference Puzzle

   May 9, 2008 5:45 PM by mustafa kara

   yes all right thanks...evden eve nakliyat evden eve nakliyat evden eve nakliyat

   Reply

10. Back to top

    Re: a using()-like block for Java

    May 14, 2008 9:16 PM by mustafa kara

    very nice...thanks ankara nakliyat ankara nakliyat ankara nakliyat ankara nakliyat ankara nakliyat

    Reply

11. Back to top

    Re: a using()-like block for Java

    May 15, 2008 10:13 AM by ankara nakliyat

    Nice site. Thank you ankara taşımacılık ankara evden eve

    Reply

• Older >