Item 30: Never cede control outside your component while holding locks





Item 30: Never cede control outside your component while holding locks

Readers of Effective Java [Bloch] will undoubtedly remember this item's title as advice from Item 49, phrased in almost exactly the same way. Fortunately, deadlock isn't quite as much of a concern in J2EE systems thanks to some of the built-in safeguards established by the EJB Specification as well as some of the transactional deadlock avoidance code built in to most modern relational databases. Despite that, for reasons that partially relate back to keeping lock windows small (see Item 29), it's still a bad idea to call out to "alien" components while holding locks open.

As a quick reminder, what exactly happens here?






public class Example

{

  private Object lock = new Object();

  public void doSomething()

  {

    synchronized(lock)

    {

      // Do something interesting

    }

  }

}


To be precise, the thread executing the doSomething method attempts to acquire ownership of the object monitor for the object instance referenced by lock. Assuming no other thread owns that monitor, this thread will acquire it.

Take it one step further: What will happen when a thread calls recursively into a synchronized region?






public class Example

{

  private Object lock = new Object();

  public void methodA()

  {

    synchronized(lock)

    {

      methodB();

    }

  }

  public void methodB()

  {

    synchronized(lock)

    {

      // Do something interesting

    }

  }

}


The JVM remembers that the thread that called methodA owns the monitor for object lock, so it breezes right through the synchronized region in methodB; it's impossible for a thread to deadlock itself in Java.

This is Remedial Java 101; why bring it up here? Take this into a different dimension for a moment and put some RMI semantics behind it. In particular, remember that the RMI Specification states that a remote method call on an exported object (i.e., a method call from outside the JVM via the RMI stack) can come on any arbitrary thread. Because the JVM remembers only the local thread that owns a monitor, it becomes possible for a thread to deadlock itself by calling outside the JVM and back in again:






// Deadlock.java

//

import java.rmi.*;

import java.rmi.server.*;



public class Deadlock

{

  public static interface RemoteFoo extends Remote {

    public void doTheCall() throws RemoteException;

  }



  public static class RemoteFooImpl

    extends UnicastRemoteObject

    implements RemoteFoo

  {

    public RemoteFooImpl() throws RemoteException { super(); }



    private Object lock = new Object();

    public void doTheCall() throws RemoteException

    {

      try {

        System.out.println("Entered doTheCall()");

        synchronized(lock) {

          System.out.println("Entered synchronized region");

          RemoteFoo rf = (RemoteFoo)

            Naming.lookup("rmi://localhost/RemoteFoo");

          rf.doTheCall();

        }

      }

      catch (NotBoundException nbex) {

        System.out.println("Not bound?"); }

      catch (java.net.MalformedURLException malEx) {

        System.out.println("Malformed URL"); }

    }

  }



  public static void main(String[] args) throws Exception

  {

    RemoteFooImpl rfi = new RemoteFooImpl();

    Naming.bind("RemoteFoo", rfi);



    RemoteFoo rf =

      (RemoteFoo)Naming.lookup("rmi://localhost/RemoteFoo");

    rf.doTheCall();

  }

}


The output from this program sends chills down the spine of any distributed systems programmer:






C:\Projects\EEJ\code\RMIdeadlock>java Deadlock

Entered doTheCall()

Entered synchronized region

Entered doTheCall()

  Window hangs


And sure enough, if we do the Ctrl-Break trick in the Windows JVM to get a complete thread dump, we can see the deadlock:






Full thread dump Java HotSpot(TM) Client VM (. . .)



other thread dumps removed for clarity



"RMI TCP Connection(3)-192.168.1.102" daemon prio=5

        tid=0x02edaf30 nid=0x7d8

        waiting for monitor entry [32cf000..32cfd8c]

        at Deadlock$RemoteFooImpl.doTheCall(Deadlock.java:29)

        - waiting to lock <0x1050d3d8> (a java.lang.Object)

        at sun.reflect.NativeMethodAccessorImpl.invoke0(

                Native Method)

        at sun.reflect.NativeMethodAccessorImpl.invoke(

                NativeMethodAccessorImpl.java:39)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(

                DelegatingMethodAccessorImpl.java:25)

        at java.lang.reflect.Method.invoke(Method.java:324)

        at sun.rmi.server.UnicastServerRef.dispatch(

                UnicastServerRef.java:261)

        at sun.rmi.transport.Transport$1.run(

                Transport.java:148)

        at java.security.AccessController.doPrivileged(

                Native Method)

        at sun.rmi.transport.Transport.serviceCall(



                Transport.java:144)

        at sun.rmi.transport.tcp.TCPTransport.handleMessages(

                TCPTransport.java:460)

        at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(

                TCPTransport.java:701)

        at java.lang.Thread.run(Thread.java:534)



"RMI TCP Connection(2)-192.168.1.102" daemon prio=5

        tid=0x02ed9238 nid=0x91c

        runnable [324f000..324fd8c]

        at java.net.SocketInputStream.socketRead0(Native

          Method)

        at java.net.SocketInputStream.read(

                SocketInputStream.java:129)

        at java.io.BufferedInputStream.fill(

                BufferedInputStream.java:183)

        at java.io.BufferedInputStream.read(

                BufferedInputStream.java:201)

        - locked <0x1006a110> (a java.io.BufferedInputStream)

        at java.io.DataInputStream.readByte(

                DataInputStream.java:331)

        at sun.rmi.transport.StreamRemoteCall.executeCall(

                StreamRemoteCall.java:189)

        at sun.rmi.server.UnicastRef.invoke(

                UnicastRef.java:133)

        at Deadlock$RemoteFooImpl_Stub.doTheCall(

                Unknown Source)

        at Deadlock$RemoteFooImpl.doTheCall(Deadlock.java:31)

        - locked <0x1050d3d8> (a java.lang.Object)

        at sun.reflect.NativeMethodAccessorImpl.invoke0(

                Native Method)

        at sun.reflect.NativeMethodAccessorImpl.invoke(

                NativeMethodAccessorImpl.java:39)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(

                DelegatingMethodAccessorImpl.java:25)

        at java.lang.reflect.Method.invoke(Method.java:324)

        at sun.rmi.server.UnicastServerRef.dispatch(

                UnicastServerRef.java:261)

        at sun.rmi.transport.Transport$1.run(

                Transport.java:148)

        at java.security.AccessController.doPrivileged(

                Native Method)

        at sun.rmi.transport.Transport.serviceCall(

                Transport.java:144)

        at sun.rmi.transport.tcp.TCPTransport.handleMessages(

                TCPTransport.java:460)

        at sun.rmi.transport.tcp.TCPTransport$Connection

                Handler.run(TCPTransport.java:701)

        at java.lang.Thread.run(Thread.java:534)


As you can see, the two threads—the one caller, the other the thread dispatched by RMI to answer the incoming request—are deadlocked because the caller is blocked waiting for the callee to return, and the callee is waiting to lock on the object locked by the caller.

This is a nasty turn of affairs; it means that the RMI Specification has no capacity to detect a logical call sequence that leaves the JVM yet circles back into the same JVM as part of that process, what Transactional COM+ [Ewald] refers to as a causality and the EJB Specification refers to as a loopback. Without this, there's no way to detect a logical self-creating deadlock, much less a deadlock created by two clients. If there's any possibility for a circular call like this, it's up to you to handle it.

Note that EJB doesn't exactly help out a whole lot here—in the event that a concurrent callback like this occurs on a stateful session bean, the container throws an exception (RemoteException if it's a remote call, EJBException if it's a local call), thereby effectively rolling back the entire chain of work. This behavior is deliberately designed, as stated clearly in the specification (Section 7.12.10): "One implication of this rule is that an application cannot make loopback calls to a session bean instance." It avoids deadlock but invalidates work, and only at runtime, to boot.

If this were the only concern, we could probably ignore it—after all, the most likely possibility for this kind of situation is a designer trying to build some sort of notification mechanism on a stateful session bean using the Observer pattern [GOF, 217], as is commonly done for Java event handling, and now that we've identified a problem with this approach, we just fall back to messaging-based approaches to avoid the problem. Given the relative rarity in which this problem arises, it wouldn't be worth discussing.

No, the larger problem comes from the fact that when you call out of your component while holding a lock, you're effectively including whatever effort the "alien" code wishes to undertake as part of your lock window (see Item 29). This means that it's entirely possible that the "alien" code, code by definition not under your control, could be doing such brain-dead things as calculating pi to the 100th decimal place, making remote service calls, or even worse behavior, all while you hold locks.

The implications of this are more than you might think at first. For starters, you need to be excruciatingly careful when choosing transactional affinity on EJBs, particularly for EJBs operating under container-managed transactions, because the window of a CMT transaction is the entire body of the method call, thereby giving you no opportunity to do nontransactional processing before opening the transaction and possibly taking out locks. Only if the EJB is entirely atomic in nature (i.e., it makes no calls to other components, beyond the database whose processing is—hopefully—well understood) is CMT a viable option.

You also need to be aware of the fact that a JMS consumer invoked asynchronously via a MessageListener (which includes the Message-Driven Bean) is invoked on any arbitrary thread, meaning that we're back to the danger of holding an object monitor needed by a different thread. In particular, when using messaging to break the request-response cycle (see Item 20), be aware that the delivery of the message could kick the asynchronous processing off immediately, and any locks that you're holding at the time you deliver the message could create contention or, in the worst-case scenario, a deadlock, as the message processor takes out a lock on something you want before trying to acquire the locks your thread is currently holding. And because it's so timing-dependent, it will happen only under severe load or during a demo to a potential million-dollar client.

Do yourself a favor and make sure the locks are all released before performing any communications outside of the component; you'll be a much happier and carefree programmer for doing so.


     Python   SQL   Java   php   Perl 
     game development   web development   internet   *nix   graphics   hardware 
     telecommunications   C++ 
     Flash   Active Directory   Windows