Direct Web Remoting

I believe in change but applications developed by our companies do not use technology in adequate measure. They do not change with the times. This is caused by fear of software. Software breaks in production and if we have inadequate unit tests we are scared.

I have written a simple test using DWR several years back but I haven’t used it anywhere. DWR enables us to communicate with our browsers using Ajax in the reverse. I experimented with it again and this code updates the ‘div’ in the browser at periodic intervals.

The code is not written very well but it works. There are techniques to shutdown the threads properly.

The main purpose is to detect broken links to the browser and expire the sessions. That is a hard problem now.

I will add details if I am able to achieve it.


package com.test;

import java.util.Collection;
import java.util.Iterator;
import java.util.concurrent.Executors;
import java.util.concurrent.ScheduledExecutorService;
import java.util.concurrent.ScheduledFuture;
import java.util.concurrent.TimeUnit;

import org.directwebremoting.Browser;
import org.directwebremoting.Container;
import org.directwebremoting.ScriptSession;
import org.directwebremoting.ServerContextFactory;
import org.directwebremoting.extend.ScriptSessionManager;
import org.directwebremoting.ui.dwr.Util;

public class ReversePublisher {

	ScheduledExecutorService scheduledExecutorService =
	        Executors.newScheduledThreadPool(5);

	    public void publish(){
	    		    	
	    	@SuppressWarnings("unchecked")
			ScheduledFuture<?> scheduledFuture =
	    	    scheduledExecutorService.scheduleAtFixedRate(new Runnable() {
	    	        public void run() {
	    	        	setDiv();
	    	        }
	    	    },
	    	    5,
	    	    5,
	    	    TimeUnit.SECONDS);

	    }

	    /**
	     * Update using reverse Ajax
	     */
	    public void setDiv(){
	    	try{

	    		Browser.withAllSessions(new Runnable()
	    		{
		    		public void run(){ 
		    	    	try{
		    	    		Util.setValue("Timeout", "Reverse Ajax"); 
		    	    	}catch(Exception e){
		    		    	System.out.println( "Detecting broken link " + e.getMessage());
		    	 	   	}
		    		}
	    		});		        
	    	}catch(Exception e){
		    	System.out.println( "Exception in Publishing in the reverse " + e.getMessage());
	 	   	}
	    }
}

Update

After asking the DWR forum I understood that when I enable ‘Reverse Ajax’ in web.xml there is an implicit polling call being made by the browser. Mike Wilson answered my questions sent to the DWR user list. This particular problem does not require a scheduled Thread to make frequent polling calls to the browser. It is implicit.

This call is made at a certain configurable interval regularly. I coded a filter to trap this call and stored the access times. If a browser crashes or closes these calls are not made for a specific period. When the browser opens again, even if cookies or caches are reinstated this filter can check the last accessed time and the current time and decide to expire the HttpSession.

I believe at this time that can solve my problem. Network latency could cause concerns though.

Google Chrome’s JavaScript console shows this implicit polling call.

reverseajax

IntelliJ IDEA anonymous class as Java 8 lambda

I recently started coding lambdas. This is partly to prevent my Java skills from getting rusted.

I found that IntelliJ IDEA has this interesting feature that allows an inner class to be shown as a lambda.

 public class Employee {

           public Employee( String name,int age){
               this.age = age;}

           interface CheckAge {boolean test(Employee p);
                 int age;
                 public int getAge() {
                       return 36;
                 }
 
           static void printEmployees( List<Employee> m, CheckAge ca ){
             for( Employee m1 : m ){
                System.out.println( ca.test( m1 ));
             } 
           }

           public static void main( String... argv ){

                List<Employee> employees= new ArrayList<Employee>();

                employees.add( new Employee( "Test", 29));

                printEmployees( employees,new CheckAge() {
                     public boolean test(Employee p) {
                         return p.getAge() >= 18;
                                  p.getAge() <= 35;}
                });
   }
 }


Lambda

The printEmployees method has an anonymous inner class and the screenshot shows
IntelliJ IDEA’s interpretation.

The product company I am working for is very far behind this type of technology curve and only technology evangelism might help.

Bounded wildcard

I have been interviewing candidates for the position of software architect. Invariably I find that the candidates’ knowledge is at a cursory level. This is surprising because the person has atleast 10 years’ experience or more. The offshore environment has muddled these people because they are never allowed to own the entire software stack including the deployment architecture. There is always a foreign architect who guides them and ensures that they don’t have wide knowledge of architectural concerns.

They also don’t have to be aware of the latest Java developments. So a question about generics which was introduced several years back elicits no response.

I am interested in learning about generics but sometimes I am befuddled by the complexity. ‘Effective Java’ by Bloch has some good advice. One of it is shown below.


public interface Parent extends Comparable< Parent> {
}

public interface Child< T> extends Parent{

}

public class Compare {

//Incorrect Test method
// public static < T extends Comparable< T>> T compare(List< T> list){
//
// return list.get(0);
// }

//Correct Test method
public static < T extends Comparable< ? super T>> T compare( List< ? extends T> list){

   return list.get(0);
}

public static void main( String... s ){

    List< Child< ?>> children = new ArrayList< Child< ?>>();
    compare( children );

}
}

The correct way to use bounded wildcards to compile this method is shown in the book. If one uses the incorrect method it does not compile. The reason is that Child instances are not only comparable to themselves but also to Parent instances.

Generics is one subject that makes me think that even after reading a very good explanation I am still at a loss. But I want to capture patterns like this.

Java Lambda

I have only lean pickings especially during this recession. But this is my first Java Lambda. I have worked with Clojure in the past and I can relate to that.

There is enormous power in lambda’s and I hope to explore it further.

package com.test.lambda;
import java.util.concurrent.Callable;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.FutureTask;

public class Lambda {

  /* @param args*/
 
  public static void main(String[] args) {

    Runnable r = () -> System.out.println("Test Lambda");
    ExecutorService executor = Executors.newFixedThreadPool(2);
    executor.execute(r);
    executor.shutdown();
  }
}

Compiled with
openjdk version “1.8.0-ea”
OpenJDK Runtime Environment (build 1.8.0-ea-lambda-nightly-h3207-20130205-b76-b00)
OpenJDK 64-Bit Server VM (build 25.0-b15, mixed mode)

aioug conference Sangam 2011

Last year I visited Bangalore with the intention of presenting at the aioug conference Sangam 2011. The turnout was poor but my slides were posted here. It is about concurrency.

‘atoi’ function code kata

I recently coded the ‘atoi’ function in Java as part of an assignment while I was reading ‘Hacker’s delight’, a delightful book full of bit twiddling tricks and techniques.
These ideas reduce the number of instructions used to execute code and should be followed by Java coders who work on critical transaction processing applications to increase throughput.

package com.atoi;

import java.math.BigInteger;

import static junit.framework.Assert.assertEquals;

/**
 * User: Mohan Radhakrishnan
 * Date: 10/21/12
 * Time: 1:19 PM
 */
public class Atoi {

    /**
     * Convert 'char' to 'int'.
     * @param chars
     * @return result final result or result till an ASCII space
     *         if encountered.
     */
    public int atoi( char[] chars ) {

        assert( chars != null && chars.length != 0 );
        int i = 0;
        int result = 0;
        char temp = 0;

        while( i < chars.length ){
            if( (chars[ i ] ^ 0x20)  == 0 ){
                return result;//Stop at the first ASCII space
            }else if ( isBetween0And9( chars[ i ] - '0')){
                //calculate
                result = ( result * 10 ) + ( chars[ i ] - '0');
            }
            i++;
        }
        return result;

    }

    /**
     * Referred 'Hacker's delight' which
     * has bit techniques that helps in reducing
     * instructions.
     * If the result of this bit twiddling is 0x80808080
     * then the input is less than 9
     * The advantage is that a extensible function to find
     * any range can be coded with slight modification.
     * @param input
     * @return result of the check
     */

    public boolean isBetween0And9(int input){
        System.out.println(  "isBetween0And9 [" + Integer.toBinaryString( input ) + "]");
        int y = ( input & 0x7F7F7F7F ) + 0x76767676;
        y = y | input;
        y = y | 0x7F7F7F7F;
        y = ~y;
        System.out.println( "~y [" + Integer.toBinaryString( y ) + "]");
        return( 0x80808080 == y );

    }
}

I also coded JUnit tests mainly to test the logic from “Hacker’s Delight’ that checks for a overflow before it occurs. That was interesting because I could check if an ‘int’ exceeds Integer.MAX_INT or underflows without using the Java libraries or checked exceptions. Since this code is not simple to understand I coded it and tested it but didn’t copy it into this entry. Later.

public class AtoiTest {
    
    @Test
    public void testAtoi() throws Exception {
        Atoi atoi = new Atoi();
        try{
            assertEquals( 5, atoi.atoi( new char[] {'5'}));
            assertEquals( 0, atoi.atoi( new char[] {'A'}));
            assertEquals( 5, atoi.atoi( new char[] {'5','A'}));
            assertEquals( 5, atoi.atoi( new char[] {'A','5'}));
            assertEquals( 0, atoi.atoi( new char[] {' ','5'}));
            assertEquals( 56, atoi.atoi( new char[] {'5','6',' ','5'}));
            assertEquals( 506, atoi.atoi( new char[] {'5','0','6',' ','5'}));
        }catch ( Exception e){
            fail();
        }
    }
}

Github code

I started to push code into Github. Skiplists were invented by William Pugh and I looked at the ‘C’ code available at ftp.cs.umd.edu and coded Java programs. I am trying to code algorithms and a Skiplist was the first one. This code might not be the best but it will be refactored.
If I am able to sustain this motivation I will code graph algorithms and other data structures. Initially I was interested in distributed algorithms but they require special skills to understand.

Apache ZooKeeper

I don’t know why management of firms that deal with financial data tend to be very conservative about new technology in my part of the world. There are varied reasons for this but the lack of experience of executives is high on the list. The vision to support large-scale deployments of mission critical software is just not there.

Company honchos should read Joel Spolsky’s opinion about the role of the executives in shaping the technology roadmap of the company.

I recently explored options to monitor and operate a simple cluster of Java socket programs and came across ZooKeeper. It is not strictly a requirement for a distributed application but these individual socket programs need to be managed and some may need to be restarted.

Initially I thought of using Google protobuffer and send simple information about the number of messages and load to the centralized ZooKeeper instance so that some decision can be taken based on it. This idea is not fully explored yet but the sample program that uses ZooKeeper nodes and proto buffer messages is shown below.

The messages sent and received from ZooKeeper nodes can be serialized in any way and here proto buffer is used for that.

Producer stores messages in the node


import com.google.protobuf.InvalidProtocolBufferException;
import message.Message;
import org.apache.zookeeper.KeeperException;
import org.apache.zookeeper.ZooKeeper;
import org.apache.zookeeper.data.Stat;

import java.io.IOException;

public class DistributedProducer {

    private ZooKeeperCoordinator zkc;

    private String root = "/Protocol Buffer";

    {
        try {
            zkc =
            new ZooKeeperCoordinator( "localhost:2181",
                                      root );
            zkc.connect();
            zkc.setUpRoot();
        } catch (InterruptedException e) {

            System.out.println( "InterruptedException" );

        } catch (IOException e) {

            System.out.println( "IOException" );
        }

    }

    public void send() throws IOException, InterruptedException, KeeperException {

        if( null != zkc ){
            ZooKeeper zk = zkc.getZookeeper();
            Stat stat =
            zk.setData( root + "/protocolbuffer",
                       getData(),
                       -1
                      );
            System.out.println( "Stat is  [" + stat + "]");
        }
    }

    private byte[] getData() throws InvalidProtocolBufferException {
        Message.Load message = Message.Load.newBuilder().setType(
                                 ( Message.Load.LoadType.HIGH)).build();
        Message.Load message1 = Message.Load.parseFrom( message.toByteArray() );
        System.out.println( "Distributed value is [" + message1.getType().toString() +"]");
        return message.toByteArray();
    }
}

Consumer consumes it.


import com.google.protobuf.InvalidProtocolBufferException;
import message.Message;
import org.apache.zookeeper.KeeperException;
import org.apache.zookeeper.ZooKeeper;
import org.apache.zookeeper.data.Stat;

import java.io.IOException;
import java.nio.ByteBuffer;
import java.nio.CharBuffer;
import java.nio.charset.CharacterCodingException;
import java.nio.charset.Charset;
import java.nio.charset.CharsetDecoder;

public class DistributedConsumer {

    private ZooKeeperCoordinator zkc;

    private String root = "/Protocol Buffer";

    {
        try {
            zkc =
            new ZooKeeperCoordinator( "localhost:2181",
                                      root );
            zkc.connect();
        } catch (InterruptedException e) {

            System.out.println( "InterruptedException" );

        } catch (IOException e) {

            System.out.println( "IOException" );
        }

    }

    public void receive() throws IOException, InterruptedException, KeeperException {

        Stat stat = null;

            if( null != zkc ){
                ZooKeeper zk = zkc.getZookeeper();
                if( null != zk ){
                    byte[] value =
                    zk.getData( root + "/protocolbuffer",
                               false,
                               stat
                              );
                    getData( value );
                }
            }
    }

    private void getData( byte[] value ){

        try {
            Message.Load message = Message.Load.parseFrom( value );
            System.out.println( "Distributed value is  [" + message.getType().toString() + "]");

        } catch (InvalidProtocolBufferException e) {

            System.out.println( "InvalidProtocolBufferException" + e.getMessage() );
            e.printStackTrace();

        }
    }
}

A client that connects to and sets up ZooKeeper


import org.apache.zookeeper.*;
import org.apache.zookeeper.data.Stat;

import java.io.IOException;
import java.util.concurrent.CountDownLatch;

public class ZooKeeperCoordinator {
    
    private String host;
    
    private String root;

    private Stat stat;

    public ZooKeeper getZookeeper() {
        return zookeeper;
    }

    private ZooKeeper zookeeper;

    public ZooKeeperCoordinator( String host,
                                 String root ){

        this.host = host;
        this.root = root;

    }

    public void setUpRoot(){

        System.out.println( "Set up root" );
        try {
            stat = zookeeper.exists( root,
                                     false );
            if( null == stat ){
                 zookeeper.create( root,
                                   new byte[ 0 ],
                                   ZooDefs.Ids.OPEN_ACL_UNSAFE,
                                   CreateMode.PERSISTENT);
            }
        } catch (KeeperException e) {
            System.out.println( "KeeperException" );

        } catch (InterruptedException e) {
            System.out.println( "InterruptedException" );
        }
    }

    public void connect() throws InterruptedException, IOException {

        System.out.println( "connect" );
        final CountDownLatch latch = new CountDownLatch( 1 );

        Watcher watcher = new Watcher(){

            public void process(WatchedEvent watchedEvent) {

                if( watchedEvent.getState() == Event.KeeperState.SyncConnected ){
                    latch.countDown();
                }
            }
        };

        zookeeper = new ZooKeeper( host, 1000, watcher );
        latch.await();
        System.out.println( "Awaiting connection" );
    }

    public void close() throws InterruptedException {
        zookeeper.close();
    }
}

The Protocol Buffers documentation has instructions to compile this simple .proto data structure

package message;

message Load {

  enum LoadType {
    HIGH = 0;
    MEDIUM = 1;
    LOW = 2;
  }


  optional LoadType type = 1 [default = MEDIUM];

}

Java Path Finder

Java Path Finder(JPF) is a model checker for Java programs and can check concurrent programs for deadlocks and data races which are more serious in muti-core environments. Java threads used by concurrent programs are context switched by the OS scheduler at run-time when we ‘yield’ a thread or due to some other reason. Even though it might be possible to test these scenarios extensively by profiling a few possible combinations of thread interleavings, it is quite tedious. So JPF tests this type of code by exploring all the code using algorithms to find errors.

JPF has quite a number of plug points and gov.nasa.jpf.jvm.VMListener is one such interface. I have implemented it to get data about the number of BLOCKED threads that the System Under Test(SUT) has and the monitor that they are blocked at. The data is still not clear but it is reasonable. I am not able to get clear data showing how many threads are blocked for a particular monitor to be released at a point in time. I think the code has to be refined.

JPF can be downloaded from http://babelfish.arc.nasa.gov/trac/jpf/ by using a mercurial eclipse plugin and it is quite easy to build and use.

I implemented gov.nasa.jpf.jvm.VMListener and added the classpath entry pointing to the class file to jpf-core.native_classpath

gov.nasa.jpf.jvm.VMListener based on JPF source code

public class LockContentionListener implements VMListener{


	@Override
	public void threadStarted(JVM vm) {
		printLocks( vm );
	}


	/**
	 * Thread and lock status.
	 * @param vm
	 */

	    private void printLocks( JVM jvm ){

		StringBuffer locksAndThreads = new StringBuffer();

		for ( ThreadInfo tf : jvm.getThreadList().getThreads()) {

		    ElementInfo ei = tf.getLockObject();
		    if (ei != null) {
		      if (tf.getState() == ThreadInfo.State.WAITING ||
				  tf.getState() == ThreadInfo.State.BLOCKED ) {
			  locksAndThreads.append( "\n");
			  locksAndThreads.append(tf.getStateDescription());
			  locksAndThreads.append(ei);
			  locksAndThreads.append(" \n Lock Count (" +                 ei.getMonitor().getNumberOfBlockedThreads()+ ")");

			    LinkedList locks = tf.getLockedObjects();
			    if (locks.isEmpty()) {
				locksAndThreads.append("  call stack:");
				    for (StackFrame frame : tf){
				      if (!frame.isDirectCallFrame()) {
					  locksAndThreads.append("\tat \n ");
					  locksAndThreads.append(frame.getStackTraceInfo());
				      }
				    }
			    }
				System.out.println( "Locks owned by Threads[ " + locksAndThreads + " \n ]");

		  }
		 }
		}
	    }

SUT

public class LockContention{
	
	private static final int NUM_THREADS = 2;
	
	
	public  static void main( String... argv ) throws InterruptedException {
		
		Thread[] readThreads = new Thread[ NUM_THREADS ];

		Locker locker = new Locker();
		
		for(  int i = 0 ; i &lt; readThreads.length; i ++ ){
			
			readThreads[ i ] = new Thread( new ReadLockContention( locker ) ){
				
				public String getName(){
					return "LockContention (Reader)[ " + getId() + " ]";
				}
				
				
			};
		}

		Thread[] writeThreads = new Thread[ NUM_THREADS ];

		for( int i = 0 ; i &lt; writeThreads.length; i ++ ){
			writeThreads[ i ] = new Thread( new WriteLockContention( locker ) ){

				public String getName(){
					return "LockContention (Writer)[" + getId() + " ]";
				}
				
			};
		}
		
		for( Thread t : readThreads ){
			t.start();
		}
		for( Thread t : writeThreads ){
			t.start();
		}
		for( Thread t : readThreads ){
			t.join();
		}

		for( Thread t : writeThreads ){
			t.join();
		}
		
	}


}

class ReadLockContention implements Runnable{
	
	private Locker locker;
	
	public ReadLockContention( Locker locker ){
		this.locker = locker;
	}

	@Override
	public void run() {
		locker.contendForReadLock();
		
	}

}

class WriteLockContention implements Runnable{
	
	private Locker locker;
	
	public WriteLockContention( Locker locker ){
		this.locker = locker;
	}

	@Override
	public void run() {
		locker.contendForWriteLock();
		
	}

}

class Locker{
	
	private final String lock = new String();

	/**
	 * Consider this as a read lock
	 */
	void contendForReadLock(){
			synchronized( lock ){
				try {
					Thread.sleep(50000);
				} catch (InterruptedException e) {
					System.out.println( "InterruptedException" );
			    }
				
			}
	}
	
	/**
	 * Consider this as a write lock
	 */
	void contendForWriteLock(){
			synchronized( lock ){
				
			}
	}
	
}

Output

The command

jpf +listener+=LockContentionListener LockContention.jpf

produces

Threads blocked on Locks[
thread id=2,name=Thread-2,status=BLOCKED,priority=5,lockCount=0,suspendCount=0java.lang.String@137
Lock Count (1) call stack: at
Locker.contendForReadLock(LockContention.java:97) at
ReadLockContention.run(LockContention.java:67)
]
Threads blocked on Locks[
thread id=1,name=Thread-1,status=BLOCKED,priority=5,lockCount=0,suspendCount=0java.lang.String@137
Lock Count (1) call stack: at
Locker.contendForReadLock(LockContention.java:97) at
ReadLockContention.run(LockContention.java:67)
]

The monitor is shown in bold. I want to know how many threads are blocked for a lock to be released.
It seems that these messages indicate that two threads(Thread-1,Thread-2) are blocked on the same monitor but I think this has to be investigated further. There is no reply to my forum question about this.

Sangam 2011

I recently visited bangalore to speak about Java Concurrency and a different set of problems like ‘False Sharing’, CAS instructions etc. Even model checkers like Java Path Finder was in the agenda. I spent quite a number of hours on the slides and the white paper. On the whole it was enlightening to investigate how Java programs are affected by multiple cores and shared memory and cache coherency.

The turnout for the conference was extremely poor but this affected only the Java tracks. It seems that the Java user groups in India are sometimes at loggerheads with each other and oneupmanship is not uncommon among the JUG’s.

I was mainly interested in the way Java problems perform on multi-core systems because there are so many tutorials on the net that show examples of other more common Java API’s. Hard problems like concurrency have their own attraction because we do not generally profile and investigate our commercial applications using Java Path Finder.

I am happy that I learnt quite a lot about relaxed memory models, java.util.concurrent and many other such topics. There is more to write about later in the blog about all this.