MindSpace

Sangam 2011

I recently visited bangalore to speak about Java Concurrency and a different set of problems like ‘False Sharing’, CAS instructions etc. Even model checkers like Java Path Finder was in the agenda. I spent quite a number of hours on the slides and the white paper. On the whole it was enlightening to investigate how Java programs are affected by multiple cores and shared memory and cache coherency.

The turnout for the conference was extremely poor but this affected only the Java tracks. It seems that the Java user groups in India are sometimes at loggerheads with each other and oneupmanship is not uncommon among the JUG’s.

I was mainly interested in the way Java problems perform on multi-core systems because there are so many tutorials on the net that show examples of other more common Java API’s. Hard problems like concurrency have their own attraction because we do not generally profile and investigate our commercial applications using Java Path Finder.

I am happy that I learnt quite a lot about relaxed memory models, java.util.concurrent and many other such topics. There is more to write about later in the blog about all this.

Fork Join

These are simple programs that might help one understand how to use java.util.concurrent.RecursiveAction which uses the ForkJoin framework. I am trying to analyze how this framework employs the work stealing algorithm so that the programs use multiple cores more efficiently.

The innards of this algorithm and its variataions do not seem to be so simple and I did not find any cheap tools that will help me understand how the cores are more efficiently used. I am looking for a way to visualize this algorithm in a simpler way. JavaFX looks appealing but the problem is that the actual algorithm should drive the UI instead of a visually appealing show that does not reveal the way the code behaves in a certain multi-core environment. So I am trying to manipulate the UI exactly when a task is stolen but how this type of code interacts with the framework is not known at this time.

MergeSort

public class SortTask {
	
    long[] aux = new long[ 40 ];

    static long[] list = new long[ 40 ];

    private void mergeSort( int lo, int hi){
 
    	if( lo < hi ){
    		
        	int mid = ( lo + hi )/ 2;
        	
        	mergeSort( lo, mid );
        	
        	mergeSort( mid + 1, hi );
        	
        	merge( lo, mid, hi );
    	}
    	
    }
    
	private void merge( int lo,
    		            int mid,
    		            int hi ){
    	 int i = lo, j = mid + 1;
    	 
    	 int k = lo;
    	 
    	 for( ; k <= hi ; k ++ ){
    		aux[ k ] = list[ k ];
    	 }
    	 k = lo;
    	 while( i <= mid && j <= hi){
    		 if( aux[i] < aux[j]){
    			 list[k++] = aux[i++];
    		 }else{
    			 list[k++] = aux[j++];
    		 }
    	 }
    	 while( i <= mid ){
			 list[k++] = aux[i++];
    	 }
     }
     
     private static long[] getArray(){
   	 
    	 Random random = new Random();
    	 
    	 for( int i = 0 ; i < list.length ; i ++ ){
    		 list[ i ] = random.nextInt(1000);
    	 }
    	 
    	 return list;
    	 
     }
     
     private void printArray(){
    	 for( int i = 0 ; i < list.length ; i ++ ){
    		 System.out.printf( "%2s ", list[ i ]);
    	 }
   		 System.out.printf( "%n");
     }
     
     public  static void main( String... argv ){
    	 list = SortTask.getArray();
    	 SortTask st = new SortTask();
    	 st.printArray();
    	 st.mergeSort( 0, list.length - 1 );
    	 st.printArray();
     }
     
}

MergeSort using ForkJoin

 
public class ForkJoinSortTask extends RecursiveAction{
	
	private static final long serialVersionUID = 1070860898589424509L;

	long[] aux = new long[ 40 ];

	private int lo;

	private int hi;

    long[] list = new long[ 40 ];

    public ForkJoinSortTask( long[] list, int lo, int hi ){
    	this.list = list;
    	this.lo = lo;
    	this.hi = hi;
    }
    
	public ForkJoinSortTask() {
   	    this.list = getArray();
    	this.lo = 0;
    	this.hi = list.length - 1;
	}

  
	private void merge( int lo,
    		            int mid,
    		            int hi ){
    	 int i = lo, j = mid + 1;
    	 
    	 int k = lo;
    	 
    	 for( ; k <= hi ; k ++ ){
    		aux[ k ] = list[ k ];
    	 }
    	 k = lo;
    	 while( i <= mid && j <= hi){
    		 if( aux[i] < aux[j]){
    			 list[k++] = aux[i++];
    		 }else{
    			 list[k++] = aux[j++];
    		 }
    	 }
    	 while( i <= mid ){
			 list[k++] = aux[i++];
    	 }
     }
     
     private long[] getArray(){
   	 
    	 Random random = new Random();
    	 
    	 for( int i = 0 ; i < list.length ; i ++ ){
    		 list[ i ] = random.nextInt(1000);
    	 }
    	 
    	 return list;
    	 
     }
     
     private void printArray(){
    	 for( int i = 0 ; i < list.length ; i ++ ){
    		 System.out.printf( "%2s ", list[ i ]);
    	 }
   		 System.out.printf( "%n");
     }
     
     public  static void main( String... argv ){
    	 ForkJoinSortTask st = new ForkJoinSortTask();
    	 st.printArray();
    	 ForkJoinPool fjp = new ForkJoinPool();
    	 fjp.invoke( st );
    	 st.printArray();
     }

	@Override
	protected void compute() {

    	if( lo < hi ){
    		
        	int mid = ( lo + hi )/ 2;
        	
            invokeAll( new ForkJoinSortTask(list, lo, mid),
                       new ForkJoinSortTask(list, mid + 1, hi));
        	
        	merge( lo, mid, hi );
    	}

	}
     
}

EasyMock IArgumentMatcher

When EasyMock is used to set expectations we might need to specify what parameters are used to call a method on the mock object. An ‘IArgumentMatcher’ is used for this purpose as this section from the EasyMock documentation describes.

Sometimes it is desirable to define own argument matchers. Let’s say that an argument matcher is needed that matches an exception if the given exception has the same type and an equal message. It should be used this way:

 
     IllegalStateException e = new IllegalStateException("Operation not allowed.")
     expect(mock.logThrowable(eqException(e))).andReturn(true);

I wanted to specifically check if the argument matcher matches a parameterized java.util.Set with a certain number of elements in it. This is how I am able to do it.

So here I pass the expected size of the Set as a constructur parameter but I use Generics reflection to get the raw type for comparison. The idea to use an anonymous class in the line

 

   SetReturnTest<ObjectInstance> returnTest = new SetReturnTest<ObjectInstance>( set.size() ){};

to preserve the raw type at run-time is from Neil Gafter.

It does seem that not all raw types are erased in all cases. This is still hard to do and I believe I get Generics wrong in many cases.

package com.monitor.test.suite.mock;

import org.easymock.EasyMock;
import org.easymock.IArgumentMatcher;

import javax.management.ObjectInstance;
import java.lang.reflect.ParameterizedType;
import java.lang.reflect.Type;
import java.util.Set;



public class SetReturnTest<T> implements IArgumentMatcher {


    /**
     * The expected reflective type of objects in the returned java.util.Set
     */
    Type objectTypeExpected;
    /**
     * The expected size of the returned java.util.Set
     */
    protected int size;
    /**
     * The reflective type of objects in the returned java.util.Set
     */
    protected Type type;

    /**
     * The type and number of the objects in the java.util.Set
     * .
     */
    public SetReturnTest( int size ) {
        /**How many are expected in the java.util.Set?*/
        this.size = size;
        /**
         * The  parameterized type is matched
         * with the actual type expected at run-time.
         *
         */
        ParameterizedType pt =
                    (ParameterizedType) (getClass().getGenericSuperclass());
        type = pt.getActualTypeArguments()[ 0 ];
    }

    /**
     * EasyMock via the static method reportMatcher(IArgumentMatcher matcher), 
     * and return a value so that it may be used inside the call (typically 0, null or false). 
     */
    public Set eqSet( Type type, Set set ) {
        objectTypeExpected = type;
        SetReturnTest<ObjectInstance> returnTest = new SetReturnTest<ObjectInstance>( set.size() ){};
        EasyMock.reportMatcher( returnTest );
        return null;
    }


    /**
     * The actual java.util.Set is compared with the
     * one expected by the mock test. There are unchecked
     * warnings here but this seems to be the best possible attempt.
     */
    public boolean matches( Object actual ) {

        /** Unchecked is a potential problem*/
        @SuppressWarnings("unchecked")
        SetReturnTest setReturnTest = (SetReturnTest) actual;

        if( setReturnTest.size != size ){
            return false;
        }
        Type clazz1 = setReturnTest.type;
        Class clazz2 = (Class)objectTypeExpected;

        boolean match =
                clazz2.isAssignableFrom( (Class)clazz1 );

        return match;
    }

    /**
     * Append a simple error to identify which
     * matcher was used.
     */
    public void appendTo(StringBuffer buffer) {
        buffer.append( "SetReturnTest matcher" );
    }
}

Mock tests for JMX

There is not much scope for this blog if code is posted without explanation but I can explain only what I understand. There is magic going on behind this code.

I had to write mocks for some JMX classes for a project that I could not convince the management to use. JMX is a useful toolkit to monitor complex pipelines of processes and is integral to DevOps. So firms that want to leverage this idea should think differently.

Does it seem logical ?

This argument fell flat on its face.

This code mocks “ManagementFactory.getPlatformMBeanServer()” and in doing so I exercised EasyMock. It is not easy to mock statics but these EasyMock annotations help. So I am able to set the expectations and make the Platform Bean Server return a mock of the MBeanServer. This is very useful because now I can register MBeans and write more mock tests.

@RunWith(PowerMockRunner.class)
@PrepareForTest(ManagementFactory.class)
@PowerMockIgnore(value = {"javax.management.*"})
public class MBeanQueryTestCase  extends MonitorTestCase {


    private MBeanServer mBeanServerMock;

    private final String CLASSNAME = this.getClass().getName();

    @Before
    public void setUp() throws Exception {
        mockStatic(ManagementFactory.class);
        expect( ManagementFactory.getPlatformMBeanServer() ).
                                    andReturn( EasyMock.createMock(MBeanServer.class) );

        replayAll();
        mBeanServerMock = ManagementFactory.getPlatformMBeanServer();
        verifyAll();

    }
}

Mock testing is another useful toolkit in the arsenal of a hardy developer but that is another idea that fell flat on its face.

Habit of reading source code

Reading code can be very enlightening but I have been following this practice only recently.One might come across nuggets that explain a particular code pattern clearly when one reads code.
Twitter has open-sourced its ‘Commons’ library and I came across the Curiously Recurring Generic Pattern and the Self-bounding pattern in it. These two are discussed in ‘Thinking in Java’ by Bruce Eckel. He also gives an exercise at the end of Page 709.

Exercise 34: Create a self-bounded generic type that contains an
abstract method that takes an argument of the generic type parameter and
produces a return value of the generic type parameter. In a non-abstract
method of the class, call the abstract method and return its result. Inherit
from the self-bounded type and test the resulting class

The code for this exercise could be

public abstract class SelfBound< T extends SelfBound<T>>{

    public abstract T get( T t );

    public T getValue( T t){
         return get( t );
    }

public class SubSelfBound extends SelfBound{

    
    public SubSelfBound get(SubSelfBound subSelfBound) {
        return this;
    }

    public static void  main( String[] argv ){

        SubSelfBound subSelfBound = new  SubSelfBound();

        subSelfBound.test();
    }

    private void test() {

        SubSelfBound subSelfBound = new  SubSelfBound();

        subSelfBound =  getValue( subSelfBound );//The sub-class's inherited method is forced
                                                 //to take only the derived type as parameter
                                                 //and not the base type. The parameter varies
                                                 //and matches the derived type( Covariant )
        
        SelfBound selfBound =  getValue( subSelfBound );//It also returns the super type.
                                                        //This is related to the covariant
                                                        //return types according to the book.

    }
}

Even though I might not have the absolutely correct solution here the idea is clear and it is that the Curiously Recurring Generic Pattern and the Self-bounding pattern are similar and the Self-bounding pattern permits a type of covariant argument types.

I used to think only covariant return types are possible in J2SE 5 but there seems to be a way to implement covariant argument types also though the practical applications of this concept could stupify the developers because not many are
used to advanced generics.

I understood all of this by reading the Twitter Commons source code that has used self-bounding types like this.

public interface Unit<U extends Unit<U>>;{

  double multiplier();
}

If one is not used to thinking about Java generics deeply this type of code will be
hard to understand and implement.