June | 2014 | MindSpace

Succinctness of Lambdas

June 30, 2014 Leave a comment

I think this is impressive. Compare it with the earlier way of iterating over the data. I am streaming a list of java.lang.management.GarbageCollectorMXBean‘s, accessing some data from each and creating an ArrayList of GCType objects in one line of code.

public class GarbageCollectionProbe implements Probe{
	
	Logger l = Logger.getLogger("collection.gc");

	public  List<GCType> getMemoryPool() {
		
		connectJMXRemote(9010);
 	
			List<GCType> pools =
					ManagementFactory.getGarbageCollectorMXBeans().stream().map(p-> new GCType(p.getName(),
							                                                                   p.getCollectionCount(),
							                                                                   p.getCollectionTime()))
					.collect(Collectors.toCollection(() -> new ArrayList<GCType>()));
			pools.stream().forEach(p-> l.debug(p));
			return pools;
	}	

}

Filed under Java, Lambda

Optional and Lambda

June 28, 2014 Leave a comment

This is a piece of code that I converted using Lambdas. It is using JMX but that is irrevelant.

I wanted to use this method initially. This is the JDK source.

    /**
     * Returns an {@code Optional} describing the specified value, if non-null,
     * otherwise returns an empty {@code Optional}.
     *
     * @param <T> the class of the value
     * @param value the possibly-null value to describe
     * @return an {@code Optional} with a present value if the specified value
     * is non-null, otherwise an empty {@code Optional}
     */
    public static <T> Optional<T> ofNullable(T value) {
        return value == null ? empty() : of(value);
    }

But once I used Lambda code I realized the API returns Optional<MemoryPoolMXBean>


	   Optional<MemoryPoolMXBean> mpx = Optional.empty();


//			   for (MemoryPoolMXBean pool : memoryBeans) {
//		       		l.debug(pool.getName());
//		               if (pool.getName().equals(METASPACE)) {
//		            	   mpx =  Optional.ofNullable(pool);
//		               }
//		       }
			   
			   mpx =
			   memoryBeans
			    .stream()
			    .filter(p ->
			        {
			       	l.debug(p.getName());
			        return p.getName().equals(METASPACE);
			        })
			        .findFirst();

I don’t want to delve too deep into this as it doesn’t break the code that calls this.

Filed under Java, Lambda

rCharts and nvd3

June 26, 2014 Leave a comment

I have been playing with some JavaScript chart libraries including nvd3 . These JS libraries are all wrapped by R code in the form of rCharts.

I have a data frame like this.

Var1 Freq
levelI 13
levelII 13
levelIII 12
levelIV 12

I want to plot a discreet bar chart using nvd3 and each bar should be of a different color depending on Var1. Var1 will have duplicate values.


#The type of <em>Var1</em> is integer in my data frame. So instead of debugging that
#I just convert it to "character" using this line.
bpf3$Var1 <- paste(bpf3$Var1,"",sep="")

#Match the first column with a hard-coded value and append a color
for(i in 1:nrow(bpf3)) {
    print(identical(bpf3[i,1],"levelI"))
    if(identical(bpf3[i,1],"levelI")){
      bpf3[i,1] <- paste(bpf3[i,1],"green",sep=":") 
    }else{
      bpf3[i,1] <- paste(bpf3[i,1],"orange",sep=":") 
    }
}
p2 = nPlot(x = "Var1", y = "Freq", data = bpf3, type = "discreteBarChart")

p2$chart(color = "#! function(d, x){ var color = d.Var1; return color.split(':')[1];} !#")

So it is possible to use a function that assigns a color to a bar in nvd3. rCharts lets you define a function like this.

The result is this. This is not an elegant way though.

Update :

The author of http://timelyportfolio.blogspot.in/ pointed out that there is a function that can be used like this.


  p2$chart(
    color = "#! function(d){
      var ourColorScale = d3.scale.ordinal().domain(['levelI','levelII']).range(['red','blue']);
      return ourColorScale(d.Event);
    }!#")

It works splendidly.

Filed under nvd3, R, rCharts

Display JVM heap details using rJava and R

June 24, 2014 Leave a comment

I finished the first full flow that pulls data from the JVM using JMX and dynamically updates a graph in a browser. I use rJava as the layer between R and Java. I code many lines of java, of course, using jdk 8.

realtimeyoungdata <- function(){
  realtimeyoungdataobserver <- .jnew("com/rxjava/jmx/YoungGenPeriodicObserver")
  .jcall(realtimeyoungdataobserver,"V","consume")
  occupancy <- .jcall(realtimeyoungdataobserver ,"Lcom/rxjava/jmx/YoungGenOccupancy;","getOccupancy")
  used <- occupancy$getUsed()
  max <- occupancy$getMax()
  print(paste("User[", used,"] Max [", max,"]"))
  result <- c(used, max) 
  return(result) 
}

Filed under Java, rJava

Interface default methods

June 24, 2014 Leave a comment

I am coding feverishly to complete a web interface to the JVM internals like memory, threads etc. This is my first set of default methods in an interface.

I have not read the JLS comparison between this and other inheritance mechanisms but this serves a purpose.

package com.memory.probe;

import java.io.IOException;
import java.io.PrintWriter;
import java.io.StringWriter;
import java.util.AbstractQueue;
import java.util.concurrent.ArrayBlockingQueue;

import javax.management.remote.JMXConnector;
import javax.management.remote.JMXConnectorFactory;
import javax.management.remote.JMXServiceURL;

import org.apache.log4j.Logger;

public interface Probe {
	
	Logger l = Logger.getLogger("generation.gc");
	
	AbstractQueue<Long> abq = new ArrayBlockingQueue<Long>(100);

	default void connectJMXRemote(int port){
	      JMXServiceURL url;
	  	try {
	  		url = new JMXServiceURL(
	  		            "service:jmx:rmi:///jndi/rmi://localhost:" + port + "/jmxrmi");
	  		JMXConnector jmxc = JMXConnectorFactory.connect(url, null);
	  		l.debug("Remote JMX connection is successful");
	  	} catch (IOException e) {
	  		l.debug(getStackTrace(e));
	  	}

	}
	
	default String getTrace(Throwable t){
		return getStackTrace(t);
	}
	
    static String getStackTrace(Throwable t){
    	StringWriter sw = new StringWriter();
    	PrintWriter pw = new PrintWriter(sw);
    	t.printStackTrace(pw);
    	return sw.toString();
    	
    }


}

Filed under Java

Real-time graph showing heap generations

June 23, 2014 Leave a comment

I am able to dynamically update a graph in real-time with test data using the code shown in the previous post. The graph is refreshed without refreshing the entire browser. Even though the particular JavaScript graph library will be replaced with another better one, at this time the tests are successful.
Now I will be able to use JMX to get the YoungGen data from the heap and show actual data as the GC collects the garbage.

All the code will be pushed to Github.

Filed under Garbage Collection, Java, R, Serviceability

Update some columns of a R data frame

June 22, 2014 Leave a comment

I wanted to code a R function that updates some columns in each row and not the entire row.

I declare a global variable to keep track of which row’s columns are being updated. I want to stop when the last row is reached.


youngdata <- function() {
    ydata <<- 1
}

This function creates a data frame which has 3 columns but valid data in only the first column. The 2nd and 3rd columns will be updated one row at a time.

youngdata <- function() {
  younggen <<- data.frame(lapply(as.Date('2014-08-06') + 0:0, seq, as.Date('2014/10/08'), '1 days'));
  younggen['eden'] <- 0
  younggen['survivor'] <- 0
  colnames(younggen) <- c("date","eden","survivor")
}

> younggen
date eden survivor
1 2014-08-06 0 0
2 2014-08-27 0 0
3 2014-09-17 0 0
4 2014-10-08 0 0


loadeddata <- function(df){
         if( ydata > nrow(df)){
             return
         }
         print("Loading new data")
         newdata <- data.frame(sample(1:40, 1, replace=F), sample(1:40, 1, replace=F),stringsAsFactors=FALSE)
         colnames(newdata) <- c("eden","survivor")    
         df[ydata,which(names(df) %in% names(newdata))]  <- newdata
 return(df)
 }

So every time the function loadeddata is called one row’s columns are updated and it stops when all the rows are finished. I increment ydata in another part of the code not shown here.

> younggen
date eden survivor
1 2014-08-06 2 16
2 2014-08-27 0 0
3 2014-09-17 0 0
4 2014-10-08 0 0

This code is used to test R-shiny’s ReactiveTimer that enables the code to dynamically update a graph with new data.

Filed under R

Compile rJava from source

June 16, 2014 1 Comment

I am trying to compile rJava from source on OSX 10.7.5 using

install.packages(“rJava”,type=”source”)

The motive is this. I am using code compiled with jdk1.8.0_05 and call it using rJava. When I do this there is a mismatch between the class file version of code compiled with jdk1.8.0_05 and the class files rJava recognizes.

llvm-gcc-4.2 -arch x86_64 -std=gnu99 -c -o rjava.o rjava.c -g -Iinclude -DRIF_HAS_CSTACK
-DRIF_HAS_RSIGHAND -mtune=core2 -g -O2 –
I/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/jre/../include –
I/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/jre/../include/darwin –
fno-common –
I/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/jre/../include –
I/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/jre/../include/darwin
llvm-gcc-4.2 -arch x86_64 -std=gnu99 -o libjri.jnilib Rengine.o jri.o Rcallbacks.o Rinit.o
globals.o rjava.o -dynamiclib -framework JavaVM -F/Library/Frameworks/R.framework/.. –
framework R -llzma -licucore -lm -liconv
ld: library not found for -llzma
collect2: ld returned 1 exit status
make[2]: *** [libjri.jnilib] Error 1
make[1]: *** [src/JRI.jar] Error 2
make: *** [jri] Error 2
ERROR: compilation failed for package ‘rJava’

I have installed xz using homebrew but that didn’t help.

rJava binaries are installed and working properly. It is the source compilation that causes this.

I have executed R CMD javareconf.

Java interpreter : /usr/bin/java
Java version : 1.8.0_05
Java home path : /Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/jre
Java compiler : /usr/bin/javac
Java headers gen.: /usr/bin/javah
Java archive tool: /usr/bin/jar
Non-system Java on OS X

trying to compile and link a JNI progam
detected JNI cpp flags : -I$(JAVA_HOME)/../include -I$(JAVA_HOME)/../include/darwin
detected JNI linker flags : -L$(JAVA_HOME)/lib/server -ljvm
llvm-gcc-4.2 -arch x86_64 -std=gnu99 -I/Library/Frameworks/R.framework/Resources/include –
DNDEBUG -I/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/jre/../include –
I/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/jre/../include/darwin –
I/usr/local/include -fPIC -mtune=core2 -g -O2 -c conftest.c -o conftest.o

llvm-gcc-4.2 -arch x86_64 -std=gnu99 -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -single_module -multiply_defined suppress -L/usr/local/lib -L/usr/local/lib -o conftest.so conftest.o -L/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/jre/lib/server -ljvm -F/Library/Frameworks/R.framework/.. -framework R -Wl,-framework -Wl,CoreFoundation

JAVA_HOME : /Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/jre
Java library path: $(JAVA_HOME)/lib/server
JNI cpp flags : -I$(JAVA_HOME)/../include -I$(JAVA_HOME)/../include/darwin

JNI linker flags : -L$(JAVA_HOME)/lib/server -ljvm
Updating Java configuration in /Library/Frameworks/R.framework/Resources
Done.

None of the solutions posted on the internet helped me until I manually downloaded xz-5.0.5.tar.gz into /usr/local and executed

rJava compiled successfully after this.

Filed under rJava

R Shiny

June 8, 2014 Leave a comment

I have recently started to code a web dashboard to show information like heap usage in HotSpot. So initially I setup a Shiny server. The part that connects to HotSpot is not ready but this is my first Shiny UI. This is a Twitter BootStrap UI.

Part of the shiny server code is this. Now the data is generated by R code and later the data will be extracted from the JVM using JMX and other serviceability API’s.

output$metaspace <- renderChart({
  metacapacity <- data.frame(lapply(as.Date('2014-08-06') + 0:0, seq, as.Date('2014/10/08'), '1 weeks'));
  metacapacity['init'] <- 300
  metacapacity['committed'] <- 700
  colnames(metacapacity) <- c("date","init","committed")
  metacapacity  <- transform(metacapacity,date=as.character(date))
  ms <- mPlot(x = "date", y = c("init", "committed"), type = "Area", data = metacapacity)
  ms$addParams(height = 300, dom = 'metaspace')
  ms$set(title="MetaSpace")
  return(ms)
  })

Filed under Java, R, rJava

CompletableFuture

June 5, 2014 Leave a comment

The short definition of java.util.concurrent.CompletableFuture<T> is

A Future that may be explicitly completed (setting its value and status), and may be used as a CompletionStage, supporting dependent functions and actions that trigger upon its completion.

Code that conforms to the Reactive manifesto is all the rage now. The Play framework embodies it.

This example uses CompletableFuture and Apache HTTP Components to download data and react to its completion. I will update this as I learn more about the API and explain the asynchronous nature of this type of code.

public class ReactiveCompletion {

    ExecutorService executor = Executors.newFixedThreadPool(4);

    private void fetch() throws IOException{

        CountDownLatch latch = new CountDownLatch(1);
        CompletableFuture future =
        CompletableFuture.anyOf(CompletableFuture.supplyAsync(() -> streamData())
        .thenAccept(content -> {
            System.out.println("Completed");
            latch.countDown();
        })
        );
        try {
            latch.await();
        } catch (InterruptedException e) {
            System.out.println("Interrupted");
        }
    }

    private Content streamData(){
        Content content = null;
        try{
            CloseableHttpClient httpclient = HttpClients.createDefault();
            HttpGet httpget = new HttpGet("http://data.gov.in/node/104089/datastore/export/json");
            CloseableHttpResponse response = httpclient.execute(httpget);
            try {
                HttpEntity entity = response.getEntity();
                if (entity != null) {
                    Scanner in = new Scanner(new BufferedReader(new InputStreamReader(entity.getContent())));
                    while(in.hasNextLine() ){
                        System.out.println(in.nextLine());
                    }
                }
            } finally {
                response.close();
            }
        }catch (  IOException io){
            io.printStackTrace();
        }
        return content;
    }

    public static void main(String... argv) throws IOException {
        new ReactiveCompletion().fetch();
    }
}

Filed under Java, Reactive Manifesto

← Older posts

Blogroll