Repeatable Annotations

It looks like the annotations can repeat according to Repeating Annotations.

I am still using

java version "1.8.0-ea"
Java(TM) SE Runtime Environment (build 1.8.0-ea-b113)
Java HotSpot(TM) 64-Bit Server VM (build 25.0-b55, mixed mode)

This code has a problem.

  1. Arrays.asList(typeMirror.getAnnotationsByType(RepeatableAnnotation.TypeUse.class))

    This does not print anything. There are two annotations.

This could be a problem with the particular JDK 8 build. The latest is b120. I have to try the latest build.

Update: This is most likely a code problem because even b120 does not help.

Update 1: This is not a code problem but b120 has some bugs according to the type-annotations-dev list.

The result is this.


Annotation [@com.test.RepeatableAnnotation.TypeUses({@com.test.RepeatableAnnotation.TypeUse(0), @com.test.RepeatableAnnotation.TypeUse(1)})] [com.test.RepeatableAnnotation.Test1]
[value()][{@com.test.RepeatableAnnotation.TypeUse(0), @com.test.RepeatableAnnotation.TypeUse(1)}]
Annotation [[]] [java.lang.Integer]

I am not able to get the annotations applied on the bound Integer

package com.test;

import java.lang.annotation.*;

public class RepeatableAnnotation {

    @Retention(RetentionPolicy.RUNTIME)
    @Target(ElementType.TYPE_USE)
    public @interface TypeUses {
        TypeUse[] value();
    }

    @Retention(RetentionPolicy.RUNTIME)
    @Target(ElementType.TYPE_USE)
    @Repeatable(value = TypeUses.class)
    protected @interface  TypeUse {
        int value();
    }

    @TypeUse(value = 0) @TypeUse(value = 1)class Test1< T extends @TypeUse(value = 2) @TypeUse(value = 3) Integer> {

        /**
         * A type annotation is permitted in front of a constructor declaration, where declaration annotations are already
         permitted.
         */
        @TypeUse(value = 0) <S> Test1(String s){

        }
    }

}

Annotation Processor


package com.test;

import javax.annotation.processing.AbstractProcessor;
import javax.annotation.processing.RoundEnvironment;
import javax.annotation.processing.SupportedAnnotationTypes;
import javax.annotation.processing.SupportedSourceVersion;
import javax.lang.model.SourceVersion;
import javax.lang.model.element.*;
import javax.lang.model.type.ExecutableType;
import javax.lang.model.type.TypeMirror;
import javax.lang.model.type.TypeVisitor;
import javax.lang.model.util.SimpleTypeVisitor8;
import java.util.*;

import static javax.lang.model.util.ElementFilter.typesIn;

@SupportedAnnotationTypes("*")
@SupportedSourceVersion(SourceVersion.RELEASE_8)
public class RepeatableAnnotationFinder extends AbstractProcessor{


    public RepeatableAnnotationFinder(){
        super();
    }

    @Override
    public boolean process(Set<? extends TypeElement> annotations, RoundEnvironment roundEnv) {

        TypeElement tye = null;

        if( !roundEnv.processingOver()){

            Iterator<? extends Element> te = annotations.iterator();

            for (TypeElement element : typesIn(roundEnv.getRootElements()) ) {

                processEnclosedElements(element.getEnclosedElements(),
                                        roundEnv);

            }
        }
        return false;
    }

    private void processEnclosedElements(List<? extends Element> enclosedElements,
                                         RoundEnvironment roundEnv) {

        for( Element e : enclosedElements ){

            if (e.getKind() == ElementKind.CLASS){

                System.out.println( "Annotation [" + e.getAnnotationMirrors() + "] [" + e  + "]");

                processClassTypeParameters(e);
            }
        }
    }

    private void processClassTypeParameters(Element e) {

        for (TypeParameterElement typeParameterElement : ((TypeElement) e).getTypeParameters()) {

            for( AnnotationMirror am  : typeParameterElement.getAnnotationMirrors() ){

                System.out.println( "Annotation [" + am + "] [" + typeParameterElement  + "]");

                for(Map.Entry< ? extends ExecutableElement, ? extends AnnotationValue> m : am.getElementValues().entrySet()){

                    System.out.println( "[" + m.getKey() +"][" + m.getValue() + "]");

                }

            }

            for (TypeMirror typeMirror : typeParameterElement.getBounds()) {

                System.out.println( "Annotation [" + Arrays.asList(typeMirror.getAnnotationsByType(RepeatableAnnotation.TypeUse.class)) + "] [" + typeMirror  + "]");

            }
        }
    }
}

So I will wait for a newer build before trying this again.

Annotation Processors

Lambdas are very popular now. But I realized I haven’t used the annotation processor API since JSE 5. None of the projects I came across has even used good generics. So even JSE 5 is not properly used.

I just compiled and executed this. It is not complete because I am trying to print all the annotations actually used in the code shown(Test.java) in my previous blog entry.

I am using

java version "1.8.0-ea"
Java(TM) SE Runtime Environment (build 1.8.0-ea-b113)
Java HotSpot(TM) 64-Bit Server VM (build 25.0-b55, mixed mode)

Now it is printing this.

Annotation [com.test.Test.TypeUse] [[com.test.Test.Test1]]
Annotation [com.test.Test.TypeParameter] [[T, S]]
Annotation [com.test.Test.TestTypeParameter] [[T, S]]
Annotation [@com.test.Test.TypeUse] [com.test.Test.Test1]
Annotation [] [java.lang.Integer]
Annotation [@com.test.Test.NonNull] [argv]

I will update the code later once I manage to fix it.

Update : Now it is printing this. Code is updated. I will explain the changes.

Annotation [com.test.Test.TypeParameter] [[T, S]]
Annotation [com.test.Test.TestTypeParameter] [[T, S]]
Annotation [@com.test.TypeUse] [com.test.Test.Test1]
Annotation [[Lcom.test.TypeUse;@2a5ca609] [java.lang.Integer]
Annotation [@com.test.Test.NonNull] [argv]
-> (java.lang.String[])void

Change 1

This method which I used earlier turns the TypeMirror back to a DeclaredType. That is not required.

    private void processClassTypeParameters(Element e) {
        for (TypeParameterElement typeParameterElement : ((TypeElement) e).getTypeParameters()) {
            //System.out.println( &quot;Annotation [&quot; + typeParameterElement.getAnnotationMirrors() + &quot;] [&quot; + typeParameterElement  + &quot;]&quot;);
            for (TypeMirror typeMirror : typeParameterElement.getBounds()) {
                System.out.println( &quot;Annotation [&quot; + ((DeclaredType)typeMirror).getAnnotationMirrors() + &quot;] [&quot; + typeMirror  + &quot;]&quot;);
            }
        }
    }

This method is replaced with the new method in Test.java shown below. typeMirror.getAnnotationsByType(TypeUse.class) is sufficient to get the annotation on Integer

Change 2

I introduced a new method to get the annotations inside a method and source on which they are applied. But that is not possible. Alex Buckley, the JSR 308 spec. lead replied that

There is no access to annotations in a method body from the Language Model API (or the Core Reflection API).

Actually I introduced this method and called it like this e.asType().accept(visitor(),null); to try to get the annotations from inside the method. This is the method that is printing

    -> (java.lang.String[])void

    private TypeVisitor<Boolean, Void> visitor() {
        return new SimpleTypeVisitor<Boolean, Void>() {
            public Boolean visitExecutable(ExecutableType t, Void v) {
                System.out.println("-> " + t);
                return true;
            }
        };
    }


AnnotationFinder.java

package com.test;

import javax.annotation.processing.AbstractProcessor;
import javax.annotation.processing.RoundEnvironment;
import javax.annotation.processing.SupportedAnnotationTypes;
import javax.annotation.processing.SupportedSourceVersion;
import javax.lang.model.SourceVersion;
import javax.lang.model.element.*;
import javax.lang.model.type.DeclaredType;
import javax.lang.model.type.ExecutableType;
import javax.lang.model.type.TypeMirror;
import javax.lang.model.type.TypeVisitor;
import javax.lang.model.util.ElementFilter;
import javax.lang.model.util.SimpleTypeVisitor6;
import javax.lang.model.util.SimpleTypeVisitor8;
import java.util.Iterator;
import java.util.List;
import java.util.Set;

import static javax.lang.model.util.ElementFilter.typesIn;

@SupportedAnnotationTypes("*")
@SupportedSourceVersion(SourceVersion.RELEASE_8)
public class AnnotationFinder extends AbstractProcessor{


    public AnnotationFinder(){
        super();
    }

    @Override
    public boolean process(Set<? extends TypeElement> annotations, RoundEnvironment roundEnv) {

        TypeElement tye = null;

        if( !roundEnv.processingOver()){

            Iterator<? extends Element> te = annotations.iterator();

            for (TypeElement element : typesIn(roundEnv.getRootElements()) ) {

                processEnclosedElements(element.getEnclosedElements(),
                                        roundEnv);

            }
        }
        return false;
    }

    private void processEnclosedElements(List<? extends Element> enclosedElements,
                                         RoundEnvironment roundEnv) {

        for( Element e : enclosedElements ){

            if( e.getKind() == ElementKind.ANNOTATION_TYPE &&

                    !roundEnv.getElementsAnnotatedWith((TypeElement) e).isEmpty()){

                System.out.println( "Annotation [" + e + "] [" +
                        roundEnv.getElementsAnnotatedWith((TypeElement) e) + "]");

            }else if( e.getKind() == ElementKind.METHOD ){

                for (VariableElement variableElement : ((ExecutableElement) e).getParameters()) {

                    System.out.println("Annotation [" + variableElement.getAnnotationMirrors() + "] [" +

                                                                                         variableElement + "]");
                    e.asType().accept(visitor(),null);

                }

            }else if (e.getKind() == ElementKind.CLASS){

                System.out.println( "Annotation [" + e.getAnnotationMirrors() + "] [" + e  + "]");

                processClassTypeParameters(e);
            }
        }
    }

    private TypeVisitor<Boolean, Void> visitor() {

        return new SimpleTypeVisitor8<Boolean, Void>() {

            public Boolean visitExecutable(ExecutableType t, Void v) {

                System.out.println("-> " + t);

                return true;
            }
        };
    }

    private void processClassTypeParameters(Element e) {

        for (TypeParameterElement typeParameterElement : ((TypeElement) e).getTypeParameters()) {

            //System.out.println( "Annotation [" + typeParameterElement.getAnnotationMirrors() + "] [" + typeParameterElement  + "]");

            for (TypeMirror typeMirror : typeParameterElement.getBounds()) {

                System.out.println( "Annotation [" + typeMirror.getAnnotationsByType(Test.TypeUse.class) + "] [" + typeMirror  + "]");

            }
        }
    }
}

JSR 308

The annotation system has been extended. I realized that there are more annotation types and more source locations where they are permitted.

I compiled a simple example. The specification is here

java version "1.8.0-ea"
Java(TM) SE Runtime Environment (build 1.8.0-ea-b113)
Java HotSpot(TM) 64-Bit Server VM (build 25.0-b55, mixed mode)


package com.test;

import java.lang.annotation.ElementType;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.lang.annotation.Target;


public class Test {

    @Retention(RetentionPolicy.RUNTIME)
    @Target(ElementType.TYPE_USE)
    @interface  TypeUse {}

    @Retention(RetentionPolicy.RUNTIME)
    @Target(ElementType.TYPE_PARAMETER)
    @interface  TypeParameter {}

    @Retention(RetentionPolicy.RUNTIME)
    @Target(ElementType.TYPE_PARAMETER)
    @interface  TestTypeParameter {}

    @Retention(RetentionPolicy.RUNTIME)
    @Target(ElementType.PARAMETER)
    @interface NonNull {}


    /**
             * ElementType.TYPE PARAMETER stands for a type parameter — that is, the declaration of a type variable. Examples
             are in generic class declarations class MyClass<T> {...}, generic method declarations <T> foo(...) {...}, and
             wildcards List<?>, which declare an anonymous type variable.
             * @param <T>
             */
    @TypeUse class Test1<@TypeParameter @TestTypeParameter T extends @TypeUse Integer> {

        /**
         * A type annotation is permitted in front of a constructor declaration, where declaration annotations are already
         permitted.
         */
        @TypeUse <@TypeParameter @TestTypeParameter S> Test1(String s){

        }
    }


    public static void main(  @NonNull String... argv ){

        Object o = ( @TypeUse String ) new String( "Test" );

        if( o instanceof @TypeUse String ){

        }
        try{

        }catch( @NonNull Exception e){

        }

    }
}
        List list1 = list.
                       stream().
                        map(p::matcher).
                          filter(Matcher::find).map(matcher -> matcher.group()).
                              collect(Collectors.toCollection(@ NonNull ArrayList::new));

Area between curves

This is what I like about ‘R’. This one line is enough to apply a shade of color
to the area between two curves. Apart from the functional programming aspects(
http://adv-r.had.co.nz/), I am interested in its powerful API’s used to visualize and parse data.

polygon( c(data$Time, rev(data$Time)),
         c(as.numeric(data$Used), rev(as.numeric(data$Committed))),
         col = "antiquewhite1",
		 border = NA )

code-cache

Error bars using ‘R’

I believe our measurements are uncertain and we need to show the errors in our capacity measurement plots. I suspect that we are making fundamental mistakes in our attempts to gather performance statistics and drawing graphs. All the more reason for showing these uncertainties. Our management and clients should not be mislead by the lack of skills of our Capacity planners.

This code and the graph are used to learn one aspect of showing such errors. I am yet to investigate the type of errors and their statistical significance.

If there is a mistake I will make corrections to this blog entry.

Updated : Code and graph.

 this.dir <- dirname(parent.frame(2)$ofile) 
setwd(this.dir)
 #Reference values plotted on x-axis. These are constant.
 #These values could be time of day. So every day at the same
 #time we could collect other measurements
 referenceset <- data.frame(c(5,10,15,20,25,30,35,40,50,60))
 colnames( referenceset) <- c("reference")

 #These are the sets of measurements. So every day at the same
 #time we could collect several samples. This is simulated now.
 sampleset <- data.frame( matrix(sample(1:2, c(20000), replace = TRUE), ncol = 2000) )
 
 sampleset <- cbind( sampleset, referenceset )
 
 #Calculate mean
 sampleset$mean <- apply(sampleset[,1:10],2,mean)
 
 #Calculate Standard Deviation
 sampleset$sd <- apply(sampleset[,c(1:10)],2,sd)
 
 #Calculate Standard Error
 sampleset$se <- sampleset$sd / sqrt(10)
 
 #print(sampleset)

	png(
	"errorbars.png",
	width =500, height = 510)
 
 plot( sampleset$reference,
       sampleset$mean,
	   las=1,
	   ylab="Mean of 'y' values",
	   xlab="x",
      ylim=c(0,3),
	  type="l",
	  lwd=1,
	   col="blue"
      );
	  
arrows(sampleset$reference,
       sampleset$mean-sampleset$se,
	   sampleset$reference,
	   sampleset$mean+sampleset$se,
	   code = 3,
	   angle=90,
	   length=0.2)

dev.off()


errorbars

Videos and articles to view and read

    Memory Barriers: a Hardware View for Software Hackers
    http://shipilev.net/pub/talks/devoxx-Nov2013-benchmarking.pdf
    Caching in: understand, measure and use your CPU Cache more effectively
    http://lwn.net/Articles/552095/

statlearning class

This will be very useful for people like me who want to apply this to Capacity Planning

Rob Tibshirani and I are offering a MOOC in January on Statistical Learning.
This “massive open online course" is free, and is based entirely on our new book
“An Introduction to Statistical Learning with Applications in R”
(James, Witten, Hastie, Tibshirani 2013, Springer). http://www-bcf.usc.edu/~gareth/ISL/
The pdf of the book will also be free.

The course, hosted on Open edX, consists of video lecture segments, quizzes, video R sessions, interviews with famous statisticians,
lecture notes, and more. The course starts on January 22 and runs for 10 weeks.

Please consult the course webpage http://statlearning.class.stanford.edu/ to enroll and for for further details.
----------------------------------------------------------------------------------------
Trevor Hastie hastie@stanford.edu
Professor, Department of Statistics, Stanford University
Phone: (650) 725-2231 Fax: (650) 725-8977
URL: http://www.stanford.edu/~hastie
address: room 104, Department of Statistics, Sequoia Hall
390 Serra Mall, Stanford University, CA 94305-4065
--------------------------------------------------------------------------------------

Simplified code using Java 8 flatMap

This code solved one of the problems. It accumulates all the matches by looping over m.find(). The main problem is still not solved. It does not work on the result of the previous match.

So the result is wrong.

[Current Usage : init:2359296, used:13913536, committed:13959168, max:50331648|]
[Current Usage : init:2359296, used:13915200, committed:13959168, max:50331648|]
[2359296, 13914944, 13959168, 50331648, 2359296, 13913536, 13959168, 50331648, 13, 31, 48, 13, 27]
[2359296, 13916608, 13959168, 50331648, 2359296, 13915200, 13959168, 50331648, 13, 31, 48, 13, 27]

import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;
import java.util.function.Function;
import java.util.regex.Matcher;
import java.util.regex.Pattern;


public class SimplerFlatMapTest {

    public static void main( String... argv ){

        List<String> source = new ArrayList<String>();
        source.add( "Peak Usage    : init:2359296, used:13914944, committed:13959168, max:50331648Current Usage : init:2359296, used:13913536, committed:13959168, max:50331648|------------------| committed:13.31Mb+---------------------------------------------------------------------+|//////////////////|                                                  | max:48Mb+---------------------------------------------------------------------+|------------------| used:13.27Mb");
        source.add( "Peak Usage    : init:2359296, used:13916608, committed:13959168, max:50331648Current Usage : init:2359296, used:13915200, committed:13959168, max:50331648|------------------| committed:13.31Mb+---------------------------------------------------------------------+|//////////////////|                                                  | max:48Mb+---------------------------------------------------------------------+|------------------| used:13.27Mb");

        List<Pattern> patterns = Arrays.asList(Pattern.compile("Current.*?[/|]"), Pattern.compile("[0-9]+(/,|/|)"));


        Function<Matcher, List<String>> matches1 = m -> {
            List<String> list = new ArrayList<String>();
            while(m.find()) {
                list.add(m.group());
            }
            return list;
        };

        patterns.stream()
                .flatMap(p -> source.stream().map(p::matcher))
                .map(matches1)
        .forEach(System.out::println);
    }
}

Java 8 flatMap

I compiled this code with

java version "1.8.0-ea"
Java(TM) SE Runtime Environment (build 1.8.0-ea-b113)
Java HotSpot(TM) 64-Bit Server VM (build 25.0-b55, mixed mode)

The documentation of flatMap was not easy to understand but there are two streams that are mixed – the source stream and the regular expression pattern stream. I am applying the regular expressions one by one on the source lines to get what I want. So here I am able to get the first value(2359296) from the line

The problem with this approach is this. The result of the first regex match is this line.

Current Usage : init:2359296, used:13913536, committed:13959168, max:50331648|

The second regex should be applied on this line. Not on the original line. But that is how this code works. Moreover it does not give me all the values returned by looping over Matcher::find

Not the other values. I have to find a way of coding it to get all.


import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;
import java.util.function.Function;
import java.util.regex.Matcher;
import java.util.regex.Pattern;
import java.util.stream.Collectors;
import java.util.stream.Stream;

public class FlatMapTest {

    public static void main( String... argv ){

        List<String> source = new ArrayList<String>();
        source.add( "Peak Usage    : init:2359296, used:13914944, committed:13959168, max:50331648Current Usage : init:2359296, used:13913536, committed:13959168, max:50331648|------------------| committed:13.31Mb+---------------------------------------------------------------------+|//////////////////|                                                  | max:48Mb+---------------------------------------------------------------------+|------------------| used:13.27Mb");
        source.add( "Peak Usage    : init:2359296, used:13916608, committed:13959168, max:50331648Current Usage : init:2359296, used:13915200, committed:13959168, max:50331648|------------------| committed:13.31Mb+---------------------------------------------------------------------+|//////////////////|                                                  | max:48Mb+---------------------------------------------------------------------+|------------------| used:13.27Mb");

        List<Pattern> patterns = Arrays.asList(Pattern.compile("Current.*?[/|]"), Pattern.compile("[0-9]+(/,|/|)"));

        //Style 1
        patterns.stream()
                .flatMap(new Function<Pattern, Stream<String>>() {
                    @Override
                    public Stream<String> apply(Pattern p) {
                        return source.stream()
                                .map(p::matcher)
                                .filter(Matcher::find)
                                .map(Matcher::group);
                    }
                })
                .forEach(System.out::println);

        //Style 2
        patterns.stream().flatMap(( Pattern p1 ) -> source.
                stream().
                map(p1::matcher).
                filter(Matcher::find).map(matcher -> matcher.group())).forEach(x -> System.out.println(x));

    }
}

‘R’ and Java Lambda

> head(y)
                                                                                                                                                                                                                                                                                                                                                                                                                                                              y
1 Peak Usage    : init:2359296, used:13914944, committed:13959168, max:50331648Current Usage : init:2359296, used:13913536, committed:13959168, max:50331648|------------------| committed:13.31Mb+---------------------------------------------------------------------+|//////////////////|                                                  | max:48Mb+---------------------------------------------------------------------+|------------------| used:13.27Mb
> y <- apply( y, 1, function(z) str_extract(z,"Current.*?[/|]"))
[1] "Current Usage : init:2359296, used:13913536, committed:13959168, max:50331648|"

The ‘R’ function ‘apply’ can operate on a data structure and apply a regular expression. It gives back a data structure with the new values.

I think the equivalent Java Lambda code could be like this. It may not be optimal but the result is similar.


import java.util.ArrayList;
import java.util.List;
import java.util.regex.Matcher;
import java.util.regex.Pattern;
import java.util.stream.Collectors;


public class ArrayListStream {

    public static void main( String... argv ){
        List<String> list = new ArrayList();
        list.add( "Peak Usage    : init:2359296, used:13914944, committed:13959168, max:50331648Current Usage : init:2359296, used:13913536, committed:13959168, max:50331648|------------------| committed:13.31Mb+---------------------------------------------------------------------+|//////////////////|                                                  | max:48Mb+---------------------------------------------------------------------+|------------------| used:13.27Mb");
        list.add( "Peak Usage    : init:2359296, used:13916608, committed:13959168, max:50331648Current Usage : init:2359296, used:13915200, committed:13959168, max:50331648|------------------| committed:13.31Mb+---------------------------------------------------------------------+|//////////////////|                                                  | max:48Mb+---------------------------------------------------------------------+|------------------| used:13.27Mb");
        Pattern p = Pattern.compile( "Current.*?[/|]" );
        List list1 = list.
                       stream().
                        map(p::matcher).
                          filter(Matcher::find).map(matcher -> matcher.group()).
                              collect(Collectors.toCollection(ArrayList::new));
        System.out.println(list1.get(0));
    }
}