Architecture conference

Enterprise Architecture does not seem to be mentioned frequently by Agilists. The definitions are not set in stone but a Technical Architect who is more concerned with working Software and programming might be required to work with another Enterprise Architect. I have not met any EA yet most probably because they seem to be way up in the hierarchy.

I perceive rather wrongly a big gulf between Technical Architects and other business types but an EA is more concerned with alignment between IT and the business. This EA conference could help us understand an Enterprise Architect’s role.

Windows Powershell I

Some of my posts have been laced with sarcasm but I could not help it. This is about the lack of interest in programming skills amongst my offshore colleagues. It is widely known that the offshore industry is driven my monetary goals. Business is the driving force here.
So we do not generally have to work through books like the ‘Structure and Interpretation of Computer Programs’ or algorithms to find employment. The interview process here favors UI frameworks and tools like Portals and ESB’s. The fundamentals are lost forever.
That is the prelude to the following drab Windows PowerShell program that I had to write recently. Not out of compulsion but out of curiosity. The PowerShell seems to pack a wealth of tricks and features and I discarded my original DOS shell script.

The shell script follows.


$script:utilScript = Get-Script | split-path -leaf -resolve

function isExists( $file ){

   
    try{
        $exists = test-Path $file
        
        if( $exists ){
        
            #log "$file exists"
            
        }else{
        
            #log "$file does not exist"
        
        }
    }catch {
    
        logException( $error )
        
    }
    return $exists
}


   
function filterFiles( $patterns, $source ){


        if( ( $patterns -eq $null ) -or ( $source -eq $null ) ){
        
            log "$utilScript The filter pattern or directory is null"
            
            return
            
        }else{
            
            $epfiles = Get-Childitem "C:\target\filter" | where {$_  -match $patterns}
            
        }
        return $epfiles

}


   
function deleteFiles( $files, $source ){

    try{

        if( ( $files -eq $null ) -or ( $source -eq $null ) ){
        
            log "$utilScript The files to be deleted or the directory is null"
            return
            
        }
        
        $files | foreach-object -process {

            if( isExists "$source\$_" ){

                log "$utilScript Deleting the file $source\$_"
                
                Remove-Item $source\$_
              
                log "$utilScript Deleted the file $source\$_"
            
            }else{
            
                log "$utilScript $source\$_ does not exist. $_ cannot be deleted"
            }
        }
        
    }catch {
    
        logException( $error )
        
    }

}



   

   
function moveFile( $files, $source, $target ){

    try{
    
            
            if( ( $files -eq $null ) -or ( $source -eq $null ) -or ( $target -eq $null ) ){
            
                log "$utilScript moveFile The file, source or target is null"
                return
                
            }
            
            foreach( $file in $files ) {
            
                if( -not ( $null -eq $file ) ){#Redundant

                    log "$utilScript Trying to move the file $source\$file to the target $target\$file"

                    if( ( isExists "$source\$file" ) -and ( -not ( isExists "$target\$file" ) ) ){

                        move-item -path $source\$file -destination $target\$file
                      
                        log "$utilScript Moved the file $source\$file to the target $target\$file"
                    
                    }else{
                    
                        log "$utilScript $source\$file does not exist or $target\$file is already present . $file cannot be moved to $target"
                    }
                }
            }
             
    }catch {
    
        logException( $error )
    }
}



   
function copyFile( $files, $source, $target ){

    try{
    
            
            if( ( $files -eq $null ) -or ( $source -eq $null ) -or ( $target -eq $null ) ){
            
                log "$utilScript The file, source or target is null"
                return
                
            }
            $files | foreach-object -process {

                if( isExists "$source\$_" ){

                    log "$utilScript $source\$_ exists in the file system"

                    log "$utilScript Copying the file $source\$_ to the target $target\$_"
                    
                    Copy-Item -path $source\$_ -destination $target\$_
                  
                    log "$utilScript Copied the file $source\$_ to the target $target\$_"
                
                }else{
                
                    log "$utilScript $source\$_ does not exist. So it cannot be copied to $target"
                }
            }
             
    }catch {
    
        logException( $error )
    }
}


function createDirectory( $directories, $target ){

    try{

            
            if( ( $directories -eq $null ) -or ( $target -eq $null ) ){
            
                log "$utilScript The directory, or target is null"
                return
                
            }

            $directories | foreach-object -process {

                if( -not ( isExists "$target\$_" ) ){

            
                    log "$utilScript Creating $directories in $target"
                    
                    new-item -path $target -name $_ -itemtype directory
                    
                    log "$utilScript Created $directories in $target"
                
                }else{
                
                    log "$utilScript $target\$_ already exists. So it cannot be created again"
                }

            }

    }catch {
    
        logException( $error )
    }

}


   
function searchFiles( $files, $characters ){

    $isSuccess = $FALSE

    try{

            if( ( $files -eq $null ) -or ( $characters -eq $null ) ){
            
                return $isSuccess
                
            }
            
            foreach( $file in $files ) {

                    log "$editScript Searching $file for $characters"

                    $reader = new-object System.IO.StreamReader -ArgumentList $file
                    
                    
                
                    do {
                        if ( -not ( $line.length -eq 0  ) ){
                        
                                if ($line -match $characters)  {
                                
                                    $isSuccess = $TRUE

                                    log "$editScript The search for $characters in the line $line in $file succeeds "
                        
                                    break

                                }     

                        }
                   }while( ( [String] $line = $reader.readline()) -ne $null  )
                    
                   $reader.Close()
            }

    }catch {
    
        //Log Exception
    }
    
    return $isSuccess

}

I picked up this function from an example and it helps me log the name of executing script


function Get-Script{

if($myInvocation.ScriptName) { $myInvocation.ScriptName }

else { $myInvocation.MyCommand.Definition }

}

Exploring the PowerShell scripting language will help you keep in touch with the duller side of the IT industry 🙂

Update : The functions to move a file and search for patterns in a file are rewritten

Technical Tidbits

  1. Microsoft and IBM researchers Ramachandran Ramjee and Shivkumar Kalyanaraman in Bangalore have figured in this year’s ACM Distinguished Scientists list. .
  2. Concurrency is a hot topic and this website 1024cores was advertised in the Java concurrency list recently.

I hope that this last post of 2010 will bode well for my technical career in the next year. Happy New Year.

Viewing Graphs

Some tools are simple and easy to use but very helpful. One such old tool is Graphviz which helped us quickly visualize the folder structures.

I could generate a pleasing graph from this simple data representation using Gvedit that is part of the download.

digraph Folders1 {
size ="1,1";
FinancialInstution [shape=box,style=filled,color=".7 .3 1.0"];
Project -> FinancialInstution [style=dotted]
FinancialInstution -> scheme -> files
files -> TYPE_OF_FILE -> MasterCard
TYPE_OF_FILE -> Visa
}

This is the simple graph.

xp2011

Lean product development

Lean Product Development practices

This diagram of the Lean product development practices from “Scaling Lean & Agile Development: Thinking and Organizational Tools for Large-Scale Scrum” copied here seems to have great advices. I am reading this book now and I have always liked Craig Larman’s interesting collection of ideas and insight.

I particularly like the authors’ dissection of a Kanban practice( http://www.agilemanagement.net/index.php/blog/Kanban_in_Action/ ) in the chapter on Lean Thinking ( Example: “Kanban System” Analysis )

We surely ignore every advice shown here but if I were to pick one that is totally anathema to the offshore industry then it would be the idea of long-lasting engineers.

Respect for Software Architecture in the offshore industry

I have been reading about the nascent – still nascent – discipline of Software Architecture from various sources but it is hard to follow because I am not able to practise it usefully. There are so many reasons for this including my lack of experience but the disrespectful behaviour of the business types is really not helping. There is always a collision between managers and technologists and we know generally on which side the top management is.

Even software engineering is not really a discipline according to David Parnas and I quote from his article that appeared in VOL. 53( 10 / 2010 ) NO. 10 of the ‘Communications of the ACM”.

Recent experiences reminded me that the activity we ( euphemistically ) call software engineering does not come close to deserving a place among the traditional engineering disciplines.

It might not be a discipline still but the work in this area is not at all insignificant. It is important enough to figure in Wharton’s list.

Software Architecture though is not in the same league yet. Agilists do not subscribe to the traditional definition of architecture and they are gaining ground fast. So new aspiring architects are pulled in different directions by traditionalists and agilists.

I have also tried to read about building architecture that many of us seem to be drawing inspiration from. They have their own Nobel prize too.

SOA presentation

I think Udi dahan’s presentation on SOA architecture is one of the most influential expositions of SOA and IT alignment that I have come across.

UML Timing Diagram

We have some Batch Applications that run round the clock. After the developer releases this to the production environment a different team is responsible for the daily operations. The complexity of this application is such that it is difficult to convey the intricacies easily to that team. A UML timing diagram like the one shown below is one way of conveying the firing schedules to other teams.

This is not such a simple task though. Sometimes it is too intricate that several
training sessions are required but a diagram like this can be used as a ready reckoner.

Bouncy Castle – Encrypt a stream

I should be forgiven for thinking that encrypting a stream of data instead of a disk file is more difficult than it is supposed to be. Actually when somebody like me who has no experience with writing secure code uses encryption/decryption libraries like Bouncy Castle, it is always hard. So even after the PCI auditor has paid us a visit many times I can still point out many loopholes in our security code like the AES key that is hard-coded. Split key technology is really hard to grok. So I have not attempted it yet. So the AES key is still inside the java class.

This example uses Bouncy Castle to decrypt a file using a GPG key and encrypt it again using another key on the fly. Some parts of the code that are supposed to be obvious are not shown but as I mentioned secure code is hard to write. The missing piece can be reconstructed from the Bouncy Castle examples.

I am planning to fix the code formatting in the blog soon.

Decrypt a file and return a stream

public InputStream decryptToStream( String file ) throws KeyNotFoundException{

PGPEncryptedDataList enc = null;

PGPObjectFactory objectFactory = null;

InputStream in = null;

try {

objectFactory = new PGPObjectFactory( new FileInputStream( file ) );

Object o = objectFactory.nextObject();

// What is a PGP marker packet ?

if ( o instanceof PGPEncryptedDataList ) {

enc = ( PGPEncryptedDataList ) o;

} else {

enc = ( PGPEncryptedDataList ) objectFactory.nextObject();

}

Iterator it = enc.getEncryptedDataObjects();

PGPPublicKeyEncryptedData pbe = null;

while ( it.hasNext()){

pbe = (PGPPublicKeyEncryptedData)it.next();

}

InputStream clear = pbe.getDataStream( getPGPSecretKey( pbe.getKeyID() ), "BC" );

PGPObjectFactory plainFact = new PGPObjectFactory( clear );

Object message = plainFact.nextObject();

if (message instanceof PGPCompressedData) {

PGPCompressedData cData = (PGPCompressedData) message;

PGPObjectFactory pgpFact = new PGPObjectFactory(cData.getDataStream());

message = pgpFact.nextObject();
}

if (message instanceof PGPLiteralData) {

PGPLiteralData literal = (PGPLiteralData) message;

in = literal.getInputStream();

}
} catch (FileNotFoundException e) {

} catch (IOException e) {

} catch (NoSuchProviderException e) {

} catch (PGPException e) {

}
return in;
}

Encrypt the stream and write to a file

It should be possible to return a stream from this method instead of writing it to a file.

public void encrypt( PGPPublicKey key,
                               String compressedfile,
                               String encryptedfile,
                               InputStream in ) throws IOException,
PGPException,
NoSuchProviderException {
byte[] buf = new byte[ 1024 ],
literalBuffer = new byte[ 1024 ],
encryption = new byte[ 1024 ];

ByteArrayOutputStream o = null;

OutputStream out = null;

FileOutputStream fos = new FileOutputStream( new File( encryptedfile ));

PGPCompressedDataGenerator compressedData = new PGPCompressedDataGenerator(
PGPCompressedDataGenerator.ZIP);
PGPLiteralDataGenerator lg =
new PGPLiteralDataGenerator();

try {

PGPEncryptedDataGenerator data =
new PGPEncryptedDataGenerator( PGPEncryptedData.CAST5,
true,
new SecureRandom(),
"BC");

data.addMethod( key );
int bytes = 0;

OutputStream writeToEncrypt = data.open( fos,
encryption );

OutputStream lgo = lg.open( compressedData.open( writeToEncrypt ),
PGPLiteralData.BINARY,
compressedfile,
new Date(),
literalBuffer
);

while ((bytes = in.read( buf )) >= 0) {
//As it is written it is encrypted.

lgo.write( buf, 0 , bytes );

}

lg.close();
compressedData.close();
writeToEncrypt.close();
lgo.close();

} catch (IOException e) {

} catch (NoSuchProviderException e) {

} catch (PGPException e) {

}
}

add to del.icio.us : Add to Blinkslist : add to furl : Digg it : add to ma.gnolia : Stumble It! : add to simpy : seed the vine : : : TailRank : post to facebook