Area between curves

This is what I like about ‘R’. This one line is enough to apply a shade of color
to the area between two curves. Apart from the functional programming aspects(
http://adv-r.had.co.nz/), I am interested in its powerful API’s used to visualize and parse data.

polygon( c(data$Time, rev(data$Time)),
         c(as.numeric(data$Used), rev(as.numeric(data$Committed))),
         col = "antiquewhite1",
		 border = NA )

code-cache

“member of the technical staff”

I was rather surprised to find these passages – copied below – in Fred Brook’s book “The mythical man-month” that was written a long time ago. I was browsing it again after a decade. So I thought I should collect my thoughts on this.

This particular organizational structure has caused me untold grief because as a technical lead I found the gap between these two groups too yawning to bridge. The two groups want to tear each other’s guts out at every meeting. Pure hatred !!

I have been specifically told by an MNC that a manager who has a large team will always get a better appraisal than a tech. arch. who builds systems with the assistance of a few. The top management is scared of equating the two groups for fear of antagonizing the project managers.

The general position “member of the technical staff” is a good way of merging the two groups. But the only way to completely abolish the difference is by making both groups work on project management as well as technical tasks, which is a tall order.

The other way is to encourage people to gain years of experience in their chosen field before moving up the ladder. This creates specialists. People in our companies don’t want to specialize in any field. So a business analyst is not a requirements expert. She should be. Actually no one is an expert requirements architect. So one of the most important fields is left without an expert. That hurts project success.

A project manager is not an expert in Earned Value Management or Statistical Process Control. Once we have specialists we will have humble people. Humility is a great leveler. Expertise gives confidence.


Plan the Organization for Change

Structuring an organization for change is much harder than designing a system for change. Each man must be assigned to jobs that broaden him, so that the whole force is technically flexible. On a large project the manager needs to keep two or three top programmers as a technical cavalry that can gallop to the rescue wherever the battle is thickest.

Management structures also need to be changed as the system changes. This means that the boss must give a great deal of attention to keeping his managers and his technical people as interchangeable as their talents allow.

The barriers are sociological, and they must be fought with constant vigilance. First, managers themselves often think of senior people as “too valuable” to use for actual programming. Next, management jobs carry higher prestige. To overcome this problem some laboratories, such as Bell Labs, abolish all job titles. Each professional employee is a “member of the technical staff.” Others, like IBM, maintain a dual ladder of advancement, as Fig. 11.1 shows. The corresponding rungs are in theory equivalent.

Figure 11.1. IBM dual ladder of advancement

It is easy to establish corresponding salary scales for rungs. It is much harder to give them corresponding prestige. Offices have to be of equal size and appointment. Secretarial and other support services must correspond. A reassignment from the technical ladder to a corresponding level on the managerial one should never be accompanied by a raise, and it should be announced always as a “reassignment,” never as a “promotion.” The reverse reassignment should always carry a raise; overcompensating for the cultural forces is necessary.

Managers need to be sent to technical refresher courses, senior technical people to management training. Project objectives, progress, and management problems must be shared with the whole body of senior people.

Whenever talents permit, senior people must be kept technically and emotionally ready to manage groups or to delight in building programs with their own hands. Doing this surely is a lot of work; but it surely is worth it!

The whole notion of organizing surgical-type programming teams is a radical attack on this problem. It has the effect of making a senior man feel that he does not demean himself when he builds programs, and it attempts to remove the social obstacles that deprive him of that creative joy.

Furthermore, that structure is designed to minimize the number of interfaces. As such, it makes the system maximally easy to change, and it becomes relatively easy to reassign a whole surgical team to a different programming task when organizational

IEEE Software issue on ‘Algorithms for Today’s Practitioner’

I am reading the Jan/Feb 2012 issue of IEEE Software that should strike a chord with Software engineers in the offshore industry. Even financial application developers are oblivious to the need for algorithms that are not always part of the J2SE kit or an astronomically priced tool that the management here favors.

The awareness is lacking and the motivation to read about and experiment with code and algorithms is missing.

I also read the interesting interview with David Chaiken, Chief Architect of Yahoo who visited the Chennai IIT and spoke about his work in Yahoo. I was in the audience and remember that he pointed out a particular bug ID that caused some Yahoo servers in the production data centre to mishebave after he deployed a release.

My question to him was about the testing procedures that Yahoo uses for such large-scale deployments and I was looking for some tips about testing distributed applications. He just replied that Yahoo tries to use agile testing principles in some cases.

I have to rememeber to urge the local IEEE chapter to invite such speakers more frequently and also involve the developer community. There were faculty and students on that day amongst the sparse audience.

Sebastian Thrun’s democratic university

I am stumped. Stanford research professor Sebastian Thrun and other professors are planning to organize and teach an entire university course in Computer Science !! I planned to but didn’t take their initial AI course but this is too good to miss. They are following it up with an entire degree course in CS. What attracts me is the open courseware concept and the self-driving car.

As a software developer I have firsthand knowledge of the lack of taste for algorithms/CS and how the entire offshore industry is stuck in a medieval mindset.
I am much obliged.

However you look at this endeavor this proves that if one is set on thinking audaciously action will follow. Shouldn’t this stir up our interest in CS and its application to seemingly mundane software engineering?

Description: This class, taught by one of the foremost experts in AI, will teach you basic methods in Artificial Intelligence, including: probabilistic inference, computer vision, machine learning, and planning, all with a focus on robotics. Extensive programming examples and assignments will apply these methods in the context of building self-driving cars. You will get a chance to visit, via video, the leading research labs in the field, and meet the scientists and engineers who are building self-driving cars at Stanford and Google.
Prerequisites: The instructor will assume solid knowledge of programming, all programming will be in Python. Knowledge of probability and linear algebra will be helpful.

Technical tidbit

  1. Microsoft and IBM researchers Ramachandran Ramjee and Shivkumar Kalyanaraman in Bangalore have figured in this year’s ACM Distinguished Scientists list. .
  2. Concurrency is a hot topic and this website 1024cores was advertised in the Java concurrency list recently.

I hope that this last post of 2010 will bode well for my technical career in the next year.

Architecture conference

Enterprise Architecture does not seem to be mentioned frequently by Agilists. The definitions are not set in stone but a Technical Architect who is more concerned with working Software and programming might be required to work with another Enterprise Architect. I have not met any EA yet most probably because they seem to be way up in the hierarchy.

I perceive rather wrongly a big gulf between Technical Architects and other business types but an EA is more concerned with alignment between IT and the business. This EA conference could help us understand an Enterprise Architect’s role.

Windows Powershell I

Some of my posts have been laced with sarcasm but I could not help it. This is about the lack of interest in programming skills amongst my offshore colleagues. It is widely known that the offshore industry is driven my monetary goals. Business is the driving force here.
So we do not generally have to work through books like the ‘Structure and Interpretation of Computer Programs’ or algorithms to find employment. The interview process here favors UI frameworks and tools like Portals and ESB’s. The fundamentals are lost forever.
That is the prelude to the following drab Windows PowerShell program that I had to write recently. Not out of compulsion but out of curiosity. The PowerShell seems to pack a wealth of tricks and features and I discarded my original DOS shell script.

The shell script follows.


$script:utilScript = Get-Script | split-path -leaf -resolve

function isExists( $file ){

   
    try{
        $exists = test-Path $file
        
        if( $exists ){
        
            #log "$file exists"
            
        }else{
        
            #log "$file does not exist"
        
        }
    }catch {
    
        logException( $error )
        
    }
    return $exists
}


   
function filterFiles( $patterns, $source ){


        if( ( $patterns -eq $null ) -or ( $source -eq $null ) ){
        
            log "$utilScript The filter pattern or directory is null"
            
            return
            
        }else{
            
            $epfiles = Get-Childitem "C:\target\filter" | where {$_  -match $patterns}
            
        }
        return $epfiles

}


   
function deleteFiles( $files, $source ){

    try{

        if( ( $files -eq $null ) -or ( $source -eq $null ) ){
        
            log "$utilScript The files to be deleted or the directory is null"
            return
            
        }
        
        $files | foreach-object -process {

            if( isExists "$source\$_" ){

                log "$utilScript Deleting the file $source\$_"
                
                Remove-Item $source\$_
              
                log "$utilScript Deleted the file $source\$_"
            
            }else{
            
                log "$utilScript $source\$_ does not exist. $_ cannot be deleted"
            }
        }
        
    }catch {
    
        logException( $error )
        
    }

}



   

   
function moveFile( $files, $source, $target ){

    try{
    
            
            if( ( $files -eq $null ) -or ( $source -eq $null ) -or ( $target -eq $null ) ){
            
                log "$utilScript moveFile The file, source or target is null"
                return
                
            }
            
            foreach( $file in $files ) {
            
                if( -not ( $null -eq $file ) ){#Redundant

                    log "$utilScript Trying to move the file $source\$file to the target $target\$file"

                    if( ( isExists "$source\$file" ) -and ( -not ( isExists "$target\$file" ) ) ){

                        move-item -path $source\$file -destination $target\$file
                      
                        log "$utilScript Moved the file $source\$file to the target $target\$file"
                    
                    }else{
                    
                        log "$utilScript $source\$file does not exist or $target\$file is already present . $file cannot be moved to $target"
                    }
                }
            }
             
    }catch {
    
        logException( $error )
    }
}



   
function copyFile( $files, $source, $target ){

    try{
    
            
            if( ( $files -eq $null ) -or ( $source -eq $null ) -or ( $target -eq $null ) ){
            
                log "$utilScript The file, source or target is null"
                return
                
            }
            $files | foreach-object -process {

                if( isExists "$source\$_" ){

                    log "$utilScript $source\$_ exists in the file system"

                    log "$utilScript Copying the file $source\$_ to the target $target\$_"
                    
                    Copy-Item -path $source\$_ -destination $target\$_
                  
                    log "$utilScript Copied the file $source\$_ to the target $target\$_"
                
                }else{
                
                    log "$utilScript $source\$_ does not exist. So it cannot be copied to $target"
                }
            }
             
    }catch {
    
        logException( $error )
    }
}


function createDirectory( $directories, $target ){

    try{

            
            if( ( $directories -eq $null ) -or ( $target -eq $null ) ){
            
                log "$utilScript The directory, or target is null"
                return
                
            }

            $directories | foreach-object -process {

                if( -not ( isExists "$target\$_" ) ){

            
                    log "$utilScript Creating $directories in $target"
                    
                    new-item -path $target -name $_ -itemtype directory
                    
                    log "$utilScript Created $directories in $target"
                
                }else{
                
                    log "$utilScript $target\$_ already exists. So it cannot be created again"
                }

            }

    }catch {
    
        logException( $error )
    }

}


   
function searchFiles( $files, $characters ){

    $isSuccess = $FALSE

    try{

            if( ( $files -eq $null ) -or ( $characters -eq $null ) ){
            
                return $isSuccess
                
            }
            
            foreach( $file in $files ) {

                    log "$editScript Searching $file for $characters"

                    $reader = new-object System.IO.StreamReader -ArgumentList $file
                    
                    
                
                    do {
                        if ( -not ( $line.length -eq 0  ) ){
                        
                                if ($line -match $characters)  {
                                
                                    $isSuccess = $TRUE

                                    log "$editScript The search for $characters in the line $line in $file succeeds "
                        
                                    break

                                }     

                        }
                   }while( ( [String] $line = $reader.readline()) -ne $null  )
                    
                   $reader.Close()
            }

    }catch {
    
        //Log Exception
    }
    
    return $isSuccess

}

I picked up this function from an example and it helps me log the name of executing script


function Get-Script{

if($myInvocation.ScriptName) { $myInvocation.ScriptName }

else { $myInvocation.MyCommand.Definition }

}

Exploring the PowerShell scripting language will help you keep in touch with the duller side of the IT industry 🙂

Update : The functions to move a file and search for patterns in a file are rewritten