Inspections and Reviews in Agile Projects



In my previous blog "Can Inspections Make Defect Prevention Effective?" , I emphasized on spending one or two hours per iteration on reviews and inspection in Agile projects. Also, I wrote that Fagan inspection can be fine tuned to suit Agile projects so that team members can perform inspections or reviews in small groups of two to four team members,  depending on the criticality of such inspections or reviews. In this blog, let us go through when and how this can be done in Agile projects.

1. Review of requirements. Group reviews makes sense in agile projects. In one of the distributed agile projects we had scheduled conference calls to walk-through requirements at the beginning of iteration. These conference calls were more beneficial than 1-1 interactions because, the entire team could understand and ask questions as group. As a group, we found gaps in requirements and related one user story to another and refined user stories. Don’t we call this backlog grooming?

2. Review of architecture and designs. This is not about ‘big-upfront’ architecture or design. I have come across situations in agile projects that required our team to do complex designs in order to move forward with reasonable modularity in code. These designs are either UI designs or UML notation etc. In this context, it is valuable to involve all team members and perform group review. This step helps us improve the quality of design.

3. Code Refactoring.  Whether you do pair programming or not, it is a good practice to convene meetings at regular intervals (typically once in a month or on need basis) to discuss some of the key code refactoring instances. This can be a part of ‘retrospectives’ too. More than this, for critical chunks of source code, group review with two to four engineers is the way to go.

4. Test Case Reviews. At regular intervals, group review of test cases can help agile teams improve the quality of requirements as well as test cases.

5. Configuration Files, Build Files, etc. When two or more engineers come together and review configuration files they can streamline the placement of configuration parameters in the right files. Also, reviews of build files can minimize build failures due to scripting or configuration errors.

Typically these reviews in traditional projects used to take 10 to 15% of efforts. Should we invest in inspections and reviews to eliminate defects proactively? Why not? Ultimately, when we find ways to implement continuous improvement through this experience, we get to improve process as well as product quality.

One may think or argue that practices such as pair programming and ‘retrospectives’ enable agile teams practice continuous improvement and that is sufficient.  'Retrospectives’ are iteration specific. Do we get to apply 80/20 rule by gathering cumulative data of all past iterations in Agile projects? If no, don’t we have to?

Can Inspections Make Defect Prevention Effective?



“Yes”, says Capers Jones. Formal inspection of requirements, architecture, designs, source code, configuration files, build scripts and test cases enable project teams in finding critical defects.
The cost of defects (related to requirements or designs) found during later stages of lifecycle is enormous. The best way to deal with this issue is to detect defects at early stages. Inspection is a way to accomplish this.

Not all Agile teams do pair programming. What about designs? Or configuration files? Do we inspect? When inspection or reviews are not done, the result can be accumulation of technical debt. Right?

When we find defects through inspections and reviews, we get an opportunity to classify defects and identify the root causes. Remember 80/20 rule? When we do this we can reduce the number of defects in subsequent iterations.

Fagan inspection can be fine tuned to suit Agile projects. Instead of performing inspection in large groups, Agile teams can perform inspections or reviews in small groups of two to four team members depending on the criticality of such inspections or reviews. One or two hours per iteration spent on reviews and inspection in Agile projects can yield immense benefits.

Have we forgotten the value of inspections and reviews? Caper Jones’s article ‘Do You Inspect?’ is an eye opener for everyone. 

In the next blog "Inspections and Reviews in Agile Projects"  I have shared 5 pointers and additional questions.

Agile Myths and Misunderstandings

This post links to a free PDF eBook for you! In our industry we do come across myths and misinterpretations related to Agile Software Development and Agile Testing. Here are some examples.

1. Take the waterfall model and add one arrow
2. Agile means 'start coding with no documentation'
3. Agile means 'ad hoc' or 'no processes’
4. Agile means 'no planning'
5. Agile team members must be super stars
6. Agile is for product engineering only
7. Changes can happen on a daily basis
8. Agile is for development projects only
9. Agile impacts work-life balance
10. Agile is just another fad
11. TDD is enough to ensure software testing
12. A chain of unit tests is a complete regression suite
13. Agile doesn’t allow for long-term planning
14. Agile testing is all about unit testing, TDD, and test automation
15. In Agile projects process compliance is a big issue

Details on each of these are available in this freely downloadable PDF. Happy Reading!

Requirement Engineering: Ten Principles



1. Quality is the top priority and it is possible to deliver high-quality software. In order to make this happen, business analysts need to focus on delivering high-quality requirements. Obviously, both the form and the substance of specifications are equally important.

2. Understand the problem before specifying requirements. When business analysts understand the problem accurately, it is highly probable that they are going to create specifications that are very close to what the user wants.

3. Use appropriate tools, models and practices. It is necessary to understand the organizational culture of customers and be aware of what tools, models and practices have worked for them. Tools, models and practices that provided results in collocated teams may not work well in case of distributed teams.

4. Awareness, skills, techniques, and discipline are more important that tools. The outcome of any practice is not determined just by the tools that you use. For example, a plumber who is in-charge of your facility requires awareness of your facility. He needs to be skilled. He has to follow certain discipline and understand relevant techniques. Without these qualities, if he claims his strength because of a powerful tool set he has, he is a dangerous plumber. This applies to team members who participate in requirement engineering activities too.

5. Establish appropriate measurement criteria. How do you measure the progress of requirement engineering activities? It is necessary to have appropriate measurement criteria. Document exchange or data collection is not requirement engineering.

6. Establish an appropriate verification and validation criteria. How do you verify requirements? How do you validate them? It is essential to have appropriate criteria for V&V activities. Doing it right is more important that making it faster.

7. Good team and good management are vital to success. Technology comes next. Start-ups succeed when they have a good team that captures product requirements and a good management that fosters product development. Technology is essential. More than technology, a good team and good management are important.

8. Understanding of customer’s priorities is paramount. This is what helps the team understand priority requirements at early stages and find ways to deliver working software early. The more the customers see, the more they will need. Understanding customer’s priority is a way to deliver valuable features early. Delivering early is the way to understand what else the customer wants.

9. Use the most efficient communication tools. Information exchange over documents can lead to misinterpretation or inadequate grasp. White board sessions, walkthroughs, using face-to-face meetings or video conferencing sessions are the effective ways of information exchange.

10. Ask questions. While this is applicable for all areas of software engineering areas, it is very appropriate for requirement engineering. Questions based on various aspects such as business need, functionality, scalability, performance, usability, test data, data volume, geography of users, compliance, etc. help business analyst understand requirements in detail. Asking context-free questions are equally important to make a positive impact.

Fixed Price Distributed Agile Projects

I will be delivering a session on 'Fixed Price Distributed Agile Projects' at Agile Tour 2011, Bangalore on 26th Nov 2011.  This session will present the challenges and approaches related to executing Fixed Price projects in Distributed Agile model.

In reality, projects are sanctioned based on pre-approved budgets. Customers prefer to get projects done in fixed-cost. Can the scope remain fixed? Can the schedule also remain fixed?  The challenges of fixed price projects  emerge from changes in requirements.  How do we stick to "Responding to change over following a plan" as mentioned in Agile Manifesto when we execute fixed price projects?  How do we value "Customer Collaboration" over "Contract Negotiation"?

If you are in Bangalore next week, plan to attend Agile Tour 2011.

Requirement Engineering: Specifications in which form?



The day before we started the new project, my team member got curious. He asked, “Are we going to receive requirement specifications from customer? How will the specifications look like? Let us make sure that there are no communication gaps.”

I responded, “In a document with text, and UML notations in our standard format. We will get wireframes too.”

We had a short pause. I continued, “We will have a look at existing data and reports. We will do our best to reduce communication gaps.” He nodded silently.

I could read his mind. We had serious gaps in understanding requirements in our recent project and the result was a huge schedule over run. We wanted to do it better this time.

Specifications come in what form? Obviously in documents with UML diagrams, wireframes, etc. When it comes to requirements, we cannot afford to say, “Let us not worry about form. Substance is critical.”  Both form and substance are equally important.

“The hardest single part of building a software system is deciding precisely what to build. No other part of the conceptual work is as difficult as establishing the detailed technical requirements, including all the interfaces to people, to machines, and to other software systems. No other part of the work so cripples the resulting system if done wrong. No other part is more difficult to go rectify later”, says Fredrick P. Brooks, Jr. in his book ‘The Mythical Man-Month (Addison-Wesley, 1995).’ He adds, “I would go a step further and assert that it is really impossible for clients, even those working with software engineers, to specify completely, precisely, and correctly the exact requirements of a modern software product before having built and tried some versions of the product they are specifying.”

Can Rapid Prototyping help? Yes. It can help to a certain extent. In fact, Rapid Prototyping can be done at the time of Requirement Gathering and Elicitation. Products meant for this purpose are available in the market these days.

When we combine the power of such tools with the way we create software using methodologies based on iterative and incremental delivery, we get an opportunity to deliver working software early and frequently. Delivering working software early at regular intervals is a way to make your specifications powerful. This is when you reduce the gap between what is specified, what is delivered and what is needed.

When you have specifications in written text, UML notations or wireframes for more than a month or two, chances are that they become obsolete. You need to get them validated. When you do rapid prototyping (on a case-to-case basis) and focus on short-iterations to deliver working software you get an opportunity to demonstrate working software and get feedback. You get to improve what you did.

It is easier said than done. I agree with Brooks. He said it right. Right?

Code Quality Metrics: Identification, Collection and Consumption



Unless we identify the code quality metrics we want to consider for a given project or product, before we start coding,  it is highly probable that we will get confused with a huge collection of metrics presented by tools and textual content such as books, white papers, blogs, etc.

We do capture functional and non-functional requirements in software projects. Can we capture requirements that can help us set a common understanding on expected level of code quality? 

When you use a static analysis tool, it is necessary to know acceptable range of values for each metric. Otherwise, how will you access code quality?

Code review tool does not serve the purpose until developers are cognizant of the semantics of code quality metrics and permissible range of values. I am sure we have heard, ‘A fool with a tool is still a fool.’ Don't you agree?


Every project or product team needs to have a code review checklist, coding standards and guidelines that can set a common understanding among developers on code review criteria. This will enable developers write good quality code to begin with.

Prevention is better than cure. Isn’t it?

Distributed Agile: Quality before Design

Distributed Agile involves geographically dispersed teams and short iterations of 2 to 4 weeks. Stakeholders of distributed agile projects need to find ways to improve quality before design by means of providing the right work environment and tools to team members and ensuring that project requirements are of good quality. There are five steps that can help us improve quality before design.

Product Vision Document: When distributed agile teams execute iteration activities without knowing the product vision or release vision, they not only miss an opportunity to understand the big picture but also fail to interact with stakeholders by asking context-free questions. Context-free questions increase project awareness among team members and provide for a strong foundation.

Goal-Oriented Requirements: The next step is to ensure that user stories are not generic statements but relate to the goals of end users. When team members receive goal-oriented requirements or users stories, they relate them to the goals of end users and come up with innovative ways to find solutions.

Not Only Functional Requirements: The third step is to include non-functional requirements such as performance requirements or security requirements in user stories. When this is accomplished, distributed agile teams optimize the time spent in defining the acceptance criteria and ‘Definition of Done’ for user stories. Also, this improves the quality of test cases and test scripts in projects.

Prioritization: Product Owners and Scrum Masters of distributed Scrum projects need to have an objective and proactive prioritization approach in order to define priority categories unambiguously. When prioritization of user stories becomes a reactive approach, it can lead to changes during iterations and hence can put lot of strain on team members. Distributed agile teams need to be aware of the rationale behind the prioritization of user stories. Lack of awareness among team members can lead to perceived notions on lack of inclusiveness.

Tool Selection: Tool selection plays a vital role in distributed agile projects. Effective tools for managing user stories as well as related communication and coordination mechanisms contribute positively to distributed agile projects. Indecisive approaches to tool implementation or introduction of new tools during project execution will hamper effective management of user stories. It is strongly recommended that distributed agile teams consider a web based tool that supports specification of user stories as well as facilitates collaboration among team members.

Finally, quality before design is quintessential to improve project success. My article, ‘Distributed Agile: Steps to Improve Quality before Design’, published by Agile Record articulate these steps with additional details.


The 7 Habits of Highly Effective 21-st Century IT Professionals

Gone are those days when monolithic IT systems were developed and maintained by exclusive communities of IT professionals confined to technology-savvy regions of the world. The challenges of software engineering during the 21st century are quite different and multifold because of factors such as globalization and technology evolution.

In order to face these challenges, 21st-century IT professionals will need to transform successful practices into habits so that these practices become second nature.

Cutter IT Journal published my article “Force of Habit: 7 Essentials for 21-st Century IT Professionals” in the September 2011 issue. In this article, I have discussed the seven habits that are essential for today’s IT professionals.

The seven habits discussed in this article empower IT professionals to prepare, act, collaborate, optimize, and influence effectively in their career.

Watch this space. I will publish a link to download the article soon.   If you are a Cutter member, you can download it from Cutter.com.    Or you can download a free copy of this article from http://www.cutter.com/offers/forceofhabit.html.

Can Unit Testing Improve Testability?



While writing unit tests software engineers tend to relook at the source code and improve it so that it becomes testable. This is an internal experience. When you do this your code becomes maintainable as well. Have you gone through this experience?

You get this experience when you write unit tests for the complex pieces of your source code.

Two recent articles on topics related to this subject are ‘Breaking Away From The Unit Test Group Think’ by Cedric Beust and ‘Chasing Code-Coverage Baubles’ by Andrew Binstock. A good read for all software engineers.

These articles reveal several misconceptions related to Unit Testing and TDD and provide answers to the following questions.

1. Do you think 90% code coverage through unit testing is sufficient?
2. Do you think you need to do TDD always in your development cycle? 
3. What is the optimal code coverage? How do you decide? 
4. Has TDD become a common practice in the industry?

Governance of Distributed Agile Projects: Critical Success Factors



On 15th October 2011, I am speaking at 'Agile Tour 2011, Pune'. India Scrum Enthusiasts Community is organizing this event. Agile Tour is a forum for Agile practitioners at all levels - from young engineers to professionals at very senior levels.

The title of my session is 'Governance of Distributed Agile Projects: Critical Success Factors'.  

This session is based on an article published in the July 2011 issue of Agile Record.

Requirement Engineering: Asking Context-Free Questions

‘Exploring Requirements: Quality Before Design’ by Donald C. Gause and Gerald M. Weinberg is a must read for students as well as professionals. In this book, the authors emphasize that project teams need to ask context-free questions in order to understand high level details that are necessary to create the right product in the right way. According to them there are two primary categories of context-free questions: a) context-free process questions and b) context-free product questions. Also, they discuss about the third broad category of questions called metaquestions or questions about questions. Context-free questions are applicable to any product design. They consume less effort but provide an opportunity to obtain valuable information.

Context-free questions can be used in Software Engineering irrespective of the lifecycle methodology used. Also, IT consulting teams can use context-free questions during portfolio analysis or assessment phase in order to better understand the customer requirements.

Context-free process questions relate to the process to be followed and the answers to context-free questions relate to the project or the lifecycle activities. Context-free product questions relate to the design of the product and the answers to context-free product questions provide details on attributes, considerations or trade-offs related to design of the product. Let me share few examples here. Most of these examples are based on my experience.

Examples of Context-Free Process Questions:

1. What are the success criteria for this project? Or how do we measure the success of this project?
2. What are the driving factors for executing this project with a collocated team (or distributed team) or a consulting partner?
3. How much time do we have for this project?
4. What is your trade-off between schedule and quality?
5. Where else can the solution to this design problem be obtained?
6. Can we reuse something that already exists? Can we use open source?
7. Who are the competitors for this product?

Examples of Context-Free Product Questions: 

1. What problems does this product solve?
2. What problems could this product create?
3. What are the operational environments for this product?
4. What kind of precision is required or desired in the product?
5. What are the interfacing standards to be followed?

Examples of Metaquestions:

1. Am I asking you too many questions?
2. Do my questions seem relevant?
3. Who is the right person to answer these questions?
4. Do you think our team needs to contact multiple members in your organization to get answers to these questions? 

Context-free questions help us identify hidden assumptions. They help us identify the right approach to design applications or products. Also they help us design the right applications or products. 

Computer Programming: Art or Science?

Donald E. Knuth, one of the world’s eminent computer scientists, published hundreds of papers and evangelized computer science through his writings and teachings. His three volumes of ‘The Art of Computer Programming’ are among the most popular books in computer science.

In an interview he said, “When you write a program, think of it primarily as a work of literature. You’re trying to write something that human beings are going to read. Don’t think of it primarily as something a computer is going to follow. The more effective you are at making your program readable, the more effective it’s going to be: You’ll understand it today, you’ll understand it next week, and your successors who are going to maintain and modify it will understand it.”   

Programming is not a mechanical activity. Successful compilation of programs is not the end to programming. Putting your programs through a static analysis tool (for eg., JTest) is just the beginning. Programmers need to understand the errors reported by such tools and develop the ability to pick and choose the right set of fixes that can improve code quality. Also, programmers need to practice refactoring in order to improve code quality.

An interesting analogy is cookery. Those who carry the passion for cooking do not blindly follow a set of instructions or restrict themselves to using state-of-the-art tools. They think of it as a work of art. 

By the way, the full version of Donald E.Knuth's interview is available at the website of Dr.Dobb's journal.

Data Mining for Environmental Protection



Data Mining is an emerging field of computer science. It involves data processing, data analysis and derivation of useful information from large databases or data marts.

Life-Cycle Assessment (LCA) is a technique used to estimate the environmental footprints of hardware products by taking a comprehensive view of multiple environmental impacts such as greenhouse emissions of products. Interestingly Data Mining can be used to automate Life-Cycle Assessment (LCA). This is a wonderful application of Software Engineering for environmental protection.

A recent article titled 'Using Data Mining to Help Design Sustainable Products' published by IEEE is a very good read on this subject.

You may ask, ‘How Green is my iPad?’. The answer to this question was elaborated by Daniel Goleman and Gregory Norris in their article published by New York Times, 4th April 2010.

Another interesting article on this subject has been published by Brazilian Journal of Oceanography. The title of this article is  'Data mining for environmental analysis and diagnostic: A case study of upwelling ecosystem of Arraial do Cabo'.

Is there a way to monitor illegal dumping of garbage and toxic material? The answer is yes. Read ‘Monitoring of Illegal Dumping Using Spatial Data Mining and Remote Sensing’ for more information.

Here is an example on how governments are funding research in this area:
Environmental data mining: learning algorithms and statistical tools for monitoring and forecasting

Isn't it interesting?

Speaking at Java Conference 2011

SiliconIndia's Java Conference 2011 is scheduled on August 20th 2011 in Bangalore. This conference is for Java professionals as well as IT experts who are into object-oriented languages, evolutionary methodologies (for example, Agile methodologies)  and emerging trends such as Cloud Computing .  This is a  1-day conference and it has 3 tracks:

     

    1) Language, Tools & Techniques

    2) Architecture and SOA

    3) Agile and Cloud.

The conference fee is very nominal. If you are in Bangalore, don’t miss this opportunity.  I am speaking on 'Agile for Legacy to SaaS Migration: Ten Key Considerations'.

Agile for Legacy to SaaS Migration



During 2000, I was associated with an Independent Software Vendor (ISV) - a small startup firm with not more than fifty employees. The engineering team of this startup firm built a Human Resources Management (HRM) product on J2EE stack with JRun application server and My SQL database. During the first 3 years the sales team sold the product to less than 25 customers. These customers were small to medium size businesses with 100 to 1000 employees. Sales cycles were longer. Every new customer had to go through an installation ceremony followed by a series of customization routine to make the product functional. Even though product customization fetched some revenue through professional services, working with customers spread across geographies and providing them product support became very tedious. Over the next several years the customer base of this product increased by 300%.

Since the past two years the complexity of product management and customer management has increased multifold. Every customer installation has had some form of customization request queue with less than 50% of the requests implemented in production. Lately, scaling up in the current model has become an insurmountable challenge.

Interestingly, this HRM product is a good example of a legacy web product or application that needs to be re-architected or transformed to SaaS paradigm. Can we use Agile for SaaS migration? The answer is “Yes”. However it is essential to reflect on a set of key considerations in such projects. You will find more info on this subject in the free white paper “Agile for Legacy to SaaS Migration: Ten Key Considerations”.

What has been your experience in this area?

Can Refactoring Improve Design ?

Yes, it can but not always!

When can refactoring improve design? Refactoring can improve design when team members who do refactoring are not only just programmers who know only language constructs but also are designers who know design principles. Team members who perform refactoring need to be experienced programmers. That is not enough. They need to know design principles in order to improve design. Else, they will not be able to figure out if the structure and association of entities or classes need to be changed in order to improve the design of software systems. They will focus only on a single program or subset of classes while refactoring.

Refactoring cannot improve design when we put programmers through a crash course on refactoring. It is essential to train them on object oriented analysis and design. They need to learn all design principles as well as programming principles. If we take this approach, yes, refactoring can improve design.

There can be extreme cases of dealing with complex or legacy code. Refactoring of legacy code is a costly affair. This requires experienced designers who have the ability to discover the hidden design.

In his article ‘Discovering Hidden Design’, Michael Feathers has articulated this very well with an example. It is a good read for all software professionals.

Challenges in Distributed Agile

While executing multi-site projects, Agile practitioners do face several challenges due to factors such as distribution of teams that results in limitations in face-to-face communication, and cultural mix of teams that impedes team bonding.   In some instances  I have seen collocated Agile teams inducting distributed teams in order to implement distributed Agile.  In other instances, several virtual teams come together to implement distributed Agile.  In both cases, the short term results are not impressive because of the inherent challenges associated with distributed Agile. Also, several questions such as the following surface as challenges.

1) How do you ensure that everyone understands the project vision and builds adequate rapport to work as a team?
2) Are engineers making the right assumptions (and validating them) and understand their expectations?
3) Are team members aware of detrimental communication loops that can impact productivity?
4) How will team members involve in efficient query resolution and ensure that they preserve the tacit knowledge exchanges?
5) How do team members avoid the trap of 'Blame Game' that could arise from product demos at the master location?
6) How do team members own quality and improve both internal and external quality of the product?
7) How will senior leaders or program mangers react to unfinished user stories?
8) How accurate can be the status checks? How will Scrum Master(s) perform status checks?
9) When and Where will Root Cause Analysis happen? Who will participate?
10)  How will senior leaders maintain team motivation across virtual teams?

There are ways to manage distributed Agile challenges and ensure early success in projects.  Governance of distributed agile projects plays a crucial role in making this happen.   What has been your experience?  What are the critical success factors?

SQL - The Past, Present and Future

In June 1970 Dr. E. F. Codd published the seminal paper, "A Relational Model of Data for Large Shared Data Banks", in the Association of Computer Machinery (ACM) journal, Communications of the ACM. This paper laid the foundation of relational databases and Codd's model got accepted as the definitive model for relational database management systems (RDBMS) across research institutes in all continents.

For the past three decades SQL has been accepted as the standard RDBMS language. There are three key reasons for the longevity of SQL. They are a) Simplicity, b) Strong Mathematical Foundation and c) Adoption (by several vendors). The power of SQL increased tremendously over the past three decades and SQL emerged as the de-facto standard for relational databases.

The NoSQL movement of 1998 supports database implementations that are non-relational data stores of several categories such as document stores, graph databases, key-value stores, multi-valued databases, etc. There are several flavors of NoSQL implementations such as Oracle Corporation’s BerkeleyDB, Google’s BigTable, Apache’s Cassandra, and Amazon’s SimpleDB.

It is very evident that SQL and NoSQL movement are complementary. RDBMS and SQL implementations will continue to thrive in ecosystems that require OLTP support on relational data stores. What Next?


Podcast on ’10 Best Influences on Software Product Engineering’

Tom Cagley (http://tcagley.wordpress.com/) interviewed me on my article ’10 Best Influences on Software Engineering’ published at SD Times.   This interview happened sometime during April or May 2011.  During this interview Tom asked me very interesting questions and engaged me in a great conversation.    This interview happened over phone across continents. He was in US (East Coast) and I was in India(Pune).   

Tom has composed this interview and made it available at http://spamcast.libsyn.com/s-pa-mcast-140-raja-bavani-the-ten-best-influences-on-software-product-engineering.

Tom runs 'Software Process and Measurement' (SPAM) casts regularly. The Software Process and Measurement Cast provides a forum to explore the varied world of software process improvement and measurement.  The SPaMCast covers topics that deal the challenges how work is done in information technology organizations as they grow and evolve.  In a nutshell, the cast provides advice for and from practitioners, methodologists, pundits and consultants!

I encourage you to visit his sites and listen to as many casts as you can.  You will find impressive interviews by several industry leaders in Software Engineering.

The Four Dimensions of Quality

This is with reference to my blog of June 3rd 2011 on The Manifesto for Software Craftsmanship Manifesto. This manifesto says

As aspiring Software Craftsmen, we are raising the bar of professional software development by practicing it and helping others learn the craft. Through this work, we have come to value:

Not only working software, but also well-crafted software
Not only responding to change, but also steadily adding value
Not only individuals and interactions, but also a community of professionals
Not only customer collaboration, but also productive partnerships

That is, in pursuit of the items on the left, we have found the items on the right to be indispensable.

When you observe the crux of this manifesto closely, it relates to four dimensions of quality, viz., Product Quality, Business Quality, Professional Quality and Engagement Quality.  Right ?

Are Software Engineering Methodologies Converging?

Are Software Engineering methodologies converging?  Or are traditional methodologies becoming extinct because of the popularity of evolutionary methodologies?   Isn’t it a reality that projects customize methodologies to suit project context?  Does it mean that every project adheres to its own flavor of a methodology?   Can we attempt to collect all engineering and project management best practices & pick and choose them to create a methodology for a given context?  Will Agile methodologies dominate the industry during the next decade?

Or do we need more methodologies?

The Ideal and the Essential

Distributed Agile projects are very challenging. Expecting the best results in your first Sprint is optimistic as well as ideal whereas the willingness to accept mixed outcomes and supporting your team in progressing over the first three or four iterations is essential.

Agile teams are as energetic as music bands that yearn for their first stage appearance. A debut of a music band comes with tremendous efforts and enthusiasm. The debut of successful musicians is not only well-rehearsed but also well-supported by the team as well as sponsors. The first Sprint of a team is as memorable as the first stage show of a budding music band.

More on this at “The First Sprint in Distributed Agile – What to Expect ?”

Manifesto for Software Craftsmanship

Robert Martin, one of the 17 founders of Agile Manifesto is one among the leaders who initiated ‘Software Craftsmanship’ movement during 2009 in order to emphasize on the importance of product quality. According to him, Agile practitioners focus more on iteration management activities and less on engineering best practices. The motivation of Software Craftsmanship manifesto is to kindle the enthusiasm in software engineers to create high quality products. Widely known among industry leaders as ‘Uncle Martin’, Robert Martin has objectively crafted this manifesto with his colleagues so that it remains aligned with Agile Manifesto and propels Agile movement. He believes that Software Craftsmanship movement will help us reinstate engineering best practices such as Coding Standards, Refactoring, Test Driven Development, Automated Unit Tests, etc. Also, he claims that several Scrum projects focus heavily on management practices and on the other hand they do not provide equal focus on engineering practices. He recommends that there has to be a balance.


Undoubtedly, Software Craftsmanship manifesto is here to stay and propel Agile movement over the next several years to come.

Agile Software Development - What Next ?

Agile Software Development - What Next ?  This is a frequently asked question.  Agile Manifesto was announced during Feb-2001.  Agile became very popular in the next few years.  

Looking back, Agile adoption has never been a cakewalk.  Immense challenges and struggles in transitioning from 'waterfall' to 'Agile' mindset were not uncommon.  Interestingly, something that happened along with Agile evolution was the growth of internet.

The growth of internet enabled computer users in collaborating on the net using collaboration tools as well as social and business networking forums.  The exchange of opinions, ideas, and best practices became easier on the internet.  Probably, I think, this is one of the key factors that propelled Agile.  Eventually, Agile could spread faster than any other methodology. 

On the 10th anniversary of Agile Manifesto, the founders had a gathering to discuss the past, present and the future of Agile Software Development. 

Couple of weeks ago, I was interacting with Michael Hugos on my thoughts on his blog 'Ten Years of Agile - What the Agile Community Must Do Now'.  

Our interactions were engaging.  The end result of our interactions was another blog 'Reflections on Four Recommendations for Agile Developers' by Michael.

When you read all these one by one, you will understand the recent happenings in Agile Software Development.   Also you will notice the power of collaboration over social and business networking forums.

Note: Every blog that I write carries a picture and this blog is not an exception. I took this picture while flying from Mumbai to Bangalore.  It is the picture of an apple offered to me by the cabin crew. 

Inventive Problem Solving for Customer Value Creation

SPIN (Software and Systems Process Improvement Network) founded by the Software Engineering Institute (SEI) of Carnige Mellon University (CMU) has over 130 chapters worldwide.  In India there are 13 chapters.   SPIN Chennai's annual conference SPICON 2001 is happening on 29th and 30th of April 2011. The theme of the conference is "Innovation in IT Processes - The Engine for Growth".

I am speaking at SPICON 2011 on "Inventive Problem Solving for Customer Value Creation."

Indian IT industry has matured over the past few decades.  The maturity of our industry has helped us encounter several challenges successfully.  However, new challenges continue to challenge us.  One such challenge is managing Service Level Agreements (SLAs) and retaining skilled engineers in projects when the work involved is technically complex but monotonous.  This session is about a case study on how we solved this problem by applying 'Inventive Problem Solving' and created value to stakeholders.

I will keep posting some interesting facts and questions here and provide my writing or a presentation on this topic to readers.

Fixed Price Agile

Agile Software Development is rooted in Agile Manifesto and Agile Principles.  It is all about delivering working software at regular intervals in order to provide visibility and predictability.  It is also about embaracing changes.  Agile requires discipline, focus, team work, conviction and courage.  I started a series of Distributed Agile Blogs couple of months ago. Going forward, I will be writing more on this.

Agile methodologies such as XP, Scrum etc. work well in Time and Material (T&M) model.   It has been a big challange to harness such projects in Fixed Price model.  How can stakeholder commit to Fixed Price and execute projects that involve agile principles?  Agile and Fixed Price: Can they go together?    These questions are just the beginning.  If we dig deeper, we come across distributed teams that work across time zones and cultures.   Executing agile projects with distributed teams has become a common theme across several companies.  However, combining agile or distributed agile with fixed price has been a bigger challenge.

My recent paper on Fixed Price Distributed Agile  shares a case study on this topic.  

The 10 Best Influences on Software Product Engineering

When we think through the past decade Software Product Engineering is one of the areas that witnessed several challenges as well as innovations. Here are the ten best influences on SPE that transformed the landscape of businesses as well as social life.

1)  Growth of E-commerce and Online Applications
2)  Service Orientation and New Business Models
3)  Collaboration Tools
4)  Agile Software Development
5)  Modern Software Engineering (or Modern Software Development)
6)  Test Automation
7)  Business Intelligence
8)  Global Software Development
9)  Open Source Movement
10) Paradigm Shift in Delivery Platforms

More information on each of these influences is available in my article 'Ten Best Influences on Software Product Engineering'.

CONSEG 2011 - International Conference on Software Engineering

CONSEG is a series of International Conferences on Software Engineering conceived and initiated by Computer Society of India in 1991.   CONSEG-2011 is going to happen in Bangalore, India during Feb 2011.   The theme of this conference is "Software Quality - The Road Ahead".   

When financial markets and economies struggle to regain their strengths,  the theme of this conference seems to be appropriate.  With cost-cutting measures and challenges in governing projects across multiple sites, delivering high quality software is a tall order.

I attended CONSEG during 2009. In my opinion, CONSEG is one of the professionally managed conferences with less commercial and promotional intents.   It offers a great value to every attendee and provides a overall perspective of the happenings in the industry, research instituties and academia.  

More info on CONSEG-2011 at the conference website.

Is IT Inclusive?

Information Technology (IT) and Software Engineering, the well known hi-tech areas once exclusive for scientists, engineers, academia and industry have transformed over years. IT has seeped through human lives in several ways.

As we enter 2011 with all the nightmares of global economic crisis, IT continues to penetrate several areas of our lives. Is IT exclusive? The answer is a definite 'No'. It is not exclusive for scientists and engineers.   Meanwhile, let us ask ourselves 'Is IT inclusive?'

These days, IT has started becoming part of physical infrastructure such as roads, power grid, television, telephone networks, and water distribution. Sorel Reisman, IEEE Computer Society 2011 President, in his article "Planning for an Inevitable Future" published in IEEE Computer magazine (Jan 2011) has expressed his views on this transformation and has emphasized on the need for organizations like Computer Society to start considering inclusive approaches. He says "It is time to consider a new transformation - one that addresses societal changes" and adds "By the end of 2011, we’ll have launched pilot Special Technical Communities in social networking, cloud computing, gaming, education, software engineering, and green computing." In this context, during Dec-2010, he initiated a social networking forum at http://www.computer.org/portal/web/the-presidents-discussion-corner for both members and non-members to participate.

Let me return to our question 'Is IT Inclusive?'. The answers are 'Yes' and 'Not Yet'. 'Yes' applies to some of the developed countries and 'Not Yet' for the rest of the world.

I can see certain visible transformations in India. During mid 80s, when I was doing my post-graduation, we used to wait for our 1-hour timeslot (couple of times a week) to use a personal computer at the computer lab of our University. It used to haunt us when a desktop crashes or has a technical problem. We had to wait for couple of days to a week or sometimes even couple of weeks for the technical support engineers to fix the problem. These days, the trend has improved. Timesharing happens among family members in sharing one or two computers. Computers are used more for other purposes than for educational activities by students.  Let me add more. Emails have become prevalent.  Computers are used for paying utility bills by bill processing executives or end users or both.  Online Education and Online Tests have become the trend setters.   Several widely accepted IT applications such as Online Banking can be added to this list.  This substantiates that we are into this transformation.  But we have not made it inclusive yet.

Finally, making this transformation inclusive in our context is a big challenge. At any cost, we are part of the inevitable future!  What about other countries?