Agile Program Management: Structure and Behavior

I believe you read my previous post.  In that post I shared a very simple lesson.   I am writing this post to discuss the next one.  It is ‘Architect your team structure and feed your feature teams!'.

 

Let me explain.  Think about a program consisting of multiple projects.  Every project has a structure according to the methodology followed.  Also at a program level there is a structure consisting of various roles and responsibilities in order to manage the program.  Every structure has its associated behavior whether it is at a project level or at a program level. Very true. Any ineffective structure at a program level can have cascading effect on almost all projects covered under the program. Obviously.

 

Hence, the lesson is ‘Architect your team structure and feed your feature teams!

 

One may think, ‘Why feed your feature teams? Why not focus on other areas as well?’ Well, a reasonable question. In my opinion, feeding the feature teams or providing them well-groomed requirements or users stories is the first and foremost need at ground level assuming that teams have required infrastructure, tools and platforms to perform their daily activities.  Providing well-groomed user stories is not the responsibility of program managers. However, being aware and taking necessary steps to ensure that well-groomed user stories are provided to teams is one of their responsibilities of program managers.  When program management teams are structured right and when they ensure that groomed user stories flow from time to time or from iteration to iteration, they get a very good view on what is happening on the ground.  They can do more. They can do more things better.

 

This is an ongoing activity.  Program management teams need to be agile. They need to go through periodic retrospective to relook at the structure and improve it.

 

How can we structure teams? We know how to structure project teams.  How do we structure program teams? We try to do this based on our prior experience or recommendations from an expert or consultant or coach – or both!  However, an important input to this exercise is to find an answer to the question, ‘What kind of behavior do we expect from this team? What do we want them to accomplish?  How do their accomplishments align with program goals?’

 

That is going to be a periodic exercise that may result in structural changes in project teams or program teams by means of role changes – either you introduce a new role, or make some changes to an existing role or create additional teams or something else to make things better.

 

When this happens, and when there is a constant focus on ensuring adequate flow of work, you have put your teams on the right track.

 

Any thoughts?

Agile Program Management: The Simplest Lesson

Program Managing a Large-Scale Distributed Agile Program involves several challenges such as team structuring, team induction, training, delivering in short-iterations, team retention and attrition management, system architecture, dependency management, risk management, software testing and automation and so on.  I am writing this post based on a large-scale distributed Agile program we undertook with more than 300 team members across two time zones with involving multiple business critical projects.   The objective of this post is to share experiential knowledge on managing large-scale distributed agile programs and present one of the simplest lessons. This is not it. I will be sharing many more lessons in my subsequent posts.

Program Management involves a higher degree of challenge and complexity as compared to Project Management.  This is because you need to cover multiple projects involving different lifecycles, pricing models, stakeholders, integration requirements, and dependencies. Of course, there are numerous other elements that add to this list.  When we move up from the role of a Project Manager or Scrum Master (in Agile world) to play the role of a Program Manager, we see program management as a role that involves managing more than one project and we stop there. We ignore the need to understand the possibilities of different technologies, stakeholders, regulatory needs, integration challenges and so on.  With over simplification we tend to ignore all the important aspects and eventually it hurts.

So, what is the simplest lesson that we need to know? Here it goes.

“Not all projects are the same. What worked in one project need not apply or work in another project.”

Obviously! We all know that.  However, how many times have we seen program managers as well as experienced project managers establishing a set of rigid rules or expectations on projects or labeling projects based on technology or lifecycle or something else? Here is an example.  Once, a program manager started opining that all agile projects must start doing Test Driven Development from the beginning.  Later, she mandated it.  It did not work because in some projects teams were getting ready to do automated unit tests right before jumping in to TDD. In some other projects teams were finding it tough to implement TDD because those projects involved data migration and the development activities were not mature enough to bring in TDD.

 

Let me present you with another example. Years ago, I came across a program manager who was very fond of three metrics - Schedule Variance, Effort Variance and Defect Density.  She ran program management meetings by looking at these three metrics. She wanted these three for projects of all sorts. It did not work because there were projects following iterative cycle. There were projects following Scrum and there were maintenance projects running in Kanban mode.

 

Well, is this simple lesson specific to Agile Program Management? No. It can be applied in Program Management in general.  Let me give you another example.  I had a Program Manager who used to manage multiple projects and most of them were following Scrum. Some of them were large-scale projects.  She recommended the same style, intensity and schedule of coaching – I mean Agile Coaching, for all Agile projects that would get added under her program no matter what.  She was ignoring the need to understand the needs of the project well and custom fit a coaching solution.  When an Agile Coach suggested this to her, she did not take it on the face of it. It took us some time to sell the idea.

 

“It worked in my previous project. It must work here as well.”  This is a typical Project Management mind block.  Many rookie project manager jump on to their second project with this belief. They bang their heads before they learn the hard lesson. And that is of course the simplest lesson.

 

Be open minded, listen to ideas, think logically and collaborate with your project managers. That is one way to keep remembering this simplest lesson and avoid fundamental mistakes.

 

Do you relate this simplest lesson to your experience or an incident happened at work? Please feel free to discuss. In my next post, I will continue with the next lesson.

 

Documentation in Agile Projects: Why? How? And What?

I am writing this blog post to provide a crisp treatment on a topic that comes up quite often when we talk about Agile implementation, adoption, transition and so on.  The first and foremost driver and a myth that causes this discussion is that Agile means ‘no documentation’.  Yes that is a myth.  Agile does not mean ‘no documentation’.  Let us begin with that understanding and move on to understand why we need documentation in software projects.    We need documentation in software because of the following reasons.

• Project stakeholders need it  (a business decision)
• To define a contract model (external interfaces and integration parameters)
• To enable communication with an external group (distributed teams, other project teams)
• For audit / compliance reasons
• To think something through – when you write or document something you think further and think deep. It helps before you get into discussions! You can save a whole lot of time by doing this!

There can be other reasons too. Here are some and these are not so good reasons. Beware!

• The requester wants it! – She has the authority and thinks that it is required.
• The requester mistakenly perceives documentation as project success
• The process says so!
• Someone wants to reassure someone else that everything is ok!
• Status Quo! We are a large organization and this is our way! 

In traditional SDLC, we used to do Big-Upfront-Documentation.  It is a costly affair.    When we know that 45% of deliver functionality is not going to be used, the corresponding cost invested in documentation goes unused too.  So, we need to be aware of the cost involved in big-upfront-documentation.



Our goal is to communicate. One of the mechanisms to enable this is to create documents.  However, documentation is the least effective mode of communication as shown below.



Let me step back.  Our goal is to communicate but that is not the only goal at a project level. There are several goals as listed here.

• Communicate Well - Documentation is not the primary issue – communication is!
• Ensure Project Success – Comprehensive documentation does not ensure project success!
• Create Consumable Documentation – Documentation need not be sophisticated!
• Create well-written, concise documentation
• Try to be flexible enough – Avoid trying to fit everything into a standardized template
• Keep the audience in mind – Usability and readability are critical.

In traditional SDLC, we have seen so many documents – High-level Plan, High-level Requirements and Architecture, Detailed Project Plan, Detailed Requirements Specifications, Detailed Design Specifications, User Manual, and so on.  And in most projects, these documents did not remain up-to-date as the software went to production and later in to maintenance phase.   Maintenance teams did not find them valuable and they started creating their own help documents.  The story went on. And this is true in most projects.

In Agile world, you ask follow Agile Principles. You know that document creation involves efforts and efforts cost money.   Why invest money in documents that go out of date?  Why not do crisp documentation and create an up-to-date set of documents that can be valuable to end users and maintainers at logical intervals?  Does it make sense?

So, Agile teams invest in documentation when they need to

• Communicate information during development
• Specify something (how-to test, what to test, etc.)
• Permanently record information (decisions and trade-offs)
• Confirm regulations

And they know that there are 3 types or categories of documentation.  These categories are not the same and documents belonging to these categories do not need the same type of treatment.

1. Work-in-progress documentation   (requirements, design, etc.)
2. Product documentation (API documentation, user manuals, etc.)
3. Handoff documentation (supports long-term viability of the project)

Above all, irrespective of the SDLC we follow, it is meaningful to ask the following questions and seek answers. That will help us know what to document and what not to.

• When? How? How much to invest in documentation? What is necessary at what timeframe?
• Do we measure the productivity of document creators? How?
• Do we measure the cost of maintaining documents? How?
• What do we do with stale or outdated documents?

So what documents do Agile teams create? It depends on factors such as complexity, strategic importance, user base across geographies, and criticality of projects. Here is a sample set.

• Executive Overview/Vision/Project Charter
• Project Overview
• Product Backlog, User Stories
• Test Cases, Test Scenarios with Test Data
• Solution Design, Contract Models(Interface Specifications)
• Design Decisions/Trade-offs
• User Manual
• Support Manual

This is what most Agile teams do.  And the documents they create

1. Maximize stakeholder ROI
2. Stakeholders know TCO of the document
3. Remain Lean and sufficient
4. Are meant for specific customer or audience
5. Are sufficiently accurate, consistent and detailed
6. Fulfill the purpose

Are you facing challenges in accomplishing these? Let us discuss.

Scaling Agile: Hone Your Anticipatory Skills

 

Working on large-scale Agile projects has made me feel as if I am challenged to solve a 10,000-piece three dimensional jigsaw puzzle!  When I think deep, the challenge is more than that.  You have some clarity and a great level of ambiguity on what you need to accomplish and you don’t know the number of pieces. As you move on some pieces are ready, and visible. Some are not visible, but you feel that you are in a dark room and you can touch and feel and create a visual piece to some extent. Other pieces evolve as you move forward.  There are multiple people involved in putting all these pieces together. Some of them are providing oversight, some are giving ideas, some other are reviewing and commenting, and the puzzle is in the process of getting solved.  This analogy stretches based on your experience. And you may say, ‘It is a lot more than that and as complex as working on multiple such puzzles and putting them together.’   Am sure, you have been through software projects like these. You need to hone anticipatory skills to sail through such projects.  Without anticipatory skills, your contribution will not be significant enough.

 

Do you know? Anticipatory expertise has been observed in a number of athletes such as Baseball players, Tennis players and so on. For more information on this, read Dan Peterson’s article 'The Anticipatory Skills of Athletes'.

1. What can we do with our anticipatory skills in large-scale Agile projects. First, we can think through and ask anticipatory questions.  Next, we can come up with potential action plan to handle what we anticipate. Here are some examples.
2. What is going to be the maturity of user story grooming for the upcoming Sprint based on past experience? How can it impact the schedule and quality of deliverables? What do we need to do about it?
3. What can be the impact of an upcoming architecture or design change on code quality and unit test coverage? What are our current measures and what can they be in future? How can we manage stakeholders?
4. What are the right sets of test cases for an upcoming Sprint? How can we optimize testing efforts and improve quality?
5. What are going to be the risks that we need to handle as we move forward? What is going to be the impact?
6. How is the current pace of work impacting team morale? What do we need to do about this?
7. What are going to be the structural changes in teams over the next two iterations? What do we need to do now to prepare for those changes?
8. How are we planning to integrate the software we produce with related projects or products? How are those projects or products performing in terms of schedule, quality and stability? What are the dependencies? How can that impact the schedule and quality of our project?
9. What are some of the leadership changes we foresee in future? How do we need to manage it effectively?
10. Based on the success rate of our build and release plans, how do we foresee success in the upcoming release? What are the corrective measures we need to put in place now?

Anticipatory skills are essential for Scrum Masters, Project Managers and the likes.  Anticipatory skills are required in all kinds of projects? When it comes to large-scale distributed Agile projects, anticipatory skills can make you play the right cards at the right time.  How well are you honing your anticipatory skills? Are they paying off?

Related Posts:

1. Scaling Agile: What Do Architects Deliver?
2. Scaling Agile: Architecting Your Agile Team
3. Scaling Agile: Feeding Your Feature Teams
4. Software Architecture and Design in Agile World
5. Scaling Agile: Emerging Architecture and Technical Debt
6. Scaling Agile: The Value of Independent Testing
7. Scaling Agile: Succeeding in Release Management
8. Scaling Agile: Disastrous Software Factories
9. Scaling Agile: Transitioning to Kanban
10. Scaling Agile: Hardening Sprint and the Problem of Procrastination

 

Scaling Agile: Hardening Sprint and the Problem of Procrastination

“Yes, we know that this is not done right yet. We have some rework to do and we have three more feature Sprints with a significant backlog. We will take care of this rework in the hardening Sprint.” 

Sounds familiar? 

What is a hardening Sprint? Why do we plan for a hardening Sprint?  Hardening Sprint is an iteration that happens before release readiness. It is a common practice irrespective of the methodology one follows. The term ‘hardening’ comes from ‘code-hardening’ – providing some finishing touches, making some minor adjustments to make things better on code base.  The goal is to improve structural quality and fix some defects too. It is not about delayed refactoring. Hardening Sprint does not include implementation of new features – obviously!  Hardening Sprint is to tie all the loose ends, fix defects and make sure that the product is ready for release. No new features but finishing touches, key improvements, and defect fixes. Period.

In large-scale Agile projects, even if we adhere to this norm, we know there are enough challenges. This is mainly because of what is involved in hardening the software created by multiple feature teams. And when we violate the purpose by introducing new features or major changes, we eventually face a whole lot of challenges.

When we go through a whole lot of challenges in a hardening Sprint because of reasons such as introduction of new features or user stories or major changes to architecture/design components or any change that rattles the compatibility of components, we must admit that probably it is because of the problem of procrastination.  Procrastination results in some of these -  insufficient planning or re-planning, ineffective prioritization, inadequate story grooming, weak design, postponed decisions, too many changes, fuzzy acceptance criteria, and so on. All these could make a hardening Sprint harder to execute.

When we are in feature Sprints to deliver user stories or features, we tend to give importance to things that are immediate and urgent (not necessarily important).  We tend to do quick fixes. We accumulate technical debt.  Eventually, we lose focus on long-term goals or vision.

“Giving up on our long-term goals for immediate gratification is procrastination.”  This is what Dan Ariely says in Chapter-6 of his book ‘Predictably Irrational’.  In this chapter he has given multiple examples on why we can’t make ourselves do what we want to do.

In large-scale Agile or projects cross-functional teams align and exhibit certain behavior.  Sometimes, the power of dissent is forgotten or ignored.

What can we do?  We can do many things to make a hardening Sprint meaningful and valuable.  Here some ideas.

1. Identify the factors that make a hardening Sprint harder.  Learn from experience. Check if certain bad smells are repeating when you are working on a subsequent release. If yes, find a fix.
2. Define the scope of Hardening Sprint 2 or 3 Sprints ahead in the release cycle.  Assess the need for any corrective action – such as one more feature Sprint or abandoning low-priority features.
3. Assess technical debt (including testing debt – such as lack of adequate testing or lack of adequate automation).
4. Assess environment issues that may lead to a backlog of work items in the hardening Sprint.

Try all these at the end of Sprints.  Don’t procrastinate and look at all these just before the hardening Sprint begins!

Related Posts:

1. Scaling Agile: What Do Architects Deliver?
2. Scaling Agile: Architecting Your Agile Team
3. Scaling Agile: Feeding Your Feature Teams
4. Software Architecture and Design in Agile World
5. Scaling Agile: Emerging Architecture and Technical Debt
6. Scaling Agile: The Value of Independent Testing
7. Scaling Agile: Succeeding in Release Management
8. Scaling Agile: Disastrous Software Factories
9. Scaling Agile: Transitioning to Kanban

Scaling Agile: Transitioning to Kanban

 

Large-scale agile projects include multiple feature teams working in rhythm over short iterations or time-boxes one after the other.  That happens when features for the first release are coded, integrated and tested in time-boxes.  When the first release or a subsequent release goes to production, we see an emerging need for forming one or two teams to work on daily iterations unlike 2 or 3-week iterations of feature teams.  This is because of the need to provide timely support and daily fixes to product owners and end-users.  Here, the transition from iteration-based or Sprint-based development into Kanban becomes essential.  Obviously, those one or two teams that support end-users cannot afford to make their stakeholders wait for 2 or 3 weeks for defect fixes and patches. They need to be even more agile, and they need to respond daily by fixing critical issues and responding to end-users.  Potentially, they must be able to understand and implement Kanban.

 

Transitioning to Kanban comes with two key aspects. The first one is about learning and implementing Kanban and the second one is about aligning the engineering processes such as configuration management, build and release management, tools etc.

 

When you and your team are transitioning to Kanban, you need to do a good mix of unlearning and learning.  This is because, when you do Kanban, you are not going to do Sprint Planning or Story Point Estimation.  You are not going to do daily stand-ups with the anticipation of delivering something at the end of the Sprint. You are going to learn about visual boards or more specifically Kanban boards. You are going to experience how team members ‘pull’ work items.  You are going to learn the concept of ‘Work in Progress’ and WIP Limit.   And of course, when you work with virtual teams, you will need tools that support distributed Kanban teams and offer electronic visual displays.

 

Coming to the second aspect on engineering processes, the first step is to think through configuration management.  How are you going to establish configuration management across multiple feature teams that synchronously work on 2 or 3-week iterations and Kanban based production support teams? An obvious solution is to have 2 branches – one for ongoing development and the other for production support activities. How frequently are you going to merge? And what is going to be the efforts involved in merging them together?   Next, how are you going to perform independent validation or testing in the new model? What are going to be the revised overheads or estimates? Finally, how will this new model impact your build and release management process?

 

Are you looking at all these aspects while transitioning to Kanban?  Are there any other key factors or issues?
 

Related Posts:

1. Scaling Agile: What Do Architects Deliver?
2. Scaling Agile: Architecting Your Agile Team
3. Scaling Agile: Feeding Your Feature Teams
4. Software Architecture and Design in Agile World
5. Scaling Agile: Emerging Architecture and Technical Debt
6. Scaling Agile: The Value of Independent Testing
7. Scaling Agile: Hardening Sprint and the Problem of Procrastination
8. Scaling Agile: Succeeding in Release Management
9. Scaling Agile: Disastrous Software Factories

Scaling Agile: Disastrous Software Factories



Quite often, we hear the buzz word ‘Software Factory’ in Scaling Agile conversations and conference sessions.  Those who promote or sell the concept of ‘Scaling Agile’ attempt to indoctrinate their audience with the belief that Scaling Agile is a definitive way to setup an Agile Software Factory. Yes, when you add ‘Agile’ to ‘Software Factory’, it becomes even more attractive.  While there are several case studies and white papers about how to go about setting up an Agile Software Factory, the challenges involved in doing so are numerous.  Because of such challenges and several other reasons, initiatives to setup such factories become disastrous.  Why do such disasters happen? This post is to bring out a list of reasons.  Here is the list.

1. Vision and Leadership Commitment - Lack of vision and leadership commitment is a primary reason.  Well, in many projects and programs there will be vision and leadership.  However, when the vision is not set right and shared among all stakeholders including team members and when there is no inspiring leadership and leadership commitment to support and nurture the vision, results can be disastrous.
2. Investment in Coaching - Not investing in Agile Coaching is a definite recipe for disasters.  In large programs it is necessary to invest in coaching at different levels right from teams to middle and senior management.  Obviously, this means that you need a coachable ecosystem. Learnability is important. When there is rigid mindset and inability to learn and collaborate, cultural issues fuel disasters.
3. Focus on Cost Savings Only - Just because your ultimate goal is to set up a software factory, your focus cannot revolve around cost savings all the time.  You cannot optimize your team members just as you optimize your machines in the production line.  You cannot replace them just as you replace your machines. You cannot operate in shifts to save costs. You cannot forget about people and their emotions.  Obviously, ‘that’ factory is different from ‘this’ factory.
4. Physical Infrastructure – Lack of infrastructure including open workspace, video conferencing, phones, meeting rooms, computers with necessary configuration, network connectivity, tools, etc., is the next reason for such disasters.
5. Intellectual and Emotional Infrastructure - Intellectual infrastructure is about skills, competencies, expertise, special groups, knowledge forums, technical excellence and so on. Emotional infrastructure is about team bonding, trusted leadership and the likes that enhance the feeling of belonging.  When intellectual and emotional infrastructure are not adequate you will see symptoms such as  loopholes in process implementation,  poor or weak automation strategy, poor execution, lack of technical excellence, etc. leading to pool quality.
6. Inclusion - In virtual teams or geographically distributed teams, lack of inclusion can create chaos. We know that there are several challenges in distributed agile.  When there is lack of inclusion, you see one site dictating or directing the other sites, double standards, and so on.
7. Governance - Though this comes as seventh, it is as significant as the first reason.  Governance can make or break things.  Collaborative, inclusive and supportive governance teams can do timely course correction and deliver value to customers.  When the members of Governance team do not have a unified agenda, they may pull each other in different directions. That is disastrous.

Do you see your team stretching or burning out? Is that becoming a trend? Are your team members demotivated? Are you calling your project or program a Software Factory? Probably you do not sit through with your team members and empathize on what is going on. You are not identifying the root causes. Revisit the seven reasons mentioned in this blog and see how you can make some course correction to come out of a disastrous Software Factory and make it a success. 

 

Related Posts:

1. Scaling Agile: What Do Architects Deliver?
2. Scaling Agile: Architecting Your Agile Team
3. Scaling Agile: Feeding Your Feature Teams
4. Software Architecture and Design in Agile World
5. Scaling Agile: Emerging Architecture and Technical Debt
6. Scaling Agile: The Value of Independent Testing
7. Scaling Agile: Hardening Sprint and the Problem of Procrastination
8. Scaling Agile: Succeeding in Release Management
9. Scaling Agile: Transitioning to Kanban

Scaling Agile: Succeeding in Release Management



This post on ‘Scaling Agile’ is to present a list of critical factors to succeed in release management.  What makes large-scale Agile teams capable of ensuring successful product releases? Here is the list.

1. Roadmap and theme for releases – Have a road map and theme for all releases or as many you can envisage.  This is one of the key responsibilities of product managers and sponsors.
2. Tech debt management -  In addition to having measurable Definition of Done and Acceptance Criteria to measure the success of Sprints, have a consistent focus on technical debt management.
3. Don’t let a hardening Sprint result in a chaos!  - Plan hardening Sprint well and execute them to preserve the integrity of your product release.  Don’t let a hardening sprint result in a big mess leading to an unexpected upsurge of defects.
4. Test Automation – The strength or weakness of test automation will reflect directly on the team experience, challenges and results of release management.  Keep this in mind.
5. Risk-based Testing – It is important to focus on risk-based testing by identifying risk prone areas or epics or modules and testing them first.
6. Continuous Integration - You may have continuous integration working very well in development and staging environment.  Make sure that continuous integration is applied in other environments too (e.g. Testing, User Acceptance, and Production).   When you don’t do this, you are going to carry out several manual steps to complete integration. That is going to be time consuming and error prone.
7. Continuous Delivery and DevOps – Do you want to increase the ‘speed-to-deploy’ in product environment? Think two things – Continuous Delivery and DevOps.  These will improve efficiency and result in cost savings.
8. Defect Management - Is your defect management process helping you in slicing and dicing data to understand product quality? Does it allow you to predict? Does it allow you to spot missing test cases?  Or are you relying on a basic defect management process that helps you understand defects by priority and severity alone?  Effective (and efficient) defect management is an enabler for release management.
9. Dependency Management – Understand all dependencies, categorize them and manage them effectively. Do not allow any dependency to create a big surprise and impact release schedule or quality.
10. Communication & Coordination and Stakeholder Management – These are the non-technical but essential aspects not only for release management but for all software engineering activities. Make sure that the way you communicate and coordinate among teams enables osmotic communication. Also, keep your stakeholders informed of what to expect and how release management activities are progressing. Manage their expectations well.

 

Related Posts:

1. Scaling Agile: What Do Architects Deliver?
2. Scaling Agile: Architecting Your Agile Team
3. Scaling Agile: Feeding Your Feature Teams
4. Software Architecture and Design in Agile World
5. Scaling Agile: Emerging Architecture and Technical Debt
6. Scaling Agile: The Value of Independent Testing
7. Scaling Agile: Hardening Sprint and the Problem of Procrastination
8. Scaling Agile: Disastrous Software Factories
9. Scaling Agile: Transitioning to Kanban

 

Scaling Agile: The Value of Independent Testing

 

In Agile projects of all sizes including large-scale projects, software testing plays an important role.  In our industry, there is a school of thought that proclaims that there are no developers or testers in an Agile team and everyone is a ‘team member’. Also, it challenges the need for independent testing.  My intention is not to argue on that but to share my thoughts on the value of independent testing in this post.

 

First, in reality we do not develop standalone applications every time. A vast majority of software applications or products we create are complex with dependencies or integration needs on internal or external systems.  They involve multiple technologies and compatibility requirements too. Second, nowadays, many projects involve two or more cross-functional teams.   As the number of teams increases the situation becomes complex in spite of cross-functional teams focusing on the ‘Definition of Done (DOD)’.  It is not enough if multiple cross-functional teams deliver working software that satisfies DOD.  We need an integrated build that works!  We need an integrated build that satisfies all functional requirements.  We need to make sure that it satisfies all non-functional requirements as well – everything related to performance, security, integration, and so on.  We need an independent testing team or independent verification team to take care of this.

 

That is obvious, known and make sense. However the questions are, 

1.       What should be the size of this independent testing or independent verification team?

2.       When should we introduce this team - in the first iteration itself or closer to the first release?

3.       Do we need this team in all iterations throughout the project?

4.       How can we justify the costs?

5.       What can go wrong even if we have an independent verification?

 

The size of the independent team typically varies between 10 to 15% of the total team size. For example if the size of your large-scale agile team is 200, that will include an independent testing team of 20 team members.  That is just a guideline.  And this team will include automation experts who focus on test automation for end-to-end functional tests, performance tests etc. The size of the team will increase based project complexity (multi-browser, multi-device, complex architecture, complex business rules), dependencies, non-functional requirements, target users (number of countries, regions, languages, etc.).

 

‘When should we introduce this team?’ is a great question. It is very important to explore test strategies and identify an optimal one.  Your test strategy exploration needs to address several questions - What is our need?  What are the possible approaches to meet our needs? What are the tools and techniques? What are the measures? What is going to be the governance?  What are the pros and cons of strategy A, B, and C?  When you do this you will get a clear idea on when to introduce this team.

 

‘Do we need an independent verification team throughout the project?’ The answer is ‘Yes’. You have multiple cross-functional teams that deliver features in short iterations.  You need an independent verification team to test the integrated builds.  You need them all the time.  How do we justify costs? Well, don’t you have to test the integrated builds? Of course, yes.  Independent verification directly relates to the cost of quality.

 

‘What can go wrong even if we have an independent verification team?’  Good question. Several things can go wrong.   First, feature team can become lax on delivering tested code to meet DOD or product owners may relax DOD to speed up feature delivery assuming that independent verification team can find all defects.  With that you enter a vicious cycle and independent verification team will slog in finding all kinds of defects.  Feature teams will do firefighting to fix defects and that will potentially open more defects. This is a huge risk on release quality as well as timelines.  Second, with no or weak test strategy, independent verification team may lose traction on test automation.  Third, with lack of focus on fundamental testing principles, the team may dilute or miss traceability and coverage. There can be many other things that can go wrong.  Please feel free to add those in your comments.

 

‘Are we going to have an Independent Verification Team? That sounds like waterfall!’  You must have heard this too. It is not waterfall. It is large-scale Agile.  All team members are involved from the initial stages. The goals are clear. They follow Agile principles and practices.  Teams do not operate in silos. Their timelines are not squeezed.  And they don’t compromise in any way in terms of quality to make a release happen!

 

Related Posts:

1. Scaling Agile: What Do Architects Deliver?
2. Scaling Agile: Architecting Your Agile Team
3. Scaling Agile: Feeding Your Feature Teams
4. Software Architecture and Design in Agile World
5. Scaling Agile: Emerging Architecture and Technical Debt
6. Scaling Agile: Hardening Sprint and the Problem of Procrastination
7. Scaling Agile: Succeeding in Release Management
8. Scaling Agile: Disastrous Software Factories
9. Scaling Agile: Transitioning to Kanban

 

Scaling Agile: Emerging Architecture and Technical Debt

In Agile projects including large-scale projects, emerging architecture offers immense benefits and it can save lot of big-upfront efforts.  That is an advantage. However, emerging architecture can lead to technical debt and significant rework when there is lack of orchestration in terms of timely decision making, validation and dependency management. I am writing this post to discuss how emerging architecture and technical debt or interconnected and how even if you ensure emerging architecture there is a possibility of accumulating significant technical debt.

 

As discussed in my previous posts (see ‘Related Posts’ at the end of this post), we need a great deal of communication, coordination and collaboration among business users, architects and designers to create and maintain a prioritized product backlog and provide groomed user stories to feature teams. That is about the four gears and they need to work synchronously.  When they are disconnected or out of rhythm, one or more of the following can happen.

1. Architects make decisions with a high-level understanding of business requirements. But those decisions may not take into account any recent changes in business requirements.
2. Solution Designers and Architects do not come together. They depend on assumptions or design-by-contract decisions that are not documented and agreed up on.
3. Decisions are not validated to firm up feasibility by constructing and demonstrating PoCs.
4. Architects and designers let features teams move forward for implementation by feature teams with placeholder decisions or designs. Later they firm up their decisions and leading to significant rework by feature teams.

These are obviously the symptoms of disconnected teams or teams that lack communication, coordination and collaboration as shown in this diagram. Yes. When they are disconnected, they do not produce a backlog that can optimize technical debt and reduce rework.

 



 

So, what do we need to do when we work on large-scale programs?  Here are some tips.

1. Look ahead architecture and design is important.  Let your architecture and design mature and be ready to cater to at least 2 to 3 upcoming iterations.  That is not big upfront design.
2. Construct architecture prototypes or PoCs to validate your architecture and design decisions.  Do not pass it on to feature teams if you want to reduce rework and technical debt.
3. Let us assume that you have four or five iterations per release in your project or program.  Most probably there will be a hardening Sprint at the end of releases.  And there will be some architecture or design work that may cascade into rework by feature teams.  Do not let this ripple effect happen your hardening Sprint. When you do this you will see code changes across all layers leading to an upsurge in defects. One way to avoid this is by firming up your architecture and design decisions at least two iterations or Sprints before your release ends.

Have you been part of large-scale programs and attempted to optimize the accumulation of technical debt through emerging architecture and designs?  How have you accomplished this? Would you like to share some practices or tips here?

 

 

Related Posts:

1. Scaling Agile: What Do Architects Deliver?
2. Scaling Agile: Architecting Your Agile Team
3. Scaling Agile: Feeding Your Feature Teams
4. Software Architecture and Design in Agile World
5. Scaling Agile: The Value of Independent Testing
6. Scaling Agile: Hardening Sprint and the Problem of Procrastination
7. Scaling Agile: Succeeding in Release Management
8. Scaling Agile: Disastrous Software Factories
9. Scaling Agile: Transitioning to Kanban

 

Scaling Agile: Feeding Your Feature Teams

 

One form of scaling Agile is about implementing Agile principles and practices in large projects or programs in such a way that you are able to deliver business value through time-boxed iterations and maintain a sustainable pace. Large projects or programs constitute multiple releases and releases comprise of multiple iterations.  In such projects or programs you will have multiple feature teams. For example, a project I was involved some time ago consisted of more than twenty feature teams distributed across two or three sites.  When a project or program of this size is in full form and happening, how do you feed your feature teams?  How do you feed your feature teams with groomed user stories and make sure that dependencies are resolved and technical debt (and consequent rework) is minimized?

 

One of the critical success factors of large-scale Agile projects is how effectively we feed the feature teams so that feature teams get quality inputs in order to ensure corresponding income. This includes

 

a)      Identification of business epics

b)      Architecture envisioning

c)       Making timely architectural decisions

d)      Identification of technical stories

e)      Validation of designs and technical decisions through POCs

f)       Breaking business epics into user stories

g)      Release planning

h)      Dependency management

i)        Sequencing the right set of stories for feature teams

j)        Story grooming 1 or 2 iterations in advance

k)      Adhering to ‘Definition of Ready’

l)        Defining Acceptance Criteria

m)    Having Definition of Done

When all these happen, feature teams will have groomed stories that satisfy the ‘Definition of Ready’ and have a clearly defined ‘Acceptance Criteria’. Doing this is a positive step towards minimizing technical debt, and avoiding rework.  Also, this can help us minimize story cycle time – the number of iteration it takes to complete an end-to-end story that is production ready.

 

Who does all these?  I hear this from hard-core Scrum practitioners – “It is the PO and Scrum Masters.  We have Scrum-of-Scrums.  There are no special roles such as Architects, Designers, and so on.”

 

What happens when you work on a large-scale complex green-field development project based on an emerging technology? What happens when none of the technical architecture defined earlier for other products or applications cannot be reused for this project? What happens when business requirements are complex and evolving?   When all these come together, I don’t think ‘Scrum-of-Scrums’ with a chief PO or PO can spend their time in turning out all these to feature teams.

 

 

You will need a team of 1 or 2 or more architects. You will need a team of two or more solution designers.  You will need a team of more than 1 PO.  And you will need a small group of technical experts to validate designs.  Besides, all these groups need to synchronize in churning out meaningful high-level release backlog and groom stories in a timely manner so that feature teams get what they want when they start iterations.  Obviously!

 

Here are the advantages of this approach.

 

a)      Effective release planning and dependency management

b)      Validation of architectural and design decisions before stories are assigned to feature teams

c)       Effective technical debt management

d)      Reduction of rework

e)      Optimization of story cycle time

f)       Improvement in quality (feature teams get quality inputs and acceptance criteria)

One of my friends asked, “All I have are 2 or 3 feature teams. It is not a very large project. Do I need such a complex structure?”  I said, “With the current structure, are you able to feed your feature teams? If no, you may need to find ways to make that happen.  May be you need to improve the way your current team operates or make some significant changes.”

 

If you want to deliver working software through multiple feature teams and deliver successful releases, don’t you have to focus on feeding your feature teams? I am sure you say ‘Yes’.

 

How have you done this or seen this happening in your projects or organization? Are there similar or different or better approaches?

 

Related Posts:

1. Scaling Agile: What Do Architects Deliver?
2. Scaling Agile: Architecting Your Agile Team
3. Software Architecture and Design in Agile World
4. Scaling Agile: Emerging Architecture and Technical Debt
5. Scaling Agile: The Value of Independent Testing
6. Scaling Agile: Hardening Sprint and the Problem of Procrastination
7. Scaling Agile: Succeeding in Release Management
8. Scaling Agile: Disastrous Software Factories
9. Scaling Agile: Transitioning to Kanban