Agile estimation is not waste per se

So you are not going to do estimations as they are a waste?

Great! How are you going to create a common understanding of what you need to deliver as a team?

You’ll probably pick a feature and discuss it in your team right? Discover what it is that needs to be build, discuss development approach, tasks and so on. The time it takes to do this depends on the story a hand. After you have had this planning discussion you’ll have a common understanding. Then you’ll discuss the next feature and so on. After your team has build up a common understanding of a feature and a plan for delivering it they also have a good idea of how much work it will be. So the team is able to give a estimation pretty fast (within 1 minute) as all the work needed for estimation is already done. Even if the estimate is completely wrong the so-called waste is almost zero!

Ok, what if you have a big list of features? then estimation of the whole list upfront is a waste? what if I have a list of work that is more or less equally sized?
Ok… but how did you get such a list? Don’t tell me you had to break it down into equally sized parts!

Lets take a look at waste.

• Type I muda is necessary waste, it is something that does not add value for the customer. The way of working of the business needs to change in order to eliminate this type of waste.
• Type 2 muda is unnecessary waste. Non value added tasks that can eliminated directly without any change.

Ok. So who is the customer in Lean?

Maybe the most important part of lean is called Respect for People.
The end customer is the most important customer but not the only one. Everybody that receives work in a value stream is a customer to the ones who deliver. Respect for people in this context also means that you offer your work in such a way that the receiver can handle it in the best way.
So e.g. Product owners, shareholders, management and so on are all customers in the lean context. The product owner is offered easy and clear indicators of progress and estimated work remaining; operations is offered a clean and high quality product; management is offered clear and honest information on costs, opportunities and risks.

So the question becomes: Does estimation not add value for any customer?

Example: You have projects in your portfolio and you have to choose which one to do? How are you going to choose?
You could take into account things like estimated added value, estimated risk, estimated duration, estimated costs etc.

Should we estimate a whole big release backlog? of-course you should as long as there is someone who needs a number to make valuable decisions. Just make sure it does not take you to long! Give input to make the budgeting decisions and then concentrate on improving your estimates and plans as you make real progress.

So if estimation adds value to workplace customers is it still a waste?
Only if you change the business in such a way that it does not need estimations anymore then estimation becomes a waste!

So in summary

• Estimation is a waste only in certain contexts.
• If you work as a team and plan your iterations you probably already did al the work needed to give an estimate anyway.

ALE Bathtub III last slot by Ester Derby

Bathtub III is on October 20th from 21:00 until 22:40 CEST.

Final agenda

21:00 – 21:15 CEST | Esther Derby – Re-thinking Management
21:20 – 21:35 CEST | Scott Barber – An overview of Performance Testing for Agile/Lean teams
21:40 – 21:55 CEST |Cesario Ramos - Lean Agile Sandwich
22:00 – 22:15 CEST | Olav Maassen – ‘Surprise surprise’
22:20 – 22:35 CEST | Filippo De Santis – Xkanban

Goto Bathtub III for more info and registration.

Just in Time details – A refresher

A backlog can contain user stories. And these user stories are just the three C’s as Ron Jeffries stated.

• Customer
• Collaboration
• Confirmation

It is written by the Customer and it is so little text that you are forced to Collaborate with the customer because otherwise you do not know what to develop.
The Confirmation enables a team to make a reasonable estimate of the user stories and so we come up with a estimated backlog.

A good thing is that the user stories naturally encourage us from adding requirement details at an early stage. Once the PO selects user stories for the next sprint the necessary details can be added Just In Time, like UI, data specifications, business rules, business process flows etc. People need time to add these details, not because the actual writing or describing takes time, but because people need to think about what they want. It is partly a creative process, partly a discovery process and this takes time!
Not all requirements should be written down in full detail, but just enough to have a good start the next sprint! Next to adding details to the scheduled user stories the backlog should also be improved by for example adding new stories, splitting stories from existing epics and estimating any new stories.
So people of the team need to start thinking about some details and adding the details to the selected user stories for next sprint during the current sprint. This of-course interferes with their activities in the current sprint, but the benefits of preparing are often higher then the losses due to e.g. task switching.

In order to start the information transfer to the team members early on, half way the sprint a requirement workshop can be organized to share and learn about the user stories scheduled for next sprint. The result of this workshop is a better mutual understanding within the whole team about what to build and what can be improved prior to the sprint planning meeting. Such a workshop should also result in a set of acceptance examples of how the PO wants to verify that a user story is build in the right way.
An important benefit is that the end result of the user story details are better because the knowledge of the whole team is used. A great side effect is that because the team is involved and contributes to the requirements and therefore the direction of the product to be developed, their responsibility and motivation is increased.

So now the sprint planning will go smoother, concurrent development during the sprint is much easier to do and WIP is under control.
In Scrum btw we expect a team to spent around 10% of a sprint grooming the backlog and preparing for the next sprint.

But isn’t this mini waterfall? No! In waterfall all requirements are detailed before development starts. But even if it was waterfall, so what! Is the goal to not do waterfall?

Retrospective results Bathtub II and peek at Bathtub III

We, Carlo Bechi, Oana Juncu, Pascal Dufour and Cesario Ramos had a retrospective on the 2nd Bathtub and come up with the following improvements for the 3rd Bathtub

1. Introduce 3 – 5 min breaks between the talks, so people have time for a break and some discussion using IRC, Twitter and so on.
2. We are going to search for a new tool that support multiple organizers
3. We will keep the IRC channel for collaboration
4. We will go back to 5 talks of 12 min + 3 min questions each.
5. We will be asking the country of participants to get a overview of the participants locations.
6. Plan retrospective with speakers and attendees upfront before the conference.
7. Survey after the conference.

Potential speakers for the 3rd Conference that wil be organized in october 2011.
- @vhazrati
- @cesarioramos
- @olavmaassen
- @jbrains
- Elisabeth Hendrikson
- @filippodesantis (shorter version of http://www.slideshare.net/p16/xkanban-xp-kanban-e-timeboxing)

The Bathtub initiative is for the ALE community and organized by the ALE community. So if you want to contribute in the organization or as a speaker please contact me.

Kanban is the new silver bullet?

Kanban is about signaling. I need a new beer because my current one is almost done, I need some more stories to test because almost all stories are tested or even bring me my new bathtub because I almost removed my old one!
Signaling creates flow which is the number one driver for reducing costs as it reduces delays, rework and excess inventories.

Kanban is also about limiting work in progress or WIP. Your next beer is ready, I will not make you a new beer because it will get warm before you drink it. I will not code any more stories because by the time you test them and find a bug it will take me lots of time to resolve it and more bugs can have accumulated. I will not bring you a new bathtub because you did not finish removing the previous one and have nowhere to place it.
Limiting WIP is the number one driver for decreasing lead time because it limits transaction costs and absorbs variation in demand more easily.

Kanban is also about improvement. Now that I have only one beer ready for you, I see that you are getting drunk and will not make you any more beers. No that I stopped coding user stories I see that testing is a bottleneck and lots of bugs are returned for fixing. Now that you cannot place you bathtub anywhere I see that we are not communicating properly.

Kanban is just an approach to realize a pull system. You can also realize a pull system without Kanban.

Using Kanban to limit WIP and exposing your bottlenecks is a great approach. You can get a real map of how work is done currently and it gives you a good indication for potential improvement areas.

There is a catch though as there always is.

It does not tell you if you are doing the right thing! And therefore with Kanban or any other approach, you could end up doing the wrong thing better!

And as Russell Ackoff tought us
“It is far better to do the right thing wrong than to do the wrong thing right.”

So using Kanban to improve your system without understanding the work as a system and redesigning it from that view will not result in any significant improvements.

The Lean Agile Sandwich

Adopting agile is a hard exercise, especially in larger organizations. All to often agile adoption initiatives come to a stop after a while and things return to the way they were before but now with some new terminology and ceremonies like a daily standup.
There are two causes that I encounter at my customers when this happens.

In the first one there has been this bottom up movement driven by XP practices and Scrum. Teams are working in iterations, are having fun and making progress, but the business does not really see any benefits or even care. “It’s their problem how they do things, this is what I need you figure it out how to deliver it”. In these situations you typically see that requirements are still written upfront but then translated into user stories by the teams, business does not accept or reject anything at the end of a sprint, prioritized backlog is development’s problem and definition of done…. not really. You also see that teams are broken up regularly, new work and change of requirements during the sprint is common practice, other departments e.g. the test department are not really participating and management keeps commanding and controlling the teams. Ever seen a situation like this?

Then there is the second scenario. Management decided to do Scrum or Lean. A book or two is bought, people are send to Scrum master trainings, …. and that’s it! What you encounter is a situation where management behaves just as before “Lean is just a new name to things we already know from classical management theorie”, scrum terminology is used, the daily standup is done and maybe there is even an estimated product backlog. But sprint after sprint no working software, teams spent lots of time on architecture, design and philosophical discussions, no automated testing, mini waterfalls in a sprint, no common understanding of quality code and maybe most importantly no team self organization and commitment. I have even seen a team have a scrum board with stickies but during the whole sprint nobody even looked at the board let alone move a sticky!

I concluded that there must be something fundamentally wrong with the common agile adoption initiatives. When I started guiding teams about 8 years ago it was good enough to ‘just start somewhere’. The idea was and for many still is ‘just start and see where it leads’. I think that we can do much better then that. This naive approach reminds me of a quote of Yogi Berra “If you don’t know where you are going, you will wind up somewhere else.”

The approach I found successful over time I is a approach where you use Agile bottom up and Lean top down. I call this the Lean Agile Sandwhich. Agile bottom up because Scrum, XP, self organizing, trust and openness appeals to many developers. Lean top down because management easily relates to things like waste removal, making full use of people’s potential, continues improvement and optimize the whole. This obviously does not mean that Lean is understood, but it helps get a very good start!

The approach I use differs from situation to situation but the main things I always plan to do are the following

Lean top down
0. Lean awareness workshops with management to discuss the change from command and control to supporting and coaching of people. These are multiple day workshops with home work and exercises to do in their own working environment.
1. Establish a change team, of people from management, that support and promote Lean over departments and solve impediments that are out of the scope of the individual teams. Ideally some people that participated in the Lean awareness workshops become part of this team.
2. Define what KPI’s you need to measure progress on adoption. Let management answer the question “when are you satisfied?”.
3. Establish Product Owner teams consisting of people from the business with support from IT people. This team prioritizes and streamlines and “evens out” work going into their development teams.
4. Make retrospectives or “Kaizen events” part of the ritual of management.
5. Provide hands-on coaching for product owners and management on leadership and lean management.

Agile Bottom up
0. Realize optimizations teams on topics like agile testing, Scrum, agile architecture and agile requirements consisting of people from IT to promote and improve their specific craft across the departments.
1. Establish a common understanding of software quality and agile development practices through leading workshops, programming kata’s, trainings and discussion.
2. Agree upon a minimal definition of done that Scrum teams can extend as they wish. (A Scrum team being the product owner, Scrum master and the team)
3. Define measures that give management insights into the capacity and performance of the IT department as a whole and performance of the individual teams. Things I often propose are, tested features delivered, WIP, code quality and variation in velocity.
4. Provide hands-on coaching for scrum masters on team building and Scrum.

A key point in all this is that you start with what is already there. No big changes at once, encourage change only when things become an obvious waste and people understand it themselves. Let people ‘remember’ what’s best to do.

In order to see how things are going I use the following ‘indicators of progress’ in addition to e.g. the Scrumbutt test.
1. Is management having productive retrospectives regularly on impediments raised by the teams?
2. Does management focus on flow instead of utilization?
3. Does management use visual management tools like a board?
5. Is management working on interactions between departments and people instead of the departments and people themselves?
6. Are there KPI’s that force cooperation among department managers to optimize globally?

If I can answer all the above questions with YES then this gives me a good indication that lasting change is taking place.

In following blog posts I will discuss more on how I do these things.

The essence of Lean development

I was hired once to help out in adopting Lean thinking. Before I started I had an interview and talked to a top level manager on Lean and how I could help the organization in their journey.
In my naivety I actually assumed that we had a similar understanding about Lean. After a few weeks I heard the man say something that rattled my brain.

”….. yes yes we are going to be Lean and Mean”…..

I thought… hmmm what does Mean has to do with Lean!!
I realized that we could have completely different definition of Lean….. unfortunately we had!

The Lean paradigm differs form Mass paradigm in interesting ways.
Lets look at some examples

Efficiency v.s. Flow

“Yes… we are going to make a year plan and allocate our engineers in such a way that all of them are utilized for 100% in all our projects. Yes…. we are going to make them work in multiple projects at the same time, Yes…we need to start all these projects otherwise they do not get done Yes…”

In mass paradigm management people are seen as resources. And these resources need to be utilized to the maximum because obviously this results in higher efficiency. So people need to be kept busy all the time operating under full load! The best way ofcourse to achieve this is to have division of work or specialization. Yes this is good…. now the testers can test all the time and the developers can develop all the time and the requirements engineer can analyse requirements all the time… very good everybody is fully utilized, very efficient indeed!! The ‘production line’ has to be busy at all times because production costs drop with volume.
But….. the problem here is that thinking in terms of resource utilization makes it more difficult to keep the true purpose of the business in mind! How does resource utilization contribute to customer satisfaction? What has resource utilization to do with the purpose of the business? how does resource utilization provide value?
You know you are working in this management style when there never seems to be time to stop the ‘line’ and fix the root cause of a problem. But there is always seems to be enough time to do things over and over again with lots of ‘workarounds’.

In Lean we do not focus on resource utilization or ‘ass to chair time’. We focus on optimizing the total time it takes for work to get done. This often means that people are not fully utilized! And this is good because in development people need to think! And if we satisfy our customer with short lead times they will be happy and that’s actually good also!! We are also happy because we get things done earlier giving a higher ROI. We then also have room to absorb variation in demand, again keeping the lead times low. In full utilization there is no room for variation in demand and the slightest variation in demand results in huge queueing of work which leads to high increase of lead time and lots of WASTE!!!!!!

People v.s. System

In Lean thinking you try to have a systemic view on things. You assume that the root cause of most ‘errors’ or sub-optimizations are in the system people work in and not the fault of the people themselves. When things go wrong you ask youself “how can we change the system in such a way that this situation cannot occur again”. A manager’s purpose is to coach and help the people learn how they can improve the way their work is done.
A senior Toyota manager once said the following

A manager knows that improving the company is not his job but the job of the people themselves. The managers’ job is to help the people understand this and make it possible for them to do so.

In Lean development the teams have authority to decide how they do their work. Teams know the deadlines, organize their work, choose who is their leader and even who is in or out of the team. This is needed because they are the ones who know best how to improve the way they work. Management accepts the decisions made by the team even if they are not happy with it.

Traditional management has a different view. They assume that the root cause of most ‘errors’ are in the people doing the work. When things go wrong they tell the people to change their behavior. You ask yourself “How can we change people in such a way that this ‘error’ cannot occur again”. Hand in hand with changing the people comes commanding how to do things and controlling what is done. It often means taking away even more responsibility and authority from the teams. “Yes we are management…. we know what is best… we know how to do things…. ofcourse that is why we are management right!”

Inflexibility v.s. Continues improvement

Standards are needed and used in both the mass as the Lean paradigm. But there is a difference.
In the Lean paradigm they are the basis for improvement. Without a standard you cannot improve. The people themselves make the standard and improve it as they learn. We acknolwedge that there are multiple ways of working better.

Taiichi Ohno once said

“There is something called standard work, but standards should be changed constantly… “

In the mass paradigm a standard is something to remain stable. It should not change because we need volume production in a standardized way. The standards can only be changed and improved by ‘specialists’ or managers. The people are only modestly involved in planning and improvements. Information is disclosed on a ‘need to know’ basis.

Some quotes I heard in a mass paradigm management companies

“We need to verify everything they do upfront, because otherwise they will build it wrong”

“We must specify as much as possible so that they do not need to think”

“You have to specify component design in detail so that they do not build the wrong thing”

“Yes they are smart people, that’s why they build their own things in their own way”

The funny part is that people often do not realize that they think like this. The way of thinking is so common that it is assumed obvisouly correct.

There are many more aspects in which the Mass and Lean paradigm differ but that is something for a future post.

The Agile V-Model?

The V Model is a well known representation of a system development life-cycle.

It’s kind of linear moving from requirements analysis, architecture, design through coding and all kinds of testing until final deployment.
It’s goal is to minimize risk, improve quality and improve communication and understanding among all stakeholders.

One good thing about it is that it proposes a verification step after each development step. In the Requirements Analysis phase for example a requirements document is written that is verified by the users and that serves as input for writing acceptance tests. The Requirement document also serves as input for the next phase of design. After Requirements Analysis the System Design starts. System design is verified by system tests that are written during system test design and then the next step etc.

One problem with the V-Model is that it suggests a linear approach with limited feedback cycles. Feedback is based on documentation and assessment on documentation but not on real working software! Risk is not handled that well because final testing is done very late in the development cycle where any unpleasant surprises, and there are usually quite a lot, can put a project in deep trouble.

In Agile we would say that the requirements are most (e.g. usability being excluded) of the acceptance tests. See Robert Martin’s paper Tests and Requirements; Requirements and Tests on this.
Furthermore we would do acceptance testing all the way during the project and let the acceptance tests drive our architecture and system design. Yes acceptance tests also cover performance, scalability and so on. Because we do acceptance testing all the time there is no need for a separate system testing or integration testing phase. System and integration tests are just part of acceptance testing!

In designing modules for realizing a particular acceptance test we would write unit tests. These unit tests would drive our module design and our coding. So, no need for Module Design as a separate phase anymore.

If we take out these phases of system design, module design, system testing and integration testing we end up with a model like

In this model we can see that requirements are expressed as acceptance tests. The acceptance tests drive architecture design through unit tests and coding. The iterations of unit testing, coding and architecture design is executed until the acceptance test completes. On the resulting system an acceptance test is performed by the customer which results in further insights of requirements.

Command & Control is Waste

People generally do not like to be uncertain and uncertainty can make people nervous.
When you hire a painter to paint your house you want him to tell you how much it is going to cost and when the job will be done, right?… or would you not?

I worked with lots of people who say “I do not have faith in this, there are to much uncertainties, why don’t we try to tackle these uncertainties upfront!”

They have questions like:
How do you control that the right thing will be build?, What is the detailed plan for the coming years?, What is the complete architecture?, How do you control this, how do you control that, etc.”

These questions have partly to do with planning for the future and partly to do with command and control!
People want to be as certain as possible before and during projects. They want to minimize risk! People feel safer when e.g. they have detailed analysis and requirements, do lots of preplanning and planning, and of course write it all down in documents.

“Now that we’ve written it down people know what to do and we have much better control…”

Doing all this upfront work increases the feeling of reduced risk and better control right, it gives people the illusion of control!
But if you do all of that what happens is that:

1. You loose a lot of time doing all this work! It can take months and months. You could have used this time to make real progress on your project.
2. You are using only a fraction of the knowledge and talent of the development staff. They are more or less treated like construction workers, “This is the plan, now you follow it exactly”.
3. The commitment, support and understanding of the people doing the actual work is poor to say the least.

So how can we deal with this uncertainty? What does Lean and Agile tell us about this?

Of course you should do things upfront to reduce risk, establish vision and a plan. But it should not be driven by realizing command and control.

For one thing, all this command and control is waste! Why check when you can prevent! An organization should not become good at command and control, NO it should completely eliminate the need for any command or control!

Organizations think they need command and control because they assume things like

• “Our workers cannot oversee how to best do their job”
• “Our workers do as little as possible”
• “Our workers do not care about quality”

Even is this is true in your organization, as sad is that would be, the solution for this is to look at the system the people work in. In Lean thinking we believe that most problems and errors are systemic in nature. Meaning that the output the people produce is a consequence of the the system they work in and are a part of.

So, if you want to eliminate command and control and let people use their talents and passion as much as possible in the good of the company what about starting with the following

• Creating a clear purpose and direction.
• Letting people closest to the work decide how to do their work and how to measure progress towards the goal.
• Expect from people to try, fail and learn.

If your best people are on the job, they share a clear purpose and direction, can decide how to organize their work, measure progress towards the common goal, have the safety to learn and as a result have intrinsic motivation to make it a success then…. what do you need to control?

Using change teams to adopt Agile

Ever helped a team in practicing agile only to see that after you left the team fell back into old habits? During your presence the team did good and team spirit was high. After you left many of the things you helped set up were abandoned. Why is that?
Unfortunately this happens quite often and it kept me wondering. What is it that you can do to leave a team that’s strong enough to keep doing the things they believe are right and not fallback into old habits?

Create optimization teams
The most important thing you should do is create optimization teams. Find the people in the organization that share a passion for a particular topic like e.g. Scrum mastery, agile testing, product ownership, TDD or simple design to name a few. Get those people together into teams and come up with short term and long term goals and guess what…. a backlog of work that needs to be done to achieve the goals. The ultimate goal of an optimization team is to ‘optimize the whole’ that is their project, department or company on their topic of interest.
Facilitate that these teams come together on a regular basis to work to transform the backlog into change. Use the meetings to share ideas and experiences from the projects they work on. Train, consult and motivate them to become highly skilled in their topic of interest so that they can go into their projects as evangelists and hands-on mentors of their project members or colleagues.

Help them understand the problem and the solution
One of the things that goes wrong is helping teams solve their problems without helping them understand the root cause of the problem first. In the role of a coach it is relatively easier to convince people and therefore also very tempting to jump in and convince teams to use this or that solution. If intelligent people do not fully understand why they are doing things and why it makes their life better then “why not stop doing them”.

Laying Agile foundations
While coaches are there working with the teams they take a lot of the ‘heat’ from the organization around the teams. After all the teams are changing the way they operate and this gives friction with the parts of the organization that are still working as usual. After a coach leaves, the team has to face the organization friction themselves and this is often to much to handle for them. Before you go you must satisfy the ‘pre-conditions to Agile’ to lay down the Agile foundations.
A very useful guideline for doing this is J.O.Coplien’s book Organizational Patterns of Agile Software Development.