Scrum Master in the Critical path of the sprint

More then a year ago myself and Eelco Gravendeel wrote the top 9 challenges of adopting Scrum . But those where just the top 9. Some did not make it to the top 9. This post publishes one that did not make it but I think is quite good. here it goes….

 

 

 

“If everything seems under control,
you’re just not going fast enough.”
Mario Andretti

The situation
You’ve got it all setup: a multidisciplinary team, a Product Owner and a dedicated Scrum Master. Deadlines are tight but seem to be do-able, scope appears to be negotiable, and the rest of the organization is well informed and prepared about what is happening. We’re good to go!
However; during the project deadlines are not met, quality doesn’t appear to be as good as expected and overall commitment of team members is not what you would like to see. And although the Scrum Master is pitching in and doing valuable project tasks to help out, things just don’t seem to improve very much per sprint passed.

The problem
The catch here is that the Scrum master is doing “valuable” project tasks. He or she is trying to do work which other team members might depend on to do their tasks. It can even go so far that the Scrum Master gets the feeling that they can’t get the Sprint done without him/her pitching in and picking up those all important / difficult tasks.
What happens next can be a combination of problems, some of these mentioned below:

 Team will not self-organize to meet deadlines
 Team does not feel committed to quality
 Scrum Master does not have (take) enough time to resolve impediments
 Scrum Master does not have (take) enough time to coach Team Members
 Scrum Master does not have (take) enough time to coach Product Owner
 Scrum Master pulls all responsibility towards him-/herself

As a result of such problems the Scrum Master then often feels the Team is letting him/her down and falls into a command & control style of managing (not leading!) the team. That way you miss out on much of the benefits self-organizing teams in general and Scrum specifically can provide.

How to fix this
It is a bit too simplistic to say the Scrum Master can never pick up any development or testing tasks. As always it depends. Here’s what worked for us:

When the project consists of one Team, one Product Owner and One Scrum Master you are best of if the Scrum Master doesn’t do any development, testing or analysis at all. Almost all time will have to be spent facilitating the team by removing impediments, coaching, etc. Some time will have to be spent on reporting about the project to management or other stakeholders. The rest of the time will go into helping out the Product Owner in managing the Product Backlog, managing Stakeholders, etc. Any time spent on doing “in Sprint” work such as development will not be spent on tasks such as mentioned above.

When the project consists of multiple teams or just has a Project Manager assigned to it, all becomes a bit different. The Scrum Master is of course still very much responsible for facilitating the team and removing impediments, but can leave the more “team outward facing” things to the Project Manager. Tasks such as Stakeholder Management, Risk Management, Product Owner coaching or removing impediments on an organizational level can be left to the Project Manager. So, if the Scrum Master has any time left he might be able to help out the team with some development tasks, or testing or whatever. But: The Scrum Master tasks always come first! The team should prevent the Scrum Master from picking up any tasks which might be in the critical path of the sprint, you never know whether some pressing impediment comes along and will require the Scrum Masters full attention!

Coaching the Good the Bad and the Ugly

Ever been in a project with this special guy that ‘helps’ the project succeed? This guy, ‘the Good’, that has the guts to stand up to management and tell them how things really are, bring discussions to a good end or has this unique technical skill?
On the other hand, have you ever worked with this ‘problem guy’, someone who acted negative, had little trust in others, was passive in his actions, did not take responsibility or any real interest in what was going on besides his own specific little piece of work or even was very cautious to show to others what he was really doing?… Let’s call this guy ‘the Bad’.

Imagine yourself being in a team with ‘the Good’ and ‘the Bad’ and frictions arise between the team members because things are not going as expected.
All to often what happens is that Mr Manager, lets call him ‘the Ugly’, who manages from behind his desk, starts talking to ‘the Good’ and encourages him to keep up the good work. “You are doing great, you are really helping the project move in the right direction”. The Good guy explains what he thinks is going on and complains a bit about the way the Bad guy behaves when it comes down to team work. “It’s very hard working with this guy, it is really hurting the team, he only sees problems and does not help at all when it comes down to team work”. The Bad and the Ugly quickly sit down together and come up with the solution to the problem.
The Bad and the Ugly decide that someone should speak to the Bad and tell him to change his behavior.

So, the Ugly goes talk to the Bad. “So, what is going on here…? why are you acting in this non constructive way?… It’s time for you to change, your lack of team spirit is causing problems in the project… I want you to change your behavior!, I want you to listen to the Good and work with him”.

This kind of Management behavior is often not the best approach from a long term perspective.
First of all the Bad is by-passed by talking only to the Good. This actually enforces the behavior of the Bad because he feels not involved and feels that anything he has to say does not matter anyways. Secondly the Ugly and the Good are jumping to conclusions and coming up with a counter measure without really grasping the situation and looking for root causes. It can be an solution but probably not the solution for the root cause of the problem. Next to that, he will not really support the countermeasure and everything is setup to stay the same again.
The third and biggest issue in this approach is that there is minimal learning involved and no real progress as the root cause is probably not tackled. The Good and the Bad do not learn much from this, so that in the future they could handle these situation better or prevent them from happening in the first place.

A different type of Manager realizes that real progress is to be found in coaching the team to learn how to deal with these kind of challenges. He realizes that most often it’s not the people who are causing the problems, but the systems in which they must work.
This Lean manager would talk to the team as a whole and make the team the owner of the issue in the long run. He would dig into the situation and see first hand what is going on at the Gemba to really understand the background and the problems. He would discuss with the team what would be appropriate goals to achieve so they could identify the gap between the current state and desired future state. Do the necessary analysis to find the root cause(s) of the issues. Guide the team to explore opportunities and propose a number of countermeasures to get team alignment and support.
He would stimulate team members to stand up and become the owner of improvement actions and ‘pull authority’ to take the team forward in becoming a constantly improving team that works thoroughly using the PDCA cycle.

WTF is this fuzz about User Stories, Use Cases and Test Cases

Over the last few years I’ve had lots of discussions with teams on how to cope with this thing we call Agile Testing. The same topics arise over and over again, with the same kind of discussions. The main question is ‘How do we handle all these Use Cases, User Stories, Test Cases and Requirement Documents in this Agile Journey’
‘First we had software requirement specification documents, from these documents we derived our test cases…..’. We worked with the RUP, used CMMI or even worse the freighting DOD or MIL standards! Now you try to ‘help’ us in using user stories, but what about the use cases and the test cases? how do they relate? what about my Master Test Plan, Feature List, Test Specification and the like? Are they not needed anymore?

First of all, an approach that worked for me several times is to start with what is already used and only propose to remove or change things when it becomes clear that things are not needed anymore. So if e.g. people are using an Master Test Plan, let them use it until it becomes clear that the way they’ve been using it does not add the value it should add because of the iterative way of working. The same holds for using user stories. If they are used to using use cases let them use it next to the user stories. If you can help the team make a successful transition they will figure out how to cope with the use cases.

User Stories work perfectly for Agile planning and estimating. Next to that, an very important aspect is that user stories force the teams to communicate with the requirements people, be it the product owner, customer or domain specialist. In the case of use cases, what you see is that people incorrectly assume that they can write everything in the use case and that face-to-face communication is not that important because the requirements are written down so ‘carefully’ and ‘precise’…..
So, if you have a mixed team with testers, requirements people and developers, why do you still need to write this pile of documentation? Why not…… just talk more to each other?
It could be that the organization needs formal requirements documents to please some institution or manager, or even better a very ‘good’ reason I heard today, ‘I want to give the test document to someone else to do the testing for me’ or ‘when we fix a bug, we need to use the test case to verify everything else still works’.

Well there is good news, with the user stories there comes acceptance criteria and from these acceptance criteria you should derive test cases to further detail the user stories. So user stories get detailed with test cases, preferably automated ones, and these can be documented in any form you want. You can even group a set of related user stories into an use case if you wish and use an use case as a container for user stories. The test cases fill up the test documents as the sprints progress and at the end of a release there you have your required test documentation….. ….. or is it not required anymore?

One Metric to rule them all

There are numerous things you can measure when it comes down to software projects. I usually see that, if things are measured, the things that are measured do not result in the expected behavior. I’ve seen KPI’s on number of bugs fixed, features solved within given estimation, documents delivered and even hours worked. Eliyahu M. Goldratt wrote in his book The Goal “Tell me how [and when] you’ll measure me, and I’ll tell you how I’ll behave”. People normally try score as high as possible on the specific metric that is used to measure them. So if you are not measuring the real problem, you’ll probably just end up with high scores on your metric but without any improvement of your real problem.

In case you are trying to improve some aspect of your software development process you need to define a KPI. The KPI should express a measurement on the root cause of the problem. Lets say e.g. that you want to improve your estimations for delivering features so that the SW department gets better info for making planning decisions. You decide to measure how many features are realized within the estimated time. Lets say that as a result of this KPI you’ll see an increase in issues that are solved within the given estimation!! Good……., problems being solved right?

but….. did the estimations really improve? or did the estimators just added more padding to be sure their estimations were ‘correct’? Taking a step back… were you unhappy with the estimations or with the time it takes to deliver features? Why why why does it take so much time to deliver a feature? What is the root cause?

If you are not that happy with the time it takes to realize a feature than you could measure just that! From Lean we know that the time it takes from the moment a feature is known until it is actually finished is called Lead Time. It is the average time it takes for one unit of work to go from start to finish through the process, including delays between sub-processes. We also know that in a stable system we have that Lead Time = CycleTime * Work In Progress + Delay Between Sub-Processes, where Work In Progress is the number of features currently in the system and Cycle Time is the average time it takes for a feature to be realized. Notice that the Work In Process strongly increases lead time. Also note that Delays Between Processes directly contributes to Lead Time.

By measuring the Lead Time, Cycle Time, Work In Progress and Delays Between Sub-Processes we are measuring the things that really matter. Next to that they give us a clear indication for potential wastes in the process so you can get to the root cause of why it takes so long to realize a feature! So once you have these metrics setup for your department you can start identifying the wastes and eliminating them by defining KPI’s on them. But remember the one metric that rules them all is the Lead Time!

the average time it takes for one unit to go through the entire process – from start to finish – including time waiting between sub-processes.

How you could deal with early estimations

Recently people have been asking me how to deal with so-called ‘expert estimations’ in Agile development. In most organisations where IT is seen as a supplier or cost centre, the business wants to know the costs and the duration of a project before it decides to fund it. This is of course perfectly normal!

The poor IT department is asked to do an ‘expert estimation’ based on the available vision, requirements and any other available information. The estimation is expected to be quite accurate with e.g. a max -10% to +10% deviation. So the IT department adds the necessary buffers and slack to the estimation. As a result the project plan is probably longer then necessary which results in longer projects and less productivity thanks to Parkinson’s Law. The project will also unnecessary claim personnel and other scarce resources for longer periods complicating their use by other projects. Task switching and hand-offs now become a necessity as people must work on multiple projects. It often also results in a project to not start at all thanks to the overestimation!

 So, how could we do this in Agile?

 Well…, what some people do not seem to realize is that in Agile we also provide estimates upfront. In Scrum e.g. we have the estimation meetings where the product backlog is estimated. We estimate just enough to be able to make the ‘right’ decision. We acknowledge that in the beginning of a project, when we know least about it, the estimates could be very inaccurate. We keep on estimating throughout the project increasing the accuracy of our estimations as we learn more about the project. We measure our progress based on actual working features and value added. This enables the business to get valuable estimation and planning info as early as possible to make the ‘right’ decision, which could be to stop the project! We use a project baseline that is based on actual data and progress. If we have 1000 points of estimated work, we work with 2 week iterations and we must be complete in 20 weeks then we must burn 100 points every iteration. This becomes our baseline and is used to measure our schedule and budget variance. We track features completed vs. features remaining, work in progress, feature cycle time and client value added. In Agile we do a lot more planning and we do it throughout the project.

We use the initial estimation to make the funding decision! We then have our estimated schedule, budget and functionality. After the decision to fund the project is made, we use the mentioned metrics and techniques to steer the project and track against the initial schedule and cost estimation. We re-estimate and re-plan based on real progress and provide the business with the most accurate information as quickly as possible.

The end of the ‘Internet’ testers

In the days of waterfall project the testers did not have anything to do during the first half of a project. During this period they would spent most of their time surfing the net and rewriting their test plans. Nowadays with cross functional teams the testers play a significant role right from the beginning of the project. 

The role of a tester is not just testing anymore. An agile tester helps the team figure out what needs to be delivered at the end of a sprint. He plays an important role in clarifying the user stories and establishing a common understanding on the requirements within the team. During sprint planning the tester helps to clarify the user stories by also paying attention to things that could go wrong (Developers often have the tendency to see only the happy flow). Next to that testers work on requirement examples prior to sprint planning together with the requirements people to start the feedback early on. During the sprint the testers continuously provide feedback to the rest of the team to ensure the right thing is build correctly the first time.

When the team finishes a story during the sprint, a tester can have a feedback session with the product owner by demonstrating the user story and showing that the acceptance criteria are met. This is specially valuable in case the product owner is discovering what it really needs! In that case the acceptance criteria or the accompanying specifications are vague and the team helps the product owner figure out what he wants. Based on what he sees he gets a better understanding on what he really needs. It’s like when you want to buy a table. You know what a table looks like, but you can’t exactly specify the table you want. That’s why you go visit stores to see various tables. One table you see helps you to decide on the table size, while another helps you decide on the color and yet another helps you see the style and material you like. By having seen these examples you steadily get a clearer and more accurate picture of what it is you really want.

So for the testers there is no more laying back during the early stages of the project, but instead being part of the team right away and working hard to help the team achieve its sprint goal.

Roles in Java using ObjectTeams

I’ve been testing ObjectTeams for a while now and think its a really cool way for using roles in the Java language. A role is a set of attributes and behavior a object can assume temporarily in a certain context. It’s like an actor behaving according to the film script. The actor plays the role as described in the script as long as he is shooting the film. After that he returns to be himself again.

Roles can help you in achieving a strict separation of concerns. This is because roles let you encapsulate context specific behavior in a role removing it from the class itself. In a collaboration you can have multiple roles working together to achieve some functionality. An example of a collaboration could be a car race. The Race collaboration exposes the role of a RaceCar for cars that want to participate in the race. A car object that assumes the RaceCar role will get additional state and behavior defined in the role RaceCar. These could be things like laps to go and car number. The Race collaboration itself can also have state like e.g. the participating race cars and the laps to go untill the race ends.

ObjectTeams is an extension to the Java language. It introduces some new language constructs that make it possible to program using roles. A collaboration is represented by a team and declared using the team keyword. Within the team you can define roles and their interaction. You can express that objects of a certain class will play a certain role using the playedBy keyword. As a result objects can be extend at run-tme with role specific behavior and state.

See below an implemtation of the SubjectObserver pattern in ObjectTeams.

public abstract team class SubjectObserver {
    …
    protected abstract class Subject {
        private LinkedList<Observer> observers = …
        public void attachObserver (Observer o) {…}
        ...
        public void notify() {
            for (Observer observer : observers)
                observer.update(this);
        }
}

    protected abstract class Observer {
        abstract void update(Subject s);
}
}

You see that we have a abstract team SubjectObserver that defines the roles of Subject and Observer. Lets assume that we have a TV and a Screen that needs to be updated when the TV changes state. We have to let TV play the role of Subject and Screen play the role of Observer.

public team class ObservingGraphical extends SubjectObserver {
    protected class Subject playedBy TV {
           notify <- after setPrice;
      }

    protected class ScreenObserver extends Observer playedBy Screen {
        update -> redraw;
}
…
}

We see that team ObservingGraphical extends SubjectObserver. Because it uses the same role name for Subject as its base, this role is implicitly extended using the same name. You can also see the use of playedBy keyword.

To couple the objects and roles the so called callin and callout statements are used. Using callin statement ‘notify <- after setPrice; ‘ you state that after method ’setPrice’ is called on TV, method ‘notify’ of Subject should be called. Using callout statement ‘update -> redraw;’ a call of the ‘update’ method on Observer is mapped to method ‘redraw’ on Screen.

How to use the collaboration and its roles is shown below

final ObservingGraphical observingGrph = new ObservingGraphical();
TV tv = new TV();
Screen screen = new Screen();
within(observingGrph) {
observingGrph.attachSubjectObserver(tv, screen);
tv.setPrice(1000);
}

The interesting part is the within statement. It defines that the role specific behavior is active. During the statement the objects TV and Screen are extended with role specific behavior. After the within statement they are back to normal again.

References

[1] E. Gamma et al (1994). Design Patterns: Elements of Reusable Object-Oriented Software.
[2] Edsger W.Dijkstra (1974).  HYPERLINK “http://www.cs.utexas.edu/users/EWD/transcriptions/EWD04xx/EWD447.html” http://www.cs.utexas.edu/users/EWD/transcriptions/EWD04xx/EWD447.html.
[3] Robert C. Martin (2002). Agile Software Development: Principles, Patterns, and Practices.
[4] ObjectTeams.  HYPERLINK “http://www.objectteams.org” http://www.objectteams.org/
[5] ObjectTeams language definition.  HYPERLINK “http://www.objectteams.org/def/1.2/index.html” http://www.objectteams.org/def/1.2/index.html.
[6] Bertrand Meyer (1997). Object-Oriented Software Construction, Second Edition.
[7] Matteo Baldoni et el. The Interplay between Relationships, Roles and Objects.
[8] Trygve Reenskaug, Role Modeling.  HYPERLINK “http://folk.uio.no/trygver/themes/roles/roles-index.html” http://folk.uio.no/trygver/themes/roles/roles-index.html.

Prioritizing the backlog on Quality

After my talk at JFall about EasyB and Agile requirements I had a discussion with a very respectful guy about some things I said during my talk. The discussion was about whether building up quality over time is a good or a bad thing!

In my talk I stated that some features like for example an engine, brakes and a steering wheel are all MUST if you are building a car.  So it’s silly to prioritize them. As far as I know, you just don’t have a car if you don’t have these features. We all know that in order to have the most business value when the budget runs out or the deadline is there, you should prioritize your backlog! So in this case, what are you going to prioritize…..?

Well, it’s real obvious, you can prioritize on quality. I am not talking about software quality here, I am talking about the quality of a feature. If we define that horse power is a quality of an engine then an engine of 400 horse power, is of higher quality then an engine of 100 horse power. The difference in this quality is expressed as a difference in time and cost to develop! So, it’s better to build a engine of 100 horse power, and the rest of the MUST things at lower quality first so you have at least a car.
Once you have that you can start building up feature quality and see how much luxury you can add within the given time and budget. If you build your features up to the most quality one by one, chances are you’ll end up with a car having 400 horse power and super brakes but no steering wheel!

In my opinion we were arguing about the definition of quality. While I was talking about feature quality, I guess he was talking about the quality of the product you deliver at the end of a sprint! In almost every situation you want your sprint result to be of the highest possible quality ofcourse. That’s what Lean and Scrum tells us right!

My Talks

LightWeight Grids With Terracotta: At the NL JUG in November 2007

LightWeight Grids With Terracotta: At the PT JUG on March 2008

Top 9 Challenges of Adopting Scrum: At the PT ScrumAlliance User Group in Lisbon on September 2008

Challenges of Adopting Scrum: At the XP Days Benelux in November 2008

Easy Requirements By Example: At the NL JUG in November 2008

Scrum at Large Enterprise Case Study: At the NLSrum in March 2009

My Papers

Lightweight Grids with Terracotta in Java Magazine, 2007

Top 9 challenges of adopting Scrum in Agile Journal, 2008

Top challenges of adopting Scrum in JavaMagazine, 2008

Objects and Roles with Java in JavaMagazine. 2009

Top 9 challenges of adopting Scrum

Scrum is a simple process, but can be quite challenging to implement. In a series of articles on AgileJournal myself and my colleague Eelco Gravendeel will discuss the top 9 challenges we encountered during various Scrum adoptions.

You can read the first part of the series at  Top 9 challenges of adopting Scrum.

You can read the second part of the series at Top 9 challenages of adopting Scrum part 2.

The third and final part is at Top 9 challenges of adopting Scrum part 3.

You can also see a brief summary at InfoQ see challenges of adopting scrum.