CHAOS AND DISCIPLINE

On our current project, the team has just spent four days of work on a requirement, to finally throw the code over and start again. We got glued into some never ending refactoring which never gave satisfaction as the code never seemed testable. What a waste! We've been through the same experience already a couple of times - and hopefully we are learning from our mistakes.

We develop near chaos. This isn't a bad thing. We follow practices very close to eXtreme Programming and Scrum, so unclear requirements and emergent systems are our lot. So, why has the team suffered so much lately?

We lacked discipline. In fact, we let go just a little on two major and very constraining disciplines. The problem is that when you develop near chaos, you cannot afford to let go of these two disciplines.

By the way, some people really misunderstand agile development. They believe agility means lack of discipline and chaos. Isn't it just the contrary? To develop in a chaotic environment (such as any software development) you need to be so extremely disciplined! Isn't eXtreme Programming such an extremely disciplined methodology?

So, what are these two so important disciplines? Very frequent synchronisation and test-driven development.

On several projects, we've noticed that shortly after a pair or a single developper gets late to synchronise and deliver his code, he gets stuck in painful code and test merging. The system emerges so quickly that members of the team must keep the rhythm. Stock of undelivered code just slows you down and requires energy to catch-up. Stock means waste! Unfortunelty, it is very difficult to learn how to develop in baby steps. Developers often assert that their current task cannot be cut into small increments.

We've also noticed that developpers tend to slowly let go of test-driven-development. The problem is that they take several successive design decisions to finally discover that their code is difficult to test! Therefore, the design is inadequate and must be refactored and tested.

OK, developers gradually let go of frequent synchronisation and test-driven-development. How do you improve your habits and maintain discipline?

The solution we try is to spread good pratices and systemize checking.

• First, the team shares the same office. Therefore, everyone detects a developper or pair struggling on painful merges. Help and advice is naturally provided.
• The team attends a daily short and focused stand-up meeting. Lack of progress and painful struggling is detected. Again, help and advice is quickly provided.
• Pairing is very frequent. The teammate is there to remind the pair to work in baby steps and write tests as it writes code.
• We have a coach on the team. He's responsible for having the practices applied, training the teammembers and helping the team to continuously improve its practices. The coach detects pairs or single developers struggling in merges or falling late to deliver. He must provide advice and help. He visits the pairs and reminds them to synchronise.
  

In a world of chaos, you just can't let go on discipline! But discipline must not be imposed on the team. The team must reflect on its struggles, understand the cause of their pains and agree on trying disciplined practices to provide scaffolding. This is another subject. For more information on continuous improvement, read this post.

TESTABILITY, NO-DUPPLICATION, CLARITY AND SIMPLICITY

When we take design and coding decisions, we most value

1. testability, then
2. no-dupplication, then
   
3. clarity, then
4. simplicity.

Applying these principles, we noticed we "grow" change-tolerant code cut into small uncoupled and cohesive classes.

INDICATOR-DRIVEN DEVELOPMENT

Every morning, the first developper to come in to work starts his day with a coffee. Then he checks the build which was automatically performed during the night.
From the generated build status web page, he collects the indicators measured during the night and writes them on our big indicator chart at today's date.
The big indicator chart shows the indicators measured every night by the automatic build.
At 9h15, the team gathers for the short daily stand-up meeting. Before running the work status, we comment the fresh indicators and detect those which have regressed and decide immediate corrective action.
Among the indicators, we track

• the percentage of code covered by tests to check it remains high;
• the number of functional requirements tested to measure progress and calculate our velocity,
• the number of lines of code in our components to check it doesn't grow awkwardly,
• that the requirement to test and requirement to code traceability matrices contain no errors or empty slots.

As these fresh indicators have an impact on the day's work, we can assert that our developpment is also indicator-driven.

ITERATION RETROSPECTIVE

Since two iterations, the team now performs iteration retrospectives.
At the end of each iteration, the team has a short and focused meeting to reflect on the past sprint (like a Scrum sprint retrospective).
The objective is to consign and track actions to establish a lightweight continuous improvement process.
The retrospectives are consigned in the minutes of meetings page of our wiki.

PROJECT LOGBOOK

The important events of the project and the decisions with their justifications are logged into a logbook, last date up.

This helps to integrate new developpers into the project, to find explanations concerning old decisions and to provide proofs for audits.

The logbook is edited in a wiki page to keep it the practice lightweight, share it among the team, keep it alive and enable easy searchs.

SHORT DAILY TEAM MEETINGS

We are again practicing short daily focused team meetings.
We have them in the team office, in front of the big workflow kanban chart.
It enables us to feel the rythm of daily work and avoid those tedious and boring long weekly meetings.

FAILURE AND CHANGE NOTIFICATIONS

We've just configured our continuous integration tool to notify all team members by email of failures in the build, and changes commited into the code base.

The failure notifications help to implement the stop-the-line culture. A failure may be a failure to build the application, a failed test case or an assertion raised in the code by a test.

The change notifications help the team members to stay informed of the changes commited by the team. This helps to share the code.

DESIGN BY CONTRACT

According to Bertrand MEYER's design by contract practice (implemented into Eiffel), we specify class responsibilities in terms of pre-conditions and post-conditions.

These conditions are evaluated in assertions.

Class invariants are also checked by assertions.

Combined with a test-driven practice, this is a great way to implement the stop-the-line culture. Furthermore, it a great way to build mistake-proof code, as a class instance may not be used without respecting its operations pre-conditions.

SHARED DESIGN

Design decisions are always taken as a team.

At least two developpers gather in front of a whiteboard and start working on a short term design.
At least one of the designer is then assigned the design to code and test.
If the design-decision is important, it is logged and justified into the design description document.

INFORMATIVE WORKSPACE

Whiteboards and large post-its show the design the team is currently working on.

Prioritized features and issues are described on post-its. They are moved along the sequential sections of a large Kanban board.

The sections are named TO-DO, ASSIGNED-TO, TO-REVIEW, TO-DELIVER and DONE.

The Kanban reprensents the team's workflow.

COMMON WORKSPACE

The software developpers share a common workspace to enhance informal communication, pairing and team-spirit.

HIGH QUALITY

Simplicity, clarity, no-duplication and design-patterns are enforced during pair-programming sessions, peer-reviews and continuous refactoring.

CONTINUOUS INTEGRATION

Developpers synchronize and deliver their work at least daily.

A continuous integration system automatically builds the application upon detection of modifications.

The build dashboard and the build artifacts may be freely consulted by managers and clients.

With this practice, integration is performed just-in-time.

ONE ACTION BUILD SCRIPTS

The applications are build, tested and measured in one-action builds.

The build

• produces the application,
• tests it,
  
• generates code to requirement traceability matrices,
  
• generates requirement to acceptance test matrices,
  
• measures code coverage by tests,
• measures feature (requirement) coverage by tests,
  
• generates detailed design documentation,
  
• generates UncleBob's depency metrics,
• generates quality metrics,
• generates to-do lists,
  
• profiles the code,
• etc ...

The build gives a boolean status : success/failure.

TEST-DRIVEN DEVELOPMENT

The requirements and the code are verified by tests.
These are organized into unit-tests and acceptance tests.
The unit-tests and as many acceptance tests as possible are automated and self-checking.
We give proof of full code coverage and full feature coverage by the tests.
The tests are used to measure progress.

REVIEWS

The main development artifacts (plans, requirements, design, code, tests, traceability matrixes, etc ...) are formally reviewed in process by peers.

REQUIREMENT BASED DEVELOPMENT

The features to develop are identified as high-level requirements.

The specific business algorithms to implement are identified as low-level requirements and traced to the high-level requirements which justify their existance.

The code is traced to the requirements.

The compliance of the application to its allocated requirements is verified by tests, formally traced to the requirements they check.

ISSUE TRACKING

Issue tracking is a practice very closely coupled to version control.
Again, for certification purposes, we clearly specify our issue tracking practices.

VERSION CONTROL

The company I work for doesn't joke with version control.

The software we build must be certified before it can be officially used and the independant auditors look seriously into our version control practices.

DEVELOPMENT METHODOLOGIES

The certification norm we must comply with requires us to clearly describe our development methodology.
The company I work for has a long legacy of waterfall processes.
However, things are changing and locally we manage to apply Agile processes with interative and incremental cycles.
Still, we must give proof of compliance with our applicable certification norm.