PRAGMATIC PROJECT AUTOMATION, Mike Clark

This book explains step by step how to automate builds, tests, continuous integration, release, installation and monitoring of software.
The examples are given for Java applications, using tools such as ant, JUnit and CruiseControl on a Unix platform.
However, the advice may (must?) be set into practice for other languages and using other tools.

EXTREME PROGRAMMING OFFICE

I've published my first article in [Pro]grammez, a french monthly magazine on software development.
It deals with the way our development team organizes its project office to optimize communication.

TEACH AS YOU DEVELOP

This short bill has been refactored by taking into account the feedback provided by the students of this year's session.

I find that a interesting way to teach software development to students is to teach as you develop: Plan, Do, Check, Act (thanks Mr.Demming).
Therefore, the course is led as a software development project.

PLAN!
Each lesson starts by a short and focused meeting. During this 15-minute project meeting, we

• recall the important matters teached in the former session, and
  
• identify what we commit to deal with in the current session.
  

Each feature to teach is then estimated to check it will fit in the day's lesson.

DO!
I never delay an answer to a student's question - even if it forces me to change and re-plan the sequence of the course. The course is driven by the students needs. It must be adaptative.

CHECK!
At the end of each lesson, the students must fill-in a short anonymous evaluation. They are asked provide the following feedback:

• What is really important in today's course?
• What did you not understand clearly (and did not dare to ask orally ;o)?
• Is there something we forgot to talk of?
  
• Do you have any remarks to improve the lesson's animation?

ACT!
With this feedback, I try to continuously improve my teaching and insist on the important matters the students have misundersood (because I was'nt clear enough).

CHECK! (again)
Then, at the end of the full course, I give the students a full anonymous evaluation.

• Rate the contents from 1 to 4.
• Rate the animation from 1 to 4.
• What did you prefer in this course?
• What did you not enjoy in this course?
  
• Do you have any suggestion to improve the course?
• Would you recommend this course to another student?
  

ACT! (again)
With this feedback I try to improve next year's session.

I believe this practice of teaching software development enables students to feel what a develoment project is like. Indeed:

• The customer (the students) are involved in the project (the lesson).

• Each lesson is an increment of knowledge.

• Each lesson is an iteration.

• Each iteration starts with a little planning, a little estimating and some short-term commitment.

• The planning is not predictive. It is adaptative as the contents of an increment (or lesson) may change according to the customers (the students) needs (questions and remarks).

• Each iteration ends with a retrospective to provide feedback to improve the next iteration.

I haven't yet managed to practice continuous testing, as the students would feel the pressure too hard ... one day maybe ;o)

WHICH DEVELOPER ARE YOU?

Of course, while the pragmatic programmer is enjoying a nice diner at 8PM, his automated build is running at work for him, building, testing, measuring the code, measuring the test coverage, generating documentation and providing fresh and complete feedback for the next morning.
Isn't it just great to know someone safe is working for you while you enjoy your time?

A PRAGMATIC DAY

This is a drawing I use to explain what an ordinary pragmatic day is for an agile developer.

TEACHING

I've just started to teach software engineering to a class of last-year engineering students.
As Chad Fowler puts it in "My Job Went To India":

When you teach, you have to answer questions that may never have occurred to you. Through teaching, we clean the dusty corners of our knowledge as they are exposed to us.

SHORT DAILY TEAM MEETINGS

We are again practicing short daily focused team meetings.
We have them in the team office, in front of the big workflow kanban chart.
It enables us to feel the rythm of daily work and avoid those tedious and boring long weekly meetings.

CONTINUOUS INTEGRATION

I use this drawing to explain how developpers contribute to the common code base (under version control) which is continuously monitored by a build machine, delivering feedback to developpers and managers on the state of the code base.

CONTINUOUS INTEGRATION = CONTINUOUS FEEDBACK

I use this drawing to explain that the build machine delivers continuous feedback on the state of the common code base. The feedback may be a website and e-mail notifications sent to developpers and managers.

CONTINUOUS INTEGRATION = STOP-THE-LINE

This drawings helps me explain how continuous integration providing continuous feedback helps to implement a stop-the-line culture in the development team.
When a change is detected in the common code base, an automated build is launched. If the build fails (compilation errors, test failures or whatever is important for you) the build machine sends failure notifications to the development team. Their job is to correct the problem NOW!

CONTINUOUS INTEGRATION

I use this drawing to explain that continuous integration is based on several practices to be most efficient.

CONTINUOUS INTEGRATION = LEAN INTEGRATION

I use this drawing to show that continuous integration is in fact based on Lean principles.

COMMON WORKSPACE

I use this drawing to explain what you gain by sharing a common workspace. In fact, this is just an illustration of my office ;o)

REVIEWS

I use this drawing to explain the efficiency of reviews and pairing.

INFORMATION RADIATIORS / SHORT MEETINGS

This drawing is meant to explain how we organize the teams' daily work using short daily meetings and a simple workflow kanban, moving around post-its.

ARCHITECT AS A GROUP

This drawing is meant to show how we have very short design group sessions.

MISTAKE-PROOF CODE

This drawing is meant to explain the practices we use to write mistake-proof code.

CHANGE TOLERANT CODE

This drawing is meant to explain the practices we us to write change-tolerant code.

BREAK DEPENDENCIES

This drawing is used to explain which practices we use to break dependencies in the code.

BREAK DEPENDENCIES BY ABSTRACTION

This drawing shows examples of how we use abstraction to break dependencies in real-time embedded software.

THE PRAGMATIC PROGRAMMER, by Andrew Hunt and David Thomas

This books exposes lots of great advice. It is a sort of catalog of tips and pragmatic practices.

I was really glad to read a book where the authors insist so much on Design By Contract, the interest of Crashing Early and the wise use of Assertions. Among all the books I have read, few deal with these effective practices.

The authors insist alot on the DRY principle: Don't Repeat Yourself and on the absolute necessity to reduce coupling and dependencies.
Great advice!

I RECOMMEND THIS BOOK.

FAVORITE PRATICES AND TIPS - SYNTHESIS

This is a synthesis of my favorite practices and tips, read in various books.

_CONTINUOUS_INTEGRATION_________________________________________

Early and at all times, the team ensures that the project is fully integrated, compilable, runnable, tested, ready to deliver and deploy.
All these instructions are automated by a one-action build script with two modes: full and incremental.

_TEST_DRIVEN_DEVELOPMENT________________________________________

"Tests are to prevent bugs, not find them."
The programmers work in very short cycles, adding a failing test, then making it work. When they get a bug report, they start by writing a unit tests that exposes the bug.
Writing tests first is a design tool. It will lead you to a more pragmatic, simpler and less coupled design.
Make sure all tests are fully automated, that they check their own results and are run on each supported platform and environment combination by a continuous integration tool.

_CUSTOMER_TESTS_________________________________________________

As part of selecting each desired features, the customers define automated acceptance tests to show that the feature is working.
These tests are run as part of the continuous integration build. They do the most with the least.

_SIMPLICITY_AND_CLARITY_________________________________________

Correctness, simplicity and clarity comes first.
The team keeps the design exactly suited for the current functionality of the system. It passes all the tests, contains no duplication, expresses everything the authors want expressed, and contains as little code as possible.
When you feel the need to write a comment, first try to refactor the code so that any comment becomes superfluous.

_CODE_REVIEWS___________________________________________________

Code reviews are invaluable in improving the quality of the code and keeping the error rate low. Review code after each task, using different developers.
Pairing is continuous reviews: All production software is built by two programmers, sitting side by side, at the same machine.
Moreover, code reviews are a great way to enhance collective code ownership.

_COLLECTIVE_CODE_OWNERSHIP______________________________________

Rotate developers across different modules and tasks in different areas of the system. Any programmer can improve any code at any time.

_REFACTORING_____________________________________________________

Fix bad designs, wrong decisions, and poor code when you see them.
When you find you have to add a feature to a program, and the program's code is not structured in a convenient way to add the feature, first refactor the program to make it easy to add the feature, then add the feature.

_STANDARDS_______________________________________________________

All the code in the system looks as if it was written by a single - very competent - individual.
Embody the current best known practices in standards that are always followed while actively encouraging evenryone to challenge and change the standards.

_TEAMS___________________________________________________________

All the contributors to a project - developers, business analysts, testers, etc. - work together in a single open space, members of one team. The walls of this space are littered with big visible charts and other evidences of their progress.
Stand-up meetings keep the team on the same page. Keep the meeting short, focused, and intense.
Innovate from the bottom up.
Real insight comes from active coding. Don't use architects who don't code - they can't design without knowing the realities of your system.
Publish your status , your ideas and the neat things you're looking at. Don't wait for others to ask you the status of your work.

_TOOLS____________________________________________________________

Costly tools don't produce better designs. Keep critical path technologies familiar.

_VERSION_CONTROL__________________________________________________

Always use version control system. If you need an element, check it in.
Never check in code that's not ready for others. Never keep files checked out for long periods.
Deliberately checking in code that doesn't compile or pass its unit tests should be considered an act of criminal project negligence.

_BREAK_DEPENDENCIES_____________________________________________

Design components that are self-contained, independant, and have a single, well defined purpose.
Tell, don't ask: Don't take on another object's or component's job. Tell it what to do, and stick to your own job.
Minimize global and shared data. Sharing causes contention: Avoid shared data, especially global data. shared data increases coupling, which reduces maintainability and often performance.
Prefer composition to inheritance: Tight coupling is undesirable and should be avoided where possible. Therefore, prefer composition to inheritance unless you know thet the latter benefits your design.

_INCREMENTS_______________________________________________________

Write code in short edit/build/test cycles.

_ROOT_CAUSE_______________________________________________________

Don't fall for the quick hack: "Quick fixes become quicksand."
Keep asking why: Keep questioning until you understand the root of the issue.

_MISTAKE_PROOF_CODE_____________________________________________

Design with contracts: Use contracts to document and verify that the code does no more and no less than it claims to do.
Use assertions to validate your assumptions and contracts.
The broken assertions enable to crash early and point to the root cause.
Compile cleanly at high level warning levels and take warnings to heart: Use your compiler's highest warning level. Require clean (warning-free) builds. Understand all warnings. Eliminate warnings by changing your code, not by reducing the warning level.

_TRACK_ISSUES___________________________________________________

Maintain a log of problems and their solutions: Part of fixing a problem is retaining details of the solution so you can find and apply it later.

_DESIGN_STYLE___________________________________________________

Extend systems by substituting code: Add and enhance features by substituting classes that honor the interface contract. Delegation is almost always preferable to inheritance.
Public inheritance is substitutability.

Consider making virtual functions nonpublic, and public functions nonvirtual. Prefer writing nonmember nonfriend functions.

DRY - Don't Repeat Yourself: Every piece of knowledge must have a single, unamiguous, authoritative representation within a system.

Program close to the problem domain. Design and code in your user's language.
Use a project glossary. Create and mainain a single source for all the specific terms and vocabulary for a project.

_BIBLIOGRAPHY___________________________________________________
[PAD] Practices of an Agile Developer, Venkat Subramaniam and Andy Hunt
[XP] eXtreme Programming
[PP] The Pragmatic Programer, Andy Hunt and Dave Thomas
[ILSD] Implementing Lean Software Development, Mary and Tom Poppendieck.
[SI] Ship It!, Jared Richardson and William Gwaltney Jr.
[CCS] C++ Coding Standards, Herb Sutter and Andrei Alexandrescu.
[R] Refactoring, Martin Fowler.

FAVORITE PRACTICES AND TIPS - COMPILATION

This is a compilation of my favorite practices and tips, read in various books.

Bibliography
[PAD] Practices of an Agile Developer, Venkat Subramaniam and Andy Hunt
[XP] eXtreme Programming
[PP] The Pragmatic Programer, Andy Hunt and Dave Thomas
[ILSD] Implementing Lean Software Development, Mary and Tom Poppendieck.
[SI] Ship It!, Jared Richardson and William Gwaltney Jr.
[CCS] C++ Coding Standards, Herb Sutter and Andrei Alexandrescu.
[R] Refactoring, Martin Fowler.

_CONTINUOUS_INTEGRATION____________________________________

[XP] Continuous Integration.
The team keeps the system fully integrated at all times.

[PAD] Keep your project releasable at all times.
Ensure that the project is always compilable, runnable, tested and ready to deploy at moment's notice.

[PAD] Integrate early, integrate often.
Code integration is a major source of risk. To mitigate that risk, start integration early and continue to do it regularly.

[PP] Don't Use Manual Procedures
A shell script or batch file will execute the same instructions, in the same order, time after time.

[PP] Test Early. Test Often. Test Automatically.
Tests that run with every build are much more effective than test plans that sit on a shelf.

[ILSD] The Big Bang Is Obselete
Use continuous integration and nested synchronization;

[SI] Script builds on day one.

[SI] Any machine can be a build machine.

[SI] Build continuously.

[SI] Test continuously.

[SI] Continuously test changing code.

[SI] It has to work for everyone.

[SI] Integrate often, and build and test continuously.

[CCS] Use an automated build system.
Push the (singular) buton: Use a fully automatic ("one-action") build systems that builds the whole project without user intervention.
Have two build modes: Incremental and full.

_TEST_DRIVEN_DEVELOPMENT__________________________________


"Tests are to prevent bugs, not find them."

[XP] Test-Driven Developement.
The programmers work in very short cycles, adding a failing test, then making it work.

[PAD] Use automated unit tests.
Good unit tests warn you about problems immediately. Don't make any design or code changes without solid unit tests in place.

[PAD] Use it before you build it.
Use Test Diven Development as a design tool. It will lead you to a more pragmatic and simpler design.

[PP] Design to Test
Start thinking about testing before you write a line of code.

[PP] Test Your Software, or Your Users Will
Test ruthlessly. Don't make your user find bugs for you.

[PAD] Different makes a difference.
Run unit tests on each supported platform and environment combination, using continuous integration tools. Actively find problems before they find you.

[PP] Coding Ain't Done 'Til All The Tests Run
'Nuff said.

[PP] Use Saboteurs To Test Your Testing
Introduce bugs on purpose in a seperate copy of the source to verify that testing will catch them.

[PP] Test State Coverage, Not Code Coverage.
Identify and test significant program states. Just testing lines of code isn't enough.

[PP] Find Bugs Once.
Once a human tester finds a bug, it should be the last time a human tester finds that bug. Automatic tests should check for it then on.

[PP] Test Your Estimates
Mathematical analysis of algorithms doesn't tell you everything. Try timing your code in its target environment.

[PP] Don't assume it - Prove it
Prove your assumptions in the actual environment - with ream data and boundary conditions.

[ILSD] Mistake-Proof Code with Test-Driven Development

[SI] Exercise your product - automate your tests.

[SI] Use a common, flexible test harness.

[SI] Don't change legacy code until you can test it.

[R] Make sure all tests are fully automatic and that they check their own results.

[R] A suite of tests is a powerful bug detector that decapitates the time it takes to find bugs.

[R] Run your tests frequently. Localize tests whenever you compile -every test at least every day.

[R] When you get a bug report, start by writing a unit tests that exposes the bug.

[R] It is better to write and run incomplete tests than not to run complete tests.

[R] Think of boundary conditions under which things might go wrong and concentrate your tests there.

[R] Don't let the fear that testing can't catch all bugs stop you from writing the tests that will catch most bugs.

_CUSTOMER_TESTS____________________________________________

[XP] Customer tests
As part of selecting each desired features, the customers define automated acceptance tests to show that the feature is working.

[PAD] Automate acceptance testing.
Create tests for core business logic. Have your customers verify these tests in isolation, and exercise them automatically as part of your general test runs.

[ILSD] Write executable specifications instead of requirements.

[SI] Mock client tests do the most with the least.

_SIMPLICITY__________________________________

[PAD] Develop the simplest solution that works.
"Keep it simple"
Incorporate patterns, principles and technology only if you have a compelling reason to use them.

[XP] Simple Design
The team keeps the design exactly suited for the current functionality of the system. It passes all the tests, contains no duplication; expresses everything the authors want expressed, and contains as little code as possible.

[CCS] Correctness, simplicity and clarity comes first.
KISS (Keep It Simple Software°: Correct is better than fast. Simple is better than complex. clear is better than cute. Safe is better then insecure.

[CCS] Don't optimize prematurely.
Spur not a willing horse (Latin proverb): Premature optimization is as addictive as it is unproductive. The first rule of optimization is: Don't do it. The secod reule of optimization (for experts only) is: Don't do it yet. Measure twice, optimize once.
It is far easier to make a correct program fast than ot is to make a fast program correct.

_CLARITY__________________________________

[PAD] Write code to be clear, not clever.
"PIE: Program Intently and Expressively."
Express your intentions clearly to the reader of the code. Unreadable code isn't clever.

[PAD] Comment to communicate.
"Communicate in code"
Document code using well-chosen, meaningful names. Use comments to describe its purpose and constraints. Don't use commenting as a substitute for good code.

[PP] Build Documentation In, Don't Bolt It On.
Documentation created seperately from code is less likely to be correct and up to date.

[R] Any fool can write code that a computer can understand. Good programers write code that humans can understand.

[R] When you feel the need to write a comment, first try to refactor the code so that any comment becomes superfluous.

_CODE_REVIEWS________________________________

[PAD] Review all code.
Code reviews are invaluable in improving the quality of the code and keeping the error rate low. if done correctly, reviews can be practical and effective. Review code after each task, using different developers.

[XP] Pair Programming
All production software is built by two programmers, sitting side by side, at the same machine.

[SI] Always review all code.

[CCS] Invest in code reviews.
Re-view code. More eyes will help make more quality. Show your code, and read others'. You'll all learn and benefit.

_COLLECTIVE_CODE_OWNERSHIP___________________

[XP] Collective Code Ownership.
Any pair of programmers can improve any code at any time.

PAD] Emphasize collective ownership of code.
"Practice collective ownership"
Rotate developers across diferent modules and tasks in different areas of the system.

[SI] Architect as a group.

Code reviews are a great way to enhance collective code ownership. Refer to CODE_REVIEWS.


_REFACTORING________________________________________

[XP] Design Improvement - Refactoring
Don't let the sun set on bad code. Keep the code as clean and expressive as possible.

[PP] Don't live with Broken Windows
Fix bad designs, wrong decisions, and poor code when you see them.

[PP] Refactor Early, Refactor Often
Just as you might weed and rearrange a garden, rewrite, rework, re-architect code when it needs it. Fix the root of the problem.

[SI] Use test driven refactoring to clean up untestable code.

[R] When you find you have to add a feature to a program, and the program's code is not structured in a convenient way to add the feature, first refactor the program to make it easy to add the feature, then add the feature.

[R] Before you start refactoring, check that you have a solid suite of tests. The tests must be self-checking.

[R] Refactoring changes the programs in small steps. If you make a mistake, it is easy to find the bug.


_STANDARDS__________________________________________

[XP] Coding Standards
All the code in the system looks as if it was written by a single - very competent - individual.

[ILSD] Standards Exist to Be Chanllenged And Improved
Embody the current best known practices in standards that are always followed while actively encouraging evenryone to challenge and change the standards.

[CCS] Don't sweat the small stuff. (Or: Know what not to standardize.)
Say only what needs saying: Don't enforce personal tastes or obsolete practices.

_TEAMS________________________________________

[XP] Whole Team
All the contributors to an XP project - developers, business analysts, testers, etc. - work together in a single open space, members of one team. The walls of this space are littered with big visible charts and other evidences of their progress.

[PP] Organize Teams Around Functionality
Don't seperate designers from coders, testers from data modelers. Build teams the way you build code.

[PAD] Use stand-up meetings.
Stand-up meetings keep the team on the same page. keep the meeting short, focused, and intense.

[SI] Use daily meetings for frequent course corrections.

[SI] Innovate from the bottom up.

[PAD] Good design evolves from active programmers.
"Architects must write code"
Real insight comes from active coding. Don't use architects who don't code - they can't design without knowing the realities of your system.

[PAD] Be a mentor.
There's fun in sharing what you know - you gain as you give. You motivate others to achieve better results. You improve the overall competence f your team.

[PAD] Keep others informed.
Publish your status , your ideas and the neat things you're looking at. Don't wait for others to ask you the status of your work.

_TOOLS________________________________________

[PP] Costly Tools Don't Produce Better Designs
Beware of vendor type, industry dogma, and the aura of the price tag. Judge tools on their merits.

[SI] Keep critical path technologies familiar.

_VERSION_CONTROL__________________________________

[PP] Always Use Source Code Control
Source code control is a time machine for your work - you can go back.

[PAD] Share code only when ready.
Never check in code that's not ready for others. Deliberately checking in code that doesn't compile or pass its unit tests should be considered an act of criminal project negligence.

[SI] Stay in the sandbox.

[SI] If you need it, check it in.

[CCS] Use a version control system.
The palest of ink is better than the best memory (Chinese proverb): Use a version control system. Never keep files checked out for long periods. Check in after your updated tests pass. Ensure that checked-in code does not break the build.

_BREAK_DEPENDENCIES__________________________________

[ILSD] Break Depenencies
System architecture should support the addition of any fearture at any time.

[PAD] Keep classes focused and components small.
"Write cohesive code."
Avoid the temptation to build large classes or components or miscellaneous catchall classes.

[PAD] Attack problems in isolation.
Seperate a problem area from its surroundings when working on it, especially in a large application.

[PP] Eliminate Effets Between Unrelated Things
Design components that are self-contained, independant, and have a single, well defined purpose.

[PP] Minimize Coupling Between Modules
Avoid coupling by writing "shy" code and applying the Law of Demeter.

[PP] Abstractions Live Longer Than Details
Invest in the abstraction, not the implementation. Abstractions can survive the barrage of changes from different implementations and new technologies.

[PAD] Tell, don't ask.
Don't take on another oblect's or component's job. Tell it what to do, and stick to your own job.

[SI] An encapsulated architecture is a scalable architecture.

[CCS] Give one entity one cohesive responsability.

[CCS] Minimize global and shared data.
Sharing causes contention: Avoid shared data, especially global data. shared data increases coupling, which reduces maintainability and often performance.

[CCS] Hide information.
Don't tell: Don't expose internal information from an entity that provides an abstraction.

[CCS] Prefer composition to inheritance.
Avoid inheritance taxes: Tight coupling is undesirable and should be avoided where possible. Therefore, prefer composition to inheritance unless you know thet the latter benefits your design.

_INCREMENTS__________________________________

[PAD] Write code in short edit/build/test cycles.
"Code in increments."
It's better than coding for an extended period of time. You'll create code that's clearer, simpler, and easier to maintain.

[PAD] Develop in increments.
Release your product with minimal, yet usable, chunks of functionality. Within the development of each increment, use an iterative cycle of one to four weeks or so.

_ROOT_CAUSE__________________________________

[PAD] Don't fall for the quick hack.
"Quick fixes become quicksand."
Invest the energy to keep code clean and out in the open.

[PAD] Keep asking Why.
Don't just accept what you're told at face value. Keep questioning until you understand the root of the issue.

_MISTAKE_PROOF_CODE_____________________________

[PP] Design with Contracts
Use contracts to document and verify that the code does no more and no less than it claims to do.

[PP] Crash Early
A dead program normally does a lot less damage than a crippled one.

[PP] Use Assertions to Prevent the Impossible
Assertions validate your assumptions. Use them to protect your code from an uncertain world;

[PAD] Treat warnings as errors.
Checking in code with warnings is just as bad as checking in code with errors or code that fails its tests. No checked-in code should produce any warnings from the build tools.

[CCS] Compile cleanly at high level warning levels.
Take warnings to heart: Use your compiler's highest warning level. Require clean (warning-free) builds. Understand all warnings. Eliminate warnings by changing your code, not by reducing the warning level.

[CCS] Prefer compile- and link-time errors to run-time errors.

[CCS] Assert liberally to document internal assumptions and invariants.

Also refer to CONTINUOUS_INTEGRATION, TEST-DRIVEN_DEVELOPMENT and CODE_REVIEWS.

_TRACK_ISSUES__________________________________

[SI] Avoid collective memory loss.

[PAD] Maintain a log of problems and their solutions.
Part of fixing a problem is retaining details of the solution so you can find and apply it later.

_DESIGN_STYLE__________________________________

[PAD] Extend systems by substituting code.
"Substitute by contract."
Add and enhance features by substituting classes that honor the interface contract. Delegation is almost always preferable to inheritance.

[CCS] Public inheritance is substitutability.

[CCS] Consider making virtual functions nonpublic, and public functions nonvirtual.
Non Virtual Interface (NVI) pattern.

[CCS] Prefer writing nonmember nonfriend functions.

[PAD] A good design is a map; let it evolve.
Let design guide, Not dictate.

[PP] DRY - Don't Repeat Yourself
Every piece of knowledge must have a single, unamiguous, authoritative representation within a system.

[PP] Program Close to the Problem Domain
Design and code in your user's language.

[PP] Use a Project Glossary
Create and mainain a single source for all the specific terms and vocabulary for a project.

FAILURE AND CHANGE NOTIFICATIONS

We've just configured our continuous integration tool to notify all team members by email of failures in the build, and changes commited into the code base.

The failure notifications help to implement the stop-the-line culture. A failure may be a failure to build the application, a failed test case or an assertion raised in the code by a test.

The change notifications help the team members to stay informed of the changes commited by the team. This helps to share the code.

CHANGE TOLERANT CODE

In most of the books I've read concerning software development, we are told to accept change. The authors share several practices to write change tolerant code.
This article is a compilation of these various practices and of personal experience.

• Use a version-control system. With such a tool, you can roll-back from a change which brought regressions into the code.
• Work in short iterations and integrate continuously. Always have an operational application, ready to be changed.
• Design-by-contract with assertions. The assertions will stop at runtime when a contract is broken by a side-effect of a change.
• Test-driven development. The automated self-checking tests will detect any regression brought into the code by a change.
• Measure code coverage by tests. You don't a change to affect a non-tested part of the code. Identify the lines of code never exercized by tests, and add some test cases.
• Apply the Single Responsiblity Principle (SRP). As Robert Martin says in AGILE SOFTWARE DEVELOPMENT:

A class should have only one reason to change.

• Apply the Common Closure Principle. As Robert Martin says in AGILE SOFTWARE DEVELOPMENT:

The classes in a package should be closed together against the same kind of changes. A change that affects a closed package affects all the classes in that package and no other packages.

• Use design-patterns. Many design-patterns organize your design to anticipate change.
• Use layering, information hiding and encapsulation. This limits coupling by regrouping and isolating cohesive code which may be affected by change, therefore limiting the impact of change into the code.
• Simplicity. It's just easier to change simple code.
• Write less code. It's just easier when there is less code to change.
• No repetition. Don't do the same change twice.
• Stop-the-line. Detect regressions brought by change as soon as possible. Then, stop work, correct the problem and resume work. A continuous-integration tool which detects changes, builds the application, runs the tests and notifies the development team when a failures occurs can really help.
• Refactor. Refactor the code and the tests to keep them healthy:eliminate complexity and repetition. The code will be easier to change.

MISTAKE-PROOF CODE

In IMPLEMENTING LEAN SOFTWARE DEVELOPMENT, the authors, Mary and Tom Poppendieck give an example of mistake-proofing (pages 6 and 7):

A properly mistake-proofed system will not need inspection. My video cable is an example of mistake-proofing. I can't plug a monitor cable into a computer or video projector upside down because the cable and the plug are keyed.

This example really puzzled me: How can I translate this into my code?
Of course there are the automated self-checking tests, but there is also design-by-contract with assertions.

Just have a look at this short example (in Java). The code illustrates the example of a video projector and a monitor cable.

public class VideoProjector {
/*************************************************
* Connect a monitor cable.
* PRE monitorCable is not null.
* POST ... some post-condition ...
*/
public void plug(MonitorCable monitorCable) {
assert monitorCable != null : "PreCond: monitorCable != null";
// some code ...
}
...

If you call operation plug with a null monitorCable argument, then the assertion halts the application and you get the following message:
run:
Exception in thread "main" java.lang.AssertionError: PreCond: monitorCable !=null
at zeroinspection.VideoProjector.plug(VideoProjector.java:14)

Therefore, the code of classes VideoProjector and MonitorCable are keyed to be a mistake-proof collaboration.

STOP-THE-LINE

I've discovered the Stop-the-line culture in IMPLEMENTING LEAN SOFTWARE DEVELOPMENT. FROM CONCEPT TO CASH, by Mary and Top Poppendieck.
The authors describe this practice as follows:

"work is organized so that the slightest abnormality is immediately detected, work stops, and the cause of the problem is remedied before work resumes."

In software development, this culture can be implemented by several practices combined:

• Test-driven development. Automated self-checking tests check the code does what it is required to do. This enables to detect an abnormality in what the application does.
• Measure code coverage by tests. Any code never exercised by tests is detected. Never-tested code is just as bad as failing code. This enables to be sure that anormalities may be detected everywhere in the code.
• Design-by-contract with assertions. The class responsabilites (operations) are described in terms of pre-conditions and post-conditions. The class state is described in term of a class invariant. The pre-conditions, post-conditions and invariants are checked by assertions coded into the application. Therefore, if a contract is broken at runtime (which means a bug is detected) the application immediately stops and requires fixing. The assertions are exercised by the tests. This enables to detect a broken contract as soon as it is introduced into the code.
• One-action build and test. A one-action script builds the whole application and runs the automated self-checking tests. This enables to dispose of test results easily.
• Continuous integration. Developpers deliver and synchronize their work as often as possible. This enables to always dispose of an up-to-date and shippable application.
• Automated build and test. A continuous integration tool automatically builds and tests the application upon detection of a delivered code or test change. This enables to automatically detect abnormalities in the build or in what the code does.
• Send abnormality notifications. If the automated build and test fail for any reason (the build fails or a test fails), then a notification is sent to the developpers. The notification may be emails or lava lamps. This enables the developpers to be aware that an abnormality has been detected.

These practices enable to detect a defect as soon as possible and to notify the development team.
Now the team has to stop and correct the problem before work resumes.
The best part is that all these practices have many other benefits!

IMPLEMENTING LEAN SOFTWARE DEVELOPMENT, by Mary and Tom Poppendieck

This is a GREAT book!

I appreciated LEAN SOFTWARE DEVELOPMENT, AN AGILE TOOLKIT so much a I had to read the next sequel.

I was glad to have read the first sequel before the second, as the second assumes the first is read and understood, and as there is no duplication between the sequels.

Reading this book wasn't pure pleasure. Indeed, it helped me realize the extend of improvement to perform in my organization. The waste and suboptimization is revealed and is now difficult to tolerate. This should be encouraging, but the book reveals some improvements are way out of control of a development team. It implies the full value stream. This is where the book may become quite discouraging: can you extend the improvements out of the development team? In a big corporate company, this can be more than a challenge...

Among many other things:

I learned that defects waiting in a list are waste, as they are unfinished work and hide other potential bugs.

I really enjoyed the part saying that tests are there to prevent bugs, not find them.

I understood that our practice of design by contract with assertions was a great way to implement the "stop-the-line" culture and to build mistake-proof code.

I learned A LOT on the use of slack to speed-up the time inside a process.

I recognized what I experienced through a CMMi certification.

I RECOMMEND THIS BOOK.

DESIGN PATTERNS EXPLAINED, Alan Shalloway and James Trott

I really enjoyed the way the design-patterns are brought into the code to solve a current problem.

This book helped me understand some patterns, like Bridge for example.

DESIGN PATTERNS, Gamma, Heml, Johnson and Vlissides

This is a must read. I didn't manage to read it in one-go. I haven't even read it all. I find this book hard to read, sometimes hard to understand - but full of great information. I use it as a reference manual, or a catalog.
What I put into practice:
+ Singleton;
+ Facade;
+ Mediator;
+ Observer;
+ Abstract factory;
+ Template method;
+ Adapter;
+ Responsability chain;
+ Proxy;
+ Command;
+ State;
+ Strategy;
+ Visitor;

UML2 ET LES DESIGN-PATTERNS, Craig Larman, Pearson Education

I read this book some years ago.
I enjoyed the GRASP analysis patterns and the lightweight modelling the author uses.
What I put into practice:
+ lightweight modelling using UML.

SHIP IT! by Jared Richardson and William Gwaltney Jr, The Pragmatic Programmers

Another great book from The Pragmatic BookShelf.

When I received the book, I was a little afraid as it is rated for readers having a skill range from beginner to medium.

Finally, I found great insights - very well explained.

What I put into practice:

+ We added a cross-platform build to our one action build;

+ I plan to build a smoke test for our current project;

+ We configured our Continuous Integration tool to send build and change notifications by eMail to the team-members.

I RECOMMEND THIS BOOK.

PROGRAMMING LANGUAGES

I program in C++, Java and Ada95.

I script with UNIX shells.

C++ CODING STANDARDS. 101 RULES, GUIDELINES, AND BEST PRACTICES, Herb Sutter, Andrei Alexandrescu

This book is full of great advice - even if you don't code in C++! I read it in parallel of a one-week training session in advanced C++. The training sessions contents and the books chapters matched quite precisely. It enabled me to put a name on a coding "pattern" I was unsing on my project: the Non Virtual Interface (NVI). Coupled with a design-by-contract practice (refer to Bertrand Meyer) implemented in code through assertions, this pattern can really improve your code.

What I put into practice:

+ NVI pattern.

+ Pimpl idiom.

+ RAII idiom.

I RECOMMEND THIS BOOK, EVEN IF YOU DON'T PROGRAM IN C++.

LEAN SOFTWARE DEVELOPMENT, by Mary and Tom Poppendieck

This book made me understand how many things in my company where far from being efficient!

The centralized top-down corporate organization for building software then seemed so inappropriate!

This book made me understand what "prepare the code for change" really means, though this is already said in so many books. (Maybe I'm just slow minded and need to read several books on the same subject to finally understand!).

And this book convinced me to re-install a Kanban in my team.

What I put into practice:

+ Kanban

+ Identify waste.

I RECOMMEND THIS BOOK.

PRACTICES OF A AGILE PROGRAMMER, Andy Hunt, The Pragmatic Programmers

I had to read this book, as it has both "practices" and "pragmatic" in the title.
It is a great catalog. It is designed in short chapters (~2/3 pages) which gives a real rythm to the reading.
What I put into practice:
+ I use CruiseControl to automate the one-action build of my current project;
+ I work from the list.

I RECOMMEND THIS BOOK.

AGILE SOFTWARE DEVELOPMENT, by Robert Martin

It is great to have the oo-principles explained in one same book.

We put into practice the part on dependy metrics on several applications.
What I put into practice:
+ I adaped Uncle Bob's dependy metrics tool (depend.sh) for Ada95 and plugged it in our automated build;
+ I pay attention to the oo-principles when writting and reviewing code.

REFACTORING, by Martin FOWLER

I really enjoyed the pragmatic approach to refactoring: run the tests, change the code in a little step, compile and run the tests, and so on.
The part on test-driven development is very clear and the article on the fact that the design is the code are just great.
This book is a must read!
What I put into practice:
+ I program in many little reversible steps;
+ I refactor ;o)

I RECOMMEND THIS BOOK.

EXTREME PROGRAMMING EXPLAINED, by Kent Beck

I read this book after starting to practice XP. It helped me understand how the practices worked so well together to achieve the principles.

What I put into practice:

+ I stopped trying to systematically apply all XP practices in an eXtreme way, but rather identified and applied specific practices which bring value to my business, such as continuous traceability, continuous code coverage mesure and continuous functional coverage measure.

I RECOMMEND THIS BOOK.

GESTION DE PROJET EXTREME PROGRAMMING, by Regis Medina, Eyrolles.

I discovered eXtremeProgramming with this book. It is very convincing. It made me want to practice XP. It even made me practice XP.
What I put into practice:
+ Information radiators;
+ Stand-up meetings;
+ Common workspace.

I RECOMMEND THIS BOOK.

CONCEPTION ORIENTEE OBJET, by Bertrand Meyer, Eyrolles

This book is pretty hard to read!
First, its a huge book. Too big for me to read in one go. So big reading it may seem discouraging. So I read chapters from time to time.
Second, it's complicated. Not because it isn't well written - but because the author explains the real issues of object-oriented programming.
The chapter on design-by-contract is fabulous. This is a practice which really made us improve our programming. This book is a reference.
What I put into practice:
+ My mentor made me apply design-by-contract with assertions.
+ I express design in source.
+ I autodocument the code.

I RECOMMEND THIS BOOK.

MY JOB WENT TO INDIA, by Chad FOWLER, The Pragmatic Programmers

This book is full of GREAT advice to improve your work on the short and long terms. Most of the advice isn't even restricted to IT jobs.
Reading it made me reconsider many aspects of my job. It helped me identify things I had to improve. Actually, the author even made me want to go work in Bengalore!
What I put into practice:
+ I suscribed to a programming magazine.
+ I made this blog.

I RECOMMEND THIS BOOK.

INFORMATION SHARING TOOL

Unfortunelty, we cannot use Wikis on a large scale to share information.

A top-down decision says the corporate choice is the QuickPlace, which we happen to dislikeafter giving them a try.

Therefore, we locally use lightweight Wikis, such as the one delivered with FitNesse.

BUILD TOOL

The automated build scripts are written in shell-script and Ant.

CONTINUOUS INTEGRATION TOOL

We use CruiseControl to automatically run build scripts upon detection of modifications in the repository.

TESTING TOOL

Thank God, we use free testing frameworks!
We test with Aunit when programming in Ada95 and CxxTest for C++ applications.

At home, I use JUnit for my Java developments.

ISSUE TRACKING TOOL

We track bugs and issues with ClearQuest.

ClearQuest is meant to be coupled to ClearCase, which is a nice feature as version control and issue tracking are such interlinked practices.

Unfortunetly, we use the pair uncoupled!

This is one of these cases when big corporate top-down decisions just don't help ...

Therefore, we violate the "Don't Repeat Yourself" practice as we identify issues and the impacted code in the ClearQuest. Samewise, we identify the impacted code in ClearCase using activities named as the reference of the issue to solve.

VERSION CONTROL TOOL

Our version control tool is ClearCase, with the UCM extension.

BUILD VS BUY VS DOWNWLOAD

After a long legacy of "Build it", my company is now into "Buy it".

It's a real pity, as the best tools are actually free to download thanks to great open-source pojects!

TESTS

One of the first practices applied on my current project was Test Driven Development.
Programming in Ada95, we downloaded the AUnit open-source test framework.
Team-members actually began to enjoy testing, even though is was traditionally seen as boring activity.
In our common workspace, you can often hear: "Thank god these tests exist! They just detected a regression I added while changing the code ...".

PAIRING

Lets face it: Some people just dislike pair-programming and you can't force them.

We've never managed to practice systematic pairing. Too bad, because pair-programming is so efficient.

However, we've sort of transformed it into a "on-demand pairing" practice: "Hey, I've got this tough refactoring to do. Can someone come help me through this one?".

Its good to have a teammate ;o)

SYNCHRONIZE AND DELIVER AT LEAST ONCE A DAY

Instinctively, many developpers will keep their code checked-out overnight, or for several days.

That's another situation when it is good to share a common workspace. Indeed, you nearly always have a teammember to ask you: "You look satisfied with your tests results. Have you checked in and delivered your work?"

It's good to have a teammate ;o)

TEST-FIRST

This is a very tough discipline!

Even though experiencing it clearly reveals better designs, it is very difficult to make it a practice.

TESTING

When a bug is detected, there is nearly controllable force pushing team-members to hack a quick correction.

That's when you need a common workspace with a colleague asking THE question: "Have you started by adding a test revealing the bug?".

It's good to have a teammate ;o)

SUB-CONTRACTING, OUT-SOURCING, OFF-SHORING

We have been through painful sub-contracting experiences.
Even though everybody was fully competent and willing, everyone has suffered from contracts which enforced early decisions and following plans over responding to change.

HAVE A MENTOR

This personal practice is described in My Job Went To India [ChadFowler].

Fortunetly I have a coach in my job, and I seek a lot of advice from a very talented peer.

I also consider the authors of my favorite software engineering books as mentors. I track the lessons of Kent Beck, Robert Martin, Bertrand Meyer, Martin Fowler, Mary Poppendieck, Andy Hunt ...