Category Archives: Agile

W. Edwards Deming quote

“Experience alone, without theory, teaches management nothing about what to do to improve quality and competitive position, nor how to do it.”

RE: The “Optimal” Fallacy

In The “Optimal” Fallacy, Jurgen Apello states that “You cannot ‘optimize the whole’. The best you can do is sub-optimize, cooperate, and iterate.”

Read his whole post for more context, but I’ll do my best to summarize.

  1. There exists a Lean principle called “Optimize the Whole”
  2. The assumption is that the customer’s view on a business is one that includes the whole system. This is not true.
  3. No observer of a system can claim to have an objective view on the whole
  4. All the parts try to optimize for themselves, and through interdependencies between the parts the whole system tends to evolve to an optimal situation.

Later, in the comments, he states, “I am referring to complex systems, which are not designed by a single authority,” as opposed to designed systems.  If that’s really what he meant, it’s a straw man that is not very interesting at all.  So emergent systems cannot be optimized as a whole? Duh! Who is the actor in that passive voice sentence? I mean, who would even do the optimization if it were possible? Mr. Emerge?

Is Toyota’s production system not complex? Or does he claim that they don’t really optimize the system at Toyota? If we are talking about theoretically optimal, I would agree.  “Optimal” means the “the most desirable possible under given restrictions.”

However, to “optimize”, at least in the most common usage among people like me, means “to make more desirable”. When a compiler optimizes your performance, we do not assume that it is as fast as it can possibly be and as memory efficient as it can possibly be, only that it is faster and perhaps more memory efficient than it would otherwise be without the optimization.

When I think of “optimizing the whole,” “optimize” does not mean mathematically optimal and “the whole” does not mean a boundless system of independent actors and discussions about Arrow’s Impossibility Theorem don’t enter into it because you just get a product manager who does the best job he can of ranking things and consider that gospel, then do the best you can to improve your throughput on that list.

Take for example the Kanban systems for software development. A major benefit of this system is the help it provides optimizing the whole system.  If you have a process of product/feature conception feeding into high level design feeding into development that feeds into QA that feeds into deployment, and you can see that QA always has more works in progress than they should, it won’t do you any good to get more efficient at development or high level design or feature conception.  You don’t need more ideas for more features, you need QA to move faster (or save money by having less development capability), but speeding up your development process without speeding up your QA process won’t get your customers features any faster (or won’t get your shareholders value any faster).  That’s just this example.  The bottleneck could obviously be anywhere – deployment, product conception, development.

Taking measures to improve the efficiency of the whole value chain is all that is meant by “Optimizing the whole”.   To claim this is futile is just silly.

Mise en Place

Mise en place is a culinary term referring to getting everything ready to cook before you start cooking. If you are going to barbecue, you make sure you have all the fuel you need (charcoal and wood chips), all the tools you need (tongs, thermometer, brushes, knives, gloves) and all the ingredients you need (the meat, seasonings, mop sauces, butters in bowls). The point of all this is that when you actually fire up the grill and get started, you want to concentrate on the task at hand, not running around trying to find your extra charcoal or looking for the pepper.

So what does this have to do with programming? It’s the way I think about all of the tools that you want to have to do your job. You have source control, IDE, automated tests and a CI server that runs those tests every time you check something in. Those are some of the tools, but it goes further.

Say you are building some web services. You need to write integration tests to test those services. If you have a maven build that loads some test data and starts up an application server, then you just need to write the tests (using something handy like HtmlUnit) and you’re good to go. If you don’t, then you probably wind up inserting some test data manually and just testing it with curl or something. That’s a lot of work. It’s enough work that you certainly aren’t going to regression test. Even worse, it may be enough work that you don’t even do it the first time and just throw it over the wall to the QA department.

It is not a perfect analogy. The mise en place is set up to get you through a recipe, but then everything is either consumed or put back away. When you’re done coding, though, everything is still there. Nothing is consumed per se. So, for example, if you go through a lot of effort to set up your integration testing automation, you don’t need to do it again the next time. I think of this as every time I finish a coding task, I’m also setting the table for the next coding task. If I have to spend some extra effort for something that wasn’t there (e.g. automated tests) I make sure to spend that effort in a way that it will serve as the mise en place for next task. If, during the next task, I find that I my automated testing executing scheme came up short, I enhance it so that it has a better chance of being sufficient for the next task. Eventually, you wind up with an environment where you can really spend almost all of your time concentrating on the task at hand and very little time distracted by having to “put everything in place”.

Take the Time to Go Fast

This week, I have been working my tail off and making what feels like slow progress, but I have been keeping in mind what Uncle Bob calls The Primal Conundrum.

Programmers face a conundrum of basic values. All developers with more than a few years experience know that previous messes slow them down. And yet all developers feel the pressure to make messes in order to meet deadlines. In short, they don’t take the time to go fast!

True professionals know that the second part of the conundrum is wrong. You will not make the deadline by making the mess. Indeed, the mess will slow you down instantly, and will force you to miss the deadline. The only way to make the deadline—the only way to go fast—is to keep the code as clean as possible at all times.

What I have been working on is relatively complex. It is taking way more time than I estimated (I’m horrible at estimating), yet I’m still taking the time to write my tests (and keep the old tests passing). But are my tests slowing me down?

The answer is not always no, but in this case, I would say it is an emphatic no! As I said, I am writing some relatively complex stuff – security related stuff. I would I want to write it and just run a couple of manual functional tests and assume I have it covered? That sounds like a bad idea. What if I ran a bunch of manual functional tests, not just a few? That might be sufficient, but now we are not saving any time over the automated unit tests and I don’t get to save the tests for all the ways the code will develop in the future. Speaking of which, all of those tests I’ve already written are really saving me a bunch of time. All of those tests that stop passing when I add security means that I have to do something. Those tests aren’t getting in my way. They are reminding me, “Don’t forget to take me into account, too!”

Furthermore, Mr. Martin is not really talking about tests per se, but clean code. If you’re writing something complex, what better way to force yourself to break it into smaller, less complex chunks than by writing the tests first?

Anyway, just wanted to dash this off before I get back to the salt mine. I’m still under pressure and I have more tests to write.

CONCAT Gotcha between MySQL and HSQLDB

Hibernate abstracts most of the data access layer so you can switch out database platforms without changing any code (unless you count configuration as code). This allows me to use HSQLDB for my tests and MySQL for production. I love HSQLDB for integration tests. Running embedded in memory it’s super fast and you don’t have to configure a server or worry about the existing state of the database when you start your test suite (it’s empty!).

There are, of course, perils to testing with it when your production database is something else (and it almost certainly is). I have a Person class with a formula name field:

@Entity
public class Person extends Party {
  private String lastName = "";
  private String firstName = "";
  @Formula("ltrim(concat(concat(first_name, ' '), last_name))")
  private String name;

  @Override
  public String getName() {
    this.name = (firstName + ' ' + lastName).trim();
    return name;
  }
...
}

It took me a while to arrive at that formula, though.  The first one I tried was concat(first_name, ' ', last_name) but HSQLDB complained that it didn’t know the concat formula.  I knew it did, so I found that strange, but tried hunting around for alternatives.  I then tried first_name || ' ' || last_name.  That worked and all of my integration tests were passing so I thought everything was great.  I deployed to MySQL and didn’t get any errors, so everything’s working like a charm, right?

Nope.  But you knew that – why would I bother to write about that? Anyway, MySQL doesn’t complain about syntax because it is treating the || as a logical ‘OR’ and is somehow able to come up with 'Jason' OR ' ' OR 'Erickson' = 0.  Why 0? Well, if you have a string where MySQL expects a number, it will try to convert the string to a number.  A string that cannot be parsed to a number is not an error.  It’s 0.  So 0 || 0 || 0 = 0.

Anyway, now I don’t have any errors, but everybody’s name now evaluates to 0.  Except that it doesn’t.  See, that getter for name? That was there because I wanted it to work whether the Person had already been persisted or not.  That means that it looked like it was working in all of the code that accessed the bean except… find by name.  The query,  “FROM Party WHERE name = ?” was never finding my people by name.  This was quite a puzzle for me.

Well, anyway, I finally figured out what the problem was, and a little “aha” flash told me that the HSQLDB version of concat did not support an arbitrary number of parameters, so concat(first_name, ' ', last_name) wouldn’t work, but concat(concat(first_name, ' '), last_name) would.  Then I added an ltrim to take care of the case with no first name and I was set.

Re: What are you rewarding

DocOnDev doesn’t like bonuses.   Here’s the lede:

It is nearing the end of the year and we at LeanDog are wrapping up our fiscal year. We’re looking at the potential tax benefits of spending some of our reserve and we’re mulling over other ideas related to the spend of money. We are not, however, discussing our bonus objectives. We aren’t discussing them because we don’t have them. I, for one, am happy that we don’t.

He points out the problem with bonus programs he’s seen, spills some ink (properly) criticizing the metrics that were used and how those incentivized dysfunctional behavior.  He then closes with:

My advice to managers, directors, and executives who want to pay bonuses? Don’t. If you must do so, then break it down based on each employee’s income. If you have a top performer, don’t give them a bigger bonus, give them a raise. Think of a raise as a bonus that lasts forever.

This seems absurd to me.  His title got it right, but the conclusion misses the mark.  It’s not the form of the reward, it’s what you are rewarding.  The problem isn’t with the bonuses (although those can exist, too), it’s with the metrics.  If you gave raises based on the same metrics he described, you would get the same dysfunctions.  You might get the same dysfunctions without bonuses or raises, simply by posting the metrics publicly.  People do what you incentivize them to do.  You are left with a choice between not measuring anything (with respect to incentive) or being very careful about what you are going to incentivize.

Now, I’m no expert on incentives, either, but here are some things I’ve observed.

Value, not behavior

You should incentivize the value you hope to achieve, not the behavior that you theorize will get you there. Measuring behavior can have great value, but should not be used to reward or punish (within the scope of normal expected behavior). I’m not talking about not punishing somebody for stealing. I’m talking about things like the classic KLOC’s. This is a classic bogeyman metric because everyone knows that more KLOC’s may mean more productivity at first, but it’s very easy to inflate your lines of code to the detriment of readability and quality. You may want to measure KLOC’s along with other things (you may want to see it going down in a particularly spaghettified module), but you need to be careful not to incentivize it.

Sphere of Influence

I’m a developer, so I’ll keep with the developer examples. Say Joe Developer is on team of 6 developers in a division with 12 teams in a company with 800 people. Tying some of Joe’s compensation to company profitability is certainly fair, but it doesn’t do much to incentivize him because it is very difficult for Joe to see how his individual contribution affected the bottom line. However, if you have some metric that measured the throughput or quality of the output of Joe’s team of 6 developers, now Joe can see how he affects things (to the benefit or detriment of the team). I don’t want to minimize the challenge of finding reliable measures at this level. (I can’t think of any that I really like for incentives.) But the point is that if Joe can’t see how he can affect it, Joe won’t do anything differently.

Avoid Individual Performance

I also believe firmly that any incentive should be tied to team metrics rather than individual metrics. You want this team working together, not competing with each other for their share of the pie.

Difficult to Game

A classic problematic metric is bug fix rate by the same individuals that can introduce bugs in the first place. If you tried to fix that by subtracting bugs for which the individual is responsible for introducing, get ready to spend a lot of energy trying to track down the ‘real’ responsible person and arguing over whose fault something is. And when a new bug is found, was it just introduced or just discovered? A better example might be if you had a mature legacy application in maintenance mode and a team that was responsible for that maintenance. Bug fix rate might make sense there, but again, you would want to incentivize the team, not the individuals, lest you get a bunch of people that don’t have time to help each other because they need to make their own numbers.

The incentives themselves

So far I have talked about metrics and started off poo-pooing criticism of bonuses. But that doesn’t mean incentives can’t be poorly designed. The problem, in my view is that any metrics can be problematic, given enough incentive. People are ingenious animals and will find very creative ways to game the system, given the ‘right’ incentive.

When people talk about deterrence of behavior, you will often see the phrase “swift, sure and severe”. I prefer “swift, sure and substantial” because that allows for carrots as well as sticks, to misuse a metaphor. What they mean is that the most effective way to change behavior is if the consequence of that behavior is swift – happens immediately – sure – always happens – and substantial – you will really care that it happened. Imagine if every time you went over the speed limit, your car sent a message to the highway department and you were issued a ticket that printed out of your dashboard. Suddenly, the speed limit would be an actual limit. You would almost never exceed it and almost surely drive below the speed limit to avoid accidentally exceeding it. (I’m not advocating such a police state, just pointing out that such a police state would be effective at controlling your behavior.)

Another case in point is smoking. The consequences of smoking can be severe (premature death) but they are neither swift nor sure, so those consequences aren’t enough to make most people quit. On the other hand, for people that enjoy smoking, the consequences are much less substantial (the enjoyment of the act of smoking) but still enough, combined with swift (immediate gratification) and sure (works every time). That’s good enough to sell a lot of cigarettes for a very long time.

So what does this mean for bonuses and other incentives? First, if you want to change behavior, keep those things in mind. If want to cause people to really get creative gaming the system, keep those things in mind. If you pick metrics that are very easy for someone to see for themselves how they are doing, they can count on a bonus (or lack of one) based on those metrics) and you make it very high stakes (for example 50% of base salary) you can bet people will find a way to make their numbers, both in ways you appreciate and probably ways that you do not.

If you don’t trust your metrics completely, dial back the consequences. Poorly chosen metrics can cause dysfunctional behavior even if the consequences are just that the metrics are displayed on a public screen. However, if the metrics are pretty good, but not perfect, put them up on a screen so people can see how they’re doing. If you really like the metric, consider tying a small bonus (or even a variably substantial one) at the end of the quarter or year based on it.

But if you’re going to give people a 50% bump (that they’re used to getting every year) you better love that metric to death, because people will maximize it one way or another.

Motivation Over Talent and Process

Jurgen Appelo has a new post over at Agile Zone that got me thinking.  The thesis is basically that motivation for increasing one’s competence is more important than raw talent any process.

It reminds me of a book I read recently: Talent Is Overrated: What Really Separates World-Class Performers from Everybody Else. This book talks a lot about “deliberate practice” (which is what I would call your studying up on how to be a good presenter). Two things about deliberate practice:

  1. It’s not just experience doing the thing. If your talking sports, it’s doing drills, lifting weights, studying game film, etc. If it’s doing presentations, it’s reading books and blogs, practicing in front of your wife or a mirror, etc.
  2. It’s work. The author thinks it’s not fun and that’s part of it, but I think it’s the motivation you are talking about that actually brings a certain amount of pleasure (if not fun) in doing that hard work.

Where does that motivation come from? How do you cultivate it in others? Or do you just have to find it in your recruiting process and build a team out of intrinsically motivated people?

I’m not sure, but I’m inclined to think some processes (or at least environments) can stifle the motivation to improve and others can nurture it.  Processes that have tight feedback loops are more motivating to improve than processes delay feedback.  Processes that are rigidly proscriptive demotivate people to change their own behavior except possibly to comply.  Financial incentives tied to individual performance can improve individuals (sometimes at the expense of the team).  Financial incentives tied to team performance can improve individuals and the team.  Incentives tied to the performance of the whole company can do little to motivate because they are seen as outside the individual’s sphere of influence.

Re: Agile people still don’t get it

I read a rather old post that was recently brought to my attention by DZone.  Cedric’s title is provocative: “Agile people still don’t get it” and he also concludes with a provocative statement:

So here is my advice to Agilists:  get real now, or you will become irrelevant soon.

Now, I’m rather a fan of Agile in general and lately, Lean Software Development in particular, so I read it with interest.  The main criticism of the article is that he clearly had a noob for a TDD trainer and he shouldn’t generalize too much to the rest of the world from that.  Take a look at this leap:

Fundamentally, I am disturbed by the Agilists’ dishonesty when it comes to presenting their arguments.  They offer you all these nice ideas such as Test-Driven Development and Pair Programming but they never — ever — disclose the risks and the downsides.  To them, Agility is a silver bullet that is applicable in all cases with no compromises.

I don’t know if I would apply the label “Agilist” to myself, but I think I’m who he’s talking about and I was surprised to learn that I never – ever – disclose the risks and downsides.  In fact, in my particular conception of Agile, it is applicable in almost all cases because it allows lots of compromise.  Let’s not confuse the principles with the practices.  Number one in the Agile Manifesto (and I’ll admit I see a lot of people act like this isn’t the case) is “…we have come to value individuals and interactions over processes and tools.”  So I’m allowed to be an “Agilist” and say, yeah, for you TDD (a practice) might not work because you have so much legacy code, or you have a high tolerance for bugs (I would examine that particular claim closely) or whatever.

There are two main sentiments with which I agree with the author.  First, I don’t like orthodoxy and he doesn’t, either (at least not Agile Orthodoxies).  The second is that Agile is no substitute for discipline – but unfortunately, neither is any other methodology.  You think a command-in-control methodology like Waterfall (which no 0ne really uses 100% either) makes people disciplined?

Take the example of documentation.  There was a theme in the article that Agilists disdain for documentation made it difficult to understand code.  Now, I’ve been a professional developer for 16 years and I have (I am pausing before I write this to see if I can think of any examples… nope) never been confused by code that was made clear by external documentation.  Comments, sure, but UML? Design spec? Often those documents lie anyway.  Either they were written lazily by someone who was just checking a box, or they were written earnestly once a long time ago and never kept up to date.  I’m not saying a high level architecture document isn’t useful, but if you’re looking at some spaghetti code and you think to yourself, “if only I had the design spec I could untangle this” then I would say that you need to get real.

Scrum and the Agile Development Process

This presentation is actually better viewed in PowerPoint format (which includes the animations!).

Seven Wastes of Software Development

Lean Manufacturing identifies a well known seven wastes. Mary Poppendieck maps these to the seven wastes of software development:

Manufacturing Software Development
Overproduction Extra Features
Inventory Requirements
Extra Processing Steps Extra Steps
Motion Finding Information
Defects Defects Not Caught by Tests
Waiting Waiting – including customers
Transportation Handoffs

Let’s always think about how ways to reduce waste.