Monthly Archives: March 2011

RE: The “Optimal” Fallacy

In The “Optimal” Fallacy, Jurgen Apello states that “You cannot ‘optimize the whole’. The best you can do is sub-optimize, cooperate, and iterate.”

Read his whole post for more context, but I’ll do my best to summarize.

  1. There exists a Lean principle called “Optimize the Whole”
  2. The assumption is that the customer’s view on a business is one that includes the whole system. This is not true.
  3. No observer of a system can claim to have an objective view on the whole
  4. All the parts try to optimize for themselves, and through interdependencies between the parts the whole system tends to evolve to an optimal situation.

Later, in the comments, he states, “I am referring to complex systems, which are not designed by a single authority,” as opposed to designed systems.  If that’s really what he meant, it’s a straw man that is not very interesting at all.  So emergent systems cannot be optimized as a whole? Duh! Who is the actor in that passive voice sentence? I mean, who would even do the optimization if it were possible? Mr. Emerge?

Is Toyota’s production system not complex? Or does he claim that they don’t really optimize the system at Toyota? If we are talking about theoretically optimal, I would agree.  “Optimal” means the “the most desirable possible under given restrictions.”

However, to “optimize”, at least in the most common usage among people like me, means “to make more desirable”. When a compiler optimizes your performance, we do not assume that it is as fast as it can possibly be and as memory efficient as it can possibly be, only that it is faster and perhaps more memory efficient than it would otherwise be without the optimization.

When I think of “optimizing the whole,” “optimize” does not mean mathematically optimal and “the whole” does not mean a boundless system of independent actors and discussions about Arrow’s Impossibility Theorem don’t enter into it because you just get a product manager who does the best job he can of ranking things and consider that gospel, then do the best you can to improve your throughput on that list.

Take for example the Kanban systems for software development. A major benefit of this system is the help it provides optimizing the whole system.  If you have a process of product/feature conception feeding into high level design feeding into development that feeds into QA that feeds into deployment, and you can see that QA always has more works in progress than they should, it won’t do you any good to get more efficient at development or high level design or feature conception.  You don’t need more ideas for more features, you need QA to move faster (or save money by having less development capability), but speeding up your development process without speeding up your QA process won’t get your customers features any faster (or won’t get your shareholders value any faster).  That’s just this example.  The bottleneck could obviously be anywhere – deployment, product conception, development.

Taking measures to improve the efficiency of the whole value chain is all that is meant by “Optimizing the whole”.   To claim this is futile is just silly.

Still talking about Spring Roo sucking

Back in June 2010, I wrote a provocative (apparently) post called Spring Roo Sucks! It was a bit of a rant, so I backed off on the tone, but I’m still defending the overall point.

Somebody commented that I had unfair expectations of this young project. I started to reply in the comments, but it got a little long-winded for a comment, so I’m putting it here.

Joseph wrote:

It’s a great project, your expectations are unreasonable because Spring roo doesn’t have release 2 or 3, but 1 … It’s a quite new project. And you expect it to be something that exist 4+ years and that has been extremely refactored in time…

If I was programming as a hobby, I might agree. But I need my tools to increase my productivity, no matter what revision.

There are plenty of smaller, less ambitious open source projects in version 0.9 that I have found to be extremely helpful. The issue is not the revision (Spring can bring a lot of resources to bear and they are working closely with Google on the GWT stuff). To me, the issue is with the scope. This tool builds a lot of my application, but it builds it exactly the way it wants to and with sensible defaults. But almost all interesting applications have something besides a CRUD view on a bunch of entities and as soon as you starting looking in the corners, you get lost. I don’t think it’s Roo’s fault exactly. It’s just the way code generation stuff works.

To me, there are similar issues with GUI drawing tools. Those are great and deliver a lot of productivity. Just lay out your screen the way you want it and presto! That works until you say, “actually, for these 5 fields, I only want to show each of them if the database says they should be visible and I want to adjust the size of the dialog accordingly, so I don’t have overflow and I don’t have a bunch of blank space.” So then you go look at all that code that got generated for you and you can pretty much understand most of it (or even all of it) but you start modifying that code to do more precisely what you want. So far, it’s still a productivity saver, but then someone says, “I want to change the theme” or “I want to add this button” and you say, “OK, that’s a few hours,” thinking you’ll impress them with your speed. Then they come back with “A day?!? How could it possibly take that long. You just open up the GUI builder and add the button!” But that doesn’t work anymore because the code isn’t code that the GUI builder understands anymore.

So, you’ve got code now that you basically understand even if it’s not structured as modularly as you would like, making future changes more expensive. You’ve also got some code that your mouse-jockey tool doesn’t recognize anymore, so you’re productivity boost from that was brief. However, I will concede that it was also substantial. You got a nice bootstrap to your project, so who’s complaining?

Not me, if it’s limited to the GUI – the outermost layer of your application on which nothing else is dependent. I don’t think it’s necessarily wise, though, to treat the foundation of your application (the domain model) and the glue (your controller layer or servlet layer or dialog layer or whatever you want to call it) the same way.

One other way Roo’s strategy is not as good a fit as GUI builders is that building a user interface is inherently visual and so is the tool. Moving stuff around to get it just so is what it’s all about. It’s a very visual exercise so a visual builder is just a good fit, even if you only get to use it for the first draft. Your domain model, however, doesn’t work quite that way. While you are building it at first, it is going to change several times as you realize this dependency would create a cycle or that field name doesn’t fit the naming convention that you really want or whatever. I find the Roo command line to be very unfriendly to this kind of refactoring work. That kind of work calls for an editor, not a command line.

As I’m writing this, I’m wondering to myself if you took the Roo command line away if I would like the tool better. You have to learn all the annotations, sure, but then you’ve learned them so you understand what you’re doing and how to change it. And you really would save yourself a ton of typing if you got good at it. So maybe in the near future I’ll come around on the annotations. We’ll see.