LINQ Providers

Well, in my last post I talked about not using stoopid kung fu to solve problems that are easily solved elsewhere. It is in direct opposition to that wisdom I have been exploring writing my own LINQ provider.

More on this later, but for now, here's a great set of articles on the topic. I read the blog, then the reference docs, then did some basic watches/breakpoints in a VS2008 test app. With those three I'm starting to get a handle on it.

Here's the blog posts.

Software Complexity as an Objective Measurement

I had a conversation last Thursday that really drove home the need to make software complexity more concrete. I had just spent 2 hours talking to one of the senior guys around the office, a person with a known propensity for making software complex. For the sake of conversation, let's call him "Mr. F". Of course, Mr. F. disagrees that his software is complex at all and most of my attempts to suggest the solution he had come up with could be done simpler were met with, "Why? I didn't even write the code. I hired contractor X to do it."

So, after two long hours, Mr. F. makes the statement that finally puts the right words to my complexity concerns. It went something like this: "So, yeah, in the last week or so I spent a lot of time at my computer coding the solution together with contractor X, he thought it was a ton of change but he learned a lot, and we ended up getting into production." So, there it is: the only reason they hit their deadline was that the one person who really understood what was going on sat down and did the work (contractor X might have been sitting beside him, but I doubt it was equal contribution if the person writing the software actually needed to be taught how the code he supposedly wrote was working). In the absence of Mr. F., the software would become unmaintainable due to complexity. Of course, as far as he's concerned, the software is "just right", and I really don't have anything but a sense that it's not.

There's lots of measurements for complexity. For example, cyclomatic complexity. One of our projects recently used this as a major benchmark. Unfortunately, an application with 14,000 classes and 150,000 functions can still have a cyclomatic complexity of 3. You can also count other items; Robert Cecil Martin has what I consider as comprehensive a list as anyone if you're talking about actual calculable metrics.

Unfortunately, these numbers aren't what I think really make a piece of code easy or hard. They certainly contribute heavily but they are usually a metric of quality as opposed to complexity. Worse scores in these concrete areas almost always stem from inexperience. In other words, a junior developer is much more likely to score higher on the complexity side than even a senior person like Mr. F. would.

The problems I have tend to be in areas where a technique is used that can be done well or poorly, but is almost impossible to do well because of its inherent difficulty. For example, if I were to take a sample of 100 developers, I could probably get them to do the following things pretty well:

1. Write a simple SQL query with a couple of joins. Everyone learns this at NAIT and it's hard to screw up. Most queries are pretty simple in practice from what I've seen, and those that are not can be abstracted away by correct use of views. Query optimization plans work well with views so this isn't a performance hit.
2. Create a screen with simple validations and bind it to a data object. Every developer knows one technique for doing this, and most can be trained to do whatever flavor you feel like documenting.
3. Write a function to do a validation or calculation that is at least somewhat reusable.
4. Create a small database.

However, most developers don't seem to be able to do the following other things:

1. Write a thread-safe application that allows multiple threads to participate on a single activity. Synchronizing global state across multiple threads, especially when there's a higher chance of two threads wanting the same thing, is officially rocket science. I'd say maybe 5% of developers can do this, and the rest may or may not be trainable if they needed to pick thread safety up. If you have implemented a business process in a way that requires multithreaded knowledge, best of luck to you.
2. Create and use a comprehensive domain model. See a recent post from me for some thoughts and counter-thoughts about domain models. You can usually get a person to correctly use a simple data-only model but if it's too complex antipatterns start to emerge, like creating giant "god objects" with everything and their dog bolted to them or "lazy load" scenarios where every object access round-trips to the database like 37 times. And creating one from scratch? Forget it. At least in this case I can say maybe one in five can do it properly.
3. Properly implement a transaction other than a default, pessimistic locked, fully isolated, single database transaction. It's virtually impossible to find people who by default know the uses and consequences of different transaction models.

You might say, "I can do all of that, and I know a ton of people who also can." Well, fine, but I know a ton more who can't. I'm talking about teams with a dozen members not having a single person who can properly deal with at least some of the three items above.

What's my point? If you design an answer to a problem, and you need knowledge of the three areas above (there's definitely more than just these three), your solution is too complex. Either you have to be solving a really difficult problem or you have just overdesigned your solution.

I'm not saying you shouldn't use these more advanced techniques. I'm saying that if you can do what you need without them, you should. Cool code kung fu is not needed to write a data entry screen. I'd think it would be valuable to put together a list of design patterns or decisions such as the three above and maybe use them as a complexity checklist, where more checks equals more complexity.