Keeping It Simple

Month: November 2007

  • In Support of the Relational Model

    Every few years, a wave of people claim that the relational model falls short of modeling data structures of modern applications. This includes some really smart people recently such as:

    – Jim Starkey
    – Brian Aker
    – MIT researchers

    This trend of criticizing the relational model started almost immediately after E. F. Codd published his seminal paper, “A Relational Model of Data for Large Shared Data Banks” in 1970.

    Some data modeling tasks are just complex by nature, and it is necessarily a difficult, time-consuming analysis to figure out how to structure the storage for a given application in a manner that is both logically complete and also efficient.

    The complexity does not originate from the technology; the complexity originates from nontrivial real-world scenarios, which are full of complexity, inconsistency, and special-case exceptions. It isn’t that the database modeling technology is insufficient, it’s that the job is genuinely hard.

    This is an example of what I call a Waterbed Problem. When you’re lying on a waterbed, and you push down with your hand to make part of the bed lower, the water inside is displaced and rises in some other area. Even if you get a lot of friends to help you with the waterbed (okay this is starting to get racy but don’t worry), the water is still mostly non-compressible, so naturally no matter how hard you push, you only displace the water, you can’t make it shrink.

    The waterbed analogy applies when we try to simplify a computer organization problem. We “push down” on the complexity of the task by inventing new methods for describing the problem. Sometimes this leads to nifty things like domain-specific languages. But generally there is a trade-off. We make one part of the problem easier to solve, at the cost of making another part of the problem less flexible, or harder to solve. The analogy is that the complexity rises in some other area, in response to our attempt to push the complexity down in the area we’re working on currently.

    We might even create a simple solution for the problem at hand, but the new solution is simply unable to address some related problems. This might be analogous to a leak in the waterbed, but probably I’ve exhausted this analogy by now.

    Flexibility of the data modeling solution is not the only criterion. It must also support rigidity where needed. If you can’t define the structure of your data model enough to disallow invalid data, then you’re asking for trouble. If the data management is just a pool of data elements and relationships, and any enforcement of structure is dependent on application logic, then your data integrity is at risk of bugs in that application, or susceptible to invalid changes made via a different client application.

    I believe the relational model strikes a good balance between flexibility and rigidity, and that’s why it has been a good choice for general-purpose data modeling for so long. Other techniques have advantages in certain areas, but they always have a balance of disadvantages in other areas.

    There is no magic bullet that can make the data modeling process easier. Not object databases, not hierarchical databases, not semantic databases, not network databases. Different data modeling paradigms can describe the data relationships differently, but they can’t make the problem simpler.

    To respond to the smart folks who say that the relational model is inadequate, I concede that the relational model is the worst form of DBMS except for all those other forms that have been tried from time to time (with apologies to Winston Churchill).

  • Less code vs. good code

    Alex Netkachov, Vidyut Luther, and Richard Heyes discuss the pros and cons of writing code that is short. Here are some thoughts from me.

    In general, I don’t think turning three lines of code into one line makes an application better, or more readable, or prettier, or whatever is your goal. The logical extension of this is to switch to writing Perl code:

    #!/usr/bin/perl
    # Valentine.pl, copyright 2001-2007 Bill Karwin
    map( ( $r=$_, map( ( $y=$r-$_/3, $l[24-$r].=(‘ ‘,’@’)[$y**2-20*$y+($_**2)/3<0] ), (0..30) ), ), (0..24) );
    print join(“n”, map(reverse($_).$_, @l)), “n”;

    The problem with such super-compressed code is that it becomes harder to code, harder to debug problems, and harder to maintain. Imagine being a newly hired developer assigned to a project that contains code all written like the above.

    Of course, we’re not talking about code as obfuscated as the above. But taking a step in the direction of compressed code for the sake of compressed code, instead of for some functional improvement, is a step toward making the code less maintainable.

    For example, think about reading diffs in your revision control system, when a bug fix occurs on such a compressed line of code. It’s harder to see the nature of the bug fix, compared to a more expanded form of coding. For this reason, I prefer to write expanded code–even when writing Perl code!

    The ternary operator is another good example. Vidyut expresses that the ternary operator should not exist. It’s true that it can be abused like any programming construct, but it is definitely useful to have the ternary operator in the language. It is a simple if/else construct, which has a return value. The conventional if/else construct doesn’t have a return value. Alex gives an example that is a perfect use of the ternary operator.

    $message = ($age < 16) ? 'Welcome!' : 'You are too old!';

    But notice I said simple. The ternary operator is inappropriate to use for any if/else blocks that contain anything more complex than a single expression. What if the application requires that we log the event when someone who is too old attempts to access the system?

    One can try to do acrobatics in code to accomplish two steps in one expression, but this is not a good practice because if a bug occurs in either step or the requirements change, you end up breaking both.

    // Wrong
    $message = ($age < 16) ? 'Welcome!' : log_and_return_message('You are too old!');

    Once either the positive or negative block becomes anything other than a single expression, you do need to change to using an if/else construct. So if you are tempted to use a ternary operator because it’s shorter or it’s prettier, consider if there is any likely scenario in which you would have to restructure it anyway. If so, that’s probably a good reason to use if/else instead of a ternary expression.

  • Enabling the Success of a Software Team

    There are three “must haves” for excellent managers, which I look for when I work for a manager, and which I try to live up to when I work as a manager.

    I thought I’d write down these thoughts, after seeing Jeremy Cole’s blog this week with some great advice about ways to attract, motivate, and retain expert MySQL DBA architects, and earlier this summer Cal Evans’ podcast Attracting Talent on the PHP Abstract site about attracting talent.

    This brings me to my ideas about how to manage excellent software developers. There are many things a manager has to do to be effective at leading a team, but to boil it down to a single principle, I like to say that the manager’s job is to enable the success of their team.

    Here is my list of three management responsibilities to support the success of the team:

    1. Give clear and achievable assignments

    The first step to making your team successful is to tell them what you want them to do. What is the goal of the project? What does the resulting software need to do? Are there constraints on schedule or technology? Who is the audience? Who approves the final deliverable? These and other high-level questions must be answered, even if all the fine details are still in “discovery phase.”.

    Counter-example: in a classic Dilbert comic strip, the boss asks Dilbert to work on a new project. Dilbert says, “great, I’m ready, what’s the project?” The boss says, “It’s not all worked out yet, so you just start coding, and I’ll stand here looking over your shoulder. If you do something wrong, I’ll scream.”

    The assignment must be achievable. Not softball — giving a developer a challenge is a good thing. Many people thrive on this, and sometimes they can pull a rabbit out of a hat and surprise everyone (including themselves). But it does no good to give someone a task that is truly impossible, this just sets them up for certain failure.

    Also, the assignment must be consistent, or at least acknowledge clearly when it changes. We all know that project requirements tend to evolve and we are okay with that. But a manager who implies that the developer should have anticipated the change is being disrespectful. Or worse, I’ve seen managers claim that the changed requirements were what was “intended” (though not stated) from the beginning, and that it’s the developer’s fault for not inferring this information. What can one say about this behavior? Let’s just say that the manager is not fulfilling his or her responsibility to make the team successful.

    2. Provide the resources needed to be successful

    I have a pretty broad definition of “resources” in this context, including hardware and software tools, enough time to complete the assignment, access to people who are needed such as IT support staff or subject matter experts, any existing technology or research that is part of the desired solution, etc.

    Counter-example: I once was told to set up a testing environment, but we had no server on which to install it. The VP’s solution was to tell me to use VMware and then I’d have as many servers as I need. But we still needed a real server on which to run the VMware software, and we had none. This is an example of being told to make bricks without straw.

    Another counter-example: a manager who won’t authorize a $250 expense for a commercial-quality code-analysis tool, but they’d rather let their highly-paid developers spend weeks debugging elusive issues. That’s not a smart use of time or money. Sure, one doesn’t want expenses to get out of control, but being either too stingy or too frivolous are both likely to put the team’s success at risk.

    3. Give constructive feedback

    The manager must communicate clearly and deliberately, instead of assuming “no news is good news.”

    Feedback doesn’t need to be full of hollow affirmations or cheerleading; it should let the developer know how close he or she is to the goal of success. Also, if the developer is off-target, it’s important to communicate about this and correct it as early as possible. Most people naturally want to do a good job, and being allowed to do the wrong job for weeks is sure to discourage them once they learn the truth.

    Ultimately, when the team completes an assignment, a manager should tell them they did so, and how well it meets expectations. An important part of enabling a team’s success is letting them know when they have done it.