Beefy Boxes and Bandwidth Generously Provided by pair Networks
We don't bite newbies here... much
 
PerlMonks  

DBI vs DBIx::Recordset vs princepawn vs chromatic

by princepawn (Parson)
on Mar 08, 2001 at 00:32 UTC ( #62820=perlmeditation: print w/ replies, xml ) Need Help??

The following is an open letter to chromatic on an issue about which many perlmonks are not completely clear on. I recently published a perl.com paper on an extension of DBI known as DBIx::Recordset.

About a week after I published this, I noticed an email from the onperl@lists.oreilly.com mailing list in which a short piece of a criticism of my article by chromatic appeared. In my eyes, the criticism is weak, but in the email, the acting editor of Perl.com write supportively of it. The aggravations for me in this are:

  • chromatic did not cc with his initial criticism
  • The acting editor released a criticism of my paper without insuring that the criticism was valid
  • Neither party made any effort to collaborate with me in analysing possible shortcomings in my article.

    Now as I write this, I am starting to think: "Tim Bunce and Alligator Descartes could say the same thing about me. I didn't tell them I was writing an article concerning DBI. And next thing you know, there it is in from 100,00 eyes for all to see." There is one difference. I did try to reach Tim and Alligator but caught pure hell trying to subscribe to the DBI mailing lists. I felt like I was interviewing for a position with the FBI or something.

    So anyway, I am now writing, as open letter what I planned to write to chromatic in email and CC to the acting editor of perl.com. I hope to hear what everyone thinks about whether DBI is really a sensible tool for application development, not scripts.

    And let's not forget we have several options besides Recordset for application database use from Perl:

  • BingoX::Carbon
  • Alzabo
  • DBIx::Abstract
    1. Before any full-fledged attack on my article occurs in public, I would like to see such attack and have a chance to respond.
    2. I attach my response to the small segement of criticism that was publicly aired without my prior notification at onperl@lists.oreilly.com
    3. Since I authored this article (1 year ago), there have been several extensions of DBI and DBIx::Recordset is one. It would be very useful to compare and contrast these offerings...
    4. I would have appreciated a CC of the initial criticisms you sent to Chris Coleman. I don't really like how much went on regarding my article without my knowledge.
    In the short blurb recently released to onperl@lists.oreilly.com, an extract of Chromatic's criticism of my recent perl.com article is offered:
    .... For example, the author is either unaware of hash slices or chooses not to use them in the first code example. While constructing SQL statements with many fields and placeholders is tedious, and the Insert() function of DBIx::Recordset is shorter, one could reduce 15 or more lines of repetitive code [in the DBI example] with hash slices .... It would be possible to improve the final example by using placeholders, as well ....
    Let's take the criticisms one-by-one. The first criticism is of the following code:
    $sql='insert into uregisternew (country, firstname, lastname, userid, password, address1, cit +y, state, province, zippostal, email, phone, favorites, remaddr, gender, income, dob, occupation, age) values (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)'; my @data=($formdata{country}, $formdata{firstname}, $formdata{lastnam +e}, $formdata{email}, $formdata{password}, $formdata{address}, $formdata{city}, $formdata{state}, $formdata{province}, $formdata{zippostal}, $formdata{email}, $formdata{phone}, $formdata{favorites}, $formdata{remaddr}, $formdata{gender}, $formdata{income}, $formdata{date}, $formdata{occupation}, $formdata{age}); $sth2 = $dbh->prepare($sql); $sth2->execute(@data); $sth2->finish();
    Chromatic is saying that by predefining the order the formdata, we can eliminate the monstrous "my @data = (...)" code completely and simply change
    $sth2->execute(@data); to $sth2->execute(@formdata{@formdata_column_ordering});
    where @formdata_column_ordering is an array indicating the order in which to choose fields from %formdata so that they match the columns indicated. And my response to Chromatic is to say that he is 100% correct he could do this. In fact, he could simplify generation of the insert sql by a couple of simple joins:
    sub make_placeholders { sprintf "(%s)", join ',', '?' x @_; } sub make_columns { sprintf "(%s)", join ',', @_; }
    After this the "$sql = ..." code becomes much smaller too:
    $sql = sprintf "insert into %s values %s", make_columns(@formdata_column_ordering), make_placeholders(@formdata_column_ordering) ;
    And so we see that Chromatic is 100% correct, you can make use of the Perl language to come up with more succinct ways to use DBI. Now let's take this statement from my paper to compare the DBI and DBIx::Recordset APIs for application-level useability:
    The key way to determine whether a particular module/library is matched to the level of a task is to count the number of lines of ``prep code'' you must write before you can do what you want.
    As we can see, to make up for the fact that DBI does not directly support commission of hashes to database, I have had to do the following:
    1 create 2 subroutines 2 use these subroutines to generate sql 3 connect to the database 4 obtain database and statement handles 5 commit the record to database.
    This is in contrast the DBIx::Recordset example:
    DBIx::Recordset->Insert({ %connection_hash, %formdata });
    Now of course, the other valid criticism is that you can write even more subroutines to allow yourself to handle steps 1-5 in one step, but by the time you have done so, you have basically re-written DBIx::Recordset! Again referring to the intro:
    In most cases the gap between DBI's API and Perl applications has been bridged by indiscriminately mixing generic application-level functionality with the specifics of the current application.
    What I mean here is suppose that had I not created subroutines to deal with placeholder and column-name generation, but instead done something like this:
    $sql= sprintf 'insert into uregisternew (%s) values (%s)', join ',', @formdata_ordering, join ',', '?' x @formdata_ordering;
    Then I would have been mixing the generic application-level functionality of SQL generation with the specifics of the data about to be inserted. In the intro I continue:
    Another maladaptive way that the DBI API has been extended for application-level databasing is by developing a collection of generic application-level tools but not publishing them.
    This would be the case if I developed large bodies of code with my make_placeholder() and make_column_name() routines but did not make them available for widespread use. Finally I state:
    The final way to misuse DBI in an application is to use it directly.
    And this is exactly what the paper showed as an example. Regarding your accusation:
    .... For example, the author is either unaware of hash slices or chooses not to use them in the first code example.
    Neither option you present is the case. This code example was lifted directly from a now-defunct dot-com that I consulted at. My goal was to show the type of kludgery that results when people who don't have the time, education, or interest to develop viable high-level APIs for their Perl programs make use of what is available and what is available (in this case DBI and it's manpages and book) provides the wrong level of abstraction for the task at hand.

    The second criticism is of the final DBI example. Chromatic says:

    It would be possible to improve the final example by using placeholders, as well ....
    How would the final example be improved by placeholders? Speed of execution? Reduction of code lines? Since the blurb did not clarify this, I assume you mean that execution speed would improve. I won't argue that the final DBI example could be made faster with fewer lines of code. But I would argue that the final DBI could be made as functional as the DBIx::Recordset example in as many lines of code. The intent of the final example was to contrast DBIx::Recordset use with DBI use in a high-level database task and show what automatic features of Recordset had to be manually coded when directly using DBI.
  • Comment on DBI vs DBIx::Recordset vs princepawn vs chromatic
    Select or Download Code
    Re: DBI vs DBIx::Recordset vs princepawn vs chromatic
    by chromatic (Archbishop) on Mar 08, 2001 at 01:25 UTC
      I would like to clear up one thing, before I respond. I had no knowledge that my e-mail to the editors of Perl.com would be reproduced in their newsletter. In fact, until another monk mentioned that it quoted me, I was not even aware of the existence of that newsletter. (While I'm not thrilled about having a paragraph bereft of context, I'd be less thrilled if the entire e-mail had appeared, verbatim, without my permission.) Without that context, one would have to use the late unlamented PSI::ESP to discern whether I called for anyone's head on a platter or even asked for the opportunity to explain some tips and tricks to make DBI usage easier (and hopefully enlighten less-experienced programmers).

      In fact, I did neither.

      That said, I believe there are two issues here. I do not disagree that DBIx::Recordset and Class::Tangram and other modules are useful. But this whole discussion boils down to:

      • "mixing the generic application-level functionality of SQL generation with the specifics of the data about to be inserted"
      • poor examples of DBI usage

      As for the first, I do not fully understand what you mean. At some point, your program must request information from the database. Something must decide to select certain rows from a particular table. If you mean that the field names should not be hardcoded in the generated SQL statement, I heartily agree. But your example is contrived, and the case is not nearly as bad as you make it seem. A reasonably experienced programmer can tell the difference between a hundred-line throwaway script and a program designed to be used continually. A decent programmer should be able to write a wrapper subroutine for the insert operation in less than five minutes. (DBIx::Recordset looks to be useful for reducing this burden, in the few minutes I've perused its documentation. I do not know if it provides a mechanism to work with anything other than a hash, which would cramp my style somewhat.)

      In your meditation above, you say "you can make use of the Perl language to come up with more succinct ways to use DBI." That is my point exactly! If your example code is such a minefield, why not take the opportunity to show a better way to do it? That would do much more good than to leave readers with the impression that DBI is too difficult to use correctly. DBI has mechanisms to quote data automatically, and Perl has mechanisms which allow you to insert the data in any order. Those are two examples of things DBI provides for that your examples did not use -- and claimed were deficiencies of DBI.

      In short, I think your advocacy would have been improved by using idiomatic code to illustrate otherwise valid issues with the DBI.

      Update: Improved the formatting slightly.

        I do not fully understand what you mean. (by "mixing the generic application-level functionality of SQL generation with the specifics of the data about to be inserted"

        That is why I called for a preliminary round of tete-a-tete between us. How can you critique what you dont understand? Anyway, let me explain again. make_placeholders() and make_insert_columns() are both subroutines which are examples of generic application-level database functionality. Now, when routines like this are needed, one of three things can happen:

        1. code similar to this shows up in a sprintf statement and is not abstracted into a library
        2. code similar to this is packaged into subroutines but not released for public use
        3. code similar to this is published on CPAN
        It is the first case that the statement refers to --- generic functionality such as that in the subroutines is mixed with the specific query about to be created instead of extracting it into a subroutine or module.

        If you mean that the field names should not be hardcoded in the generated SQL statement, I heartily agree. But your example is contrived, and the case is not nearly as bad as you make it seem. No, but things are worse with DBI and they don't need to be if you don't use it directly. Further, I don't know how much rapid prototyping you have done, but DBIx::Recordset was of immense use during this phase for this very reason. Instead of going back to fiddle with my SQL (or a hash slice to generate SQL), Recordset took care of it for me.

        A decent programmer should be able to write a wrapper subroutine for the insert operation in less than five minutes. This was another point of my article. Why should every programmer run about writing these wrappers when a viable wrapper exists within DBIx::Recordset? ANSWER ME THIS! It is sheer stupidity and a crass violation of the first rule of software engineering to reinvent wheels this way. Further, your problem set becomes more complicated once the insert must work across databases. This is not something I focused on in this article, but DBIx::Recordset calls work for several major databases --- Oracle, MySQL, and Access to name a few.

        I do not know if it provides a mechanism to work with anything other than a hash, which would cramp my style somewhat. It does not

        In your meditation above, you say "you can make use of the Perl language to come up with more succinct ways to use DBI." That is my point exactly! If your example code is such a minefield, why not take the opportunity to show a better way to do it? Because this is not a tutorial on DBI usage. Did you read my statement in the open of this thread? I clearly stated why such brain-dead code such as the example used was used: because it is being done and it can be done because the API is far too low-level for app-level programming. It's impossible to write code this brain-dead with Recordset because the API use high-level Perl structures and high-level functions not dissimilar from what you might see in a design spec for a database-aware application.

        DBI has mechanisms to quote data automatically, What can you tell us about this authoritatively? This is a serious question. The DBI book stated that DBI uses heuristics, not database metadata (which Recordset does) to determine how to quote fields. Thus while DBI's quoting mechanisms are faster, they are more error-prone.

        Perl has mechanisms which allow you to insert the data in any order. Given the two, a pre-written library which handles order-independent inserts and the necessity to write Perl code for it, which is more sensible from a software engineering perspective? The answer is clear and is the reason I wrote the article.

          Examining the examples at the end of the DBIx::Recordset documentation, it appears that the simple and consistent syntax that appeals to you is only available when you select all rows and fields from a specific table.

          The rest of the examples are not nearly so simple. They need to be expressed in language not far removed from SQL itself (presumably because the module must convert it internally). That's where my confusion comes on the idea of mixing 'application-level functionality of SQL generation' with the rest of the code. At some point, something has to generate SQL -- that's what a database speaks! Whether you generate it by hand, abstract it away in a wrapped function, or use a module which does this for you, at some point, at some level, you'll have to express details about the data you want.

          The one counter example is the exact example in your article -- the equivalent of the SQL statement 'SELECT * FROM (table)'.

          One's preferred layer of abstraction depends on the nature of the program being written and the programmer's experiences and knowledge. Additionally, the appropriate database access depends on the internal data structures of the program. In my experience, only the smallest programs I've worked on had data structures corresponding closely enough to the row/fields concept of a relational database that they didn't require an additional translation layer.

          The rest of my projects tended to need a bit of code modifying the returned rows into a data structure more appropriate for manipulation by other sections of code. With that in mind, it was an easy and conscious engineering decision to use DBI's fetch* methods, as the returned results lent themselves nicely to the necessary transformation.

          Going through an additional layer of abstraction that wouldn't transform the data appropriately, rewriting the SQL query components into the format needed for this additional layer, and giving up the speed and flexibility were not appropriate for these projects.

          For your projects, DBIx::Recordset appears to be the best solution. Good for you. I wish you the best of luck with it.

    Re: DBI vs DBIx::Recordset vs princepawn vs chromatic
    by danger (Priest) on Mar 08, 2001 at 01:43 UTC

      Surely you must realize that when you publish something, others are free to review and criticize it as they see fit, with or without your permission or your input. That's just life. If you have a rebuttal to a review or criticism then submit it to the forum where that criticism is published (and again, realize that they are under no obligation to do anything). I think there are far more appropriate ways to handle your discontent than to drag it over here and call it a Perl Meditation.

        danger, www.perl.com does not lend itself to discussion as well as PM. Admittedly, princepawn could have fought this battle out on the mailing list turf where it appears to have begun. But then again, from chromatic's entry, it appears that he wasn't aware that he had been published in the mailing list in question.

        I think that this is exactly the sort of thing Meditations is designed for, though. In this case, it's a tad bit more combative, but nonetheless it's designed to make you, the viewer, think about the two sides here (everything has at least two sides here) and decide which one is better for you, the viewer.

        I do not see this posting, or chromatic's reply as any other sort of attack than a technical one. Sure it seems personal at first glance, but when it comes down to it, princepawn is evangelizing one side and chromatic is rebutting. What's more natural? Do you eat everything that your OS vendor spoonfeeds you? No, you're technical (assumption based on the site's purpose) so you test to determine your actual limitations.

        I'm watching this because I find it fascinating, but I'm reserving judgment until I feel like I can contribute. This is a good meditation because it actually forces you to consider choices.

        ALL HAIL BRAK!!!

          PsychoSpunk, I am not saying that a discussion of the relative merits of two different ways of doing something isn't appropriate. Unfortunately, that is not how the original post appears to my eyes. Though it easily could have been written in such a fasion, instead it appears as a defensive position for an article not published here, against a criticism, also not published here. This kind of defensive posturing is hardly conducive to open minded discussion and more like a pissing match (even if only one party is actually pissing). If princepawn merely wanted to open a discussion about the relative merits of DBIx::Recordset vs. DBI, and evangelize the former, why bring in the excess baggage of the article and criticism in the first place (because, it appears, he didn't come to open a discussion about DBIx::Recordset vs DBI, he came to defend his article -- and his follow-up to chromatic looks like more of the same). I don't know about you, but thus far I haven't seen anything really new here beyond arguments for, and restatements of, the original perl.com article's content. I am glad you find it so fascinating though.

    Re: DBI vs DBIx::Recordset vs princepawn vs chromatic
    by mirod (Canon) on Mar 08, 2001 at 03:01 UTC

      As danger said, everything you publish will attract opinions, some good and some bad. After one of my articles for xml.com they set up a forum to discuss it, and the title of he first post was "This is a terrible advice for Perl developers". That's OK, if people disagree then they should say so. I learnt a thing or two in the ensuing discussion.

      Now there might have been a problem in how you got to read chromatic's opinion, but why not send him an email, or /msg him and get this out of the way so you (and we) can discuss calmly the technical disagreements you have.

      On that point I am certainly not an expert in using DBI, but when reading your article I could not help but notice that it was quite unfair with DBI. If DBIx::Recordset is such a great module you don't have to use deliberately clumsy DBI constructs to prove it. This certainly detracts from your goal. And believe me, if I noticed it, I am probably not the only one. Being enthusiastic about a module is one thing, not being fair when comparing it to other modules is another.

        On that point I am certainly not an expert in using DBI, but when reading your article I could not help but notice that it was quite unfair with DBI. If DBIx::Recordset is such a great module you don't have to use deliberately clumsy DBI constructs to prove it. This certainly detracts from your goal. And believe me, if I noticed it, I am probably not the only one. Being enthusiastic about a module is one thing, not being fair when comparing it to other modules is another.

        That's a good point and it's not the first time that it has been made. In fact, the very article you see on perl.com was rejected at The Perl Journal with the major complaint being I didn't need to spend time slamming DBI, but could instead just show how great Recordset was and attract interest that way.

        Your point is now taken, but please understand: when I used that crufty example, my intent was not: "oh boy let me come up with some super-crappy DBI code and scare everyone off from it." In fact, until I read this thread, it never occurred to me that that would be the way that people would take that example.

        That example came about simply because I had to fix something and it was right there staring me in the face. And then I began to think: "if only Recordset were the de facto application-level interface, things like this could not be written." And then things continued from there. I kept writing and piecing together ideas until it turned into an article.

    Re: DBI vs DBIx::Recordset vs princepawn vs chromatic
    by princepawn (Parson) on Mar 08, 2001 at 04:08 UTC
      danger's writing style tickles me. His sentences are thorough, well-formed crystallizations of methodical, highly analytical thought processes.

      Anyway, you are right. I haven't said much other than what was in the original article. In fact, I worked very hard on that article for about 4 weeks, making sure that it said everything that I wanted it to say. And I rewrote it from scratch a few times in the process. So, because it was an attempt to comprehensively cover a particular topic, I am not surprised that all I am doing now is restating parts of the article. In the process, I am hoping that certain things that were overlooked or not understood are becoming clearer.

      And I am disappointed that many questions in the chatterbox have not shown up here. And I am disappointed that no one is discussing the relative merits of these 2 modules or other related modules.

    Re (tilly) 1: DBI vs DBIx::Recordset vs princepawn vs chromatic
    by tilly (Archbishop) on Mar 08, 2001 at 06:32 UTC
      FWIW when I saw the original article I had a similar reaction to chromatic. The quality of the code was such that I would object to code like that at work, and I simply do not think it appropriate that a site like http//www.perl.com should be posting articles that show as examples of how to use something code which I wouldn't like to see in production.

      And yes, I identified many of the same items that chromatic did. If you are lining up long chains of variable names and hoping that you have the same number, in the same order, then you are going about things in the wrong way. I am quite sure that there is plenty of code in the world that does that. But they should not be appearing in prominent Perl sites as examples of how to code.

      In fact most of your complaints about things that involve "manual coding" are things that I look at and would never code that way. If you are writing large amounts of code that looks like repetitive cut and paste, there are usually better ways to do it (quite easily) in Perl. I would expect a good programmer to find those ways. And conversely a programmer who does not find those ways will usually benefit from a discussion about how to do it better, and why it is important to do so.

      Other complaints I simply disagreed with. More on that later.

      This is not even getting into nitpicks. For instance a good editor should have caught your using Angryman as a data structure, and asked you to use a less provocative name for that.

      Until those issues are addressed, it will be hard for me to read that article and get a sense of how those modules would compare in the hands of a competent programmer. It is not hard for me to look at the list of things that you are saying are improved in DBIx::Recordset and tell that the abstraction, at least as you are describing it, would not be appropriate for the actual situations that I am coding against at work. Likewise your list of things that you are complaining about having to do repetitively with DBI are things that I don't have to do repetitively in the situation I am working with.

      And now I just went to look at what chromatic was talking about about placeholders. What I saw simply convinced me that there is a lot about programming well that you either have not bothered to learn, or you saw it and thought was bunk.

      Here is some commentary of one repeated obvious mistake in your code example at the end. Those who want to see the code that I am about to criticize can go here. Those who want to keep any illusions of me being a generally polite nice guy should skip this. Those who read on should note that this is far from a comprehensive criticism of everything that I saw which needs improvement. And princepawn, I sincerely hope that you read this, get off your high horse, and learn something useful.

      You use gratuitous globals everywhere. Not only do you use globals with very generic names, you are forcing them into package main where they can conveniently collide with any other gratuitous globals you write. In mod_perl you would have just gone out of your way to create a perfect setup to have stale data from one request pollute the next. Congratulations.

      Some of the globals are conceivably reasonable globals. They are registration data that you might have a reason to use elsewhere in the program. But some of the globals are just stupid. For instance you have an insanely_long_function_name which has no purpose other than to set up $::INSERT_TERM so that you can do an insert in the next function.

      Does this variable really need to be global? How about global and in package main? Is creating conflicts so important to you that you need to work so hard to defeat the hapless maintainer that tries to introduce a little sanity with package? Have you heard of this little innovation called returning a value or another silly idea that a few clearly out of touch programmers use called passing a parameter?

      One almost can imagine that in the distant past some hapless programmer faced with your code tried to introduce some sanity with a package command. After that broke everything, you decided to defend yourself by always using package main rather than learning how to program better.

      An alternate theory is that you heard somewhere that strict was a good idea, but never saw the point of using vars rather than fully qualifying your variable names. So you fully qualified everything into main (less typing, and makes it easy to violate every rule you are ignoring about encapsulation etc) and promptly lost most of the practical benefits of strict. But you didn't actually understand why strict is beneficial, so you never realized that.

      You know, if you really need to share variables between packages, that might be what this handy module known as Exporter is for? I know you have heard about it, because I mentioned it to you before.

      Now that mention is interesting. Your entire idea of selecting * from a table and having your code figure out how to handle that is a stupid idea. I spent a good deal of energy in that post trying to explain to you why that was bad and what a saner approach is.

      Apparently you didn't listen. Somehow I am not surprised. You don't appear to spend much time listening. You think that the rest of the world should listen to you though.

      Now if you want an answer to the question I ask there, along with a few other comments that would be good for you to take to heart, I would recommend reading the end of About white shoes. My comments there would probably have been even harsher if I knew exactly how little you appreciate about basic things like why you should avoid globals everywhere.

      The fact is that right now you clearly have so little appreciation of basic principles of how to program that I don't care about your opinion on what modules you like. You simply lack the skill level necessary to make judgements on that that I would find even remotely useful.

        And yes, I identified many of the same items that chromatic did. If you are lining up long chains of variable names and hoping that you have the same number, in the same order, then you are going about things in the wrong way.I don't agree with you. Such programming is possible in DBI and in some cases is appropriate. I see DBI just like I see assembly language or something. It isn't very structured, but it is very fast. In some cases, where you want optimal speed, then a "portable database assembly language" like DBI is the tool of choice

        It is not hard for me to look at the list of things that you are saying are improved in DBIx::Recordset and tell that the abstraction, at least as you are describing it, would not be appropriate for the actual situations that I am coding against at work. Please describe these situations in detail and show why, otherwise I am afraid I have nothing as hard data showing the inadequacy of Recordset in application-level programming. OPEN CHALLENGE. PLEASE FILL IT.

        Furthermore understand, that DBIx::Recordset inherits from DBI so you can always call retrieve the underlying DBI database handle and make DBI method calls.

        And princepawn, I sincerely hope that you read this, get off your high horse, and learn something useful. what high horse? I do agree with your discussion of misuse of globals... and the newer version of this article corrects this, but under editorial pressure to get the article out the door fast, I left the old code in there. I have learned a lot about globals since then, especially from you. As a result, all my programs use strict; use warnings; use diagnostics;

        Your entire idea of selecting * from a table and having your code figure out how to handle that is a stupid idea. I spent a good deal of energy in that post trying to explain to you why that was bad and what a saner approach is. "Stupid idea" is a qualitative, unsubstantiated assertion. Maybe you could instead show why it is a bad idea and show all of us what would be better. I am afraid that you are under-informed here, and part of it has to do with how I presented Recordset perhaps. The fact is that binding a typeglob to a select leads to a tie of a scalar, array and hash within the namespace of that typeglob. The scalar is used for object-oriented access to the table, the array is used to (lazily and memory-efficiently) iterate through the set of records matching the query and hash is used to access the columns of the current records in the recordset. This is in contrast the the manual decision-making required with DBI in which you must decide whether to get your data back as an array, arrayref, or hashref. So the bottom line is the process is automatic.

        Apparently you didn't listen. Somehow I am not surprised. You don't appear to spend much time listening. You think that the rest of the world should listen to you though. Then why waste your energy talking to what you believe is a brick wall? I don't ignore people, I am here, particularly in this thread, to hear WITH LIVING PROOF AND EXAMPLES, of what you think about the two modules and their relationship and I would like to hear about other related modules, such as DBIx::Abstract and BingoX::Carbon. What is your goal in responding to me? I am perplexed

        The fact is that right now you clearly have so little appreciation of basic principles of how to program that I don't care about your opinion on what modules you like. You simply lack the skill level necessary to make judgements on that that I would find even remotely useful. Who was that talking about a high horse? <grin>

          This will be heated, but I really do not intend it to be a flame. However those who like to skip heated discussions should skip this post. If you think that heated discussions are always inappropriate for the Monastery, please drop a -- in the bucket first. But I actually think this is going somewhere. I believe this is constructive.

          (UPDATE In case it isn't obvious, this is a reply to princepawn. "You" means princepawn, not the whole monastery.)

          First of all I don't care whether you disagree with me. I am giving you my professional judgement of what the facts are. Take or disregard as you will. But as far as you disagree with my rant, I believe you either are mistaken or are missing key details.

          For instance a case where you are out and out wrong is that repetitive code is sometimes necessary for efficiency. If you really need the efficiency of repetitive code, then preprocess your code to autogenerate the repetitive stuff. But the code that you maintain should avoid having that kind of fragility.

          For instance a key detail you miss is that I work in a small company. While there is considerable differentiating of what we do with data, there is little differentiation between what data we work with and what access we need. Therefore everything you said about authentication and permissions is completely irrelevant to me.

          Now I admit it. I find dealing with you very, very frustrating. You are obviously not a stupid person. You go out, put a lot of work in, learn about a lot of stuff. But you really seem to have no appreciation for how to rate what you learn, what principles to pay attention to, what goals you can aim for. I covered part of this in Re (tilly) 1: Topics in Perl Programming: Table-Mutation Tolerant Database Fetches with DBI.

          Perhaps I can summarize it by saying that you appear to be motivated by the desire to accomplish neato things in a gee whiz manner. That may be fun, but that isn't what software engineering is about. What software engineering is about is managing complexity. That does not mean avoiding complexity, although it is good to avoid unneeded complexity. That means that as programmers we are asked to accomplish complex tasks, and over time we will be asked to both build on what we did, and modify it often at the same time! This is inherently hard and inherently leads to confusion.

          The aim of software engineering is to make dealing with these necessarily complex situations a managable task. Being straightforward is good. Being straightforward to the point of having an unwieldy amount of code that has to be maintained just so is bad. Hiding information under an interface is good. Having interfaces multiply like rabbits so that nobody can ever learn the system is bad. Not reinventing wheels is generally a good idea. But avoiding doing so when the wheel doesn't work, is inappropriate, or leads to the above issue about having too many components is bad. Having idioms, ways of saying things that are instantly recognizable and allow people to "chunk" code to higher levels is good. Turning your idioms into cut and pasted mistakes scattered through your code is bad. Having policies and procedures and change control is good. Having the paperwork reach a point where you do not get the job done is bad.

          Every good thing you can do has a corresponding cost. The key is to understand both the benefits and the costs so that you can evaluate situations, understand the trade-offs, then find and implement an appropriate solution. That is what we are paid to do. We have jobs because somebody thinks we can be trusted to do that, and we should try to do it well.

          However you have not a hope of succeeding unless you understand that that is the goal, it is the point. It is what is important.

          Now you are not stupid. You know that, I know that. I honestly believe that you are capable of staying aware of this principle, and paying attention to the trade-offs. Apply some of that desire to go off and learn lots of stuff, go read some stuff by Steve McConnell. But most importantly, just constantly try to evaluate how every tool and idea you run across helps and hinders (they all do both) the ability to manage the inherent complexity that we have to work with. Think less about what you want to do, and how you are going to do it. Think about why you are doing things, and think through ways it can go wrong.

          I really believe that you can do that. I would love to see you do that. I promise you that if you do do that, you will improve. You will develop a sense of what things are design mistakes and what are good ideas. You will develop an ability to anticipate what is going to lead to trouble and head it off. You will be able to write code that isn't painful for people like chromatic and myself to read. Some of these things will happen faster than others.

          But until you do that you will continue to crank out code, designs, and questions that make good programmers wince. And without you knowing how to judge good from bad, we will continue having trouble explaining to you the whys and wherefores of where we disagree with you.

    (redmist) Re: DBI vs DBIx::Recordset vs princepawn vs chromatic
    by redmist (Deacon) on Mar 08, 2001 at 12:58 UTC

      princepawn: I have seen some good posts from you lately, and think that it's great that you have had such a great turn around, but this post left me with a bitter taste in my mouth. I downvoted princepawns post for one reason: private issues belong in the private domain (see this post). I think that if you have an issue with chromatic or anybody else, you need to bring it up with them, and them only (unless, of course, it gets to the point that an arbitrator or mediator is needed).

      A side issue that I would like to agree with danger about is the fact that once something is on the Net, a mailing list, spoken, whatever, it's out there! Written or spoken statements are the same as ideas or thoughts in that once released into the world, you can never put them back in your head, and forget about it. The world is forever altered, no matter how significant (or insignificant) the alteration.

      redmist
      Silicon Cowboy
        I think you are right. I mixed my interest in hearing about people's feelings on various database modules with my disgust for the lack of protocol followed by chromatic and the Perl.com editor.

        Now regarding my open publication of vroom's squelching of my right to have my picture in the upper-right corner, I am afraid that certain things when said in private must become public. And since what he did is truly hinting at cultish-ness (you must remold yourself to be liked by everyone else before I will publish your picture), I felt I had a right to bring it forward.

          Please view source if you wish to see original text.

          Edited 2001-03-09 by Ovid.
          The post has been commented out (i.e. <-- post goes here -->). I didn't delete the comments as I don't care for direct censorship.

    Re: DBI vs DBIx::Recordset, etc.
    by footpad (Monsignor) on Mar 09, 2001 at 03:25 UTC

      Like PsychoSpunk, I've been following this thread with some interest, though I'm withholding certain votes until I see chromatic's full response to the original article.

      In reading the various replies, it occurs to me that perhaps we should step back a moment, take a deep breath, try to pull together a few recurring themes, hopefully without offending or angering any participants so far.

      If I'm reading things correctly (and I freely confess that I may not; I've been trying to reduce the caffeine and, well, withdrawal and all), you have three main concerns:

      • It would have been courteous to discuss the merits of your original article privately before presenting them to the general public.

      • You find DBIx::Recordset to be a far superior way to populate databases than other approaches, including (but not limited to) handing rolling INSERT queries using hashes (or slices), forcing all Perl programmers to build general subroutines for common atomic operations, calling the DBI API directly, and so on.

      • You're more than a little frustrated that criticism being levelled against your example code, as opposed to discussing the merits of the previous point.

      With regard to the first point, I agree with your sentiment. In an ideal world, you would have been given the opportunity to discuss the technical merits privately before having them aired publically. As you're well aware, though, this isn't a perfect world and different people have different ideas about professional and personal courtesy. As I've said previously, the only thing we can really do to improve the situation is to demonstrate these qualities ourselves. As one Master discussed many moons ago, it is better to treat others the way you would have them treat you.

      You and I have previously discussed the difficulties with effective online communication. Other monks have tried to suggest similar ideas.

      Please understand; I'm not trying to dredge up the past or be offensive. I'm trying to offer some food for thought (this *is* a meditation, after all). Tilly raises some valid points with regard to listening and I think it's very important that you not let your frustration over certain points prevent you from seeing and meditating on those points.

      Personally, I'm glad you chose to rejoin us and I've been pleased to be able to learn from your recent posts. I hope you'll continue to contribute to the Monestary and help others learn from your experiences. I believe you are a valuable member of the community. Indeed, one of your threads helped me though a problem I was working on some time back (though I should update that node to include my latest incarnation of that project).

      I would offer one piece of advice that chipmunk recently offered me during a CB discussion. It may help to tone down the rhetoric a few notches. It may even help to use a trick I use at times: compose the message off-line, let it sit for a couple of hours, and then revisit it. You may find that certain phrases or choices that aren't completely relevant to the points you really want to make.

      I do think danger raises an important point regarding the criticism given to authors. In certain cases, generally columnists for certain titles, it's possible to allow the author to respond before publication. Apparently, the editor of that list you mentioned doesn't operate that same way. There's not much to be done, save perhaps by emailing said editor and (as nicely as possible) describing your disappointment. Perhaps you'll make a difference, perhaps you won't. But at least you tried.

      Also, keep in mind that, in many cases, it's difficult or impossible to allow authors the opportunity to review criticism before publication. You mentioned deadline difficulties when composing your article. All editors face the same problems on a magnified scale. I noted, for example, that at least two typographic errors appeared in your article. This suggests that the editor that approved its posting was under similar pressures.

      There are a set of skills all sucessful authors learn, such as writing actively, careful selection of supporting material/arguments, trimming unnecessary prose that does not directly support the goal of the work at that moment. (You can tell these are skills I'm still learning.) You only learn these through through practice and experience. You may wish to continue to practice those skills, much the same way you continue to practice your Perl skills by writing more code. You won't get everything published, but the act of writing an article stretches and improves your skills.

      Taking a time out between the composition and the submission of an article may help in this regard.

      In previous posts, I've mentioned that I have an interest (and degree) in Drama. I've acted in several community theatre productions (~50) that have been subsequently reviewed in local papers. I've gotten good notices and I've gotten crushing ones. It took some time for me to learn that all feedback is valuable. When I was able to accept that person's opinion as just that and to actively look for ways to improve my performances through the feedback provided in the negative notices, my acting and my reviews got better. I'm no Olivier by any means, but I have my moments.

      In a way, you're going through a similar process in your participation in this community and the larger Perl community. You clearly want to help and you clearly have information to share. You're also making clear progress; please don't stop now.

      I cannot comment on your second point, primarily because you have far more experience with DBI than I do. Your article did succeed in one way, however. It let me know of another way to do it. I would have written it differently, but that's to be expected. (I'm not sure, but I believe that both arturo and PsychoSpunk mentioned similar results.)

      That said, I believe it's important to point out that you and I each have different experiences. This doesn't make you right and me wrong or vice-versa. If we allow it, it can help us grow as individuals, as professionals, and as Perl programmers. Every monk can learn and (except with one or two exceptions) has learned from their peers. This can only happen, however, when we approach each other's words and ideas with an open mind, one allowing and celebrating alternate points of view. I believe this is part of what tilly was trying to express.

      This is also part, I believe, of the unspoken messages in other replies. Remember the Perl mantra: There Is More Than One Way To Do It. By sharing our experiences and the ways we've done it, we learn more about Perl and more about the art of programming in general. Please try to take all feedback, no matter how heated or abusive, in that light. Now that NodeReaper is slouching about, you'll rarely get flamed simply out of spite.

      I say this because tilly is trying to help. He's also trying to help the lurkers that read your meditation after it's scrolled off Newest Nodes. Chromatic, likewise, is trying to help. As near as I can tell (having not read his original reply), he's trying to make sure you've thought things through and he's also trying to prevent those that follow from believing that there's only one way to put data in a table.

      You say that you have thought things through. Great; I'll accept that. Consider, though, why that thought wasn't clear in your original presentation of your ideas. Personally, I can see your enthusiasm for DBIx::Recordset, but I don't fully understand it. Clearly, both chromatic and tilly missed it, too. It might have helped to provide more details about why you dislike the DBI API. You gave some; more might have helped. That's only one suggestion; others are certainly possible. In any event, remember that "obvious" and "intuitive" mean different things to different people. What's crystal clear to you may be impenetrably opaque to me.

      There are a lot of unique ideas in our community. Each is appropriate for certain cases and completely useless in others. Part of the TIMTOWTDI mantra obliquely refers to the programmer's responsibility to review each solution with an eye for the current context, to find the right idea for the specific context in question. You've made the same point in other posts. I think you agree with these folks more than you may realize.

      With regard to your final point, I have to confess I agree with the sentiments that you might have chosen better example code. Yes, it does illustrate a good point, but it may have helped to clearly note in the text that your examples are (and were) extreme cases that should not be used in a production environment. Heck, I've awarded you brownie points for a) confessing that it was from one of your older applications and b) providing a second example based on your current experience level that illustrated the same point. I realize you suggested that obliquely, but I think those points could have stated more clearly.

      Again, the time out trick may help you with that in future articles. I know you were under deadline pressure, but there's nearly always time to review content before submission. As an example, I wrote the rough draft of this reply, saved it, and then went to a dance lesson. When I returned, I edited it as carefully as I could. While I'm sure there are typos and more to be cut, they're a lot fewer and the points are a lot clearer (I think) than they were in the rough draft. Even if you do nothing more than takes 25 minutes to get a sandwich and walk around the block, as you did during yesterday's CB discussion, it will still help clear your head and focus on the actual message you want to communicate.

      Please understand I am not trying to offend, upset, anger, frustrate, or otherwise raise your blood pressure. Like chromatic, tilly, and the other monks, I'm trying to help. While my technical expertise with Perl cannot hope to match that of the masters or even you, I am trying to share some lessons I've learned in my life and while serving in the Monastery.

      I hope they help...I think they can.

      --f

      P.S. Minor feedback: I found your reply to tilly a little hard to read because your normal style seems to italicize quoted material from your reply, which appears in normal, not bolded text. Just an FYI, that's all.

        I do want to help. If I didn't then I would just walk away and I wouldn't spend energy typing.

        What I see is someone who puts in a lot of work and learns lots of facts, but is not integrating all of that together into becoming a good programmer. I want to see that transition, and I felt it necessary to make it extremely clear that there was something being missed, and then to try to figure out and explain what I thought the cause was...

    Re: DBI vs DBIx::Recordset (or Ghosts in the Machine?)
    by PsychoSpunk (Hermit) on Mar 09, 2001 at 13:17 UTC
      princepawn I'm still in no position to argue who is correct. I haven't used the DBIx::Recordset, so this reply will primarily be a critique of your article. I think that your position on the DBI is a bit hasty to point out detriments that may be simply ghosts in your machine.

    Log In?
    Username:
    Password:

    What's my password?
    Create A New User
    Node Status?
    node history
    Node Type: perlmeditation [id://62820]
    Approved by root
    help
    Chatterbox?
    and the web crawler heard nothing...

    How do I use this? | Other CB clients
    Other Users?
    Others chanting in the Monastery: (14)
    As of 2014-10-20 19:16 GMT
    Sections?
    Information?
    Find Nodes?
    Leftovers?
      Voting Booth?

      For retirement, I am banking on:










      Results (89 votes), past polls