Beefy Boxes and Bandwidth Generously Provided by pair Networks
go ahead... be a heretic

Reflections on the design of a pure-Moose web app...

by sundialsvc4 (Abbot)
on Mar 23, 2009 at 03:26 UTC ( #752486=perlmeditation: print w/replies, xml ) Need Help??

I just finished a rather interesting – one might even say rather elegant – web application that I think has a lot of future in it. I'd like to briefly describe that design here and see what you think. (Full disclosure: I'd be even more delighted if you want stuff like this and if you like telecommuters!) :-D

The app is built, as it were, on the Mojolicious framework, partly because that framework is small and mostly because this application is basically being done entirely with Moose object-oriented programming. But, one of the “rather different” attributes of this application is... “it's not all about the framework.” This application design is really, from stem to stern, all about the objects.

What Moose really gives you, really for the first time, is a first-class object structure that allows you to leverage a lot of existing (off the shelf) functionality in very creative ways. The key to this seems to be especially in its concept of Roles. For instance, the single statement: with Storage ( format => 'JSON' ); gives any object (or any descendent thereof) the instant ability to be reliably serialized and unserialized ... and you don't need to do anything particularly beyond that for “it just works.” When you combine this with other stuff like MojoX::Session, you have persistent objects without the headache.

Many web-designs seem to be very “database centered,” with the most-extreme example of course being "CRUD" designs. But really, an (SQL) database is really just another expression of “persistence.” The database is a source of information that objects may refer-to when building themselves, and it is also something that the objects may from time to time update. But it isn't the center of the application-user's world ... nor should it be the center for the designer's.

This particular application's “world” is a seminar registration system. This world therefore contains a slew of persistent, serializable objects – I call them Things – such as:   Seminar, Person, Invoice, CreditCardTransaction, Instructor, Venue, and Seat. Importantly:

  • A Thing has no direct correspondence to any particular SQL table, although it may load its initial information from one or more tables, and it may likewise update them.
  • Once a Thing has been instantiated, it can be serialized into the session-data (automagically!) and retrieved at any future point. (I built custom methods into the application's MojoX::Context object to easily retrieve the base Things. When one Thing refers to another, and all of them are serializable, all of them are magically serialized and restored. “It Just Works™.” Sweet! So, you're not constantly hitting the SQL database to build an object once you've already built it.
  • Any suitable form-handling and HTML-templating system can be used, and since Things are well-behaved objects you can simply refer to them in your forms and templates. They work, when used that way, just as they do when accessed in any other situation. If your app uses AJAX or SOAP or what-have-you, “Things still Just Work.”
  • The resulting logic is extremely orthagonal and expressive. You find yourself, a little ways down the line, asking... “is this really all I have to say here?” And the answer, in most cases, is “yes.”

I find that this application design really rattles the venerable foundation concept that we call, “MVC.” It rattles that foundation so much, in fact, that several of the letters fall off... especially “M” and “C.” Although various forms of data-model do exist, the application isn't centered around data anymore. Nor is it centered around an application-specific “controller.” My app has a number of scheduled scripts, for example, that are able to use these Things in exactly the same way as the web-app code does. The rules are not different. The web-app does have controller code which drives a particular HTML form (or subform), but those controllers (and their parent classes) are almost reduced to GUI-style “button pushers,” which of course they are.

Over the years, I have dealt with a lot of “web application code” that I really felt had zero future. (Especially a nightmarish year spent with an e-commerce company's twelve year old code ... they should have known better.) This code really seems to be different. It's loosely coupled, and I think that it will therefore continue to “have a future” for many years to come.

What do you think?

Replies are listed 'Best First'.
Re: Reflections on the design of a pure-Moose web app...
by ELISHEVA (Prior) on Mar 23, 2009 at 19:20 UTC

    I also was having a bit of trouble following your post, but it sounds to me like the "ah-ha" for you is the idea that an object is something separate from its representation - whether that representation be JSON, relational database tables, a Moose object, or a standard Perl object. By turning storage into yet another role, Moose has given you the ability to mentally decouple your notion of "thingness" from its various representations.

    However, as BrowserUk hints at, this is not something specific to Moose, or even this application of Moose. Once you have latched onto the idea that the representation is not the "Thing", you are set free in many ways - even from a specific OOP framework (which is itself yet another representation). One thing that has come clear to me over the years is that a very great deal of programming is about representational changes: moving round and round, back and forth from visual image on the screen or browser to representation in program memory to database records and SQL statements to IPC stream to various document formats (XML, PDF, MSWord, RSS feeds, etc, etc) and back again.

    Best, beth

      The “ah ha” is not so much the idea, but how cleanly Moose (and Moose roles) allowed that idea to be expressed in Perl. The speed and reliability of the development was also a nice big win.

      (Knowing that Perl-6 will be plowing the same furrows, was also nice.)

      Yes, you're right:   there aren't so many great new ideas in this world; just good, serviceable implementations of them that are found to be suitable to a particular occasion.

      To say that “a very great deal of” programming is devoted to representational-changes ... that is an understatement! I've dealt with many applications that were positively tied-into-knots not only by the data representation but by its architecture. Apps (especially web apps built by amateurs in Drupal ... ;-) ) just sorta “start down the primrose path” of what is euphemistically but accurately called CRUD, evolving without a particular plan until ... wham! ... someone, for a perfectly legitimate and important reason, needs to restructure the database. You know the drill.

      We all talk about “object-oriented programming” and data representations, and of course I'm very much talking about the selfsame thing here. The implementation was fast and clean and reliable, partly because so much of it could be “out of sight out of mind,” and these design ideas, instead of making the code-modules cruftier and more complicated, actually improved< them. I could have done the thing in “traditional Perl,” but this was, in my opinion, substantially better.

        Did you find a (publically accessible) simple starting point for your architecture, or did you evolve it yourself? (If not, would you consider writing one? :)

        The whole thing sounds very interesting (despite my "non-WEB programmer" status), and I'd love to dig around in the guts of a simple but complete example of what you've described.

        Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
        "Science is about questioning the status quo. Questioning authority".
        In the absence of evidence, opinion is indistinguishable from prejudice.
Re: Reflections on the design of a pure-Moose web app...
by perrin (Chancellor) on Mar 23, 2009 at 10:59 UTC
    I'm not following this. Are you saying that you aren't using an ORM, or built your own? Are you just caching all of your objects in user sessions? Maybe a little description of the object lifecycle would help.
Re: Reflections on the design of a pure-Moose web app...
by sundialsvc4 (Abbot) on Mar 23, 2009 at 13:31 UTC

    Sure... the first post was “a bit of a late night ramble,” so let me explain a little bit.

    All of the database-access in the application does use an ORM tool: DBIx::Class. But the term, “object relational mapper,” in this case is entirely a misnomer. DBIx::Class is merely the tool that a Thing uses to avoid writing erroneous SQL queries.

    An object ... a Thing ... is not “a database table,” nor “a query” nor “a view.” It is ... a Thing. You can instantiate it at any time, and having done so, because it is with Storage and somehow directly-or-indirectly associated with a session, you can keep it as long as you like. Each web-page request, or each AJAX request or what-have-you, can get access to “any Thing,” just as it was, without thinking about it. (The application actually uses a separate SQL database just for session and Thing persistence.)

    In the application I've just finished, for example, a Seminar is something that you can sign-up for. The underlying database has several different tables that describe different kinds of seminar-offerings, but conceptually “a Seminar is a Thing that People can sign-up for.” It is not “conceptually, a database table or tables.” It is a well-formed, persistent Perl (Moose...) object, with methods to do any task and to answer any question that you may have with regards to it, anywhere in the application.

    When you are dealing with a particular Seminar, the user may or may not ask to look at “the complete brochure.” If he does, we ask the Seminar object for its brochure information ... which the object retrieves (and remembers in its own properties) if it has not already done so. Some volatile information, like attendee counts, is always retrieved from the SQL database on-request, but most of the information is retained in the object itself. Information that you have not yet referred-to (like detailed brochure information) is probably not yet retrieved and may never be ... but if and when you do, the Thing knows it doesn't yet have the information you seek and the Thing knows how to get it. To the object making the request, “It Just Works.™”

    The “shopping cart” is probably a better overall example. What is a shopping-cart, really? Well, it's a container in which you store things that you might eventually buy ... or it might be a container that you leave on Aisle 7, having walked out of the store. But it is a Thing that contains, among other things, a collection of other Things, and that can do things to that collection and/or answer questions about them. So, a Shopping-Cart, like all Things, is just a persistent, storable Perl object. And you work with it as a Perl object. Like all Things, it automagically serializes and de-serializes itself such that “it's always there when you need it.” When a sale is completed, the shopping-cart is responsible for “becoming a Real Boy” by creating appropriate entries in the SQL database.

    I believe that the real “win” of this approach is that you are neither modeling an application that revolves around its database, nor an application that revolves around being a web-site. Both the registration application and the internal staff applications will use the same set of objects (and their descendents). The same coding techniques can be used in any context. And if the SQL database structure changes (as it inevitably must do from time to time), those changes don't ripple throughout the code.

    The ORM, along with the database(s), has been kicked out of its center-of-the-universe position, and the web-framework has been reduced to being “just the engine that drives one-of-many possible user interfaces.” The central precept of this universe is: persistent Perl objects, built in Moose, which I call Things.

    Does this help?

      So it sounds like the thing you're doing that's unusual is to store objects as JSON in a denormalized sessions table instead of normalized database tables. That can be good for some things. In general, I don't recommend it. Here are some reasons why:

      • Databases work hard to provide a sane model for concurrent access. You break that by using denormalized storage. It's usually ok if the cached objects are all related to a single user or read-only, but the chances for lost updates and other problems become high for shared read/write data. They can also get stale and miss updates in the underlying tables.
      • Serialized perl objects aren't easy to search. If you have no normalized storage of your shopping cart and you want to find all the carts that contain seminar 75, you have to walk through all carts in the system.
      • One reason that people often design from the database up is because the database is nearly always the bottleneck and the schema design has the largest affect on your application's scalability and performance. Trying to hide or bypass the database tables can lead to designs that don't take advantage of the details of your database's implementation.
      • Large sessions often perform badly. They can easily end up needing to serialize/deserialize lots of objects that aren't even used by the current request.

      I don't mean to rain on your parade. Just pointing out some messes I've made before with sessions and caching of database objects. Apologies if I've misunderstood what you're doing.

        The application is built on top of a fully normalized and very well-constructed database. (That I didn't even have to construct! It was already there!)

        The advantage of the approach that I took in this application is that “almost all of the web application” did not have to be concerned with the vagaries of the structure of that database. The “Things” know how (and when...) to retrieve information from it, and to post information back to it. Having done so, they can retain the information, and make not-yet-permanent contemplated changes to it, without “hitting” the production database at all.

        I posit that such an approach provides for excellent separation-of-concerns within the application, and that it strictly avoids the problem of having the software design be subtly linked-to, hence dependent upon, the schema of that production database. Unstructured information of a reasonable size (i.e. a Perl object) can be stored separately for a reasonable time, in an altogether separate data-store, and then posted to the database only when, and if, necessary. Information is retrieved only on-demand, yet only the object need be aware of when such a “demand” has actually taken place.

        So... am I arguing any of your points? Absolutely not. Instead, I am just here to say:   “Hey! Looky! This approach worked really well for me!” It is a very noticeable improvement that is going to stay at the top of my tool-bag for a while.

        The distinct advantage that I found in using Moose was the ease with which the job could be accomplished reliably. The concept of “roles,” in particular, allows for a lot of useful functionality – in this case, particularly serialization – to be leveraged very easily and reliably.

        Since Moose is built on regular Perl 5.8, clearly anything that can be done in it can also be done without it, but this is a better mouse-trap.

      Isn't ORM a method for getting 'persistent Perl objects'?

        No, it is a way to take Objects and a Relational storage of some kind and Map them to one another. The (very well known) impedence mismatch between the O and the R makes truly generalize object persistence difficult.

Re: Reflections on the design of a pure-Moose web app...
by Your Mother (Archbishop) on Mar 26, 2009 at 21:33 UTC

    Wet blanket time. This does sound interesting but without code and docs to peek at it also sounds very much like the legacy system I'm working with now -- an object, meta-data repository API -- which is awful and strongly resistant to improvement or feature additions because anything that doesn't go through the API as expected will break something.

    So, while you write "it's loosely coupled," I read "it's entirely and completely self-bound." I don't argue that that's a design mistake or could even be beautiful and extensible allowing one follows its rules; without seeing it in action or seeing what it does when it hits a corner case or an unexpected feature request it sounds a bit vaporesque.

    Post some code when you can. :) I'd love to start to learn to do Moose/MOP correctly. I've only tinkered with it and I know I'm not getting as much out of it as it can provide.

Re: Reflections on the design of a pure-Moose web app...
by metaperl (Curate) on Mar 23, 2009 at 19:15 UTC
    Mojolicious doesnt seem to be written in Moose...
    use base 'Mojolicious'; sub startup { my $self = shift; my $r = $self->routes; $r->route('/:controller/:action') ->to(controller => 'foo', action => 'bar'); }
    I guess your app which uses Mojolicious is?


      And Mojolicious, as a framework, well... I'm not trying to sell it here. Did the job okey.

Re: Reflections on the design of a pure-Moose web app...
by xyzzy (Pilgrim) on Mar 24, 2009 at 21:04 UTC
    ++, This seems very relevant to my interests... I have a project that's mostly in a conceptual stage right now but some elements that it has in common with what you presented are the high-level generalization of all objects ("Things"), "automagical" integration of object data into a central persistent data structure, and independence from any specific controller. I'm interested to see and learn something from your implementation.

    Everything is true." "Even false things?" "Even false things are true" "How can that be?" "I dunno man, I didn't do it."
Re: Reflections on the design of a pure-Moose web app...
by jdrago_999 (Hermit) on Mar 27, 2009 at 20:40 UTC

    You know, or should know, that the multiple layers of abstraction between *your* code and the database will one day come back to bite you. Badly.

    It's good that you now see that CRUD is not the pinnacle of the web application user experience. However, it appears that rather than developing from the Database up, you've developed from the User experience down. You mentioned the problems you've run into when the database structure had to be changed...what happens when the user experience changes?

    Although you now see that Users do not necessarily need to interact with a poorly-disguised database layer (i.e. simple CRUD), there is often more to the relationships between "Things" than we typically express in our Perl code. That's why we use the database to help define those relationships. That's why we call it a Relational Database.

    Moose is no free lunch. While it does offer some grace and power, it does not abrogate the need for a clean database design and a sane MVC architecture. If you somehow saved yourself from writing the database and MVC, then great - that's sort of the whole point of using modules other people write. However, storing things in the Session or in RAM and just expecting things to Just Work Somehow is not how development works in the long run.

    At some point, your code will need to be changed. If you have 100% code-coverage from a well-designed set of unit tests, then you will be able to make those changes without worry that the changes you make have broken something else. If you don't yet have a full set of unit tests, now is the time to write them.

      That's why we call it a Relational Database.

      Nope, that's not why. It's called "relational" because the data is stored in the form of relations. Why those math folks called this thing relations I'm not sure. However, I do know that the relation in relational is not about relationships.

      The fact that a relational database can store information related information is an important part of what makes it useful, of course.

        Maybe there is a difference between why "we" call it a relational database and why "they" call it a relational database? :-)
A reply falls below the community's threshold of quality. You may see it by logging in.

Log In?

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlmeditation [id://752486]
Approved by planetscape
Front-paged by Arunbear
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others perusing the Monastery: (1)
As of 2023-06-07 21:30 GMT
Find Nodes?
    Voting Booth?
    How often do you go to conferences?

    Results (29 votes). Check out past polls.