Beefy Boxes and Bandwidth Generously Provided by pair Networks
Do you know where your variables are?

From database views to abstract SQL to NOSQL

by metaperl (Curate)
on Nov 15, 2011 at 14:46 UTC ( #938188=perlmeditation: print w/replies, xml ) Need Help??

I do not like NOSQL because I think relationally, but I've come to a realization due to remaining open to them:

Aggressive use of views in database-driven apps will lead to code that is easily selected via a simple SQL::Abstract call AND the query language for SQL abastract is very similar to what you might use in a NOSQL tool like Redis or MongoDB, thus making the transition easier, if it becomes necessary
Instead of having Perl code with complex SQL, the views immediately make your pure SQL simpler. Then you get away from SQL altogether with views and then you make your move to the performance advantages of NOSQL easily.
  • Comment on From database views to abstract SQL to NOSQL

Replies are listed 'Best First'.
Re: From database views to abstract SQL to NOSQL
by BrowserUk (Pope) on Nov 15, 2011 at 21:36 UTC
    I do not like NOSQL because I think relationally, ... Then you get away from SQL altogether with views and then you make your move to the performance advantages of NOSQL easily.

    You are kidding yourself.

    Firstly, no one "thinks relationally". Relational algebra is as an unnatural an act as free-diving. With self discipline and training, and huge gulps of the pure oxygen of pre-planning, a very few can become very proficient at it. But even they have a tendency to push their luck with disastrous consequences. The fact that we can all dip our heads in a few centimetres of water for a few seconds, just gives us false hope and encouragement.

    Secondly, allowing yourself to fall into the mindset that using views over an RDBMS will adequately prepare you for the move to a NoSQL (ie. Distributed DBM) system is to fall into the clutches of one or more of the Fallacies of Distributed Computing.

    Views are inherently consistent, and any update made through a view will either ensure that all tables underlying the view are updated; or that none of the changes will be made.

    With NoSQL (of most flavours), the different fields that make up a single logical record will be managed by entirely separate distributed nodes which may or may not be able to (or even attempt to) communicate with each other. Hence, an update that modifies multiple fields within a single logical record may succeed in communicating with some of those nodes and not others, which means that some parts of the update may be completed and others not. Updates requests to parts of that logical record that fail in their first attempt to reach the appropriate D-DBM node will (should) not be lost, and will eventually be actioned, (perhaps by fail-over to another backup node), thereby bringing the logical record back to a consistent state. But this may happen some time later, and your front end processing model has to account for this possibility.

    NoSQL requires a different mindset to RDBMSs. And initially, that mindset may appear to be somewhat laissez faire. But to get the best from it, means taking a different tack to the possibilities of failure. With RDBMSs, we have become used to the idea that failure is not an option, and any hint of failure is treated as a "cannot happen" event with the result that when it does, the software metaphorically throws it hands in the air shouting "I don't know what to do now" and just stops.

    The essence of NoSQL is that you don't wait for confirmation of success. You assume it and move on to the next customer. Failure will only be discovered -- or even noticed -- when another request is made -- probably by a different front-end server, possibly thousands of miles away in a different datacentre -- to retrieve the logical record, and when its constituent fields are gathered together from their D-DBM sources and reassembled, they are found to be inconsistent.

    At this point, the front-end doesn't just give up the ghost and stop, but rather anticipates this possibility and does something quite unusual. It tries gain. Or rather, it re-queues the inbound user request (eg. HTML get) and moves onto the next customer request. At some point in the near future it or another front-end will dequeue that request and re-try the logical record fetch. And with luck, the back-end queues of distributed updates will have caught up, and this time when the logical record is reassembled, it will be in a consistent state.

    And whilst the individual user may have noticed some delay in the response to his request, many other users will have received timely responses to theirs without having had to wait for the problem of one user who got in a few milliseconds before them to be resolved.

    As I said, it requires a different mindset. One that recognises that machines fail, connections go down and routing goes awry; that peaks and troughs of activity occur and sometimes the most statistically unlikely of coincidences will happen. By recognising these truths and expending far less effort on trying to guarantee that nothing will ever fail, it gives you licence to plan to deal with them when they inevitably do, in a manner that doesn't slow down your response to new requests. By making your front-end and back-end systems run asynchronously through the use of queues, and divorcing the need to detect inconsistencies from the write/update stage to the subsequent read stage, you give the back-end systems time to reach consistency before that subsequent critical read is reached.

    All the time you saved waiting to see if everything was okay, you can now spend doing real work. And along the way, you avoided needing to size your DB manager system to cope with an occasional-going-on-never slash-dot event.

    With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.
Re: From database views to abstract SQL to NOSQL
by jdrago999 (Pilgrim) on Nov 29, 2011 at 21:52 UTC

    I think of NoSQL as a gigantic Scripting.Dictionary from the Olden Days of Visual Basic. Except I can't find out what keys are there. And there's some kind of magic going on under the hood that may-or-may-not get all my nodes synced up with one another.

    NoSQL and RDBMS are as different as Apples and Footballs. Two different problem domains - and both solve their intended appointments well, when used correctly.

    Every few years there's a new database gimmick. Ten years ago it was "XML Databases" and "Object Databases" - but the salespeople I spoke with trying to sell them to my company was trying to convince me that they would replace traditional RDBMS like MS SQL Server, Oracle and the like. A decade later and now we've got NoSQL "databases" which are great for caching, but not really great for replacing your RDBMS.

    I say, "Use Only as Directed" and you'll be fine.

    About distilling your SQL::Abstract 'statements' into keys and caching the results from the RDBMS in a NoSQL database, keyed by some kind of hash of those SQL::Abstract statements - that's a great plan. So long as your data doesn't change much, otherwise you'll be playing whack-a-mole with cache invalidation.

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: perlmeditation [id://938188]
Approved by herveus
[thepkd]: Hi, in trying to create an Array of Hash of Array of Hash. You read it right. Is it possible to dereference it without using temps?
[GotToBTru]: what are temps?

How do I use this? | Other CB clients
Other Users?
Others lurking in the Monastery: (11)
As of 2016-12-06 13:14 GMT
Find Nodes?
    Voting Booth?
    On a regular basis, I'm most likely to spy upon:

    Results (104 votes). Check out past polls.