Beefy Boxes and Bandwidth Generously Provided by pair Networks
Problems? Is your data what you think it is?
 
PerlMonks  

Re^3: Seeing Perl in a new light: Epilog

by TGI (Vicar)
on Sep 25, 2010 at 18:56 UTC ( #861987=note: print w/ replies, xml ) Need Help??


in reply to Re^2: Seeing Perl in a new light: Epilog
in thread Seeing Perl in a new light

Speed is a valid concern. But before you get to worried about it, try it the change and see if it is actually a problem. It may not be as slow as you think. And as my mom always said "Don't borrow trouble." But if speed is a problem, there are ways to move forward.

Two common solutions exist for the problem of having to open and parse too many files. 1. Use a database 2. Pre-process the files and cache the results. I'm just guessing, but I believe that you have zero interest in digging into the world of databases right now. That leaves preprocessing.

Your current method is a form of pre-processing where you are the processor. You manually convert the list of monsters in your CSV file into a group of files. Then the menu script looks into the directory and makes the menu based on the file names. When you add or remove a monster, you manually add or remove a .pl file.

Instead of this manual process, have a script that does whatever needs to happen to generate a data structure that represents your menu. Probably series of nested arrays. Then save that data structure to a file, using Storable is probably the simplest and best way since it is core and fast.

Now when you want to generate the HTML for your menu, instead of redoing the work of processing all the files, simply load the data from your file and generate html.

# Generate this by doing what you do now to make your menus. # Then save it to a file by using Storable's lock_store function: # use Storable qw( lock_store ); # # lock_store( \%menu_data, 'menu_file.store' ); # In the code you use to render your html menu, load the data from # the file using Storable's lock_retrieve function; # use Storable qw( lock_retrieve ); # # my $menu_data = lock_retrieve( 'menu_file.store' ); # Here is an example of a data structure that compactly represents you +r # menu. # # Item format [ 'Text', Link, children ] my $menu_data = [ [ 'About', undef, [ ['About me', 'About/About_me.pl'], ['About my user names', 'About/About_my_usernames.pl'], ['About this site', 'About/About_this_site.pl'], # And so forth ], ], [ 'Books', undef, [ ['Fiction', 'Books/Fiction.pl'], ['Non-fiction', 'Books/Non-fiction.pl'], ['Role-playing fiction', 'Books/Role-playing_fiction.pl'], # And so forth ], ], [ 'Fiction', undef, [ ['My Life as a Witch', 'Fiction/My_Life_as_a_Witch.pl'], # And so forth [ 'Erotic fiction', undef, [ [ 'The Angel', 'Fiction/Erotic_fiction/The_Angel.p +l' ], [ 'The Assassin', 'Fiction/Erotic_fiction/The_Assa +ssin.pl' ], # And so forth ], ], ], ] ];

This approach should lead to a couple of benefits:

  • Less manual work - any number of updates can be handled by simply running the preprocessor script.
  • Simple HTML generation code. You have several different ways that you represent data on your account. Each of these must be processed in its own way. My guess is that each different sort of data processor directly generates HTML. This means that any changes to your desired HTML output must be repeated in many places. By generating a standard structure you can consolidate all of the code that HTML generation for menus in one place, you make changes to the final HTML easier. Also, this approach allows you to produce multiple distinct outputs from the data structure, a possible real world example would be RDF generation.
  • You can break the relationship between file names, URLs and document titles.

I am assuming that you can write code to traverse the $menu_data structure. If not, let me know and I can demonstrate the required concepts.


My comments here are based on my understanding of the following statement. If they don't make sense, then I probably misunderstood you.

Currently, the Monster directory has a .pl file for each monster. Those are loaded into the menu script.

I understand this to mean you scan the file names and use those to populate the menu.


TGI says moo


Comment on Re^3: Seeing Perl in a new light: Epilog
Select or Download Code
Re^4: Seeing Perl in a new light: Epilog
by Lady_Aleena (Chaplain) on Sep 25, 2010 at 21:15 UTC

    If you had caught me before I wrote any part of Base::HTML, your way could be used. However, what you are suggesting is that I totally destroy everything I have now and start from scratch. That fills me with a lot of anxiety. It has taken a lot of time to get to where I am now, and to start from the beginning for the second time makes my stomach knot up. I would have to delete everything and start with a blank slate and that is just not palatable. Base::HTML is part of everything I write to display scripts in a browser. Base::Menu would have to be tossed, and that was the key to making the site possible in the first place. The code for that took me a long time to get. I will muddle through with my current structure even if it isn't ideal.

    I know you are doing your best to explain this concept to show how it would make my life easier, however I don't want to spend several more agonizing years trying to get it to work. With everything hinging on those four Base:: modules, a rewrite is only possible if I delete them and everything dependent on them and start all over. I just can't do it right now.

    There are non-code real life factors that are clouding my judgment at the moment too, making it doubly hard for me to see any silver linings. I'm sorry that I can't get the courage up to do it your way, but my way is easy enough to understand for me. I am still using ANSI encoding instead of the more popular UTF-8 encoding because I have encountered too many problems with that conversion.

    I hope that someone who is just starting out sees your advice before they get too deep into a structure that is like mine. Hopefully, it will keep them from making my mistake of having everything so dependent on everything else. Thank you for trying to help me understand.

    Have a cookie and a very nice day!
    Lady Aleena

      The first rule (or at least an important one) of software engineering is that a complete rewrite is never as good an idea as it seems.

      You are correct to be concerned about making any moves that lead to a complete rewrite.

      Look for ways that you can work these ideas in edgewise, at the margins. Move code around, change what happens behind the scenes, but keep the interfaces the same.

      Having everything going through those 4 modules may be what makes it possible to change.

      I'm willing to take a look at your Base::* modules and see what I think is reasonable to target for refactoring.

      For example, to use my suggested approach for menus, you can start by changing your existing print_menu() in Base::Menu.

    • First put the logic that traverses your website into one routine, call it build_menu(). Build the menu data structure in build_menu().
    • Simply insert build_menu() at the top of your print_menu() routine and let it create the structure on the fly, exactly as it does now.
    • Then modify print_menu() to use the results of build_menu().
    • Now you can make a separate script that uses the build_menu() routine to build the menu data and save it to a file.
    • Modify Base::Menu::print_menu() to use data loaded from the file.
    • Now you can modify the build_menu() routine to handle the case where you have an all-in-one file, for example, Monsters
    • Now you can convert each directory to use an all-in-one approach.

    These steps don't require a rewrite, and at each part of the process you never have a broken website.

    Also, while it may have taken you several years to get where you are now, most of that work was not in the production of code, but rather, the acquisition of the knowledge required to produce the code.

    I think you may find that a better architecture will pay off handsomely. A good structure will come together with much less effort than a patched one.

    BTW, don't worry too much about UTF-8 until you need it. I haven't needed to worry about it, so I haven't bothered to learn anything other than the basics. I know that when I need to know, I will be able to find the resources I need to make it work. That is enough.


    TGI says moo

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://861987]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others studying the Monastery: (6)
As of 2014-08-31 05:01 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    The best computer themed movie is:











    Results (294 votes), past polls