Beefy Boxes and Bandwidth Generously Provided by pair Networks
Just another Perl shrine

Re: Seeing Perl in a new light: Epilog

by TGI (Parson)
on Sep 20, 2010 at 07:14 UTC ( #860780=note: print w/replies, xml ) Need Help??

in reply to Seeing Perl in a new light: Epilog
in thread Seeing Perl in a new light

I am writing to you about your approach for multiple similar pages. I am going to focus on the Monsters section of your site, because that is the part I've been looking at and am familiar with.

You have a section on Role Playing, with a subsection titled "Monsters". You've got a CSV file with all the information for all the monsters you want a page for.

Right now, it appears that each monster has its own Perl script. So there is "Dark" for a dark centaur. And "" for a Daemar and so forth. If you were to add a new monster you would add the monster to the CSV file and add a "" script.

What makes much more sense is to make one script that handles all monster related issues. Call it Then when you want to access a list of all monsters, I've put together a CGI script that does exactly that. One script will handle all the monsters and also provide a summary page with links to detail view of specific monsters.

With this approach, to add a new monster, you simply add it to the CSV file. Everything else happens automatically.

Let me emphasize this key point; <bThere is no need to have a one-to-one relationship between pages displayed and scripts that generate them. A single script can produce many different pages.

Without any other messing about, here's a script that demonstrates how a single script can handle all your monsters.

use strict; use warnings; use CGI (); use URI::Escape; use Text::CSV; use Data::Dumper; use constant DATA_FILE => 'Monsters.csv'; use constant ATTRIBUTES => ( 'Climate/Terrain', 'Frequency', 'Organization', 'Activity cycle', 'Diet', 'Intelligence', 'Treasure', 'Alignment', 'No. Appearing', 'Armor Class', 'Movement', 'Hit Dice', 'THAC0', 'No. of Attacks', 'Damage/Attack', 'Special Attacks', 'Special Defenses', 'Magic Resistance', 'Size', 'Morale', 'XP Value', ); use constant SECTIONS => ( 'Appearance', 'Combat', 'Habitat/Society', 'Ecology', 'Variants', 'Note', ); use constant FIELDS => ( 'Monster', ATTRIBUTES, SECTIONS, ); use constant SUMMARY_FIELDS => ( 'Monster', 'Climate/Terrain', 'Frequency', 'Organization', 'Activity cycle', 'Diet', 'Intelligence', 'Treasure', 'Alignment', 'No. Appearing', 'Hit Dice', ); my $q = CGI->new; my @monsters = $q->param( 'name' ); print @monsters == 1 ? monster_detail($q, $monsters[0]) : monster_summary($q, @monsters ); # -------------------------------- sub page_start { # Generate the standard HTML for the start of a page. # Returns html text. my $q = shift; # CGI Object my $title = shift; # Title of the page. my $html = <<END_HTML; <html> <head> <title>$title</title> </head> <body> <h1>$title</h1> END_HTML return $html; } sub page_end { # Generate the standard HTML for the end of a page. # Returns html text. my $q = shift; # CGI Object my $url = $q->url(-relative=>1); my $html = <<END_HTML; <p><a href='$url'>Monster List</a></p> </body> END_HTML return $html; } sub monster_detail { # Format a monster detail page # Returns html text. my $q = shift; # CGI object my $monster = shift; # Name of monster to display my $monster_data = load_monsters( $monster ); return error_page( $q, "'$monster': Monster not found.") unless @$monster_data; my $html = join '', $q->header, page_start($q, $monster_data->[0]{Monster} ), gen_detail( $monster_data->[0], [ ATTRIBUTES ], [ SECTIONS ] ) +, page_end( $q ); return $html; } sub monster_summary { # Format a monster summary page. # Returns html text. my $q = shift; # CGI object my @monsters = @_; # List of monsters to summarize. my $monster_data = load_monsters( @monsters ); return error_page( $q, "No matching monsters found (@monsters)." ) unless @$monster_data; my $html = join '', $q->header, page_start($q, 'Monster Summary'), gen_table( $monster_data, [ SUMMARY_FIELDS ], { Monster => sub { my ($attr, $item) = @_; my $name = $item->{$attr}; my $esc = uri_escape( $name ); return "<a href='?name=$esc'>$name</a>"; }, }), page_end( $q ); return $html; } =head3 load_monsters Accepts a list of monster names to load. If no list is provided, all +monsters will be loaded from the data file. Returns an array ref containing hash refs, each containing data from o +ne monster. =cut sub load_monsters { my %wanted_monster; @wanted_monster{@_} = (); my $csv = Text::CSV_XS->new ({ binary => 1, quote_char => '~', sep_char => '|', }) or die "Error creating CSV parser: ".Text::CSV->error_diag; open my $fh, "<:encoding(utf8)", DATA_FILE or die "Error opening data file: $!"; my @monsters; while (my $row = $csv->getline ($fh)) { my %monster; @monster{ FIELDS() } = @$row; push @monsters, \%monster if !%wanted_monster || exists $wanted_monster{ $monster{Mo +nster} }; } $csv->eof or $csv->error_diag (); close $fh; return \@monsters; } sub gen_table { # Generate monster summary table. # Returns html text. my $data = shift; # Array of Monster hashes my $fields = shift; # Array of fields to use my $formatter = shift || {}; # Hash of subs used to preprocess +fields my $html = '<table><tr>'; $html .= join '', map "<th>$_</th>", @$fields; $html .= '</tr>'; for my $item ( @$data ) { $html .= '<tr>'; $html .= join '', map "<td>$_</td>", map { exists $formatter->{$_} ? $formatter->{$_}( $_, $item ) : $item->{$_}; } @$fields; $html .= '</tr>'; } $html .= '</table>'; } sub gen_detail { # Generate monster detail view # Returns html text. my $data = shift; my $attr = shift; my $sections = shift; my $html = '<dl>'; $html .= join '', map "<dt>$_</dt><dd>$data->{$_}</dd>", @$attr; $html .= '</dl>'; for my $section ( @$sections ) { my @paras = split /\\/, $data->{$section}; $html .= "<h2>$section</h2>"; $html .= join '', map "<p>$_</p>", @paras; } return $html; }

I know this code doesn't use your Base elements. I didn't want to download, configure and troubleshoot them. I thought it was more important to get this example together for you. You should be able to massage your standard infrastructure into this code.

As you do, be aware that it has got a big, fatal bug--I didn't define an error_page routine, you'll see the error if you specify a bogus monster as the name parameter. You'll need to define an error page routine, or replace it with one that already exists in your libraries.

The big theme of the discussion the other night was abstraction and generality. This is a good example of abstraction. Instead of one script for each monster, we have one script that can handle all the monsters by treating them as if they were identical. By doing so we can replace 20 scripts with one script.

Start by converting the monsters to this method. Then repeat with spell books and weapons. As you build similar scripts, you might find similarities that point to opportunities for code reuse. Even better you may be able to work on an even more abstract level and simplify your code even more.

First and foremost, you really need to move away from the SSI "one file per page" approach. Your life will be much simpler, and your stated goal of reducing your site size will be met.

TGI says moo

Replies are listed 'Best First'.
Re^2: Seeing Perl in a new light: Epilog
by Lady_Aleena (Curate) on Sep 24, 2010 at 02:34 UTC

    I have been thinking hard about your approach, and I don't think it will work. However, I may not have grasped the concept, so my thinking below may be completely wrong.

    You speak of creating an "all-in-one" script; and going with your example of my monsters, it would really slow up the loading of each page. Since the menu is generated solely by the directory and file structure of the site, every "all-in-one" would have to open up the data file(s) to populate that section.

    Currently, the Monster directory has a .pl file for each monster. Those are loaded into the menu script. However, a singular would require that Monster.csv be opened to create links to each monster akin to <a href="">monster</a>. For other sections, there isn't a singular data file, but a list. So, would have to go to the Spellbook data directory, read all the files there, and populate the list on the menu with <a href="">character</a>. I am not too keen on opening almost all of my data files or directories for every page loaded.

    Sample from menu

    <ul> <li class="inactive"> <a href="http://localhost/Role_playing/Monsters/Chaos_elemental-ki" title="Chaos elemental-kin">Chaos elemental-kin</a> </li> <li class="inactive"> <a href="http://localhost/Role_playing/Monsters/" title=" +Daemar">Daemar</a> </li> <li class="inactive"> <a href="http://localhost/Role_playing/Monsters/" t +itle="Dark centaur">Dark centaur</a> </li> </ul>

    After "all-in-one"

    <ul> <li class="inactive"> <a href="http://localhost/Role_playing/Monsters/ +os_elemental-kin" title="Chaos elemental-kin">Chaos elemental-kin</a> </li> <li class="inactive"> <a href="http://localhost/Role_playing/Monsters/ +mar" title="Daemar">Daemar</a> </li> <li class="inactive"> <a href="http://localhost/Role_playing/Monsters/ +k_centaur" title="Dark centaur">Dark centaur</a> </li> </ul>
    Have a cookie and a very nice day!
    Lady Aleena

      Speed is a valid concern. But before you get to worried about it, try it the change and see if it is actually a problem. It may not be as slow as you think. And as my mom always said "Don't borrow trouble." But if speed is a problem, there are ways to move forward.

      Two common solutions exist for the problem of having to open and parse too many files. 1. Use a database 2. Pre-process the files and cache the results. I'm just guessing, but I believe that you have zero interest in digging into the world of databases right now. That leaves preprocessing.

      Your current method is a form of pre-processing where you are the processor. You manually convert the list of monsters in your CSV file into a group of files. Then the menu script looks into the directory and makes the menu based on the file names. When you add or remove a monster, you manually add or remove a .pl file.

      Instead of this manual process, have a script that does whatever needs to happen to generate a data structure that represents your menu. Probably series of nested arrays. Then save that data structure to a file, using Storable is probably the simplest and best way since it is core and fast.

      Now when you want to generate the HTML for your menu, instead of redoing the work of processing all the files, simply load the data from your file and generate html.

      # Generate this by doing what you do now to make your menus. # Then save it to a file by using Storable's lock_store function: # use Storable qw( lock_store ); # # lock_store( \%menu_data, '' ); # In the code you use to render your html menu, load the data from # the file using Storable's lock_retrieve function; # use Storable qw( lock_retrieve ); # # my $menu_data = lock_retrieve( '' ); # Here is an example of a data structure that compactly represents you +r # menu. # # Item format [ 'Text', Link, children ] my $menu_data = [ [ 'About', undef, [ ['About me', 'About/'], ['About my user names', 'About/'], ['About this site', 'About/'], # And so forth ], ], [ 'Books', undef, [ ['Fiction', 'Books/'], ['Non-fiction', 'Books/'], ['Role-playing fiction', 'Books/'], # And so forth ], ], [ 'Fiction', undef, [ ['My Life as a Witch', 'Fiction/'], # And so forth [ 'Erotic fiction', undef, [ [ 'The Angel', 'Fiction/Erotic_fiction/The_Angel.p +l' ], [ 'The Assassin', 'Fiction/Erotic_fiction/The_Assa' ], # And so forth ], ], ], ] ];

      This approach should lead to a couple of benefits:

      • Less manual work - any number of updates can be handled by simply running the preprocessor script.
      • Simple HTML generation code. You have several different ways that you represent data on your account. Each of these must be processed in its own way. My guess is that each different sort of data processor directly generates HTML. This means that any changes to your desired HTML output must be repeated in many places. By generating a standard structure you can consolidate all of the code that HTML generation for menus in one place, you make changes to the final HTML easier. Also, this approach allows you to produce multiple distinct outputs from the data structure, a possible real world example would be RDF generation.
      • You can break the relationship between file names, URLs and document titles.

      I am assuming that you can write code to traverse the $menu_data structure. If not, let me know and I can demonstrate the required concepts.

      My comments here are based on my understanding of the following statement. If they don't make sense, then I probably misunderstood you.

      Currently, the Monster directory has a .pl file for each monster. Those are loaded into the menu script.

      I understand this to mean you scan the file names and use those to populate the menu.

      TGI says moo

        If you had caught me before I wrote any part of Base::HTML, your way could be used. However, what you are suggesting is that I totally destroy everything I have now and start from scratch. That fills me with a lot of anxiety. It has taken a lot of time to get to where I am now, and to start from the beginning for the second time makes my stomach knot up. I would have to delete everything and start with a blank slate and that is just not palatable. Base::HTML is part of everything I write to display scripts in a browser. Base::Menu would have to be tossed, and that was the key to making the site possible in the first place. The code for that took me a long time to get. I will muddle through with my current structure even if it isn't ideal.

        I know you are doing your best to explain this concept to show how it would make my life easier, however I don't want to spend several more agonizing years trying to get it to work. With everything hinging on those four Base:: modules, a rewrite is only possible if I delete them and everything dependent on them and start all over. I just can't do it right now.

        There are non-code real life factors that are clouding my judgment at the moment too, making it doubly hard for me to see any silver linings. I'm sorry that I can't get the courage up to do it your way, but my way is easy enough to understand for me. I am still using ANSI encoding instead of the more popular UTF-8 encoding because I have encountered too many problems with that conversion.

        I hope that someone who is just starting out sees your advice before they get too deep into a structure that is like mine. Hopefully, it will keep them from making my mistake of having everything so dependent on everything else. Thank you for trying to help me understand.

        Have a cookie and a very nice day!
        Lady Aleena

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://860780]
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others contemplating the Monastery: (5)
As of 2021-05-13 16:52 GMT
Find Nodes?
    Voting Booth?
    Perl 7 will be out ...

    Results (137 votes). Check out past polls.