Beefy Boxes and Bandwidth Generously Provided by pair Networks
We don't bite newbies here... much
 
PerlMonks  

Module design for loadable external modules containing data

by Discipulus (Abbot)
on Oct 14, 2020 at 16:31 UTC ( #11122832=perlquestion: print w/replies, xml ) Need Help??

Discipulus has asked for the wisdom of the Perl Monks concerning the following question:

Hello folks,

The title shows well the confusion I'm feeling about the matter :)

For my Perl::Teacher project (see here and there) I plan to write a base class Perl::Teacher::Lesson and a Perl::Teacher::Course one aimed to just contain lessons logically grouped.

In my intention me and others will write and release various courses in the form of Perl::Teacher::Course::EN::PerlIntro (for example) containing many different lesson modules like Perl::Teacher::Course::EN::PerlIntro::01_foreword etc.

This is because the main Perl::Teacher object must be able to load a course and sequentially all its lessons at runtime.

But I'm very confused about how to write these main base classes. Basically each lesson will be full of steps containing discourses, assignements, questions and, more importantly, tests to be run against student's perl programs (see the proof of concept to have an idea).

The Perl::Teacher::Course::EN::PerlIntro::01_foreword module must be able to call methods from the main distribution but will contain a lot of data (texts and tests).

Basically Perl::Teacher::Course::EN::PerlIntro::01_foreword can just have a mere @EXPORT exporting a big array/hash of data to be processed by the main Perl::Teacher object, but this sounds a bit primitive to me.

How to provide a nice and easy to use interface to course writers?

I have no experience with Moo and friends, so if you want to provide some sketch please take the patience and explain it plainly.

I'm totally of the road? Complete redisign suggestions are also welcome if well explained.

L*

There are no rules, there are no thumbs..
Reinvent the wheel, then learn The Wheel; may be one day you reinvent one of THE WHEELS.

Replies are listed 'Best First'.
Re: Module design for loadable external modules containing data
by kcott (Bishop) on Oct 15, 2020 at 03:15 UTC

    G'day Discipulus,

    The title of your post struck a chord: I went through a similar design dilemma a couple of years ago. Although adding the data to the module code seemed like it would work, mixing (mostly static) code with (potentially volatile) data went against the grain.

    The solution I came up with used File::ShareDir to access the data files at runtime. Its companion module, File::ShareDir::Install, was used for installation.

    File::ShareDir::Install is used in Makefile.PL. It is not used or required in any of your runtime modules. The documentation has details and it's fairly straightforward.

    Here's a very rough, ASCII-art, UML representation of how I used File::ShareDir:

            +-------------+  [compose] +----------------+
            | ParentClass |<>----------| File::ShareDir |
            +-------------+            +----------------+
              A         A [inherit]
              |         |
      +-----------+  +-----------+
      | SubClass1 |  | SubClassN |
      +-----------+  +-----------+
          ^           ^         ^ [manifest]
          |           |         | 
    +----------+ +----------+ +----------+
    | <<file>> | | <<file>> | | <<file>> |
    +----------+ +----------+ +----------+
    

    The File::ShareDir - DESCRIPTION discusses some of the bad ways to tackle the design; e.g. using giant data structures or adding data after __DATA__. It then has: "The problem to solve is really quite simple. ...".

    I hope that's enough to get you started. I have to get back to $work :-(

    — Ken

      Hello kcott and thanks,

      maybe I'm too much confused, but the interesting module File::ShareDir you suggested to me does not solve my doubts. If I read it correctly (and just glanced it atm) it is used to share and expose pure data stored at install time somewhere in the filesystem and make the program to be able to find them at runtime. My situation is more complex, or I'm too blind to see a clean solution.

      Maybe a rough piece of pseudocode will show it better:

      # # modules of my distribution: # Perl::Teacher load_course (then in the program using this module I can call $course +->get_description) load_lesson (then in the program using this module I can call $lesson +->get_steps) Perl::Teacher::Course new creates a slot for description and one for [lessons] get_description get_lessons Perl::Teacher::Lesson new creates a slot for abstract and one for [steps] get_abstract get_steps # # modules outside my distribution, intended to be written by someone e +lse and installed separately # (note that these modules should have their own tests under /t folder + as any module should) # Perl::Teacher::Course::EN::PerlIntro isa Perl::Teacher::Course $__PACKAGE__::description = 'a fair intro to perl', $__PACKAGE__::lessons = [Perl::Teacher::Course::EN::PerlIntro::01_for +eword , ...] Perl::Teacher::Course::EN::PerlIntro::01_strictures isa Perl::Teacher::Lesson $__PACKAGE__::abstract = 'introducing the safety net' $__PACKAGE__::steps = [ # a simple text 001=>{type=>text,content=>'We now introduce strictures'}, # check_script_compiles is a method exposed by Perl::Teacher 002=>{ name=>'script compiles', action=>\check_script_compiles() } # a complex test to be run against student's code 003=>{type=>test,content=>{ name => 'strictures', # select_child_of is a method exposed by Perl::Teacher select_child_of => { class => 'PPI::Statement::Include', tests => [ ['PPI::Token::Word', qr/^use$/], ['PPI::Token::Word', 'strict'] ], }, hint => "search perlintro for safety net", docs => ['https://perldoc.perl.org/perlintro.html#Safet +y-net'], } } ]

      As you can see it is very ugly. I'd like to provide a nice and usable interface to course authors.

      One solution I have is to abstract as much as possible the lesson content and provide an higher level language to define a lesson, probably YAML will be enough. I dont like very much this solution because it will force the author to write YAML instead of perl code. If I go for YAML then File::ShareDir can be an option: the Perl::Teacher::Course::EN::PerlIntro simply write 01_strictures.yaml to the disk then the Perl::Teacher object load the course and all its yaml from the disk.

      Thanks for the help!

      L*

      There are no rules, there are no thumbs..
      Reinvent the wheel, then learn The Wheel; may be one day you reinvent one of THE WHEELS.

        You are correct in your assessment of File::ShareDir: it is predominantly for access to static data.

        Your modules should still contain program logic as usual; tests should still be written as usual. All of your static data (e.g. "discourses, assignements, questions") can be handled by File::ShareDir.

        I envisioned some sort of get_text() method implemented at the ParentClass level (from my rough UML example). So, your 001 line would become something like:

        001 => { type => 'text', content => 'strict_intro.txt' }

        Blocks of embedded text can be removed from the module code and replaced with filenames. These can be accessed along the lines of: get_text($hash{001}{content}).

        Now, your text can be available to multiple modules. Changes to that text — whether a simple typo fix, addition of new paragraphs, or a complete rewrite — need only be done in one place (i.e. strict_intro.txt). No code changes are required in any modules. This eliminates a very real source of potential errors: introducing new typos in one or more places; forgetting to modify one or more modules; breaking source code with some newly introduced punctuation character in the text; and so on.

        In addition, this will reduce the size of modules and therefore the memory footprint when they are loaded. This can be further improved upon with just-in-time mechanisms; for instance, only accessing strict_intro.txt when the student clicks on "Show the strictures intro" (or otherwise requests that information).

        "As you can see it is very ugly."

        Removing hard-coded text would certainly be a step in the right direction.

        Looking at your pseudocode, another improvement presents itself. You have your steps implemented as an array; the contents of that array is really a hash; the keys of that hash are a sequence of numbers which could easily be derived from the array index. That would look a lot cleaner as:

        [ { ... simple text ... }, { ... check script ... }, { ... complex text ... }, ]

        While I do appreciated that what you presented was pseudocode, this would be the type of thing to look for on the path to beautification.

        — Ken

        This is just my two cents and very opinionated but ...

        One solution I have is to abstract as much as possible the lesson content and provide an higher level language to define a lesson, probably YAML will be enough. I dont like very much this solution because it will force the author to write YAML instead of perl code.

        You're just getting started here. You can't even just tell them to write YAML because you're really creating a Domain Specific Language. It still won't be all of YAML, it will be a specific subset with expressions that mean something to Perl::Teacher but not YAML. (etc etc, I'm not trying to prove the point, just hoping you can see it)

        The other point I'd like to raise is that, despite OOP, in practice, we (programmers) almost always segregate data and code. IMHO, Perl::Teacher should take a cue from things like Text::CSV and XML::LibXML and process data rather than contain it. The code can live on CPAN and the data on github or whatever and just get pulled via https or whatever.

        I hope my post isn't discouraging. I think it's great that you're experimenting with an interesting blend of semantics and code.

Re: Module design for loadable external modules containing data
by Fletch (Chancellor) on Oct 14, 2020 at 16:53 UTC

    Completely off the cuff and haven't thought too deeply about it but:

    This (at initial blush) sounds like you want to define a role (maybe Perl::Teacher::Roles::CourseProvider) which enumerates what things you expect a "course" to be able to do (e.g. maybe course_abstract gives the overview what it's about; list_topics returns a list of topics covered in the course; topic_abstract( NAME_OR_NUMBER ) takes a name or index (in the list returned by the prior) and returns the details on that topic; ETC). The individual course modules would then implement those methods. If you do it right you probably could even then get a base parent class which would read YAML/JSON from the DATA section and concrete modules would inherit from that.

    </handwaving>

    The cake is a lie.
    The cake is a lie.
    The cake is a lie.

Re: Module design for loadable external modules containing data
by perlfan (Vicar) on Oct 14, 2020 at 16:44 UTC
    My recommendation is to provide content in a presentation neutral way (e.g., markdown, SGML, etc). The provide the tooling around it to render it. This puts the "output" of any kind on the same footing, be it PDF, HTML, or interactive commandline coolness. This data can still be encapsulated in the module (e.g, in a ./data directory, but provides the separation that you should have in content versus presentation.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://11122832]
Approved by davies
Front-paged by kcott
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others examining the Monastery: (4)
As of 2020-10-31 10:16 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    My favourite web site is:












    Results (287 votes). Check out past polls.

    Notices?