Beefy Boxes and Bandwidth Generously Provided by pair Networks
Think about Loose Coupling

Overall Directory Organization

by Xiong (Hermit)
on Nov 10, 2009 at 04:02 UTC ( #806110=perlquestion: print w/replies, xml ) Need Help??
Xiong has asked for the wisdom of the Perl Monks concerning the following question:

What overall strategy do you employ when creating a directory structure for development, management, and delivery of a web application?

Over the years, I've employed several different strategies; I've become dissatisfied with each one. As my projects become more advanced and I attempt to formalize my development process, it seems that directory structure becomes more complex and opaque, when the ideal is to be straightforward and transparent.

Here are some of the most important goals I have in mind:

  • Orthogonality between my local desktop platform and my remote server. The same relative directory structure should work for both. That is, there should be some local folder projectname/ which corresponds to /home/myaccount/ on server. In the past, I attempted to maintain an exact mirror of www/ on server. I still want this but I also have seen the light and want to store files "below" www/ (those that don't need to be directly accessible via web, such as Perl modules). Still, from the point of view of any given script, the relative structure should be identical. This goes (especially) for test scripts.
  • Compatibility with my VCS. I'm using git-gui to manage version control. I need to decide which files to track, which means to settle on a particular folder as version control root. I want to have other folders within the project path, such as backups and scratch folders, which should be excluded from version control.
  • Compatibility with module-starter and CPAN. On the offchance that I attempt to upload a project module to CPAN, I intend to use module-starter and try to maintain correct Makefile.PL and .t files, etc., for each module. A difficulty is that module-starter wants to put each module (with its related files) in a directory all by itself. For project development purposes, all the modules should(?) be together, in the hierarchy implied by their names (Project::Base, Project::This, Project::That). For that matter, if ever I do distribute the project, all its modules should go together. Also, the tests should be available to me during development and not buried in deeply nested folders.
  • Flat hierarchy. I abhor deeply nested directories. There's merit only up to a point; then nesting gets to be an issue. I especially do not like repetitions of the same word in multiple path parts, such as lib/project/projectname/lib/project/module. When these long paths are displayed or must be typed, they lead to confusion.
  • Narrow hierarchy. I don't like to see my project distributed among dozens of folders at the same level. After awhile, it gets to be a chore simply to remember where anything is. I realize that there is a tradeoff: One either has deep nesting, a large number of nodes at each level, or a large number of leaves. I prefer more leaves.
  • Maintainability and ability to branch. I like to duplicate the main development code and "stuff" intermediate working versions out of the way, for possible disinterment later. I like to create short what-if scripts; I often mine them for production code. I want all the files I create to be accessible but not in the way. That is, one directory should include only the deliverable project, in its current state.
  • Add 2009-11-10: Compatibility with taint mode. I overlooked this in my original post as taken for granted (but to ass-u-me...). Since taint mode removes cwd from @INC, this must be considered when planning directory structure.

All the approaches that I've used have led to some nasty choppery-about. Perhaps when it comes time to upload to server, I have to go hunting for local files and do mulitple cd on server to put everything where it's supposed to go. Perhaps I find myself copying local files mulitple times, or dropping a lot of symlinks around. Often, I spend far too much time just digging around, trying to figure out where I put some file. I feel a more disciplined approach to directory structure would help.

Directory structure (for an individual developer, not part of a team) is a personal decision and I don't expect a one-size-fits-all answer. But I would like to see what others have found that works for them.

Replies are listed 'Best First'.
Re: Overall Directory Organization
by dsheroh (Prior) on Nov 10, 2009 at 11:53 UTC
    Here's what I'm currently using, although it's admittedly not CPAN-compatible, since I use git to deploy my webapps, so CPAN isn't a concern for them. Assuming I'm building My Web App:
    • ~/src/my_web_app/
      • TODO
      • MyWebApp/
        • (general project-specific modules defining the underlying object/data model)
        • CGI/
          • (plugin modules for connecting the object/data model to the web interface)
      • cgi/
        • index.fcgi
        • tpl/
          • (page-generation templates)
        • (other misc files which need to be accessible by web)
      • local/
        • (misc host-specific support files which, for whatever reason, aren't put into the main git repo)
      • t/
        • (test scripts)
      • util/
        • (any command-line tools specific to the project, including cron scripts)
    With this structure, installing on a new host is a simple matter of cloning the project out of git and symlinking the cgi/ directory into the host's web root. Depending on how the host's operating system handles paths doing a ../ from a symlinked directory, it may also be necessary to symlink MyWebApp/ into one of the dirs in @INC, but that's usually not required, thanks to FindBin. (Obviously, I only work with *nix-derived OSes. I don't know whether Windows has a suitable equivalent to symlinks or, if it does, what that might be.)

    If I were to want to build a CPAN distribution out of one of these projects, I would expect it to mostly just be a matter of creating a distribution dir (call it dist/ and symlinking the relevant bits into it - MyWebApp/ into dist/lib/, t/ into dist/ to replace the default dist/t/, and cgi/ and util/ to... well... I don't know where because I've never packaged executables for CPAN, only modules.

Re: Overall Directory Organization
by bibliophile (Parson) on Nov 10, 2009 at 15:24 UTC
    Here's what I use:
    • ~/prj/myproject/trunk
      • bin - cgi-bin stuff
      • conf - config files
      • devdocs - just like it sounds
      • externals - catch-all for support programs
      • logs
      • modules - module hierarchy
      • templates - mostly mimics modules hierarchy
      • updates - sequentially numbered SQL files to update database structure to "current version"
      • - single command to move everything necessary into "live" /opt/myproject, unlink/relink conf files, reload apache, etc
    All under version control (svn), and Trac.

    This layout (copped mainly from some CGI-App docs) works well enough for me, but then my current projects aren't particularly large; YMMV

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://806110]
Approved by NetWallah
Front-paged by NetWallah
and all is quiet...

How do I use this? | Other CB clients
Other Users?
Others surveying the Monastery: (7)
As of 2017-01-22 12:29 GMT
Find Nodes?
    Voting Booth?
    Do you watch meteor showers?

    Results (187 votes). Check out past polls.