Beefy Boxes and Bandwidth Generously Provided by pair Networks
Welcome to the Monastery

Re^7: [JOB] The Perl Foundation seeks Windows Developer

by adamk (Chaplain)
on Apr 03, 2006 at 04:19 UTC ( #540846=note: print w/ replies, xml ) Need Help??

in reply to Re^6: [JOB] The Perl Foundation seeks Windows Developer
in thread [JOB] The Perl Foundation seeks Windows Developer

Even if certain Windows-specific modules were added to the core _today_, these fixes would still need to be done to deal with older installed Perl versions.

Whether or not something should be in the core is an orthogonal problem to this grant, but I'll address it in my reply to one of your other comments, below.

Comment on Re^7: [JOB] The Perl Foundation seeks Windows Developer
Replies are listed 'Best First'.
Re^8: [JOB] The Perl Foundation seeks Windows Developer
by demerphq (Chancellor) on Apr 03, 2006 at 06:42 UTC

    Even if certain Windows-specific modules were added to the core _today_, these fixes would still need to be done to deal with older installed Perl versions.

    Ive said it a lot of times Adam, a lot of these fixes can't be done without proper access to the Win32 API. And as long as that API is reachable only through a non core module set the issues involved with adding better support become much more difficult. You can't just say "am I on Win32? Yes ok, better use the Win32 tools" you have to say "am I on Win32 AND do i have the required modules available". Which complicates things and means that less Win32 folk are going to bother to try to add proper Win32 support to core modules.

    Not only that, but look at the mantra of P5P, it basically says "fix what you like, but you can't use non-core modules to do it". Which means for instance that functionality in EU-I simply cant be used or tested under a core build. (Although what I find really amusing is that you can added Win32 API support to core modules, but only via XS, for some reason providing that access through a perl layer is verbotem. So we have Win32API calls for specialized purposes scattered throughout the core, but always hidden so that end users can't make use of them in interesting new ways. Which is bad because it raises the skills requirement for improving Win32 perl, not only do you need to be conversant with the API, you need to be a good perl programmer, and a good C/XS programmer.)

    As an example using core perl there is only one sharing mode that you can open a file with, using Win32API::File there are a variety, including the mode that Windows itself uses to lock DLL's but still allow them to be renamed. (Annoyingly the default open mode for Cygwin is different from the default open mode of native builds. This is why you can delete files opened with Cygwin perl, but not AS perl)

    Similarly since Win32API::File is omitted from Core it is unsurprising to me that File::Temp uses broken *NIX based logic to do its thing on Win32. Wheras if Win32API::File were included in core then the author of F::T (apparently not a win32 developer) might have looked at the documentation for Win32API::File and found out that Windows supports an OS level temporary file which will be autodeleted once all filehandles to it are closed. Or with even better API support he might have found the API call that windows provides to produce a guaranteed unique temporary file.

    But since this stuff isn't available to Core, and since non Win32 developers seem to have this strange aversion to using MSDN library search, (maybe because they know that since there is no support for such in core there isnt any point in searching) the fact is that non-win32 programmers will never even know that there are better ways to do things on Win32.

    So I stick to my guns: adding proper win32 API support to core is not a nice to have, its an essential part of improving Perl support on Win32. And doing it in blead even will be a signal to the community that the investment in making the code work better on Win32 will not be a wasted one.

    And as a closing thought consider this: In VB you can bind to a Win32 API call with one line of code. In Perl you cant bind to API at all out of the box. That VB can provide better OS access than Perl does is just wrong.


      You've hit the nail on the head.

      The biggest problem (for the none *nix developer), that I've seen in my forays into Perl sources, is that (understandably given it's origins), most of the Perl sources are written directly to the architecture and semantics of *nix system calls and methods. Where the underlying OS does not directly support those architectures and semantics, it is up to each developer on each non-*nix OS to add conditional code in-situ, at the site of each use, to emulate the *nix behaviours.

      It is the combined effect of the conditional codes for all the non-*nix OSs, (along with variations of minor tweaks and fixes for the myriad not-quite-*nix-compatible *nix-like platforms), that is responsible for a large part of the complexity and fragility of the sources.

      It is made even worse by the need to apply the same conditional fix/work-around in-situ at each place in the sources where it is used. This just exacerbates the potential for the multiple implementations of the same fix to get out of synch. It's the cardinal sin of c&p code reuse. Your example of file open & sharing modes above relects the problem exactly.

      It gets worse where a pair (or more), of related but otherwise atomic posix system calls, which get used directly in-line in the Perl sources, require a combination of 2 or 3 Win32 (or other platform) system APIs to achieve the same function. And it gets even worse when there is a need to convey some information (like OS handles) between the open and close sequences of a bracketing pair of calls. If the posix pair do not require the passing of a handle, then there is no provision in the architecture for this.

      The architectural solution to this problem, for all platforms, not just Win32, would be to isolate OS dependant calls from the perl sources. Essentially, this would require building a Perl virtual OS layer and removing all system calls from the core sources in favour of calls to the virtual OS layer. The Perl sources would call the virtual OS functions and the platform dependant equivalents are conditionally compiled in to perform that function. In many places this has already been done--PerlIO is a good example.

      The problem is that in a lot of cases, the virtualisation is done at too low a level. They require a one to one correspondance between the virtualisation and the underlying POSIX apis. When the OS is unable to easily provide that one to one correspondance, the need to cut&paste platform dependant fixes all over the source tree returns.

      Of course, it is entirely the wrong time, too late, to attempt to address this properly in P5. Unfortunately, unless a cross-platform white knight has become involved in Parrot recently, that was headed in exactly the same direction.

      However, I think there is a solution (for win32), that could be retro-fitted to the P5 sources without creating wholesale mayhem. It would require considerable effort; at least 3 or 4 commited, knowledgable, Win32 developers--though not necessarily all full time; and the involvement and support of 1 or 2 of the deeply knowledgable p5p source gurus. Essentially, there are already a bunch of Win32 OS specific virtual functions conditionally build into and exported from perl5x.dll. All the stuff that resides in win32\Win32.c.

      The project would involve removing as much of the Win32 conditional code from the core of the sources and replacing it with calls to new virtualised functions added into Win32.c (or probably better Win32v.c or similar). This would benefit all p5 programmers. Core *nix programmers would have considerably less Win32 conditional code to work around. Win32 p5p developers (like yourself & Steve) could concentrate your efforts in one place more easily. And cross-platform developers and module writers would gain access to the Win32 equivalents of *nix functions exported from the Win32\* sources rather than needing to invent their own workarounds or cut&paste existing code.

      Like you, I absolutely applaud and encourage any moves/projects that will enable Perl on Win32 to

      1. Be less of a problem for non-Win32 authors to write modules that will easily port to Win32.
      2. Be less of a source of bug-reports and grief to the p5p efforts in general.
      3. Give me greater access to the facilities and semantics of my chosen platform.

      My fear is that, as currently envisaged and described, the Vanilla/Strawberry projects, and this particular effort are not likely to achieve any of these.

      I feel that the situation is analogous to General Motors designing and proposing a solution to the current fuel crisis and expecting Toyota (and Honda and Ford and BMW) to implement their solution without consultation or input.

      The current proposal seems to revolve around allowing more modules written originally for non-Win32 platforms to be seen as functional on Win32, as measured by some statistic (currently the AS autobuild process). However, I think that as currently conceived, the project will achieved very little in the long term. Perhaps the problems that prevents the AS autobuild process from passing more modules is that it is automated. And as the endemic solution to compatibility problems is to hand-code in-situ, conditional work arounds to each problem as it arises, the automated approach cannot do that.

      Whilst a short, concentrated effort now may result in a bunch of immediate fixes to individual incompatibilities, without an architected solution that allows those fixes to be virtualised and embedded into the core, the efforts would only allow a short term hiatus in the problems because new authors and new modules would be forced back into writing individual fixes to known problems with known solutions as soon as the manual 'bug hunt and fix' effort subsided.

      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      Lingua non convalesco, consenesco et abolesco. -- Rule 1 has a caveat! -- Who broke the cabal?
      "Science is about questioning the status quo. Questioning authority".
      In the absence of evidence, opinion is indistinguishable from prejudice.
        Unfortunately for this idea though, you have to find a way to make it happen.

        The reason the grant committee will probably end up getting behind this, is that it will work, today.

        You are proposing something without a clear path to the end point. The Strawberry Perl grant is saying "Here's something I want to do, here's how it can be done. I could do it ANYWAY, but we could use some resources to make it happen faster".

        Allowing more modules written originally for non-Win32 platforms, or for non-specific platforms to work on Win32 is a good thing. It can be done now, and doesn't require a multi-year, multi-developer effort to entirely re-architect the way Perl works.

        Which is why it is happening.

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://540846]
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others exploiting the Monastery: (5)
As of 2015-11-29 18:11 GMT
Find Nodes?
    Voting Booth?

    What would be the most significant thing to happen if a rope (or wire) tied the Earth and the Moon together?

    Results (752 votes), past polls