Are all .cgi helper modules unfit for http methods common in REST?

[isync]

Are they?
So far I had a look at CGI.pm and CGI::Simple.

The trouble starts when you are handling non-mainstream requests that contain payload, like the PUT method. That means, the HTTP message has a body or content attached, which is not the result of a POST action or is not labeled as being 'application/x-www-form-urlencoded'.
My experience is that CGI.pm 3.49 does not read in the put data correctly, although it should do so and store it in param('PUTDATA') (after a lot of effort).

Other non-standard methods do send content as well, for example the PROPFIND method common in WebDAV. CGI.pm simply ignores it. CGI::Simple at least tells me it skipped it by raising the '.cgi_error' => '400 Unknown method PROPFIND' error flag.

In both cases I can resort to reading in STDIN. But things with STDIN get messy when your helper module *thinks* it did the right thing, garbled everything into key/value param nonsense or similar. And then, STDIN is cleared.

Building on the best there is, you decide to use CGI::Simple, but what about large payloads, gigabytes? Reading all this into memory is a bad idea. CGI.pm has hooks to process chunk by chunk or send everything to a file - CGI::Simple doesn't (right?).

Or have you every tried to output all capitalized/partly capitalized headers like "ABC-Value: test"? The header(-abc_alue => 'test') interface is your only chance and it only ever capitalizes the first letter!

So, is there a .cgi helper module which I can use as a replacement for CGI.pm which doesn't try to marshall which method is okay or not and lets me handle method evaluation. Is there a helper which reads in whole HTTP requests while providing hooks to do it in slurps, or in chunks, to a file, etc.

In one sentence: is there a .cgi helper module which is fit for REST?

[hossman]

I haven't drunken a lot of REST Kool-Aid, but i know enough to know that REST != CGI. REST isn't a subset or a superset of CGI for that matter. They are largely orthogonal concepts, overlapping only in as much as they both use HTTP for transport.

Note in particular that RFC 3875 specifically identifies only 3 valid HTTP Methods for CGI (HEAD, GET, and POST). Quite simply: if a CGI related Perl module is moderately useful for writing REST resources, that's a perk above and beyond the scope of it's primary function as a CGI library. Complaining that it doesn't support everything you need to implement a REST resource is like complaining that an XML Parsing module doesn't support everything you need to parse crufty HTML found in the wild.

Have you perchance considered using REST::Resource or REST::Application?

[isync]

That was a good hint!

I was too centered on this CGI::Application / (it can do anything, so why not provide a REST API) idea that I didn't realize it might make sense to look around cpan in the REST namespace... Thanks!

(interesting, REST::Application is built around CGI.pm.., I'll see how this behaves.. see above 'keywords' observation)

[Anonymous Monk]

My experience is that CGI.pm 3.49 does not read in the put data correctly, although it should do so and store it in param('PUTDATA').

Have you submitted a bug report?

Other non-standard methods do send content as well, for example the PROPFIND method common in WebDAV

Usually for WebDAV you run a dedicated webserver

In one sentence: is there a .cgi helper who is fit for REST?

Yes, its called mod_perl :)

[isync]

Have you submitted a bug report?

No account. And who solves these bugs anyway? ;)

dedicated webserver .. mod_perl

Already in the pipeline (as everything else turned out to be a wall). But does that mean there is no equivalent for the CGI interface?
Further, running a dedicated HTTP webserver for us perlheads means building something around HTTP::Daemon (which for me seems to be unstable, see this node or Net::Server with a completely unusable ::HTTP implementation.

Second, mostly you'll end up describing your requests as HTTP::Request objects, which is great, but these aren't procedural as well. So you'll end up waiting for a HTTP::Request object to complete, with a 2GB content variable - and you've got no access to it while it arrives. No buffered reading, no partial XML parsing possible...

[Anonymous Monk]

No account. And who solves these bugs anyway? ;)

You claim there is a bug, but you won't submit a bug report?

But does that mean there is no equivalent for the CGI interface?

No, it doesn't mean that, that would be jumping to conclusions.

...see this node ...No buffered reading....

Not very authoritative ... see Re^2: POST'ing a large File with LWP::UserAgent for one example, if LWP::UserAgent can use callbacks to chunk file uploads and downloads, so can you

[Anonymous Monk]

There is this thing called Plack/PSGI that seems relevant ... 10 years experience doesn't make you a noob :)

[isync]

I seems relevant! Good hint.

Relying on CGI::Application for years now, I will see when I'll find the strength to see how Plack works and how/if it fits into my CGI::Application code...

btw: thanks for soothing my noob pain ;)

[Corion]

I would look at how Net::DAV::Server does things, or HTTP::DAV (but that's only a client I think).

[derby]

Check out REST Webservices and CGI.pm ... CGI should be handling PUT and POST (and DELETE and GET) correctly. Can you provide code examples? As for PROPFIND ... meh ... I would vote against polluting CGI with support for it (but that's not my call). As for not having an account to submit bugs, come on it takes about 30 second to create a bitcard account.

Update: Okay missed the link to REST Webservices and CGI.pm (it was more work to write it up than actually create a patch) ... code please ... and failing test cases.

[isync]

Edited my post to include this node while you wrote your post...

You are right with bitcard, but who wants all these bug reports from a noob who is now banging his head against the wall for two days before finding out that these proven decade-old modules were the initial bug that caused endless rewrites, commenting in/out, design changes, etc.
My strategy is to hope that the (active) module maintainers follow discussion here anyway and they are best to decide if what I am observing is a bug, or simply incompetence.

[derby]

Gotcha. Okay ... I normally use bare-bones CGI::Application and do something like this:

package MyApp;

use base 'CGI::Application';

....

sub setup {
  my $self = shift;
  my $cgi  = $self->query();

  $self->start_mode( $cgi->request_method() );

  $self->run_modes(
    'PUT'       => 'create',
    'GET'       => 'retrieve',
    'POST'      => 'update',
    'DELETE'    => 'delete',
    'AUTOLOAD'  => 'method_not_supported'
  );
 
  return;  
}

....

1;

I have no idea if PROPFIND would work or not (I'm pretty much WebDAV ignorant). I do believe you still have problems with marshalling (CGI will try to copy that 2G POST/PUT into a scalar and that will probably not go well.

[isync]

Puh, test scripts basically done. Still, the test-server seems to refuse to serve PROPFIND requests. I don't know if this is HTTP::Server::Simple or something else, yet. But again - this "we don't like lesser known methods" sucks..

What already can be observed is that CGI.pm strangely parses PUT data into a 'keywords' param(). (??)

Minor update to the code: uncommenting the CGI::Simple part should enable you to see how CGI::Simple beahves, but for me it simply hangs. I'm giving up for today...

test-client.pl
Read more... (3 kB)

[derby]

If you're getting keyword set when processing a PUT request than your code (or HTTP::Request) is setting the content-type of the request to either multipart/form-data or application/x-www-form-urlencoded. You should set the content-type to something else (maybe text/xml).