Beefy Boxes and Bandwidth Generously Provided by pair Networks
P is for Practical
 
PerlMonks  

Re^3: is there a way to (temporarily) switch off mod_perl caching/variable storing? (contrast compare)

by Anonymous Monk
on Jul 11, 2013 at 08:27 UTC ( [id://1043652]=note: print w/replies, xml ) Need Help??


in reply to Re^2: is there a way to (temporarily) switch off mod_perl caching/variable storing?
in thread is there a way to (temporarily) switch off mod_perl caching/variable storing?

So the next question is, what can I do now?

compare and contrast

Each proc comes with STDIN/STDOUT/STDERR/@ARGV/%ENV/CWD so you dump %ENV/Cwd between the mod_perl and cgi versions, between the successfull and failed ones, and then you check to see whats different

Do the same for each function ( open or die ... ; chdir or die ...; ...)

I don't know, but that cd /test/repo could fail for some reason ... so verify everything, Git::Wrapper can help ( How to script git with perl and Git::Wrapper | David Golden )

If what you've posted is really the misbehaving code, it might be that you need to sanitize %ENV

General notions you should be aware of as well as lists of things to check are found in Coping with Scoping , CGI to mod_perl Porting. mod_perl Coding guidelines , brian's Guide to Solving Any Perl Problem, CGI Help Guide , Troubleshooting Perl CGI scripts , On debugging, verify everything, talk to teddybear ...

Replies are listed 'Best First'.
Re^4: is there a way to (temporarily) switch off mod_perl caching/variable storing? (contrast compare)
by tomgracey (Scribe) on Jul 11, 2013 at 12:47 UTC
    Hi anonymous(?) monk - thanks for the suggestion

    I checked the 'cd' in a separate script - it's definitely not this.

    Not sure if Git::Wrapper is helpful - I tried originally with this but was getting similar errors which was what drove me to simple system commands. I figured Git::Wrapper was just adding unnecessary complexity (since I don't know how it works) ?

    I am now running the following:

    #!/usr/bin/perl use strict; use lib '/var/www/perl/vbsite/packages'; my @out = `cd /var/www/test_repo; strace git commit -m 'msg' 2>&1`; print "Content-type: text/plain\n\n"; print "ENVIRONMENT VARIABLES:\n\n "; foreach( keys %ENV ){ print "$_: $ENV{$_}\n"; } print "OUTPUT:\n\n"; print "@out\n\n";

    The environment variables look like this:

    When it fails under mod_perl

    ENVIRONMENT VARIABLES: SCRIPT_NAME: /perl/site/git_test/test_commit2.p +l SERVER_NAME: xxx.xxx.xxx.xxx SERVER_ADMIN: root@localhost HTTP_ACCEPT_ENCODING: gzip, deflate HTTP_CONNECTION: keep-alive REQUEST_METHOD: GET HTTP_ACCEPT: text/html,application/xhtml+xml,application/xml;q=0.9,*/* +;q=0.8 SCRIPT_URI: http://xxx.xxx.xxx.xxx/perl/site/git_test/test_commit2.pl SCRIPT_FILENAME: /var/www/perl/site/git_test/test_commit2.pl SERVER_SOFTWARE: Apache/2.2.15 (CentOS) QUERY_STRING: REMOTE_PORT: 57313 HTTP_USER_AGENT: Mozilla/5.0 (X11; Ubuntu; Linux i686; rv:21.0) Gecko/ +20100101 Firefox/21.0 SERVER_PORT: 80 SERVER_SIGNATURE: <address>Apache/2.2.15 (CentOS) Server at xxx.xxx.xx +x.xxx Port 80</address> HTTP_ACCEPT_LANGUAGE: en-US,en;q=0.5 REMOTE_ADDR: xxx.xxx.xxx.xxx SERVER_PROTOCOL: HTTP/1.1 MOD_PERL_API_VERSION: 2 PATH: /sbin:/usr/sbin:/bin:/usr/bin REQUEST_URI: /perl/site/git_test/test_commit2.pl GATEWAY_INTERFACE: CGI/1.1 SCRIPT_URL: /perl/site/git_test/test_commit2.pl SERVER_ADDR: xxx.xxx.xxx.xxx DOCUMENT_ROOT: /var/www/html/ HTTP_HOST: xxx.xxx.xxx.xxx MOD_PERL: mod_perl/2.0.4

    When it succeeds under mod_perl (ie first run after restarting apache)

    ENVIRONMENT VARIABLES: SCRIPT_NAME: /perl/site/git_test/test_commit2.p +l SERVER_NAME: xxx.xxx.xxx.xxx SERVER_ADMIN: root@localhost HTTP_ACCEPT_ENCODING: gzip, deflate HTTP_CONNECTION: keep-alive REQUEST_METHOD: GET HTTP_ACCEPT: text/html,application/xhtml+xml,application/xml;q=0.9,*/* +;q=0.8 SCRIPT_URI: http://xxx.xxx.xxx.xxx/perl/site/git_test/test_commit2.pl SCRIPT_FILENAME: /var/www/perl/site/git_test/test_commit2.pl SERVER_SOFTWARE: Apache/2.2.15 (CentOS) QUERY_STRING: REMOTE_PORT: 57332 HTTP_USER_AGENT: Mozilla/5.0 (X11; Ubuntu; Linux i686; rv:21.0) Gecko/ +20100101 Firefox/21.0 SERVER_PORT: 80 SERVER_SIGNATURE: <address>Apache/2.2.15 (CentOS) Server at xxx.xxx.xx +x.xxx Port 80</address> HTTP_ACCEPT_LANGUAGE: en-US,en;q=0.5 REMOTE_ADDR: xxx.xxx.xxx.xxx SERVER_PROTOCOL: HTTP/1.1 PATH: /sbin:/usr/sbin:/bin:/usr/bin MOD_PERL_API_VERSION: 2 REQUEST_URI: /perl/site/git_test/test_commit2.pl GATEWAY_INTERFACE: CGI/1.1 SCRIPT_URL: /perl/site/git_test/test_commit2.pl SERVER_ADDR: xxx.xxx.xxx.xxx DOCUMENT_ROOT: /var/www/html/ HTTP_HOST: xxx.xxx.xxx.xxx MOD_PERL: mod_perl/2.0.4

    When it runs in CGI

    SCRIPT_NAME: /cgi-bin/test_commit2.pl SERVER_NAME: xxx.xxx.xxx.xxx SERVER_ADMIN: root@localhost HTTP_ACCEPT_ENCODING: gzip, deflate HTTP_CONNECTION: keep-alive REQUEST_METHOD: GET HTTP_ACCEPT: text/html,application/xhtml+xml,application/xml;q=0.9,*/* +;q=0.8 SCRIPT_URI: http://xxx.xxx.xxx.xxx/cgi-bin/test_commit2.pl SCRIPT_FILENAME: /var/www/cgi-bin/test_commit2.pl SERVER_SOFTWARE: Apache/2.2.15 (CentOS) QUERY_STRING: REMOTE_PORT: 57336 HTTP_USER_AGENT: Mozilla/5.0 (X11; Ubuntu; Linux i686; rv:21.0) Gecko/ +20100101 Firefox/21.0 SERVER_PORT: 80 SERVER_SIGNATURE: <address>Apache/2.2.15 (CentOS) Server at xxx.xxx.xx +x.xxx Port 80</address> HTTP_ACCEPT_LANGUAGE: en-US,en;q=0.5 REMOTE_ADDR: xxx.xxx.xxx.xxx SERVER_PROTOCOL: HTTP/1.1 PATH: /sbin:/usr/sbin:/bin:/usr/bin REQUEST_URI: /cgi-bin/test_commit2.pl GATEWAY_INTERFACE: CGI/1.1 SCRIPT_URL: /cgi-bin/test_commit2.pl SERVER_ADDR: xxx.xxx.xxx.xxx DOCUMENT_ROOT: /var/www/html/ HTTP_HOST: xxx.xxx.xxx.xxx

    I couldn't see any noticeable differences there? (but I really dont know what I'm looking for!)

    In fact I did investigate environment variables earlier but couldn't find any references in the documentation to things that git needed setting (beyond the path to the git binary - let me know if anyone knows more on this?). At that time I compared command line/browser.

    I discovered 'strace' on my google travels and added this. Not really sure how to analyse the output though! I wont post the full output as it is very long - but when it fails it starts

    execve("/usr/bin/git", ["git", "commit", "-m", "msg"], [/* 16 vars */] +) = 0 brk(0) = 0x26ee000 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, + 0) = 0x7f26e51ae000 access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or +directory) open("/etc/ld.so.cache", O_RDONLY) = 0 fstat(0, {st_mode=S_IFREG|0644, st_size=32187, ...}) = 0 mmap(NULL, 32187, PROT_READ, MAP_PRIVATE, 0, 0) = 0x7f26e51a6000 close(0) = 0 open("/lib64/libz.so.1", O_RDONLY) = 0 read(0, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0 !\0\0\0\0\0\0 +"..., 832) = 832 fstat(0, {st_mode=S_IFREG|0755, st_size=88600, ...}) = 0

    I dont know if this is significantly different from when it succeeds (first run)

    execve("/usr/bin/git", ["git", "commit", "-m", "msg"], [/* 16 vars */] +) = 0 brk(0) = 0x1daf000 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, + 0) = 0x7f9b43975000 access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or +directory) open("/etc/ld.so.cache", O_RDONLY) = 3 fstat(3, {st_mode=S_IFREG|0644, st_size=32187, ...}) = 0 mmap(NULL, 32187, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7f9b4396d000 close(3) = 0 open("/lib64/libz.so.1", O_RDONLY) = 3 read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0 !\0\0\0\0\0\0 +"..., 832) = 832 fstat(3, {st_mode=S_IFREG|0755, st_size=88600, ...}) = 0

    or in CGI:

    execve("/usr/bin/git", ["git", "commit", "-m", "msg"], [/* 29 vars */] +) = 0 brk(0) = 0x935000 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, + 0) = 0x7f9969bd7000 access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or +directory) open("/etc/ld.so.cache", O_RDONLY) = 3 fstat(3, {st_mode=S_IFREG|0644, st_size=32187, ...}) = 0 mmap(NULL, 32187, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7f9969bcf000 close(3) = 0 open("/lib64/libz.so.1", O_RDONLY) = 3 read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0 !\0\0\0\0\0\0 +"..., 832) = 832 fstat(3, {st_mode=S_IFREG|0755, st_size=88600, ...}) = 0

    Below are the last few lines of the strace before it falls over:

    close(0) = 0 open(".git/objects/info/alternates", O_RDONLY|O_NOATIME) = -1 ENOENT +(No such file or directory) access(".git/objects/99/564f15c5a06c44e372d03489c59a32ebcf9ec9", F_OK +) = 0 access(".git/hooks/prepare-commit-msg", X_OK) = -1 ENOENT (No such fi +le or directory) access(".git/hooks/commit-msg", X_OK) = -1 ENOENT (No such file or +directory) open(".git/COMMIT_EDITMSG", O_RDONLY) = 0 read(0, "msg\n", 8192) = 4 read(0, "", 12308) = 0 close(0) = 0 open(".git/objects/99/564f15c5a06c44e372d03489c59a32ebcf9ec9", O_RDON +LY|O_NOATIME) = 0 fstat(0, {st_mode=S_IFREG|0444, st_size=52, ...}) = 0 mmap(NULL, 52, PROT_READ, MAP_PRIVATE, 0, 0) = 0x7f26e51ad000 close(0) = 0 munmap(0x7f26e51ad000, 52) = 0 access(".git/objects/f8/8d4f7df95a7c0b3842bf98533e01472efab496", F_OK +) = -1 ENOENT (No such file or directory) open(".git/objects/f8/tmp_obj_pW5OIL", O_RDWR|O_CREAT|O_EXCL, 0444) = + -1 ENOENT (No such file or directory) mkdir(".git/objects/f8", 0777) = 0 open(".git/objects/f8/tmp_obj_pkqRHL", O_RDWR|O_CREAT|O_EXCL, 0444) = + 0 write(2, "error: unable to create temporar"..., 66error: unable to cr +eate temporary file: No such file or directory ) = 66 write(2, "fatal: failed to write commit ob"..., 37fatal: failed to wr +ite commit object ) = 37 exit_group(128) = ?

    When it succeeds on the first run, it appears to execute more commands before arriving at what I guess is the same place, which looks like:

    close(3) = 0 munmap(0x7f9b43974000, 137) = 0 access(".git/objects/99/564f15c5a06c44e372d03489c59a32ebcf9ec9", F_OK +) = 0 access(".git/hooks/prepare-commit-msg", X_OK) = -1 ENOENT (No such fi +le or directory) access(".git/hooks/commit-msg", X_OK) = -1 ENOENT (No such file or +directory) open(".git/COMMIT_EDITMSG", O_RDONLY) = 3 read(3, "msg\n", 8192) = 4 read(3, "", 12308) = 0 close(3) = 0 open(".git/objects/99/564f15c5a06c44e372d03489c59a32ebcf9ec9", O_RDON +LY|O_NOATIME) = 3 fstat(3, {st_mode=S_IFREG|0444, st_size=52, ...}) = 0 mmap(NULL, 52, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7f9b43974000 close(3) = 0 munmap(0x7f9b43974000, 52) = 0 access(".git/objects/0f/7f57fee43d1fa25cfc7f0d7fd1bf5826b22fcc", F_OK +) = -1 ENOENT (No such file or directory) open(".git/objects/0f/tmp_obj_RAN8sj", O_RDWR|O_CREAT|O_EXCL, 0444) = + -1 ENOENT (No such file or directory) mkdir(".git/objects/0f", 0777) = 0 open(".git/objects/0f/tmp_obj_rnREwj", O_RDWR|O_CREAT|O_EXCL, 0444) = + 3 brk(0x1df9000) = 0x1df9000 write(3, "x\1\255\216I\n\3030\f\0{\366+\374\201\4/R\22A(\355STUY\16\2 +16C\354\26\372"..., 138) = 138 brk(0x1de9000) = 0x1de9000 brk(0x1dd9000) = 0x1dd9000 brk(0x1dd7000) = 0x1dd7000

    I will continue my investigations but just on the off chance this means something to someone. Many thanks for your help so far!

      Very big hat off to the real wizard on google groups "git for human beings" who solved this conundrum (I wont say who as I haven't asked for permission - but you can find the thread under the title 'frustrations trying to control git through apache'). Here is an extract of his diagnosis:

      So we see that:

      1) The Git code is poorly documented, and there is no coding control to ensure that the code is internally documented.

      2) The test "if (fd > 0) return fd;" is incorrect. It should be "(fd >= 0)".

      3) The error is triggered if fd 0 (standard-input) is closed when Git starts and Git needs to put a file into the object store. In your situation, that is determined by the particular Git operation and the details of how Git is invoked by Apache.

      From this, we can construct a simple test case that demonstrates the bug:

      $ git --version git version 1.7.7.6 $ git status # On branch master nothing to commit (working directory clean) $ echo This is a test >ffff $ git status # On branch master # Untracked files: # (use "git add <file>..." to include in what will be committed) # # ffff nothing added to commit but untracked files present (use "git add" to +track) $ git add ffff $ # The notation "0<&-" means "close standard input (fd 0) in the proc +ess that $ # executes this command. It may be specific to bash. $ git commit -m xxxx 0<&- error: unable to create temporary sha1 filename : No such file or dire +ctory error: Error building trees $ git status # On branch master # Changes to be committed: # (use "git reset HEAD <file>..." to unstage) # # new file: ffff # $ git commit -m xxxx [master 54c146c] xxxx 1 files changed, 1 insertions(+), 0 deletions(-) create mode 100644 ffff $ git status # On branch master # Your branch is ahead of 'origin/master' by 1 commit. # nothing to commit (working directory clean) $

      So there you have it. Nothing to do with environment variables, and mod_perl was a red herring

      The fix?

      Add </dev/null to any (bash) line being executed that involves a write (or otherwise ensure standard input is not closed when git tries to open the file). Unfortunately this appears to mean *any* git module (Git::Wrapper, Git::Repository... ) is unusable for me (and I would be surprised if not for other people aswell?), and I must use home grown code...

      Btw the bug has been reported

      And now it works! I am so happy =D

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://1043652]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others exploiting the Monastery: (5)
As of 2024-03-19 09:13 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found