Beefy Boxes and Bandwidth Generously Provided by pair Networks
Your skill will accomplish
what the force of many cannot
 
PerlMonks  

Re^4: is there a way to (temporarily) switch off mod_perl caching/variable storing? (contrast compare)

by tomgracey (Beadle)
on Jul 11, 2013 at 12:47 UTC ( #1043715=note: print w/ replies, xml ) Need Help??


in reply to Re^3: is there a way to (temporarily) switch off mod_perl caching/variable storing? (contrast compare)
in thread is there a way to (temporarily) switch off mod_perl caching/variable storing?

Hi anonymous(?) monk - thanks for the suggestion

I checked the 'cd' in a separate script - it's definitely not this.

Not sure if Git::Wrapper is helpful - I tried originally with this but was getting similar errors which was what drove me to simple system commands. I figured Git::Wrapper was just adding unnecessary complexity (since I don't know how it works) ?

I am now running the following:

#!/usr/bin/perl use strict; use lib '/var/www/perl/vbsite/packages'; my @out = `cd /var/www/test_repo; strace git commit -m 'msg' 2>&1`; print "Content-type: text/plain\n\n"; print "ENVIRONMENT VARIABLES:\n\n "; foreach( keys %ENV ){ print "$_: $ENV{$_}\n"; } print "OUTPUT:\n\n"; print "@out\n\n";

The environment variables look like this:

When it fails under mod_perl

ENVIRONMENT VARIABLES: SCRIPT_NAME: /perl/site/git_test/test_commit2.p +l SERVER_NAME: xxx.xxx.xxx.xxx SERVER_ADMIN: root@localhost HTTP_ACCEPT_ENCODING: gzip, deflate HTTP_CONNECTION: keep-alive REQUEST_METHOD: GET HTTP_ACCEPT: text/html,application/xhtml+xml,application/xml;q=0.9,*/* +;q=0.8 SCRIPT_URI: http://xxx.xxx.xxx.xxx/perl/site/git_test/test_commit2.pl SCRIPT_FILENAME: /var/www/perl/site/git_test/test_commit2.pl SERVER_SOFTWARE: Apache/2.2.15 (CentOS) QUERY_STRING: REMOTE_PORT: 57313 HTTP_USER_AGENT: Mozilla/5.0 (X11; Ubuntu; Linux i686; rv:21.0) Gecko/ +20100101 Firefox/21.0 SERVER_PORT: 80 SERVER_SIGNATURE: <address>Apache/2.2.15 (CentOS) Server at xxx.xxx.xx +x.xxx Port 80</address> HTTP_ACCEPT_LANGUAGE: en-US,en;q=0.5 REMOTE_ADDR: xxx.xxx.xxx.xxx SERVER_PROTOCOL: HTTP/1.1 MOD_PERL_API_VERSION: 2 PATH: /sbin:/usr/sbin:/bin:/usr/bin REQUEST_URI: /perl/site/git_test/test_commit2.pl GATEWAY_INTERFACE: CGI/1.1 SCRIPT_URL: /perl/site/git_test/test_commit2.pl SERVER_ADDR: xxx.xxx.xxx.xxx DOCUMENT_ROOT: /var/www/html/ HTTP_HOST: xxx.xxx.xxx.xxx MOD_PERL: mod_perl/2.0.4

When it succeeds under mod_perl (ie first run after restarting apache)

ENVIRONMENT VARIABLES: SCRIPT_NAME: /perl/site/git_test/test_commit2.p +l SERVER_NAME: xxx.xxx.xxx.xxx SERVER_ADMIN: root@localhost HTTP_ACCEPT_ENCODING: gzip, deflate HTTP_CONNECTION: keep-alive REQUEST_METHOD: GET HTTP_ACCEPT: text/html,application/xhtml+xml,application/xml;q=0.9,*/* +;q=0.8 SCRIPT_URI: http://xxx.xxx.xxx.xxx/perl/site/git_test/test_commit2.pl SCRIPT_FILENAME: /var/www/perl/site/git_test/test_commit2.pl SERVER_SOFTWARE: Apache/2.2.15 (CentOS) QUERY_STRING: REMOTE_PORT: 57332 HTTP_USER_AGENT: Mozilla/5.0 (X11; Ubuntu; Linux i686; rv:21.0) Gecko/ +20100101 Firefox/21.0 SERVER_PORT: 80 SERVER_SIGNATURE: <address>Apache/2.2.15 (CentOS) Server at xxx.xxx.xx +x.xxx Port 80</address> HTTP_ACCEPT_LANGUAGE: en-US,en;q=0.5 REMOTE_ADDR: xxx.xxx.xxx.xxx SERVER_PROTOCOL: HTTP/1.1 PATH: /sbin:/usr/sbin:/bin:/usr/bin MOD_PERL_API_VERSION: 2 REQUEST_URI: /perl/site/git_test/test_commit2.pl GATEWAY_INTERFACE: CGI/1.1 SCRIPT_URL: /perl/site/git_test/test_commit2.pl SERVER_ADDR: xxx.xxx.xxx.xxx DOCUMENT_ROOT: /var/www/html/ HTTP_HOST: xxx.xxx.xxx.xxx MOD_PERL: mod_perl/2.0.4

When it runs in CGI

SCRIPT_NAME: /cgi-bin/test_commit2.pl SERVER_NAME: xxx.xxx.xxx.xxx SERVER_ADMIN: root@localhost HTTP_ACCEPT_ENCODING: gzip, deflate HTTP_CONNECTION: keep-alive REQUEST_METHOD: GET HTTP_ACCEPT: text/html,application/xhtml+xml,application/xml;q=0.9,*/* +;q=0.8 SCRIPT_URI: http://xxx.xxx.xxx.xxx/cgi-bin/test_commit2.pl SCRIPT_FILENAME: /var/www/cgi-bin/test_commit2.pl SERVER_SOFTWARE: Apache/2.2.15 (CentOS) QUERY_STRING: REMOTE_PORT: 57336 HTTP_USER_AGENT: Mozilla/5.0 (X11; Ubuntu; Linux i686; rv:21.0) Gecko/ +20100101 Firefox/21.0 SERVER_PORT: 80 SERVER_SIGNATURE: <address>Apache/2.2.15 (CentOS) Server at xxx.xxx.xx +x.xxx Port 80</address> HTTP_ACCEPT_LANGUAGE: en-US,en;q=0.5 REMOTE_ADDR: xxx.xxx.xxx.xxx SERVER_PROTOCOL: HTTP/1.1 PATH: /sbin:/usr/sbin:/bin:/usr/bin REQUEST_URI: /cgi-bin/test_commit2.pl GATEWAY_INTERFACE: CGI/1.1 SCRIPT_URL: /cgi-bin/test_commit2.pl SERVER_ADDR: xxx.xxx.xxx.xxx DOCUMENT_ROOT: /var/www/html/ HTTP_HOST: xxx.xxx.xxx.xxx

I couldn't see any noticeable differences there? (but I really dont know what I'm looking for!)

In fact I did investigate environment variables earlier but couldn't find any references in the documentation to things that git needed setting (beyond the path to the git binary - let me know if anyone knows more on this?). At that time I compared command line/browser.

I discovered 'strace' on my google travels and added this. Not really sure how to analyse the output though! I wont post the full output as it is very long - but when it fails it starts

execve("/usr/bin/git", ["git", "commit", "-m", "msg"], [/* 16 vars */] +) = 0 brk(0) = 0x26ee000 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, + 0) = 0x7f26e51ae000 access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or +directory) open("/etc/ld.so.cache", O_RDONLY) = 0 fstat(0, {st_mode=S_IFREG|0644, st_size=32187, ...}) = 0 mmap(NULL, 32187, PROT_READ, MAP_PRIVATE, 0, 0) = 0x7f26e51a6000 close(0) = 0 open("/lib64/libz.so.1", O_RDONLY) = 0 read(0, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0 !\0\0\0\0\0\0 +"..., 832) = 832 fstat(0, {st_mode=S_IFREG|0755, st_size=88600, ...}) = 0

I dont know if this is significantly different from when it succeeds (first run)

execve("/usr/bin/git", ["git", "commit", "-m", "msg"], [/* 16 vars */] +) = 0 brk(0) = 0x1daf000 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, + 0) = 0x7f9b43975000 access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or +directory) open("/etc/ld.so.cache", O_RDONLY) = 3 fstat(3, {st_mode=S_IFREG|0644, st_size=32187, ...}) = 0 mmap(NULL, 32187, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7f9b4396d000 close(3) = 0 open("/lib64/libz.so.1", O_RDONLY) = 3 read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0 !\0\0\0\0\0\0 +"..., 832) = 832 fstat(3, {st_mode=S_IFREG|0755, st_size=88600, ...}) = 0

or in CGI:

execve("/usr/bin/git", ["git", "commit", "-m", "msg"], [/* 29 vars */] +) = 0 brk(0) = 0x935000 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, + 0) = 0x7f9969bd7000 access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or +directory) open("/etc/ld.so.cache", O_RDONLY) = 3 fstat(3, {st_mode=S_IFREG|0644, st_size=32187, ...}) = 0 mmap(NULL, 32187, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7f9969bcf000 close(3) = 0 open("/lib64/libz.so.1", O_RDONLY) = 3 read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0 !\0\0\0\0\0\0 +"..., 832) = 832 fstat(3, {st_mode=S_IFREG|0755, st_size=88600, ...}) = 0

Below are the last few lines of the strace before it falls over:

close(0) = 0 open(".git/objects/info/alternates", O_RDONLY|O_NOATIME) = -1 ENOENT +(No such file or directory) access(".git/objects/99/564f15c5a06c44e372d03489c59a32ebcf9ec9", F_OK +) = 0 access(".git/hooks/prepare-commit-msg", X_OK) = -1 ENOENT (No such fi +le or directory) access(".git/hooks/commit-msg", X_OK) = -1 ENOENT (No such file or +directory) open(".git/COMMIT_EDITMSG", O_RDONLY) = 0 read(0, "msg\n", 8192) = 4 read(0, "", 12308) = 0 close(0) = 0 open(".git/objects/99/564f15c5a06c44e372d03489c59a32ebcf9ec9", O_RDON +LY|O_NOATIME) = 0 fstat(0, {st_mode=S_IFREG|0444, st_size=52, ...}) = 0 mmap(NULL, 52, PROT_READ, MAP_PRIVATE, 0, 0) = 0x7f26e51ad000 close(0) = 0 munmap(0x7f26e51ad000, 52) = 0 access(".git/objects/f8/8d4f7df95a7c0b3842bf98533e01472efab496", F_OK +) = -1 ENOENT (No such file or directory) open(".git/objects/f8/tmp_obj_pW5OIL", O_RDWR|O_CREAT|O_EXCL, 0444) = + -1 ENOENT (No such file or directory) mkdir(".git/objects/f8", 0777) = 0 open(".git/objects/f8/tmp_obj_pkqRHL", O_RDWR|O_CREAT|O_EXCL, 0444) = + 0 write(2, "error: unable to create temporar"..., 66error: unable to cr +eate temporary file: No such file or directory ) = 66 write(2, "fatal: failed to write commit ob"..., 37fatal: failed to wr +ite commit object ) = 37 exit_group(128) = ?

When it succeeds on the first run, it appears to execute more commands before arriving at what I guess is the same place, which looks like:

close(3) = 0 munmap(0x7f9b43974000, 137) = 0 access(".git/objects/99/564f15c5a06c44e372d03489c59a32ebcf9ec9", F_OK +) = 0 access(".git/hooks/prepare-commit-msg", X_OK) = -1 ENOENT (No such fi +le or directory) access(".git/hooks/commit-msg", X_OK) = -1 ENOENT (No such file or +directory) open(".git/COMMIT_EDITMSG", O_RDONLY) = 3 read(3, "msg\n", 8192) = 4 read(3, "", 12308) = 0 close(3) = 0 open(".git/objects/99/564f15c5a06c44e372d03489c59a32ebcf9ec9", O_RDON +LY|O_NOATIME) = 3 fstat(3, {st_mode=S_IFREG|0444, st_size=52, ...}) = 0 mmap(NULL, 52, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7f9b43974000 close(3) = 0 munmap(0x7f9b43974000, 52) = 0 access(".git/objects/0f/7f57fee43d1fa25cfc7f0d7fd1bf5826b22fcc", F_OK +) = -1 ENOENT (No such file or directory) open(".git/objects/0f/tmp_obj_RAN8sj", O_RDWR|O_CREAT|O_EXCL, 0444) = + -1 ENOENT (No such file or directory) mkdir(".git/objects/0f", 0777) = 0 open(".git/objects/0f/tmp_obj_rnREwj", O_RDWR|O_CREAT|O_EXCL, 0444) = + 3 brk(0x1df9000) = 0x1df9000 write(3, "x\1\255\216I\n\3030\f\0{\366+\374\201\4/R\22A(\355STUY\16\2 +16C\354\26\372"..., 138) = 138 brk(0x1de9000) = 0x1de9000 brk(0x1dd9000) = 0x1dd9000 brk(0x1dd7000) = 0x1dd7000

I will continue my investigations but just on the off chance this means something to someone. Many thanks for your help so far!


Comment on Re^4: is there a way to (temporarily) switch off mod_perl caching/variable storing? (contrast compare)
Select or Download Code
Update: problem solved = bug in Git
by tomgracey (Beadle) on Jul 12, 2013 at 09:18 UTC

    Very big hat off to the real wizard on google groups "git for human beings" who solved this conundrum (I wont say who as I haven't asked for permission - but you can find the thread under the title 'frustrations trying to control git through apache'). Here is an extract of his diagnosis:

    So we see that:

    1) The Git code is poorly documented, and there is no coding control to ensure that the code is internally documented.

    2) The test "if (fd > 0) return fd;" is incorrect. It should be "(fd >= 0)".

    3) The error is triggered if fd 0 (standard-input) is closed when Git starts and Git needs to put a file into the object store. In your situation, that is determined by the particular Git operation and the details of how Git is invoked by Apache.

    From this, we can construct a simple test case that demonstrates the bug:

    $ git --version git version 1.7.7.6 $ git status # On branch master nothing to commit (working directory clean) $ echo This is a test >ffff $ git status # On branch master # Untracked files: # (use "git add <file>..." to include in what will be committed) # # ffff nothing added to commit but untracked files present (use "git add" to +track) $ git add ffff $ # The notation "0<&-" means "close standard input (fd 0) in the proc +ess that $ # executes this command. It may be specific to bash. $ git commit -m xxxx 0<&- error: unable to create temporary sha1 filename : No such file or dire +ctory error: Error building trees $ git status # On branch master # Changes to be committed: # (use "git reset HEAD <file>..." to unstage) # # new file: ffff # $ git commit -m xxxx [master 54c146c] xxxx 1 files changed, 1 insertions(+), 0 deletions(-) create mode 100644 ffff $ git status # On branch master # Your branch is ahead of 'origin/master' by 1 commit. # nothing to commit (working directory clean) $

    So there you have it. Nothing to do with environment variables, and mod_perl was a red herring

    The fix?

    Add </dev/null to any (bash) line being executed that involves a write (or otherwise ensure standard input is not closed when git tries to open the file). Unfortunately this appears to mean *any* git module (Git::Wrapper, Git::Repository... ) is unusable for me (and I would be surprised if not for other people aswell?), and I must use home grown code...

    Btw the bug has been reported

    And now it works! I am so happy =D

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://1043715]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others pondering the Monastery: (8)
As of 2014-07-26 07:19 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    My favorite superfluous repetitious redundant duplicative phrase is:









    Results (175 votes), past polls