Beefy Boxes and Bandwidth Generously Provided by pair Networks
No such thing as a small change
 
PerlMonks  

Re: Inserting large chunks of data into MySQL

by g0n (Priest)
on May 04, 2008 at 21:28 UTC ( #684482=note: print w/ replies, xml ) Need Help??


in reply to Inserting large chunks of data into MySQL

1) Please don't update your post by replacing the original content. It means that the answers people have already given no longer make sense. If you want to post an update, put 'update' at the bottom of your post and add more details below it, or something like that.

2) Please give us an error message. The error you've posted tells us that you are having trouble with a CGI script, and that's about all. Run your script from the command line, not via a web browser, and report the error generated, and maybe someone can help.

3) "here the problem wasn't from mysql command it was from perl it doesn't seems like to split string that is large over 20,000 arrays or about 5MB"

Nope, sorry, you're making a false assumption about the problem. The Perl interpreter can handle split on much bigger strings than that. In fact, I ran the following very simple code to demonstrate that on my laptop PC

use strict; use warnings; my $string=""; my %charmap = ( 0 => "a", 1 => "~" ); for (1..50000000) { $string .= $charmap{int(rand(2))}; } open (my $testfile,">","testfile.txt") or die $!; print $testfile $string; my @arr = split "~",$string; for (@arr) { print "$_\n"; }

it took a while and slowed my laptop down, but it ran without a problem, and that was a string of approximately 50MB - ten times what you're working with.

OK, so by commenting the sql line out you're on the right lines - try to narrow the problem down to the simplest case that still generates an error. But first, find out what the error is - run it locally and see what it does.

Update You could also get useful error messages by putting:     use CGI::Carp qw(fatalsToBrowser); at the beginning of your script.

--------------------------------------------------------------

"If there is such a phenomenon as absolute evil, it consists in treating another human being as a thing."
John Brunner, "The Shockwave Rider".


Comment on Re: Inserting large chunks of data into MySQL
Select or Download Code

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://684482]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others perusing the Monastery: (11)
As of 2014-10-23 20:22 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    For retirement, I am banking on:










    Results (129 votes), past polls