Beefy Boxes and Bandwidth Generously Provided by pair Networks
Syntactic Confectionery Delight

Re^2: How to process with huge data's

by arivu198314 (Sexton)
on Oct 05, 2011 at 13:39 UTC ( #929801=note: print w/replies, xml ) Need Help??

in reply to Re: How to process with huge data's
in thread How to process with huge data's

Thanks for your reply

After consulting all the replies, I have moved into array in hashes

Please find my code below, but still i'm struggling lot since it is taking long time to finish this task.

#!/usr/bin/perl use Data::Dumper; my $queryCount=0; my $answeredCount=0; my @connectionValues; open(INP, $ARGV[0]); while(my $inputtext=<INP>) { chomp($inputtext); my ( $cmd, $fnum, $snum ) = split ' ', $inputtext; if ($cmd eq 'q') { for( my $arrVal=0; $arrVal < @connectionValues; $arrVal++) { if (exists $connectionValues[$arrVal]{$fnum} and exists $c +onnectionValues[$arrVal]{$snum}) { $answeredCount++; } } $queryCount++; } elsif ($cmd eq 'c') { if ($fnum ~~ @connectionValues or $snum ~~ @connectionValues) { my $fHashVal; my $sHashVal; my $valid=0; for( my $arrVal=0; $arrVal < @connectionValues; $arrVal++) { next if (!$connectionValues[$arrVal]); if (exists $connectionValues[$arrVal]{$fnum} and not e +xists $connectionValues[$arrVal]{$snum}) { if ($valid) { $connectionValues[$arrVal]={%{$connectionValue +s[$arrVal]}, %{$connectionValues[$sHashVal]}}; delete $connectionValues[$sHashVal]; $sHashVal=0; } else { $connectionValues[$arrVal]{$snum}=1; $fHashVal=$arrVal; $valid=1; } } elsif (exists $connectionValues[$arrVal]{$snum} and no +t exists $connectionValues[$arrVal]{$fnum}) { if ($valid) { $connectionValues[$arrVal]={%{$connectionValue +s[$arrVal]}, %{$connectionValues[$fHashVal]}}; delete $connectionValues[$fHashVal]; $fHashVal=0; } else { $connectionValues[$arrVal]{$fnum}=1; $sHashVal=$arrVal; $valid=1; } } } } else { push(@connectionValues, {$fnum=>1, $snum=>1}); } } } close (INP); print "$answeredCount,".($queryCount-$answeredCount)."\n";

Replies are listed 'Best First'.
Re^3: How to process with huge data's
by TomDLux (Vicar) on Oct 05, 2011 at 19:12 UTC

    Let's start by generating specifications for what you're trying to achieve.

    First start with a short, one-line summary of the project. I'll make one up ... Summarize activity of users at my web site.

    Then generate a description, in English, of what you want to achieve. I'll make stuff up, guessing what i think you might be thinking

    • Read the data file, line by line, specifying the edges of a graph.
      • Split the line into a command, the 'from' node, and the 'to' node.
    • For a connection 'command', if the 'from' node and 'to' node have not been seen, add them.
      • If the 'from' node has been seen, but the 'to' has not, then ... do something
      • if the 'from' node has not been seen, but the 'to' node has, then ... do something

    Once you have an idea of what you want to achieve, then you can consider how to implement it.

    At the moment, if you are asked to "connect 1 2", you create a hash { 1 => 1, 2 => 1}. How do you know whether that is "connect 1 2" or "connect 2 1"?

    If this is homework, you should ask your teacher or TA.

    As Occam said: Entia non sunt multiplicanda praeter necessitatem.

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://929801]
and all is quiet...

How do I use this? | Other CB clients
Other Users?
Others making s'mores by the fire in the courtyard of the Monastery: (4)
As of 2018-08-17 18:37 GMT
Find Nodes?
    Voting Booth?
    Asked to put a square peg in a round hole, I would:

    Results (182 votes). Check out past polls.