Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl Monk, Perl Meditation

Re^3: How to process with huge data's

by TomDLux (Vicar)
on Oct 05, 2011 at 19:12 UTC ( #929858=note: print w/replies, xml ) Need Help??

in reply to Re^2: How to process with huge data's
in thread How to process with huge data's

Let's start by generating specifications for what you're trying to achieve.

First start with a short, one-line summary of the project. I'll make one up ... Summarize activity of users at my web site.

Then generate a description, in English, of what you want to achieve. I'll make stuff up, guessing what i think you might be thinking

  • Read the data file, line by line, specifying the edges of a graph.
    • Split the line into a command, the 'from' node, and the 'to' node.
  • For a connection 'command', if the 'from' node and 'to' node have not been seen, add them.
    • If the 'from' node has been seen, but the 'to' has not, then ... do something
    • if the 'from' node has not been seen, but the 'to' node has, then ... do something

Once you have an idea of what you want to achieve, then you can consider how to implement it.

At the moment, if you are asked to "connect 1 2", you create a hash { 1 => 1, 2 => 1}. How do you know whether that is "connect 1 2" or "connect 2 1"?

If this is homework, you should ask your teacher or TA.

As Occam said: Entia non sunt multiplicanda praeter necessitatem.

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://929858]
and all is quiet...

How do I use this? | Other CB clients
Other Users?
Others chanting in the Monastery: (1)
As of 2018-05-27 23:54 GMT
Find Nodes?
    Voting Booth?