Beefy Boxes and Bandwidth Generously Provided by pair Networks
XP is just a number
 
PerlMonks  

Re^3: How to process with huge data's

by TomDLux (Vicar)
on Oct 05, 2011 at 19:12 UTC ( #929858=note: print w/ replies, xml ) Need Help??


in reply to Re^2: How to process with huge data's
in thread How to process with huge data's

Let's start by generating specifications for what you're trying to achieve.

First start with a short, one-line summary of the project. I'll make one up ... Summarize activity of users at my web site.

Then generate a description, in English, of what you want to achieve. I'll make stuff up, guessing what i think you might be thinking

  • Read the data file, line by line, specifying the edges of a graph.
    • Split the line into a command, the 'from' node, and the 'to' node.
  • For a connection 'command', if the 'from' node and 'to' node have not been seen, add them.
    • If the 'from' node has been seen, but the 'to' has not, then ... do something
    • if the 'from' node has not been seen, but the 'to' node has, then ... do something

Once you have an idea of what you want to achieve, then you can consider how to implement it.

At the moment, if you are asked to "connect 1 2", you create a hash { 1 => 1, 2 => 1}. How do you know whether that is "connect 1 2" or "connect 2 1"?

If this is homework, you should ask your teacher or TA.

As Occam said: Entia non sunt multiplicanda praeter necessitatem.


Comment on Re^3: How to process with huge data's

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://929858]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others browsing the Monastery: (10)
As of 2014-08-20 09:17 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    The best computer themed movie is:











    Results (108 votes), past polls