Beefy Boxes and Bandwidth Generously Provided by pair Networks
Think about Loose Coupling

Re^3: How to process with huge data's

by TomDLux (Vicar)
on Oct 05, 2011 at 19:12 UTC ( #929858=note: print w/replies, xml ) Need Help??

in reply to Re^2: How to process with huge data's
in thread How to process with huge data's

Let's start by generating specifications for what you're trying to achieve.

First start with a short, one-line summary of the project. I'll make one up ... Summarize activity of users at my web site.

Then generate a description, in English, of what you want to achieve. I'll make stuff up, guessing what i think you might be thinking

  • Read the data file, line by line, specifying the edges of a graph.
    • Split the line into a command, the 'from' node, and the 'to' node.
  • For a connection 'command', if the 'from' node and 'to' node have not been seen, add them.
    • If the 'from' node has been seen, but the 'to' has not, then ... do something
    • if the 'from' node has not been seen, but the 'to' node has, then ... do something

Once you have an idea of what you want to achieve, then you can consider how to implement it.

At the moment, if you are asked to "connect 1 2", you create a hash { 1 => 1, 2 => 1}. How do you know whether that is "connect 1 2" or "connect 2 1"?

If this is homework, you should ask your teacher or TA.

As Occam said: Entia non sunt multiplicanda praeter necessitatem.

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://929858]
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others drinking their drinks and smoking their pipes about the Monastery: (16)
As of 2016-10-25 14:16 GMT
Find Nodes?
    Voting Booth?
    How many different varieties (color, size, etc) of socks do you have in your sock drawer?

    Results (320 votes). Check out past polls.