Beefy Boxes and Bandwidth Generously Provided by pair Networks
Problems? Is your data what you think it is?

Re^3: Out of Memory when generating large matrix

by Anonymous Monk
on Mar 06, 2018 at 17:51 UTC ( #1210421=note: print w/replies, xml ) Need Help??

in reply to Re^2: Out of Memory when generating large matrix
in thread Out of Memory when generating large matrix

I'm not the Anon above, or you-know-who, but I'll gladly oblige!

Tally up the counts of tokens that appear on field #2.

$ cat datafiles/* | sort -k2 | uniq -c -f1

Can we please see your code now? Or maybe yet another diatribe on how the approach is simply wrong, does not scale, requires expensive disk i/o, is inelegant, hazardous, broken, politically incorrect or in some other way unsuitable.

Replies are listed 'Best First'.
Re^4: Out of Memory when generating large matrix
by Your Mother (Bishop) on Mar 06, 2018 at 22:08 UTC

    Well, I was doing a bit since SunnyD cannot answer any question where a technical answer is required and the spec as stated is risible. It demands no "custom programming." Your command pipeline is customized for the problem. It violates the solution spec.

    As to "see your code now," was that for me or the anonymous monk formerly known as sundialsvc4? I made no assertions at all about any of the code or that I was one with an answer here. I merely inquired as to the ingredients of the snake oil being advertised. This problem domain is quite a ways from anything I'm good at. :P

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://1210421]
NodeReaper says "Shhhh! Be vewy vewy quiet, I'm hunting wumpus"

How do I use this? | Other CB clients
Other Users?
Others lurking in the Monastery: (4)
As of 2018-04-20 22:47 GMT
Find Nodes?
    Voting Booth?