|laziness, impatience, and hubris|
Re: Graphing HUGE Social Networksby mattr (Curate)
|on Mar 20, 2008 at 17:14 UTC||Need Help??|
Certainly the human eye can process a 1 megapixel photo, so it's the way you represent it. First why are you doing it? (besides that it's fun).
You might be interested in googling for: visualizing large graphs
So these different approaches tend to use interactive zoomable charts, and show details that can be made sense of when seen even from far away.
FWIW I remember once I was involved with a company doing Y2K remediation and I saw the visualizations IBM had cooked up. They may not have been as complex as yours but resembled your data, it was to describe safety of code in different programs. Each program was represented by a circle of dots and each circle was drawn like a pie chart in a way, with colorings of segments indicating their safety. It was just a bunch of these pie charts in a big table but from far away you could tell which programs were most complex or dangerous (more red). So maybe you need to think about what is most important to represent.
That, and also to realize that 100M edges is extremely complex. Probably if you really want to grasp what is going on you should try some different ways, some which shrink groups down to a small number of pixels and other ways that allow you to zoom in more and forget people outside the group.
For example look at the picture on the large-scale rdf graph visualization page. The Software Options section half-way down explains that GraphViz is aimed at making nice pictures of reasonably sized graphs. Since yours isn't reasonable you might want to consider using another tool or at least not drawing straight graphs but doing something else. It says Walrus is good for around 100K nodes and Tulip for about 1M nodes/edges (they have a social network graph sample).
Sorry I couldn't be of more help.