enhering has asked for the wisdom of the Perl Monks concerning the following question:
Hi. I'm building a data aquisition system in Perl that colects data at the rate of 10 cicles per second, and I'd like to show that in a graph on a Tk window.
If I use Tk::Canvas and create boxes or dots for each aquired data point, soon I'll fill all the memory because, if I understood correctly, each of those points is an entity on Tk.
Using Imager with Tk::photo was tried too, but the convertion from Imager format to PNG, as stated on Tk::Photo example, using base64 takes too long.
GD was also tried, but we need more functionality for the generated graphics.
In fact, what I'm looking for is a suggestion of a way to draw primitives on a canvas that are not items or objects, as was done in the past on VGA screens. The Tk::Photo + Imager solution would be fine if the conversion did not take so long.
Can you suggest me a way of solving this problem?
The following piece of code examples the problem. In a athlon64 notebook it takes 70% of the CPU to run.
#!/usr/bin/perl
use strict;
use Tk;
use Tk::Photo;
use MIME::Base64;
use Tk::PNG;
use Imager;
my $xsize = 200;
my $ysize = 200;
my $cicle;
my ($x, $y) = (int($xsize/2), int($ysize/2));
my $image = Imager->new(xsize=>$xsize, ysize=>$ysize);
my $blue = Imager::Color->new( 255, 255, 255 );
my $image_data;
$image->write(data =>\$image_data, type=>'png')
or die "Cannot save image: ", $image->errstr;
$image_data = encode_base64($image_data);
my $main = MainWindow->new;
my $tk_image = $main->Photo(-data => $image_data);
my $screen = $main->Label(-image=>$tk_image)->pack;
my $id = $main->repeat(100,\&move);
MainLoop;
sub move {
my ($dx, $dy);
my $i = int(rand(100));
if (($i/5) == int($i/5)) {
$dx = 0;
}
elsif (($i/2) == int($i/2)) {
$dx = 1;
}
else {
$dx = -1;
}
my $j = int(rand(100));
if (($j/5) == int($j/5)) {
$dy = 0;
}
elsif (($j/2) == int($j/2)) {
$dy = 1;
}
else {
$dy = -1;
}
if (($x + $dx) > $xsize) {
$x = $xsize;
}
elsif (($x + $dx) < 0) {
$x = 0;
}
else {
$x += $dx;
}
if (($y + $dy) > $ysize) {
$y = $ysize;
}
elsif (($y + $dy) < 0) {
$y = 0;
}
else {
$y += $dy;
}
$image->setpixel(x=>$x, y=>$y, color=>$blue);
$cicle++;
print "Cicle $cicle. Coords: $x, $y. Diference $dx, $dy.\n";
$image->write(data =>\$image_data, type=>'png')
or die "Cannot save image: ", $image->errstr;
$image_data = encode_base64($image_data);
$tk_image->configure(-data => $image_data);
$screen->configure(-image=>$tk_image);
}
Re: Real time data graphs.
by RobPayne (Chaplain) on Oct 07, 2006 at 19:06 UTC
|
enhering, have you tested using Tk::Canvas? How much memory did it take up to graph the number of points that you expect to plot? It doesn't make logical sense that you will necessarily fill all available memory because each acquired data point takes some amount of memory, because at some point you are going to hit "the end" of your graphing period.
The length of your graphing period limits the amount of memory that you will need. (Points that are no longer graphed should no longer take up memory for Tk::Canvas.) That might mean that the memory requirements are very large, if your graphing period is long, but it will be finite. Whether your very long graphing period (assumption) fills all of memory is a question of how much memory is required to graph all of the data points that you have at that length.
That's why I asked what you had measured in terms of memory requirements for any testing that you had already done. | [reply] [Watch: Dir/Any] |
|
RobPayne, I may need to monitor the temperature of a system for days, at 864000 points per day. The only possibility I can see of using Tk::Canvas in this situation is by working with as many points as there are horizontal pixels in the canvas and reposition them when the graph fills up.
| [reply] [Watch: Dir/Any] |
|
| [reply] [Watch: Dir/Any] |
Re: Real time data graphs.
by BrowserUk (Patriarch) on Oct 07, 2006 at 20:49 UTC
|
If I use Tk::Canvas and create boxes or dots for each aquired data point, soon I'll fill all the memory
How about using a line item, and use the itemconfigure method to add new points to the line? That way you would only use one entity and get a line graph of the temp against time.
GD was also tried, but we need more functionality for the generated graphics.
What beside plotting points did you need for your graph that GD didn't provide?
If GD really cannot provide what you need for legends, axis and stuff (though I cannot see why it wouldn't), the you might consider positioning the Tk::Photo with just the points plotted over the top of a tk::Canvas on which you draws the axis, legends etc.
Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
Lingua non convalesco, consenesco et abolesco. -- Rule 1 has a caveat! -- Who broke the cabal?
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.
| [reply] [Watch: Dir/Any] |
Re: Real time data graphs.
by fmerges (Chaplain) on Oct 07, 2006 at 21:39 UTC
|
Hi,
I'm using rrdtool, for meteorological data an also for general telemetry data, as for example, hidrostatic system, telescope tracking, and a large of etceteras...
Take also a look at PGPLOT and PGPLOT::Simple if you need to do some special temperatures related graphs...
Regards,
fmerges at irc.freenode.net
| [reply] [Watch: Dir/Any] |
Re: Real time data graphs.
by Joost (Canon) on Oct 07, 2006 at 18:02 UTC
|
Unless you can find a pre-build solution for drawing graphs quickly in Tk ( Tk::Graph looks promising, but I haven't tried it ), you might want to take a look at SDL.
| [reply] [Watch: Dir/Any] |
Re: Real time data graphs.
by Khen1950fx (Canon) on Oct 07, 2006 at 21:12 UTC
|
If you like Tk::Photo but just want to speed it up, then how about trying Tk::Task. It'll break up the data into smaller pieces to "allow multiple tasks to proceed at once"and handle events between pieces. It would be more responsive. See: Tk::Task | [reply] [Watch: Dir/Any] |
Re: Real time data graphs.
by Koosemose (Pilgrim) on Oct 08, 2006 at 03:31 UTC
|
Just an idea for an alternate way to go about it. Consider having Tk::Canvas only show a finite time period, say the last hour for example, Continue to save the data points in an array, and at program completion use Image::Magick or some similar graphics generating package generate a graph containing the full graph.
Not certain if it would work for your particular task, just figured I'd offer another potential way to attack the problem, and it might help solve the problem RobPayne mentioned of only a limited amount of space to draw in.
Just Another Perl Alchemist
| [reply] [Watch: Dir/Any] |
|
The idea is to show only the last n minute period of aquisition and n should be configurable by the user. With the resources available now, the best way to do this seems to be by working with as many Tk points as there are horizontal pixels in the graph and make a transformation on the data so that the data points are represented by Tk points.
But this is makes things unnecessarily complex. The best would be to be able to use imager primitives directly on a canvas, in some way.
| [reply] [Watch: Dir/Any] |
|
While poking around in the Tk Docs, I noticed Tk::Photo can take a -data option, I was attempting to see if this could lead to a solution by using Imagemagick to generate an image in memory, and use it as the input for the -data option, and just update it via a configure, unfortunately I can't figure out how to get Imagemagick to output the image data in a compatible format (it's ImagetoBlob method doesn't seem to do it quite right) I'm not familiar with any image packages other than Imagemagick, but if you are this may be a valuable route to explore, as you will only have a single image to update.
Also tried to do a simple writing an image with Imagemagick, as it seems fairly quick at it, and can draw primitives directly as you want. Eh, I'm a little to tired to try offering advice, can't even get my test script to function properly. If you're still having issues by tomorow I'll see if I can't beat Tk and Imagemagick into submission :)
Happy Coding
Just Another Perl Alchemist
| [reply] [Watch: Dir/Any] |
|
|
Re: Real time data graphs.
by zentara (Archbishop) on Oct 08, 2006 at 11:23 UTC
|
| [reply] [Watch: Dir/Any] |
Re: Real time data graphs.
by vkon (Curate) on Oct 08, 2006 at 17:38 UTC
|
BLT package is graph expert for Tk, see http://vkonovalov.ru/cgi-bin/perl-tcltk-wiki.cgi/38.
This requires Tcl/Tk (via Tcl::Tk CPAN module), so you will have much better Tk library only if you have resources to move further, so this is not always an option for perl/Tk people who afraid to upgrade. | [reply] [Watch: Dir/Any] |
Re: Real time data graphs.
by dwhite20899 (Friar) on Oct 08, 2006 at 21:19 UTC
|
How often do you actually need to view the real time data?
When collected data every 32 ms for a project, I had a thread for collection that kept that cycle, and a thread for visualization that refreshed every 1, 2, or 5 seconds depending on how often a human needed to see it.
Can you afford to show the data points at a lower rate than you collect it? Or graph every 10th point? | [reply] [Watch: Dir/Any] |
Re: Real time data graphs.
by tonyc (Pilgrim) on Oct 08, 2006 at 23:34 UTC
|
I've looked at adding a faster mechanism to get an Imager image into a Tk::Photo, but Tk's internals scared me. Maybe it's time to look again.
I'd planned on adding an Imager wrapper (at the C level) around a Tk::Photo object which would allow you to use Imager methods to draw directly to a Tk::Photo.
I've done something similar with the unreleased Imager::SDL (or viewcvs).
| [reply] [Watch: Dir/Any] |
|
| [reply] [Watch: Dir/Any] |
Re: Real time data graphs.
by zentara (Archbishop) on Oct 09, 2006 at 14:30 UTC
|
See Tk Realtime data aquisition for a demo. It's just the basics, but it shows how to use the coords method to update a curve, and breaking it into segments. It would be nice to add a "floating y-axis" so as you scrolled, you could still read the y-axis value.
| [reply] [Watch: Dir/Any] |
|
|