Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl Monk, Perl Meditation

Persistent HTTP connection

by mst226 (Initiate)
on Apr 17, 2014 at 09:31 UTC ( #1082614=perlquestion: print w/replies, xml ) Need Help??
mst226 has asked for the wisdom of the Perl Monks concerning the following question:

Hi everyone.

I am trying to write a web client that connects to an API using HTTP. The API server is the Gephi software ( with the "streaming plugin" which accepts commands to modify a graph in real-time. My goal is to visualize Internet traffic showing end-stations as nodes and connections as edges.

The API was not written by me and there is no way to change its behaviour. (There might be, but this is out of scope)

This API is somehow special since the HTTP connection stays open the whole time (even if it is idle).

So, my plan is to:
1. make a http connection to the server
2. send commands (POST) in JSON format using the established connection
3. wait for a reply which might take a while
4. send more commands
5. wait for replies
and so on.

Only steps 1. and 2. are fixed. There is no order in the following steps. Some commands expect a reply while others do not. Also it is possible to send hundreds of commands without waiting for reply.

I tried to build the web/api client using LWP and IO::Socket::INET, but the connection is closed at some point in time.

Basically it is possible to open/close the connection for every single command, but this limits the performance of the web/api client.

The API is described here:
Is it focused on Java but on the bottom there are examples of the command syntax.

Can you give me some hints how to continue?


Replies are listed 'Best First'.
Re: Persistent HTTP connection
by jellisii2 (Hermit) on Apr 17, 2014 at 11:27 UTC
Re: Persistent HTTP connection
by RonW (Vicar) on Apr 17, 2014 at 16:01 UTC
    Web servers often close connections for various reasons, so your client application has to be prepared to open a new connection as needed, probably also retrying commands. Retrying commands can be tricky if they don't always get a response. Perhaps part of the problem is sending too many non-response commands consecutively.
Re: Persistent HTTP connection
by pvaldes (Chaplain) on Apr 21, 2014 at 21:54 UTC
    If you want to visualize nodes and edges you'll probably want to add graphviz to your plan
      Hello pvaldes.

      Thanks for your answer. Graphviz is an excellent tool which I already use for other projects. In this case my focus is on *real-time* visualization. The information, which become nodes and edges is only known at runtime.

      I think that GraphViz is not capable of that, but not 100% sure...
Re: Persistent HTTP connection
by mst226 (Initiate) on Apr 25, 2014 at 12:36 UTC
    Unfortunately I can't find a good solution for this, so I make every API call a separate http connection using LWP. That was exactly what I did not wanted, but I turns out that my client/server hardware performs at 160 API calls per second.

    Hopefully this is enough...

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://1082614]
Front-paged by Arunbear
[Discipulus]: uh uh.. scent of gods..

How do I use this? | Other CB clients
Other Users?
Others chanting in the Monastery: (11)
As of 2018-06-22 08:41 GMT
Find Nodes?
    Voting Booth?
    Should cpanminus be part of the standard Perl release?

    Results (122 votes). Check out past polls.