Re (tilly) 1: Future of FBP

[tilly]

I believe that I am the one who most often brings up the idea of flow-based programming around here. I mostly heard about it from Stephen White in a Ruby mailing list, and thought it was an interesting idea. Another which looked related to my eye was toontalk, which was mentioned at Re: Any interesting philosophy of programming articles to recommend?. Unfortunately while interesting, the overhead to get it going in my current job didn't make it look like it would pay off.

If you want to get more of an uptake in Perl, Ruby, etc I would suggest that the best first step is to try to create a nice programming environment (in Java or otherwise) for the hooking together of components, and then document what exactly you need to do to be able to build components in any language. That will allow people to do the component building in any language that they want, and then put them together in your toolkit. If people are interested in duplicating the basic programming environment, they can.

An alternate approach is to instead of re-creating the IPC based approach, instead take a look at what people are doing with other programming areas, and see if you can add to them what makes flow-based programming work as a concept. For instance take a look at this article on XML::SAX::Machines and see whether you can implement the basics of flow-based programming in terms of XML filters talking to each other. (My gut feeling is that it is doable, though possibly the idea of SAX machines will need enhancement.) If so, then engage key people involved in that in a dialog and try to meet in the middle. I think it is easier for people to get excited if they see themselves running towards a future goal, than if they think they are giving up what they have done to start all over.

[Masem]

XML::SAX::Machines is probably half of the framework one needs to get something like this going. I've looked at FBP (or what I've previously considered in perl as Perl Beads), and while the concept is easy to conceptualize and start to put together, the more difficult part is working on the standard that you would use for the data transfer. Using XML as intermediates makes great sense here, so this might be the best way to go.

Right now, without having done anything yet with X:S:Machines, the problem that I envision is the issue of synch vs asynch operation. My understanding is that with Machines, say you chain 100 filters on it with one generator object. When I send any SAX event from the generator object, that event has to be handled by each filter in order, assuming that those filters process and resend the event upstream. But now that I type this out (don't you just hate that? :-), I realize that one can have filters that get sax events but don't do anything until certain conditions are met. This would be necessarily in a case where one might have a Machine that worked line-by-line on a file, but diverged at some point to two different processing paths, and then remerged back to a single path. At some point, at the merge, you need to make sure that you don't send the data upstream until you get a matching set of elements from the divergant point. Again, here's where XML comes in handy, because these bits of XML that 'flow' around would contain history of where they came from and where they're going.

I was thinking about FBP earlier this week and saw the Machines article on perl.com, so I'm considering trying it this weekend to see if something reasonable can be done.

-----------------------------------------------------
Dr. Michael K. Neylon - [e-mail] || "You've left the lens cap of your mind on again, Pinky" - The Brain
"I can see my house from here!"
It's not what you know, but knowing how to find it if you don't know that's important

[jpm]

Here is an example of how JFBP (the Java implementation of FBP) might specify a network to read XML, parse it and throw away the result (yes it was a test):

public class ProcXML extends Network {

    
    protected void define() throws Throwable {
        connect(component("Read", ReadText.class),
            port("OUT"),
            component("XMLToObj", XMLToObj.class),
            port("IN"));

        connect(component("XMLToObj"),
            port("OUT"),
            component("Discard", Discard.class),
            port("IN"));        
        
           initialize(new FileReader("c:\\com\\jpmorrsn\\eb2engine\\te
+st\\data\\myXML3.txt"),
            component("Read"),
            port("SOURCE")); 

        
        }


    public static void main(String[] argv) {
        new ProcXML().go();
        }
}

It names 3 components, and links them together using named ports. It is actually not too dissimilar from the SAX machine syntax, but your network can be as complex as you like.

XMLToObj is an XML parser I wrote that runs in the JFBP environment.

Your comment in the second para seems to suggest that you can use such a specification to link components written in different languages - I didn't know you could do that in Java. Where would I look to find out?

[tilly]

I don't know how you would do it in Java. But if your components live in different processes that communicate through IPC, there is no reason that you can't implement each component in a different language.

In Perl you can lift the requirement of separate processes through liberal use of the Inline module.

[jpm]

I really liked what you had to say in one of the discussions about not having a feeling for application development until you had been doing it for a while (maybe that's my interpretation of what you said). Is there a watering-hole (on the Net) where people who think this way hang out?!

I don't understand what a SAX event is. To me, an event is something happening at a point in time - in FBP, it's usually the arrival of a data chunk (IP) at a network node, but it could be an external event, like completion of I/O. Also, sending anything upstream is un-FBP-like!

Last comment: most of the SAX:Machine functions have counterparts in FBP, but in a more general way. I don't see a need for special functions like Manifold....