Future of FBP

[jpm]

A number of monks have said kind things about Flow-based programming (FBP), which is near and dear to my heart. I am a very new novice in the Monastery, and am not yet a Perl user (blush), but I thought I would ask for feedback on what direction we should take FBP. So far, neither the business world nor academia seem terribly interested, although, as I have tried to show in my book, I believe FBP could really revolutionize the way we build and maintain business systems.

We do have a Java implementation of FBP - should we have Perl, Python, Ruby implementations...? Or go in a completely different direction?

TIA

PS I went to an English public school, so I do have some monastic experience...

[tilly]

I believe that I am the one who most often brings up the idea of flow-based programming around here. I mostly heard about it from Stephen White in a Ruby mailing list, and thought it was an interesting idea. Another which looked related to my eye was toontalk, which was mentioned at Re: Any interesting philosophy of programming articles to recommend?. Unfortunately while interesting, the overhead to get it going in my current job didn't make it look like it would pay off.

If you want to get more of an uptake in Perl, Ruby, etc I would suggest that the best first step is to try to create a nice programming environment (in Java or otherwise) for the hooking together of components, and then document what exactly you need to do to be able to build components in any language. That will allow people to do the component building in any language that they want, and then put them together in your toolkit. If people are interested in duplicating the basic programming environment, they can.

An alternate approach is to instead of re-creating the IPC based approach, instead take a look at what people are doing with other programming areas, and see if you can add to them what makes flow-based programming work as a concept. For instance take a look at this article on XML::SAX::Machines and see whether you can implement the basics of flow-based programming in terms of XML filters talking to each other. (My gut feeling is that it is doable, though possibly the idea of SAX machines will need enhancement.) If so, then engage key people involved in that in a dialog and try to meet in the middle. I think it is easier for people to get excited if they see themselves running towards a future goal, than if they think they are giving up what they have done to start all over.

[Masem]

XML::SAX::Machines is probably half of the framework one needs to get something like this going. I've looked at FBP (or what I've previously considered in perl as Perl Beads), and while the concept is easy to conceptualize and start to put together, the more difficult part is working on the standard that you would use for the data transfer. Using XML as intermediates makes great sense here, so this might be the best way to go.

Right now, without having done anything yet with X:S:Machines, the problem that I envision is the issue of synch vs asynch operation. My understanding is that with Machines, say you chain 100 filters on it with one generator object. When I send any SAX event from the generator object, that event has to be handled by each filter in order, assuming that those filters process and resend the event upstream. But now that I type this out (don't you just hate that? :-), I realize that one can have filters that get sax events but don't do anything until certain conditions are met. This would be necessarily in a case where one might have a Machine that worked line-by-line on a file, but diverged at some point to two different processing paths, and then remerged back to a single path. At some point, at the merge, you need to make sure that you don't send the data upstream until you get a matching set of elements from the divergant point. Again, here's where XML comes in handy, because these bits of XML that 'flow' around would contain history of where they came from and where they're going.

I was thinking about FBP earlier this week and saw the Machines article on perl.com, so I'm considering trying it this weekend to see if something reasonable can be done.

-----------------------------------------------------
Dr. Michael K. Neylon - [e-mail] || "You've left the lens cap of your mind on again, Pinky" - The Brain
"I can see my house from here!"
It's not what you know, but knowing how to find it if you don't know that's important

[jpm]

Here is an example of how JFBP (the Java implementation of FBP) might specify a network to read XML, parse it and throw away the result (yes it was a test):

public class ProcXML extends Network {

    
    protected void define() throws Throwable {
        connect(component("Read", ReadText.class),
            port("OUT"),
            component("XMLToObj", XMLToObj.class),
            port("IN"));

        connect(component("XMLToObj"),
            port("OUT"),
            component("Discard", Discard.class),
            port("IN"));        
        
           initialize(new FileReader("c:\\com\\jpmorrsn\\eb2engine\\te
+st\\data\\myXML3.txt"),
            component("Read"),
            port("SOURCE")); 

        
        }


    public static void main(String[] argv) {
        new ProcXML().go();
        }
}

It names 3 components, and links them together using named ports. It is actually not too dissimilar from the SAX machine syntax, but your network can be as complex as you like.

XMLToObj is an XML parser I wrote that runs in the JFBP environment.

Your comment in the second para seems to suggest that you can use such a specification to link components written in different languages - I didn't know you could do that in Java. Where would I look to find out?

[tilly]

I don't know how you would do it in Java. But if your components live in different processes that communicate through IPC, there is no reason that you can't implement each component in a different language.

In Perl you can lift the requirement of separate processes through liberal use of the Inline module.

[jpm]

I really liked what you had to say in one of the discussions about not having a feeling for application development until you had been doing it for a while (maybe that's my interpretation of what you said). Is there a watering-hole (on the Net) where people who think this way hang out?!

I don't understand what a SAX event is. To me, an event is something happening at a point in time - in FBP, it's usually the arrival of a data chunk (IP) at a network node, but it could be an external event, like completion of I/O. Also, sending anything upstream is un-FBP-like!

Last comment: most of the SAX:Machine functions have counterparts in FBP, but in a more general way. I don't see a need for special functions like Manifold....

[rjray]

Could you provide some of us with a brief overview/definition of FBP? In fact, I wouldn't be offended by a link to your book-- you can link directly to the purchase page at fatbrain.com by means of the ISBN number as such:

    [isbn://XXXXXXXXX]

Where XXXXXXXXXX is of course the ISBN code, minus the dashes. If you do, use the preview facility to make sure that you have the ISBN the same way FB does by following the link (I've referred to books where a leading 0 was optional, and others where FB required it). Plus, if anyone buys your book via that link, PerlMonks gets a spiff off of it.

--rjray

[grinder]

hmm, taking a wild guess, www.jpaulmorrison.com/fbp/ might have something to do with it. The name fits :)

Let's see know, that would make the fatbrain link something like New Impoved Data Factory, although it appears that Fatbrain doesn't stock it directly.


print@_{sort keys %_},$/if%_=split//,'= & *a?b:e\f/h^h!j+n,o@o;r$s-t%t#u'

[jpm]

You're right about the web site, and FB has the ISBN correct (ISBN:044217715), but the book is called "Flow-Based Programming" - although their title does sound familiar....

I suspect nobody stocks it 'coz AFAIK all the remaining copies are in a couple of closets at home - but I would be delighted to send a copy to anyone who is interested - price negotiable. Amazon also lists it and uses me as a supplier, but it sounds like they're not too popular in certain quarters.

[Matts]

If XML::SAX::Machines isn't what you're looking for, then take a look at Pipeline.pm (on CPAN). It's basically like SAX, but without the tie to XML-like data structures, and with potentially pluggable flow-control.

[toma]

Agilent Technologies sells a program called VEE that implements flow-based programming. Many non-programmers like it. Experienced programmers tend to prefer to write code. It is easy to draw a bowl of spaghetti!

The underlying implementation of VEE looks like LISP. If the underlying implementation was XML, it would be more stylish.

Circuit designers typically use flow-based design in the form of schematics. When circuit complexity gets high, schematics become problematic. The electronics industry solved this problem by converting almost all complex digital designs to Verilog or VHDL, which is code instead of flow.

Dominus tought a course in regular expressions that used flow-based diagrams to show flow in a regex. I thought it was helpful and intuitive. It would be nice to make such a tool for designing and documenting perl regexes, perhaps taking advantage of automatic graph layout with GraphViz.

Update By "underlying implementation," I mean that the program that does the work behind the scenes of the UI appears to be written in LISP. I surmised this because if you look at a VEE program file, it looks like LISP code.

Flow based diagrams help with visualization, but they do not work with large problems. Eventually the diagrams become too complex to see on the screen or print out. Hierarchical diagrams help with this, but do not solve the problem as well as a well-designed textual description. This is not an intuitive result. It was discovered when the diagrams became very large, such as happened when integrated circuit designs reached about 1_000_000 transistors.

It should work perfectly the first time! - toma

[jpm]

I'm sort of glad I didn't see VEE before building my picture-drawing tool - I would have given up before I started! But I needed a way to diagram FBP networks - I am very visual, although I have known some programmers who are not...

I don't understand the distinction between code and diagrams: you can code FBP networks without drawing pictures, but I wouldn't be able to visualize them. A chap I know built a network of 200 nodes using stepwise decomposition and never once drew a picture! I can't do that, but it worked for him!

I also don't understand what is meant by "underlying implementation". You can encode your network in XML-style, and in fact we did this on a project I worked on, but basically it's just a list of connections, so I don't see where it matters much what notation you use... Maybe I'm missing something?