Flow-based Programming

Panta rhei (Panta rhei) - Everything flows.

Animation of simple diagram

In computer programming, Flow-Based Programming (FBP) is a programming paradigm, discovered/invented by J. Paul Rodker Morrison in the late '60s, that uses a "data processing factory" metaphor for designing and building applications. FBP defines applications as networks of "black box" processes, which communicate via data chunks (called Information Packets) travelling across predefined connections (think "conveyor belts"), where the connections are specified externally to the processes. These black box processes can be reconnected endlessly to form different applications without having to be changed internally. FBP is thus naturally component-oriented.

FBP is a special case of dataflow programming characterized by asynchronous, concurrent processes "under the covers", Information Packets with defined lifetimes, named ports, "bounded buffer" connections, and definition of connections external to the components - it has been found to support improved development time and maintainability, reusability, rapid prototyping, simulation, improved performance, and good communication among developers, maintenance staff, users, systems people, and management - not to mention that FBP naturally takes advantage of multiple cores... without the programmer having to struggle with the intricacies of multitasking!

Bottling factory

FBP involves a significant "paradigm shift" from conventional programming, as it uses an "assembly line" image of data processing, which really has to be used to experience its advantages. Not surprisingly given the hold that the so-called "von Neumann" paradigm has over programming technology, over the last 40+ years, a number of companies have come out with FBP-like or "FBP-inspired" systems (to borrow Joe Witt of Hortonworks' term), which do not involve this paradigm shift, including some of the biggest and most successful companies in the industry, such as IBM, Microsoft, Facebook, Google, and the New York Times (see FBP External References). Unfortunately, without the FBP paradigm shift, these systems cannot fully achieve the advantages listed above, although most of them combine a visual notation with some degree of componentry. We have therefore been forced to use the term "classical FBP" to refer to the FBP architecture described in this web site. It is hard for someone reading documentation to tell whether a given product calling itself "FBP- like" is "classical" or not, but, in general, asynchronism is the litmus test. While the FBP paradigm shift may cause some trouble at first, given the prevalence of the older programming approaches, in fact it embodies a simple mental model (not "simple" as in easy but "simple" as in elegant, to quote Joe Witt), and has proven over several decades to result in applications that are more understandable and maintainable than those built using the conventional (von Neumann) paradigm.

FBP is described in J. Paul Morrison's book, "Flow-Based Programming: A New Approach to Application Development", now in its 2nd edition, which can be obtained in paperback form from Amazon, or in 2 e-book formats: Kindle and epub.

Further down on this web page you will find a bunch of useful links, including some videos demonstrating the use of the FBP diagramming tool, DrawFBP. During FBP's early days, networks were coded by hand, and initially we used pencil and paper, followed by more sophisticated graphical tools, whether or not they played well with FBP software. However, with the advent of good graphical support, we now have a powerful graphical tool (DrawFBP), and in recent years the emphasis has been shifting to doing the diagramming on-line, and then generating the network code (we can see this progression with other flow tools on the market). Now that compile and run functions have been added to DrawFBP, the developer can do much of the development graphically, and on-line.

If you have come across Flow-Based Programming by way of NoFlo or Flowhub, it is recommended that you read FBP vs. FBP-inspired Systems, which attempts to highlight the differences and similarities between FBP proper, or "classical" FBP, (the subject of this web site) on the one hand, and NoFlo and other similar FBP-inspired systems on the other.

We are in the process of "growing" a tutorial which will introduce the reader to the basic concepts of FBP by means of a series of "steps", which use the diagramming tool DrawFBP to progressively expand an application diagram and generate and run code at many of those steps.

FBP Tenets (open-ended)

  • If work is boring, let a robot do it!
  • No one language should try to do everything!
  • The world is asynchronous - don't try to force the systems we build into a synchronous framework!
  • The von Neumann paradigm is only appropriate within a single process! Duh!

Links

  1. Introduction
  2. History
  3. Concepts
  4. Tutorial (under development)
  5. Software:

  6. Examples
  7. Comparison with OO
  8. Comparison with other paradigms and methodologies
  9. PowerPoint presentation on FBP (2007)
  10. Justin Bozonier's "knol" re FBP (2008?)
  11. FBP vs. FBP-inspired Systems
  12. Old C2-style FBP Wiki
  13. Bibliography
  14. Chapters from the first edition - online
  15. External links

Video Interview with J. Paul Rodker Morrison, Summer 2013

In addition, there are six Youtube videos about DrawFBP, illustrating a number of basic ("classical") FBP concepts (what we are now calling "FBP-inspired" or "FBP-like" systems do not necessarily contain all of these, although DrawFBP should be able to support most of these systems):

  1. DrawFBP video #1
  2. DrawFBP video #2
  3. DrawFBP video #3
  4. DrawFBP video #4
  5. DrawFBP video #5 - concept of "subnets", both at design and implementation time
  6. DrawFBP video #6 - simple interactive systems using WebSockets, with demo of JavaFBP-WebSockets (JavaFBP and HTML5)

Finally, a video presentation at a Meetup in Toronto in Sept. 2013 organized by Paul Tarvydas, CEO of Tarvydas-Sanford Controls Inc., and Dann Toliver, also containing a few jokes and reminiscences...

Google group on Flow-Based Programming

C2-style Flow-Based Programming Wiki (last maintained in 2010)

More general FBP Wiki by Vladimir Sibirov, et al., intended to cover both "classical" FBP and FBP-inspired approaches (still under development )

FBP Specification wiki - under development by Vladimir Sibirov

Twitter hashtag: #FlowBased