I really enjoyed reading Secret Geek's (AKA Leon Bambrick) post today: Workflow software: I'm calling the bluff. Leon has a major gripe about workflow tools, especially the graphical tools that are their real selling point. Non-programmers love graphical tools. Why? Because code scares them, whereas the pretty pictures seem to make so much more sense. Non only that but, wonder of wonders, you can get business users to design the software. Wow, think of the savings of removing those expensive, scruffy awkward coders. Leon throws down a challenge:
"Show me a working 'business analyst' -- one, who is not now nor has ever been a coder -- who successfully designs 'business workflows' using an off the shelf tool, and who didn't require *any* expensive training, and who achieves their task in less time and with more precision than a coder. And who doesn't need to call technical support for help at the time.
Show me just one.
World wide.
I can wait. I give you one month. Nah, screw it. I give you eternity."
Somehow, I don't think he's going to see any challengers.
Visual tools make for great marketing. That's why so many companies spend so much money on Enterprise Workflow/Service Bus/Business Rule Engines. It's simple evolutionary psychology, Humans are evolved from fruit eating primates, we instinctively like anything with lumps of fruity colors. Code doesn't have that, it looks like something scary and mathematical. It's very hard to sell. But show some management type a colorful tool where you can simply drag and drop a few shapes to make a simple workflow and they're sold.
So what's the problem? It's simple: software is complex. Now that's not just a bit tricky, like sodoku say; it's seriously complex. Most visual tools give you little boxes that do an operation like a decision or maybe a calculation and those boxes are joined together with little lines. Hey, it's just like a programming language, except prettier. The problem is it's just like the simplest programming language. In fact it's a bit like the orginal BASIC:
I used to write very simple BASIC programs on my TRS-80 as a child. But even they sometimes had hundreds of lines of code. Imagine a picture with a hundred little boxes and several hundred little lines connecting them. You can only see the whole thing when it's zoomed too small to read any of the captions, and if you can read the captions you'll only see at best a dozen boxes.
The thing is, we've moved on a little since TRS-80 BASIC. Modern programming languages are fundamentally the distillation of fifty years of lessons learnt; pretty much every language innovation: loops, conditionals, stacks, functions, and all the technology of object oriented programming, not to mention the functional stuff, is there to help us manage the awesome complexity. There are hundreds of books and a million web pages written on the subject: algorithms, object orientation, design patterns, refactoring, testing, the list is endless. A good programmer has to work really hard to keep up with all this stuff. But the result is that with the current state of the art we can do a reasonable job at keeping this horrendous complexity under control.
If you use a visual programming tool, you're throwing all this away. You cannot write anything other than a simple demo program with a visual tool. If you try you will fail.
There's one place where I think visual tools succeed and that's when they're used to represent static structure. I would much rather look at a data model in SQL Server's Database Diagram tool than trying to grep a long list of create table statements, and of course, it's far easier to work on a user interface with a graphical designer than by hand coding the construction and mapping of loads of objects. I actually rather like the class designer in Visual Studio. But I will never use a graphical tool to create program flow or logic.
So you started in a TRS80 as well, but to the main point. In general I agree that I would prefer to work in non-graphical environments. However, my experience is that we are being forced down the visual route due to tooling in many environments. This said I do think the graphical has a place, they can be very productive especially if you can componentise your handwritten functionality and use the graphical tool for final logical assembly, but I do doubt anything useful will come out of letting business user wire the code together. Testing alone would be a nightmare. I like the idea of the business user validating a system ‘on their own’ aka Fitnesse, but I think that is about the limit
ReplyDeleteI still think there are far too many issues with most visual tools. Only if they are a view onto underlying program structure, like the class designer in VS or the database diagram tool in SQL Management Studio are they acceptable. Something that gives you a useful view on your code.
ReplyDeleteTools which persist themselves as binary formats, or even illegible XML and require complicated server based runtimes are always a headache. The issue of source control and continuous integration is enough to rule them out in most environments.
Hi
ReplyDeleteWell, your screenshot appears to be BizTalk, in which case yes I would agree - not many people using that without some training. And being honest I'm not sure I disagree overall - yet. But i'm not sure it's a failing of the tools.
Your analogy seems to have a slight flaw - there's not a one to one mapping of lines of code to boxes. To take an example from that Microsoft world you're showing, one "box" may symbolise transferring data between databases within a transaction - while transforming formats. If you look at something like Windows Workflow Foundation you find graphical representations of concurrency, etc.
All of these would be hundreds of lines of code in Basic. Less in other languages, sure. But it's a higher level of abstraction, and graphical representation is just another of those.
Will we ever see drag and drop programming for the business user which is as powerful as letting a good developer loose? Probably not. But that's not something I'd bet on either.
It has its place. That level of abstraction is still dramatically better than the TRS-80 you refer to. GOTOs and all :)
(If you want to argue about this over a pint, I see you're in Brighton as well!)
Hi Andrew,
ReplyDeleteOK, so you're saying that the 'statement' (the boxes) that Biztalk provides are more powerful than the statements in TRS-80 basic. That's true of course, but it doesn't change the point that the graphical environment itself is a hugely limited programming language.
Anyway, I'm looking forward to meeting you at the geek dinner. It should be a lot of fun.
Mike
Anyone else out there done any technical computing with LabVIEW? I would say that fits the bill of a visual tool that actually works. 10 years ago I worked with it for a few months. Having used BASIC, Modula 2, FORTRAN, C, VB, C# and Python, I still think of LabVIEW as my favourite development environment.
ReplyDeleteUnlike BizTalk (from what I've heard) it doesn't take weeks to get up to speed. Yet it is pretty powerful, effectively supporting inheritance and aggregation (not so sure about polymorphism and overloading, it's been a long time).
The company I worked for had a completely automated factory, making 1000+ refrigerators a day, that ran on LabVIEW. The entire staff of the place was about half a dozen developers, a handful of packers to put packaging onto the fridges, and two old blokes doing earth testing.
By and large I tend to agree with you, but I don't think we are dealing with a basic principle. Rather, I think the vast majority of real-world examples suffer from "solution-seeking-a-problem" syndrome wrapped in a "wouldn't-it-be-cool-if" usability breakdown. That is, overengineered pipedreams with no real end-user connect.
ReplyDeleteYou probably already know about Yahoo! Pipes, which is, by any practical definition, programming. So is DabbleDB, being visual in a very different sense of the word. Central to both is a focus on prototyping from real-world data, assemble-by-example and providing a limited set of simple but powerfull tools.
I'm not sure what to make of it, and we are certainly not anywhere near Business Analyst Nirvana. We probably won't get there on this side of The Singularity anyway, but certain programming-like tasks are going to be a lot easier for the non-coding public.
Well, I don't know about you, but I'm enjoying things like WF that let you design state machines and sequences visually. Of course it's never as simple as the model you have made, but at least you have a reference point which you can then customize or, if you are really pissed off, to take out completely. But having the option there certainly helps, escpecially if you have, e.g., a very simple state machine that fits the constraints of WF precisely.
ReplyDeleteYou're joking right. I'm as hardcore as programmers come, but a business analyst using Excel would kick my ass any day on most business systems I've worked. Many skilled analysts just wish they could use spreadsheets to run their business (without the suck, that is).
ReplyDeleteThe simple fact is that there are more spreadsheet programmers than all other kinds of programmers combined. They are using a declarative, functional programming language and they don't even know it. Obscured in plain view.
Hi SimonTeW, I've not encountered LabVIEW. Checked out their web site, it's a specialist environment for industrial automation right? But as with any graphical programming tool, how do you do source control for example? Also what about refactoring or code generation?
ReplyDeleteHi Jesper, yes I've seen Yahoo pipes, it's pretty cool. But it still suffers from the same problems as all other graphical programming tools, fine for the simplest possible scenarios, but you wouldn't want to try and scale it. Compare it to Ayende's ETL tool which does a similar thing but is based on a DSL style API and the difference in power and flexibility is tremendous.
ReplyDeleteHey, after the singularity you won't need business analysts, well not of the unaugumented flesh and blood kind :)
Hi Dmitri, you're right, the only good graphical programming tool is one that provides a visualization of code, then at least you can ignore it if it doesn't suit what you're doing. As I said in the post, I think tools which allow you to visualize static structure are great. I wouldn't design a WPF UI without Blend for example.
ReplyDeleteHi Anonymous, spreadsheets are wonderful tools, but you can hardly compare them to the box and connector style graphical programming environments that I was ranting about. Have you seen resolver? It's a really interesting take on spreadsheet-as-code-generator that might be just the thing your analysts are looking for.
ReplyDeleteBut actually it is just visualization.
ReplyDeleteThat is never bad.
Think about an IDE, think about touchscreens where you can drag objects around and rearrange them, composite them and so on.
Basically just because some people like to do things in an awkward way, doesnt mean that these things are in general bad.
Wouldnt it be the best to have a computer/fully integrated operating system where people can command everything voice based? Or text based? Or however-they-like? As flexible and powerful as it needs to be?
"Show me a working 'business analyst' -- one, who is not now nor has ever been a coder -- who successfully designs 'business workflows' using an off the shelf tool, ..."
ReplyDelete1975: "Show me a working business executive who successfully sums a list of numbers on a computer ..."
"I used to write very simple BASIC programs on my TRS-80 as a child. But even they sometimes had hundreds of lines of code. Imagine a picture with a hundred little boxes and several hundred little lines connecting them."
BASIC is practically useless at creating abstractions, which are the core of programming. A visual language is only useless if it's similarly incapable of creating abstractions (but then, so is any language, visual or not).
"pretty much every language innovation: loops, conditionals, stacks, functions, ..."
All existed long before the TRS-80 -- it just took a while for it to make it to cheap PCs. As did powerful visual programming systems, BTW.
"If you use a visual programming tool, you're throwing all this away. You cannot write anything other than a simple demo program with a visual tool. If you try you will fail."
Are you referring to a specific visual programming tool? On the occasion I've used visual tools, I've not had this spectacular mandatory failure you keep mentioning. (OTOH, I've seen spectacular failures with C++ and Perl.)
"There's one place where I think visual tools succeed and that's when they're used to represent static structure."
I think the common thread is not so much "static" but "declarative". And functional programmers have no problem with declarative program flow or logic.
Your entire argument seems to apply to high-level languages, too. Why is your hundreds of lines of BASIC (or other low-level languages) inherent?
LabView. PD (pure data). Max/MSP. These are visual dataflow tools the prove (in my opinion) that if you have the right abstractions and agree to certain limitations, you can do some pretty powerful things very quickly and easily (once you understand the system).
ReplyDeleteHowever, your beef seems to be more directed at selling visual tools as a way for non-programmers to do programming. I'll agree with you: Non-programmers are probably not going to suddenly be able to program using these tools. But they can be very valuable to people who are programmers who want an easier way of working in a particular domain.
One of the biggest problems is that most of the tools you are talking about are probably written by people who know a lot about programming and not much about the business domain that their users are operating in, and therefore end up writing something that looks more like a traditional programming language. Other programmers come along and look at the tool and say, "This isn't programming. How am I supposed to do X?" where X could be "implement a red-black tree" or "create a dictionary and look up key-value pairs". Meanwhile, a business user looks at it and still can't figure out how he's supposed to tie together all the data and math operators to calculate his total expenses for the year.
Your criticism also sounds a bit like the criticism of high-level languages that you would hear a few decades ago. People who did assembly language programming believed that the power that they had to create things like highly optimized loops was very important. And sometimes it is. But for most applications, programmers have accepted that it is easier to tell the complier to generate a loop (using "for" or "while" or "foreach") and trust the computer to do the right thing. In the same way, I don't think it is impossible to see a day in the future where some (but not all) programming tasks can be done in some sort of visual environment, and it will generally be accepted that the loss of absolute control is more than compensated for in the increase in speed and reliability.
Hi,
ReplyDeleteI am using a tool that is between code and visual tools. It helps me a lot because is easier to see complex logic, but at the same time it shows me the code, so as a programmer it helps me to develop my ideas and at same time it gives me the total control of the code.
This tools is something in the middle, you can create your logic (using a flow chart), generate the code, and then you can choose to continue using the tool or just use the generated code.
checkout the tool at http://code.google.com/p/logiccodebuilder/
This is interesting. I was wondering why this post was so popular and after googling I find that someone's posted it on reddit.com. That's very nice, thanks gst!
ReplyDeleteAnyway, a few people have commented on this post on reddit and one, Mazin, points to a post by Gregor Hohpe, one of the authors of Enterprise Integration Patterns. Gregor calls visual tools Doodleware, an excellent name I think.
http://subtextual.org/
ReplyDeleteCheck out the video of the prototype.
Just because it's a difficult egg to crack doesn't mean we shouldn't try to figure out how to instruct our machines in a more visual manner.
I won't dispute the idea that boxes and lines fail to magically impart programming rigor. However, I will say that in my experience boxes and lines are an excellent interface for non-programmers to configure the simple, interesting parts in the middle of complex systems. I work in videogames. At our company we have a "shader network editor" that is basically a boxes and lines code generator. We have dozens of artists generating hundreds of examples each containing hundreds of lines of complex math. Most of them do it by basically fiddling with it until the results look right. What they can do (and break) is limited and there is a lot of magic hidden behind the boxes. The results are not optimal, but they are very cost-effective.
ReplyDeleteExcellent article, I love it.
ReplyDeleteI agree with you entirely. The 'mapping of boxes to LOC' argument is moot, good programming practices like keeping code DRY cover that completely.
Here's the bit of rhetoric that I use on this topic:
ReplyDeleteWe progressed from drawing cave paintings to writing mathematical formulae for a reason.
Most visual tools are tantamount to doing rocket science in rebus drawn by fingerpaints. And it's not because the tools are limited; it's because the medium is limited. See Why looking isn't always seeing for more details on the limitations of visual programming.
So you've never thought "Hmmm, this would look a lot more understandable as a matrix/petri-net/directed-graph."?
ReplyDeleteI would love to be able to create simple visualizations on the fly that represent actual code.
Show me a tool that can visualize my models, views and controllers as three layers and let me see some code pass through them and I'll pay handsomely for it.
I think visual programming is _great_. There are simply a few problems:
1) Programmers are text junkies. We like to write and read, not draw and look.
2) We store code in text-files and it's pretty darn difficult to store the meta-data for the visualizations without immensely cluttering the code.
3) We are stuck on an all-or-nothing top-down approach instead of incrementally adding visualizations to our existing code.
Anyhow, that's my $.02 ...
Totally agree with you!
ReplyDeleteStill, there are people who claim they've had huge successes with their WF implementation. My question is what do they mean by "success"? Most of the WF groupies are MS MVPs who are dying for any new technologies that they can sell to their clients, and their definition of success is "successfully getting a client to pay a hefty bill for a project done using WF". Who cares if the same project could have been done with half the cost? I've personally seen quite a few consultants trying to squeeze WF into every project they come across and they make me sick. WF does have its place in certain scenarios, but for a garden variety 5-month ASP.NET project, it's just not worth it.
"Visual diagrams" definitely have their place, and here's what it is... (read carefully)...
ReplyDelete"A diagram is not sequential in time."
Got that? It's REAL Important!
On your TRS-80, an "IF statement" actually means "IF (AT THIS POINT IN TIME) A > B THEN..."
That's "a fork in the road," not "a rule."
A visual diagram, on the other hand, is "a rule." It is a complex set of constraints, predecessors and successors, expressed in a visual form that "we humans" happen to be genetically programmed to deal with very well.
A suitably-designed (human or computer) algorithm can apply this information to obtain a generalized solution to a particular problem.
The technical term for this is declarative programming. For instance, I can express "a solution for any Soduku problem" in less than 20 lines of gprolog "programming," and this "program" will consist only of a declaration of the problem and the constraints. The gprolog core (written in "C", of course) can then generically be applied to "any such problems," thereby making very short work of Sudoku.
So, whether or not a particular GUI is judged (by you) to be successful at expressing a particular problem, the essential notion of "declaring a solution, rather than an algorithm," is a sound one.