While I don’t share Tim O’Reilly’s enthusiastic belief that Yahoo!’s new Pipes service is a milestone in the history of the internet, it is an interesting modern day implementation of an old and reliable Unix construct. The premise is that web services produce syndication feeds, which can then be modified as queries for other web services. Considering the openness of web services, the standardization of data structure, and the ubiquitous nature of syndication feeds, it is a concept whose time has come.
Yahoo’s* Pipes is a way for non-programmers to create a means of taking the output of a feed or group of feeds and transmitting them to or ‘piping’ them to another service or services. It doesn’t require programming language, but it does require understanding of programming constructs, such as For Each, filters, as well as understanding what each web service requires as to parameters. I would say its a long way from being the way of enabling mash-ups for everyone, but it’s a good step in the right direction.
I played around with it this morning and could get some pipes to work, but not others, even though I more or less re-created pipes examples like those given as examples. The alpha state of this tool was demonstrated by the fact that a pipe would work one moment, and then stop working the next. I would also say that the application isn’t as intuitive as it would first seem, and the promised tutorials will be needed for programmers and non-programmers alike. However, if you use the debugger, your job is immeasurably simplified.
Danny Ayers mentions about pipes only supporting RSS 2.0 feeds, but I was able to get an Atom feed to work. I agree with him that the pipes metaphore is near perfect on the concept, and the UI is rather nice. However, The UI is also something that works when browsers support a common graphical API, such as SVG, or at a minimum, the Canvas object. You can do almost anything in a web page browser once we have universal agreement on one or the other, or both. It does work in IE, so I would say the application is making use of the SVG/VML plug-in from Adobe. Whatever you do, don’t use Firebug in Firefox on the Mac, to try and peek into the innards, because you’ll crash your browser.
The major area of failure with this example is it utilizes a graphical interface when it doesn’t require a graphical interface. The ability to create a pipe between applications could be managed using traditional forms or even a text editor. Expanding the interface would enable this application to be open to all people, rather than just those with scripting turned on and working eyes, working arms, fingers to push mouse buttons, and so on. We have to separate the concept from the ‘coolness’ of the UI: if we marry the two, we’re heading down the wrong path.
One other area where the example is limited is that it does go to the original source for the feeds, when it would be better if it had the ability to store feed requests in a ‘cloud’ for each use of the tool, and then check the cloud, first, before going directly to the source. That way the application doesn’t unduly hammer endpoints.
A last area of improvement is to provide an API to access that content analysis module. I haven’t worked with Yahoo’s APIs over much (I have the UI objects), and perhaps this already exists? All the other components are ones that we can implement ourselves, but that Yahoo content analysis seems like its a direct shot into the Yahoo search engine functionality.
Yahoo is to be commended for dancing nicely on the edge. Once we’re past the alpha state, and hopefully once we’re past the accessibility problems, it will be interesting to see if the only people who use the service are programmers, or if it will reach a more general audience. I’d like to see this extended to a more distributed solution that’s not dependent on one service.
I look for a hook-in into this from Flickr, which is Yahoo’s naive user test bed. All of us interested in the semantic web should get some real insights into how non-techs ‘mash-up’ data.
update
Yup it’s using the canvas object, and Google’s open source Explorer Canvas work around for IE.
*I don’t use Yahoo! anymore because the exclamation point irritates me.
