Creative Synthesis

Pipes are fascinating. It’s no coincidence that our favorite video game legend is a plumber or that the internet is actually a collection of tubes.

Pipes are all about connections. Taking something from one place and sending it someplace else.

The recently launched Yahoo Pipes is, as eloquently stated by Read/Write Web, a giant relational database that uses the web as the data source. That sounds great and really exciting (and it is). While it is an incredible tool for creative synthesis, there are some problems lurking under the concept.

Mashups are about taking something that is one place, and putting it somewhere that is another place. Pipes are actually a great metaphor here, because literally we want to create connections between different parts of the web, manipulate them (add fluoride) and then send them home. Yahoo pipes sort of gets this. You can take all kinds of data from the web (a huge variety of data in fact), you can do cool stuff with it using lots of cool tools - then you can put it somewhere. However, that somewhere is always yahoo pipes.

A quick scroll through the discussion board highlights this problem:

Basically the question is, what can we do with these pipes. You know, apart from listing them on yahoo’s pipe portal. Yahoo’s response is:

I could understand this from a new web startup, or a slightly more experimental feeling application (like a lab app) but this appears to be a finished published service. The fact that sharing is not at the forefront of this application is rather suggestive about yahoo’s priorities.

This actually raises some problems I’ve been having with the end state of mashups in general (i.e. what we do with cool data and content after we’ve manipulated, aggregated, and organized it. There aren’t a lot of solutions for what you can actually do with this stuff outside of blog integration, general site integration or maybe (for the location stuff) posting it on a map. There is a strange dearth of open sharable visualization platforms like Google Maps.

Building (and selling, commercially or socially) web applications involves giving some stuff away and keeping other stuff to yourself. It turns out that the stuff that is kept is often the stuff I wanted the most in the first place (not that the other stuff isn’t nice). It sort of looks like this:

1. Base development tools. Open!
2. Nifty components built with development tools, and used in cool application. Closed
3. Cool application. Webtooey Open!
4. Profit!

I wonder if there is some way to get the components too, without compromising the user collection, profit, community building etc. that lead to people creating the cool application. Lots of companies do this, but some of them seem to have longer component chains than others (37 Signals although also a hosting only establishment, seems to avoid this problem because with their applications the application == component, and when it doesn’t the components for large applications are actually other applications).

To be fair, not everything needs to be given away. Gmail is cool and sure, I’d install it on my server if I could but I don’t really feel like I need too or it would benefit future development (and if I was a big company I could still get it privately for my domain). That seems to cover the bases pretty well. Google Maps, well that makes a lot of sense to stick on other applications and pages. That’s exactly why you can do that.

I don’t think that the pipes visual editor is something that could be directly given away, but I do think there are a lot of elements of it that suggest interaction and visualization capabilities that could be explosively useful if they were distributed or API’d like the maps are (and suggest even more visualization mediums). It turns out that Yahoo Pipes is like a relational database for the web. The only problem with this is that good applications don’t look like databases. They live in a place that is visual and interactive. Unfortunately you can’t warp there yet.