Yahoo! have just released an amazing new service named Pipes. The short explanation of Yahoo! Pipes is that it lets you combine multiple streams of XML data into a single XML stream with contextual filtering and other magic along the way.

The name of the new Yahoo! service is inspired by the Unix pipe command, which allows you to seamlessly send the output from one program in as the input of the next program in the command line. A simple example of using the pipe command in the shell might be:

• ps -A | grep firefox

In the above example, the first command (ps -A) is asking for a list of all the running processes on the machine. The output of that command is then sent into the next command (grep firefox), which filters the input and only returns information relating to the string firefox; you could think of the grep utility as a crude search facility. The little vertical bar (|) in between those two commands is the pipe.

If you then apply that type of logic to the Yahoo! Pipes service, the name makes a lot of sense. Yahoo! Pipes allows you to combine various XML feeds into a newly created feed. As you add or combine each feed of information, you can perform all sorts of processing along the way (just like the grep example above).

To give you an idea of the processing you can do along the way, some of them include:

• adding as many XML feeds into a single stream as you want
• adding filters, so that only items with a particular criteria are passed through
• stripping out duplicate items
• sorting the output of the combined XML stream
• looping over each item of a feed and replacing blocks of content with other content or adding new information to each item based on the content of each one.
• combining XML feeds which you have previously filtered.

Other than this being a really great utility for the geek or power user among us; I can see some amazing applications to certain businesses. Imagine for a moment you want to publish breaking news from around the world about a particular topic on your web site. Through combining various XML streams from other pertinent sources, you could then apply specific filtering to give you exactly what you want. You’ve then got your own targeted news reporting service that you only hear from when it has something to report; what a great signal to noise ratio.