I’m not going to knock the Dewey Decimal system, but if I never again have to endure a paper cut from sorting through dusty old index cards in wooden boxes, I’ll live a much happier life. Thank goodness that the world of search has changed significantly since the days when rummaging through card catalogs at the local library was the common way of finding information on topics ranging from bird species of the Pacific Northwest to instructions for building a radio from scratch. I’m not asking this to be some kind of smart alec — it’s an honest to goodness question: Do kids today even know what the Dewey Decimal system is? It it even necessary for navigating the computer-equipped libraries of today?
Thanks to the World Wide Web and search engines like Google and Bing, we pretty much take it for granted that we can find out just about anything from the comfort of home, with minimal effort, in seconds. Gone are the days when we’d have to put three pairs of woolen socks, bread bags, and giant boots on our feet and march through the slush in inclement weather to the library to research our papers on killer bee invasions and the facial hair patterns most common to vice presidents of the United States over the past 220 years. Sure, I still like to go to the library for fun, but it’s not essential for most of my current research needs.
So we’ve got text searches covered. Images, too, are increasingly easy to look up on the Web thanks to innovations like Google’s Search by Image, where an image can be dragged and dropped into a form that will then be used to find other instances of the image on the World Wide Web. This came in handy the other day when I was trying to discover the identity of a mysterious figure from a picture I’d taken at the Whaley House in San Diego last year. (He turned out to be a fellow by the name of Philip Crosthwaite, for the curious.) And because Google’s Search by Image is so much fun to play with and distracted me for hours, I was also able to discover that a photo where I was sporting a very bad haircut was used to illustrate an article where a very bad haircut was mentioned. (It got better — my haircut, that is. My shame is perpetual.)
Another neat feature is that it will detect images that have visual similarities to the one you’re using to search, so you can track down matching color schemes (which can be helpful for designing if you can find public domain and Creative Commons images for personal use) as well as other views of the same subjects — whether animal, vegetable, or mineral. Maybe you were at a costume party and you know someone there was taking pictures. Out of curiosity, you want to see how they turned out, so you search using a photo that you took of yourself with your camera phone. If there are other pictures posted of you online from that same costume party, there’s a possibility that they may show up. Because why should the world settle for one measly picture of you dressed like Henry Kissinger in drag when there are possibly dozens out there for the finding?
So text and image searches are pretty well represented (and getting more sophisticated all the time). But what about sounds? Where can you turn when you need to dig up a sample of hoodlums exchanging gunfire, a specific snippet of chamber music, or Yoda belittling that dorky Skywalker kid in the swamps of Dagobah? Sure, there are search engines out there that focus on hunting down sounds, but they often rely on sound files being labeled properly for what they are — which, on a basic level, still makes them text search engines. But audio engineers at Imagine Research in San Francisco have been hammering away at what they call “the world’s first sound object recognition Web service.” Dubbed MediaMined, the service is actually driven by artificial intelligence that is able to “listen” to sound files — whether they’re properly labeled, mislabeled, or not labeled at all — and analyze what they actually are.
“Let’s say you are working on a movie, and the director needs some explosions,” says Imagine Research founder Jay LeBoeuf. “The state of the art for searching for sounds in multi-terabyte audio collections is to search on the text — usually the filename — of the sounds. So, the sound editor could find explosion — but would never find tracks that were labeled big bang, huge blast, detonation, nuclear blast, bomb, etc. MediaMined is capable of grouping those sounds together — you would give us an example of what you are looking for (the sound of an explosion) and we are able to return things that sound like an explosion — regardless of their underlying metadata, name or text content.”
Of course, that’s just one example of the many possible uses for an artificial intelligence driven sound search like MediaMined. Musicians, podcasters, radio broadcasters, and audio engineers would obviously benefit from this kind of technology, but some other unexpected applications could make use of it, as well. Mobile devices could use a MediaMined type of system to detect their surroundings and present new ways to interact with their users based on this incoming data (think augmented reality cranked up to 11). Medical professionals might be able to use this technology in order to gather data based on sounds made by patients — such as sneezing, snoring, coughing, and wheezing — to help with more keenly diagnosing their condition. One wonders what other creative uses for MediaMined will be discovered as it expands its database and becomes even more refined. Heck, I could imagine sitting in front of a computer for days at a time searching for cool sound effects for the sheer fun of it without any practical or constructive need to do so. How would you use something like MediaMined?
So it looks like we’ve got a handle on ways to effectively search for (and, one would hope, find!) text, images, and now sounds. If only Melvil Dewey could see us now.