Searching by Drawing

How do you search for something if you don’t have the words to describe it? What are alternate ways to search besides the ubiquitous text box?

There are a variety of tools that let you search based on an image. Google Googles lets you use the camera on your mobile phone to do a search for the object it recognizes in the image. Amazon provides similar search functionality to search through its product inventory. Google image search lets you upload an image (or images) to use as your search. And the results can be refined by adding text keywords. It’s pretty cool.

Searching based on an uploaded image

Refining the search with a keyword

Franz Enzenhofer’s hack adds a drawing interface on top of it. You do a rough sketch, and the program sends the image to Google image search — which then shows you images that match what you just drew.

But these are still searches for images. What about searching for other types of things — less tangible than an image, perhaps emotional or temporal?

Paul Lamere recently built a really cool hack that lets you search for music by drawing a picture of it. It’s rough (it was built in less than a day), but the idea is awesome: you draw a loudness profile that you’re interested in, and the tool returns songs that match it (as well as a more accurate profile of what’s available in its database). It’s a way of asking “what songs start soft gradually build?” or “what song starts with a bang and goes out with a whimper?” This is definitely something I want for my music library!

Leave a Reply