June 16, 2006

Google has developed an application that lets your PC actually listen to your television set, in an effort to display targeted Internet advertisements on your computer. What will they think of next!

According to Google, its "mass personalization" software can correctly determine the nature of a specific TV program from as little as five seconds of sound. It then presents related information in the form of ads in your Internet browser.

Engineers at Google believe it could as well be used to monitor audience size or create certain social networks around other viewers watching the same television show. "The new system could keep up with users while they channel surf, presenting them with a real-time forum about a live political debate one minute and an ad-hoc chat room for a sporting event in the next," wrote Google researchers Michele Covell and Shumeet Baluja on the Google research blog.

"Of course, all of this would be done without users ever having to type or to even know the name of the program or channel being viewed," they wrote. Although the product has been tested and shown to work in trials, Google are keen to stress that there are still technical hurdles to overcome and that it may never be launched.

All researchers at Google are given time to work on their own pet projects. Mass personalisation aims to combine television viewing with a personalised web experience. "Mass-media channels typically provide limited content to many people," the researchers wrote. The Web provides vast amounts of information, most of interest to few."

The core of the system works in a similar way to the Shazam music service that identifies music played into a mobile phone and then sends the listener a text message with the name of the track. We could collect snippets from the Internet describing the actors appearing in a movie or present local maps within the movie as it takes place.

Instead of music and mobile phones, Google's system uses a laptop or computer to listen to the ambient sound in a room. Each snippet of TV audio has a unique pattern, like an audio fingerprint, that can be queried against a database of television soundtracks. By recording the background sound and regularly analysing five second chunks, the system can identify the program 'genre' being watched.

It then starts to look for related information and content. This could include personalised advertiments. For example, if the system recognised that you were watching a trailer for a film, it might show adverts for cinemas or DVD rental shops. The researchers envisage that retailers and advertisers would bid for television segments for their adverts in the same way as they bid for keywords on the Google search engine at the moment.

The Google scientists believe the system could also be used to direct other information to a television users computer. "We could collect snippets from the web describing the actors appearing in a movie or present maps of locales within the movie as it takes place," they wrote.

The system could analyse any background sound in the future. The team says that mass personalisation could also allow viewers of the same shows to link up with on social networking sites. This would allow real time chat and allow fans to discuss and comment on shows as they are broadcast.

Other applications include providing broadcasters with accurate, real-time viewing figures or allowing viewers to create video bookmarks of their favourite shows. In the latter example, viewers watching on demand services would press a button to identify a programme or particular segment as being of interest.

The audio collected at the time the button was pressed would allow a viewer to retrieve entire shows or clips from Internet movie databases. Although the system has many potential applications, it also has many potential problems. Background sound and chatter does interfere with the system making the retrieval of accurate information difficult.

However, the researchers believe that they can glean enough "clean" snippets of sound, enough of the time, to make it work correctly. In their "in-living-room" experiments the researchers showed the system could correctly identify the right program every time.

But the tests used a database of just 24 hours of audio. The biggest difficulty for a real world system could be creating the audio database itself. With thousands of hours of television broadcast around the world every day, keeping track of all of the soundtracks and having the processing power to search the massive database could be a problem.

But despite all these technical glitches, Google's researchers are confident that the new system would work and could be used on devices like cell phones or PDA's and for different content.

"The mass media content can originate from other sources like radio, movies or scenarios where viewers share a location with a common auditory background (e.g., an airport terminal, party, or music concert)," they write.

Source: BBC