It doesn’t take Edwin Hubble to recognize that the search universe is expanding. Instead of studying faraway galaxies to see the shifts in the cosmos, it only takes a glance at the home page of any major search engine to realize that search is moving at light speed.

The stars of search – America Online, Google, MSN and Yahoo – are attempting to extend their reach by launching a stream of search tools that provide custom filters of online information. The rate of change has sharply accelerated during the past year, and it seems that with every fortnight comes a new personalized, localized or visualized search method aimed at speeding up the delivery of relevant results.

A decade ago it was assumed that most users would find companies and information through portals that organize content into easy-to-navigate sections. However during the past few years search engines, led by Google, have become the primary resource for finding information.

According to an April 2005 Harris Interactive survey, Web surfers said they use a search engine during more than 90 percent of their online sessions.

“Google’s sneak attack was quality,” says Jon Cooper, vice president of interactive services at search marketing firm UnREAL Marketing. Instead of trying to direct users to content partners or handpicking links, Cooper says offering quality search results is the best model for satisfying surfers.

Google’s model of throwing open the doors through advertising-supported search has won out over trying to provide premium content. “As long as the content is pretty good and free, people will take the path of least resistance,” Cooper says. Google’s ad-supported search model has helped search engine marketing grow to a $4 billion industry in 2004, according to the Search Engine Marketing Professional Organization (SEMPO).

Tools of the Trade

Basic search tools provided by all of the big four include standard search, image search and news search, although the depth of the search results can vary widely among engines. For example, AOLSearch’s news tool generates results from news wire services only, while all of its competitors include links to articles from newspapers and online media outlets.

This year’s flurry of new search tools will generate additional volumes of Web traffic (and therefore advertising opportunities) by adding utility, increasing the level of competition and enhancing the significance of search in daily online activity.

Google and Yahoo have been the most active during a frenetic 2005 in rolling out new search tools, while AOL and MSN are also rapidly increasing the profile of search on their portals. Instead of taking away traffic from others, the new features will prompt more searches, and advertisers are expected to increase their search engine marketing spending by 41 percent in 2005, according to SEMPO. “The pie keeps getting bigger,” says David Berkowitz, director of marketing at search advertising agency icrossing.

Google and Yahoo have added personalization features that tailor results so that the most appropriate links for the individual are delivered at the top of the results page. Google’s Personalized Search enables users to scan their past searches to “re-find” information and uses the search history to refine the results. Yahoo’s personalization service, My Web 2.0, similarly uses past searches to refine results, as well as enabling friends to share pages that they have visited.

According to Nielsen NetRatings, nearly 70 percent of all search traffic flows through Google (48 percent) and Yahoo (21.2 percent). Personalized search could increase Google and Yahoo’s market leadership because it produces better results without asking users to change the way they search.

Most people use relatively simple one-or-two-word search terms that lack the context to filter out inappropriate results. For example, someone who searches on “Lincoln” will get results about the car, city, university and the president, but a personalized search relying on previous experiences would automatically narrow the results.

“Changing user behavior is a challenge,” says Gary Price, news editor of SearchEngineWatch.com and editor of ResourceShelf.com, because even after many years of searching, people still make the same mistakes. Since people won’t change, “search engines have to do things to make results more relevant,” he says. If what they are looking for is not delivered in the first 20 results, users will give up on a search, according to Price.

Getting Googled

Price says it’s much easier for the market leaders to get users to experiment with new search features than it is for their smaller competitors. When Google introduces a new vertical service, such as a search of academic papers or product catalogs, Web users and the press provide plenty of coverage.

“Google is a PR juggernaut,” says Price, adding that the word of mouth the company gets from enthusiastic supporters puts competitors at a disadvantage. Yahoo similarly generated considerable buzz when it launched tools for searching subscription content and comparison-shopping sites, even though similar services existed from lesser-known competitors.

The challenges for search engines not named Google or Yahoo in spreading the word will likely further the current trend toward consolidation in the search engine industry. Smaller companies that fail to distinguish themselves are likely to be acquired, according to Price.

Microsoft has become more serious about the importance of search on MSN, which previously served as more of a shopping and news portal and showcase for emerging Microsoft media technologies than a top-tier search engine. Microsoft decided in 2003 to replace the Yahoo search technology it had been using with its own search technology, which went online in February this year, according to MSN product manager Justin Osmer.

Osmer says MSN Search’s product development is focused on giving factual answers and not just links. When users type in a question, MSN searches Microsoft’s Encarta database as well as external resources for the answer, an approach similar to that of niche search engine AskJeeves.com. For example, typing in “Phillies score” will yield the score of the team’s latest game as the first result, while “population of Seattle” displays the latest statistics from the U.S. Census Bureau.

Google and Microsoft are further enhancing the importance of search in everyday computing by integrating Internet and desktop search. Both companies have launched free desktop search utilities, and Google’s Gmail email service replaces folders with a search model.

America Online is beta testing a new home page highlighting search tools that makes available to everyone a portion of the content that was previously restricted to subscribers. In addition to reference material and product search utilities, AOL now provides multimedia searches that enable users to tap into its considerable content partnerships.

AOL Search’s video search uses technology from fellow Time Warner subsidiary SingingFish and includes clips from television shows, movies and music videos, while the audio search displays radio program segments and music tracks.

Yahoo’s AltaVista also includes audio search technology, and Google is developing technology to search the text of audio, according to a report in the New York Post. Searching the spoken word currently requires developing faster and more accurate speech recognition technology, but eventually “will become just as important as the written word,” according to SearchEngineWatch’s Gary Price.

Steam Behind the Local Motion

According to Chris Henger, vice president of marketing and product development at Performics, the new tools will propel search marketing to become a $13 million industry within four years. Matching consumers with local sellers is expected to be one of the largest areas of search growth, according to Henger. “Local search is the biggest thing and opens up the door to a whole new s

et of advertisers,” he says. (See “Think Global, Search Local” on page 40.)

All four of the top search sites have launched local search services that look for nearby businesses selling services or items that match the search term. Henger says that working with smaller regional companies poses some challenges for search companies. Search engines will be interfacing with smaller companies that may be inexperienced in the business model, which may require search engines to augment their existing sales teams with a network of local sales representatives, according to Henger.

MSN’s beta local search service attempts to match the searcher with relevant local information by automatically scanning IP addresses, according to Microsoft’s Osmer. The location of the searching computer is used to call up nearby business listings, and the same technology is also used to identify the location of the results pages so that nearby websites are given priority.

The search engines will have to contend with established phone directory companies such as Verizon’s SuperPages.com, YellowPages.com, YellowBook.com and Amazon.com’s A9, which recently launched a visual search tool that provides images of the actual storefronts.

All of the search engines are experimenting with RSS (really simple syndication) search capabilities, which could further boost the amount of advertising opportunities. RSS is a method of formatting content used by many news sites and bloggers to share information with other publishers. Tracking feeds currently requires RSS reader applications, but search engines are likely to integrate RSS into search in the not-too-distant future.

Google rotates RSS feeds into its Gmail service, which could pave the way for broader RSS searching. MSN Search enables users to save any search query as an RSS feed, eliminating the need to repeat searches. Yahoo is integrating RSS into its news search, and AOLSearch includes video content formatted with Media RSS, which describes the content of the video.

Connecting search advertising with bloggers and news content through RSS would take advantage of some of the Web’s fastest growing segments. Google is currently posting some RSS advertisements on Gmail, and in July WashingtonPost.com became the first major news organization to include ads with its RSS feeds. Yahoo is testing RSS as a medium and looking into the viability of RSS advertising, according to senior manager of communications Gaude Paez.

Too Much of a Good Thing?

Keeping up with all of the search options and learning the benefits of each presents challenges to advertisers and users who must determine which variations will work best for them. The home pages of the top search engines now include a half dozen or more search options, and beta search technologies are often listed on their own pages.

When initiating a search, users have to keep mental notes as to which search tool will work best for each occasion. Users who would like to speed up searches through personalized search must remember yet another ID and password, and also have to remember to sign out before performing searches that they would rather not have saved on a search engine’s servers.

Marketers have to decide which of the multitude of search tools from a given search engine they want to participate in, and then figure out how to track their return on investment. For example, a search marketer might be getting good returns on standard search, but might not do as well on local search and generate no returns from news searches associated with their keyword or contextual ad.

Since each new tool increases the magnitude and complexity of search marketing, the need for interactive agencies will greatly increase, according to Chris Henger of Performics.

“Companies will need to go to specialists and third parties” to sort through the dozens of search marketing options, he says. Specialized agencies will track the new search tools for volume, user demographics and potential ROI.

Icrossing’s David Berkowitz says the rise in search tools “is phenomenal for interactive agencies because it makes it very difficult to keep track of everything that is going on.”

Search marketers also have to consider if they want to exclude having their ads show up on specific websites that include content that they consider objectionable. Berkowitz says large advertisers will call on agencies to protect their brands from unwanted associations, particularly with the rise in video and audio searches.

The search engines are committed to extracting the maximum value from the growing universe of content by producing personalized packets of information. New customized tools that anticipate the intent of users’ queries or automatically refine the scope of the search will further entrench search as the de facto first step in the quest for online information.

JOHN GARTNER is a freelance writer in Portland, Ore. He is a former editor at Wired News and CMP. His articles regularly appear on Wired.com, AlterNet.org and in MIT’s TechnologyReview.com.