Formulating predictions, such as the movements of the stock market or the likelihood of a movie’s success, have traditionally been costly, and unevenly successful, endeavors. Prediction research often involves labor-intensive efforts to understand geographically localized social trends and “on-the-ground” conditions. Now, as reported in Knowledge@Wharton , two Wharton professors, Albert Saiz and Uri Simonsohn, have found a cheaper way to deliver some of the same benefits as this type of resource-intensive research: an Internet search.
Using Exalead as their Internet search tool of choice, they chose to study political corruption as a test case. They found that the Internet search results for this topic on Exalead showed a strong correlation to ‘real world’ facts regarding corruption, namely, the frequency and proximity of the word ‘corruption’ alongside various locality names and socioeconomic indicators matched known ‘real-world’ corruption linkages.
This reliable correlation means social scientists are likely to use Internet search statistics as a proxy for measuring local social trends that are otherwise difficult to assess (such as measurements within relatively closed societies), and certainly astute market researchers will be adding Internet search results analysis to their arsenal in determining the best markets for product launches or the best geographical distribution for campaign election funds.
Of course at Exalead, we’re as interested in innovative ways to use Internet search as we are pleased that these two professors assessed all the major search engines over the course of their research, and selected Exalead as the most reliable (giving high marks on reliability to Ask.com as well). The others, Simonsohn stated, either couldn’t support a single automated search or were simply too unreliable, producing radically different results from week to week. You can download the complete paper from the Social Science Research Network site.