I had the opportunity to discuss the trends that one can discover with our News Patterns algorithms and visualizations. I was talking with a group of sophisticated managers from the financial services industry. It was not long before I realized that they were very skeptical that any useful patterns could be derived from the data we call news. (News in this blog can include traditional news, message boards, blogs, RSS feeds, Google Alerts, videos, social media and Twitter.) They were convinced that news was mostly biased and dominated by ignorant contributors, or Garbage In = Garbage Out.
In answering their skepticism, I started by referred to a 2004 book by James Surowiecki called The Wisdom of Crowds - Why the Many Are Smarter than the Few and How Collective Wisdom Shapes Business, Economics, Societies and Nations. At the time of publication, James Surowiecki was a staff writer at the New Yorker, with other publications including the New York Times, the Wall Street Journal, Artforum, Wired and Slate. I started my explanations with two references from the "Wisdom of Crowds."
- In 1884, at the International Exhibition in London, a British scientist named Francis Galton collected all the individual entries for a weight judging contest of a slaughtered and dressed ox. Eight hundred contestants of various abilities entered the contest. "Galton undoubtedly thought that the average guess of the group would be way off the mark. After all, mix a very few smart people with some mediocre people and a lot of dumb people, and it seems likely you'd end up with a dumb answer. But Galton was wrong. The crowd had guessed that the ox, after it had been slaughtered and dressed would weigh 1,197 pounds." The actual weight was 1,198 pounds, or an error or 0.08%!
- John Craven was a naval officer who was tasked to locate the U.S. submarine Scorpion that sank in the Atlantic Ocean in 1968. Craven assembled a team of various experts who were to independently offer sinking and location scenarios without consulting with each other. Craven compiled the individual scenarios and with some calculations, determined the "collective" location of the submarine. This location was unique from any individual guess. The "collective" location was 220 yards from the actual location.
- In 2009, Nokia had continued its dominance of the cell phone and smart phones markets, often kissing a 40% share of handset sales. Yet starting in the Fall of 2009, two new competitors entered the smart-phone market. The first was Apple with its iPhone. The second was not really a distinct competitor but a new competing operating system from Google called Android. We at News Patterns collected and patterned hundreds of thousands of articles with special algorithms over many months. The collective wisdom of thousands of news creators soon discovered that iPhone and Android displaced Nokia, causing Nokia to fall from a central market relevancy to a peripheral relevancy position by November of 2009. A competitive fall from news pattern relevancy often correlates with an actual decline in market or campaign relevancy. In November of 2009, a significant divergence of stock price versus news pattern relevancy occurred. This drop in competitive news pattern relevancy was not noticed by the stock market. In April of 2010, Nokia announced that it was falling far short of market expectations with its smart-phones due to competition from iPhone and Android. In a short period of time in April, Nokia's stock price dropped 52%, a move that started to realign Nokia's stock price with the competitive news pattern relevancy (or irrelevancy) of Nokia among its competitors. The collective wisdom of thousands of news creators, as part of a mass of 200,000 plus articles over 5 months was a front-runner to the new competitive pressures on Nokia.
See more details about the collective wisdom of news creators in this paper: Anticipating Financial Market Moves Using Long Term Patterns in the News .
No comments:
Post a Comment