The data tsunami has arrived. According to recent research from the MIT Center for Digital Research, digital Information is doubling every 1.2 years and will exceed 1000 Exabytes in the next year. And the size of the largest data warehouses is tripling every two years. There is so much data now related to our “own lives” that it is overwhelming and almost unmanageable. This is especially true for the social customers who are becoming more powerful every day through the new word of mouth marketing platform called the social web.
Let’s take a short walk down data memory lane. Before the Internet we had tons of data collected and managed first on punch cards, large hard drives, flat file data bases and then came one of the most powerful technological innovations in the form of the relational data base. This was quickly followed by the advent of the business intelligence market and data warehousing platforms which enabled CRM.
The MIT researchers also point out that an inflection point in information technology use began right around 1995 when the Internet took hold. At that point purchasing of industrial transportation and other business equipment leveled off as information technology purchasing skyrocketed with business process reengineering that enabled the digitization of business processes. In 1993 Michael Hammer and James Champy’s book, Reengineering the Corporation delivered the thought leadership and acted as a catalyst in those smart companies and organizations that understood IT is a competitive advantage.
Pre-Internet Data
- Flat file and RDBMS
- OLTP transaction
- Business process transaction
- Email messages
- RFID
- Real-time machinery diagnostics/engines/equipment
- Bar code scanner data
- Stock market transactions
Enter the Internet, a massive disrupter that changed the way we do everything. Corporate web sites flew up with reckless abandon in hopes of innovating business models and creating more sustainable profitable growth. It was all about clickstream data, site visits, stickiness, and the data for the most part was rudimentary in nature, but the big search engines knew more about everyone and everything. There was even a search engine called Alta Vista, but they became road kill very quickly because of geographical cultural issues. More importantly, however, there was one company that thought that it could quickly build its own Internet on its own standards and we know who that is now. As Socrates once said “the very thing that makes you great will be your undoing.” Amazon was the first to begin sniffing the digital exhaust and began to harvest the gold in ways that none had visioned with one click shopping and predictive analytics.
Internet 1.0 Data
- Clickstream/Page views/Web transactions
- Web links
- Alta Vista/Google/Yahoo Searches
- Wikipedia updates
Now a tsunami of social media data is upon us. During my evolution to Social predator I left such a large plume of digital exhaust to be honest I couldn’t believe it. Most of it wasn’t through Twitter, although our most recent SNCR research clearly shows it as an Amazon of social media. What surprised me was what I wrote about in the Crescendo effect, and in the Ecosystem of Influence blogs, was that hundreds of people read my reviews of restaurants and hotel stays. Now we have even more powerful data to work with, like the customers or a potential customer’s location in addition to preferences likes and dislikes about food and environments. This leads us further down the experience economy path where brand experience reins king.
Social Customer & Web 2.0 Data
- Mobile phone/GPS/Location data
- Blog references
- Facebook updates
- Twitter feeds
- LinkedIn Updates
- Yelp Reviews & TripAdvisor reviews
Net/Net
In the New Symbiosis of Professional Networks research respondents told us that LinkedIn, Twitter and Facebook where the top professional social networks. What we have going on here is a dramatic sea change in human collaboration and interrelationships that is already digitized and available for harvesting. Other findings I thought worth mentioning are:
- A strong 60% of the population used social networking to increase the own professional network
- And 60% of those surveyed were using it to monitor brand.
- Nearly 50% of the respondents indicated that they also participated in specialized or industry specific social networks.
- Apple is the portal to social behavior as 67% of respondents used an Apple mobile device to access a social network. Apple IPhone apps for social networking rose from 180 in 2009 to 736 in 2010.
So what does this all mean? It means that those “Amazon like” companies that understand the basics of analytics and data harvesting can gain a massive competitive advantage over those in their industries that do not go “web farming.” This is an incredible bonanza for data miners who can essentially know everything about us and now where we are. In 1999 I ran across a book entitled Web Farming for the Data Warehouse by Richard Hackthorn, that provides a cook book for web farming the information resources of the web, a great book way ahead of its time.
The Game Changer
In Memory data management is one of the newest technology innovations that portends to change the way we collect, manage and make decisions on data. Along with predictive analytics and other business intelligence tools, this brings a new way of looking at data and will enable “real time data mining,” even of OLTP data bases. This is something that many companies asked for in numerous surveys that I conducted during the 1999-2000 period. The good news is that now memory is cheap. In the simplest of terms, In Memory computing enables you replicate and mirror data sets from any data base in memory on a separate hardware device or within the BI system and/or platform. That means that I can look at data as its collected in real time and watch it trend enabling “real time business decision making.” These technologies are not out of the reach of small and midsized businesses that will leverage the cloud.
The disciplines of data warehousing and analytics are now taking on special meaning in our society from predicting where and when parolees recommit crimes, to enriching our customer experience at the theme park. Companies that harvest the rich web of social media will undoubtedly gain competitive advantage in their industries and governments can also serve their citizens with enhanced services. Until next time I wish you great selling and marketing in the millennium.
Comments