Skip to main content

Author: Igor Shraybman

Big Data Analytics and Social Media – Are We There Yet?

Having worked in financial services, with insurance companies and a number of software companies, while being very active on social media in my free time, I noticed several interesting trends that I would like to share.

Networking with professionals in Boston and NYC areas and discussing issues facing Big Data Analytics (BDA), it seems the common conclusion is that many companies are talking about BDA, many are claiming they do it, but not many know how to do it. In fact, there are 3 big aspects of BDA:

  1. How to get the data
  2. What to do with it
  3. How to present it

Some companies are very good at collecting big data from a variety of sources and have gigantic databases or data warehouses filled with data, something I saw in the life and disability insurance industry and in the medical trials. But their product management departments have yet have to come up with good ideas for analytical processing of data they have in order to create breakthrough product offerings for the market.

The issue related to it is reporting on big data. I knew a company that had access to many statistical data sources, had developed a robust analytical engine for calculation of risks, and a SaaS product to let client companies to use it over the web, but reporting capabilities were not on par with sophistication of the calculation engine and quality and quantity of data it had. When it tried to invest in industry-standard business intelligence packages, they were not customizable enough for BDA wide range of requirements that their clients had.

Other companies I have seen or spoke to were very instrumental processing data and presenting it, and were looking hard to identify sources in order to get a variety of clean and reliable customer data.

With the rise in popularity of social media (SM) sites, about 1 billion people registered on Facebook, and big crowds using Twitter, Instagram, Pinterest, and a number of ethnic overseas sites, it is not surprising these companies have turned to acquiring data from those sites that offer it for sale.

In the case of Facebook, prior to this year, the users had a choice to opt out of having their personal data collected. Starting on 01/01/2015, the act of logging on to Facebook constitutes the acceptance of new terms for data collection. For most users it would mean that companies could now buy this personal data, information about their location and travels from their smart phones and all of their images posted and hosted on Facebook servers.. It provides companies with a wide variety of information, such as social habits and shopping and personal style preference, and even an option to use profiling of the users. Several companies have developed analytical tools that allow scanning of someone’s Facebook posts and, based on what the user posts, report on his or her personality characteristics.

In conclusion, thecompany that will be able to successfully combine all 3 aspects of BDA – how to get it, what to do with it, and how to present it – and mine social media data to successfully implement a robust BDA solution, will be golden, in my personal opinion.

Don’t forget to leave your comments below.