You may have heard the phrase data mining, especially in social media, but do you know what it is?
A good explanation is as follows:
Obtaining information from observed patterns and relationships of large data sets.
Perhaps that is over simplifying a bit, but that is the general idea.
Data mining is really nothing new. It can be traced back to tracking migrating animals with cave drawings. People have been looking at patterns and relationships to gain advantages and insights for centuries. The thing that makes this hot topic exciting now-a-days are computers. Never before could we process such large data sets to look for those patterns and relationships. Also, never before has so much data been collected. With the internet and the “Internet of things” there is an unfathomable amount of data being collected on people, places, and things. Enough information is gathered every day that patterns can be derived that would otherwise have remained in the shadows.
If you can get past the “big brother” aspect of all the data collection and processing, you can start to see valuable non-NSA uses for all of this. UPS for example, gathers data recorded by their trucks to help optimize routes and predict maintenance needs. Walmart discovered that before hurricanes, the most popular item was strawberry Pop-Tarts, so they were moved to a more convenient location before storms hit. Those are just a couple examples. Social networks are a great source of big data. Consumers freely give their opinion on people, places, and things all over the Internet. That information can be mined. Through the use of computer science (programming languages, machine learning, pattern recognition), information can be used by companies big and small for any number of purposes such as marketing.
My talk during @Midwest will more specifically look at how to use data from social networks such as Twitter and Facebook.