big data
/
what is big data?
At its core, big data is just an extremely large quantity of information. Data is collected and analysed arithmetically to uncover new things about human behaviors and interactions through associations. It is the amount of information that those conclusions are based off of that makes big data so powerful and accurate.
Big Data ": an accumulation of data that is too large and complex for processing by traditional database management tools"
- Merriam-Webster
big data's powers
​
​
​
​
​
​
Number One: New Types Of Data
Big data offers us insight into areas that may have never even been considered data. Before the internet was invented in 1983, there was no way to easily get large quantities of information about any topic. For example, for the U.S. government to calculate nationwide unemployment, they have to figure out how many people are currently employed, then subtract that number from the amount of people eligible to be working. This information takes a while to collect and release, and can’t be entirely accurate. Big data can possibly change this. Davidowitz describes in his book his use of Google Correlate. After studying, he concluded that “Slutload” and “Spider Solitaire” were the phrases that correlated most to the unemployment rate. This represents a new way that google searches and big data have unlocked solving problems and crunching large numbers. It is important to understand that this can be both a wonderful tool to analyze information, solve problems, and learning more about human behaviors, but, it is also important to recognize that this information could be used in a negative light if used poorly. I encourage you to think about how your data is used and if you feel it is doing more good than harm.
Number Two: Big Data Provides Honest Data
Digital truth serum is a real thing. As Davidowits explains, when we are prompted to answer questions from anyone, especially in person, we are driven to lie. If someone walked up to you and asked what the weirdest thing you ever searched on Google was, my guess is you wouldn’t answer the question at all, or you wouldn’t answer honestly. But guess what, Google already knows. Google doesn’t just know this, they know everything you have ever typed into the search bar. Whether it was you trying to figure out who would be winning the current season of the bachelor, or trying to self-diagnose yourself using WebMD, Google knows it. A lot of this information is stuff you would never outwardly express verbally to others, or even through a survey. That is why big data is the most honest form of data collection. People don’t connect the dots between their information on the internet and people. That is why it is so honest, and why big data can be so powerful.
Number Three: Big Data Allows Us To Zoom In All Small Groups Of People
Written into big data is the word big. Big data has incredibly large quantities of information because of the sheer amount of people who use the internet, as well as different social platforms daily. According to broadbandsearch.net,, “there are approximately 4.79 billion people worldwide who have access to and use the internet frequently.” That amount of data is larger than any other collection method worldwide! With all of this information, you can pick extremely specific groups and subgroups of people, as well as extremely specific questions. This could be anywhere from people who live in the United States who like dogs, to people who live in a suburban neighborhood in California who like dogs, make over 50,000 a year, have 1 or more children, are married, and are between the ages of 25-35. Big data is the only data in the world that can do that. Big data is what allows targeted advertisements to be so exact, make it seem like your phone is watching everything you do, and for data scientists to use your information for experiments on the human psyche and behaviors. This once again brings up the question of if this does more good than bad? Big data allows us to learn a lot about very specific groups of people, but is it right the way that the data is used?
Number Four: Big Data Allows Us To Do Many Casual Experiments
As mentioned through the previous powers, big data allows scientists and everyday people to do casual experiments. It is because of the previously mentioned powers that this is possible. We all have access to many ways of accessing big data. Sites like Google Trends allow anyone and anyone to see popularity of searches through graphs, maps, and statistics. Anyone can conduct an experiment, and it can be as elaborate or simple as you want. It is through these small, simple experiments that we can learn more than ever before. When all of these studies are considered and reviewed is when they can be groups, combined, and cross referenced to get even more accurate information. One of the most popular casual experiments is A/B testing. A group is the control group and the other is the test group in which a variable is changed. This allows you to see if that change makes a difference, and further if that change is caused by or correlated to the non-change group. This paired with the immense power of big data allows even more questions to be answered. These tests are incredibly common as well. On page 231 of Everybody Lies, this is noted by saying, “Facebook now runs a thousand A/B tests per day, which means that a small number of engineers at Facebook start more randomized, controlled experiments in a given day than the entire pharmaceutical industry starts in a year.” So, not only can we do it, but when it is scaled up you start to see the amount of answers companies like Facebook get daily. This is important to understand and recognize, to know what you can do with big data, and also how many experiments are run with it daily.
key point: big data has unlocked solving problems and crunching large numbers.
key point: Digital truth serum is a real thing. When we are prompted to answer questions from anyone, especially in person, we are driven to lie.
key point: With all of this information, you can pick extremely specific groups and subgroups of people, as well as extremely specific questions to answer.
key point: Anyone can be a data scientist, and can conduct an experiment as elaborate or simple as they want.
As a part of this project, I read Everybody Lies by Seth Stephens-Davidowitz. The book explores four specific powers of big data. Number one, big data offers new types of data. Number two, big data provides honest data. Number three, big data allows us to zoom in on small groups of people. Number four, big data allows us to do many casual experiments. Below, I have deep dived into what these powers do.
what I learned about big data
key point: It is important for us to understand both the good and the bad around big data to construct our own opinion on our data rights and big data as a whole.
Over the course of this project, I learned a lot about big data. For me, it was really important to hear both sides of the story. A lot of what we hear in the media is negative against the “unrestricted” usage of our privacy and data usage, which is incredibly important to understand as the internet and social media are some of the core platforms that dictate how we communicate, work, and stay entertained. I came into this project with that point of view. All big data is bad, I should have full control over my information, and we need to change this now. But, after doing my research, I wouldn’t say I still feel the same way. Big data is a powerhouse in scientific studies, behavioral analysis, and overall the learning of the human psyche. Seeing the work that data scientists are doing with this information forced me to think about the good benefits of big data. I think it is important to recognize the risk as well as the rewards, and just be more careful about the information we intake and output on the internet. I do believe that the information regarding what data is being collected and how it is being used for or against us should be more widespread, and every digital account you sign up for should simplify the terms of service to make it more readable, and actually allow users to gain more insight into what rights they are signing away by checking the box. Big data is clearly a tool that will continue to become more powerful, which is why we need to understand it, and ask larger corporations to reveal what they have been and plan to do with our data.
the future of big data
To put it simply, big data’s future looks big. As the internet continues to become even more of a basic necessity for all of us to live our lives, large corporations such as Google, Facebook, Amazon, Youtube, Apple, Samsung, and many more will get even more information about our individual habits and lives. With an even better idea of who we are, it will become even more difficult to distinguish what is being targeted at us and what is not. But, with more information will also come even more studies about our internet use, beliefs, habits, correlations, decisions, and more that can prove to be extremely beneficial. This could be anything from finding a better way to calculate unemployment, to being able to see if people respond more to the word “my” or “I.” The possibilities are endless. It is also key to know that our data that is used, we get absolutely no reimbursement for. We are actively participating in studies, and having our data sold, and we get nothing for it. So, take the time to read through the terms of services and look at your state’s regulations for data privacy to understand where we are now, and develop your plan of action to what you want the future of your and our data to look like.
key point: As time goes on, big data will become even more powerful of a tool than it already is, which is why it is so important that we figure out how to move forward now.
additional resources
for more information on big data, please visit these resources:
​
Big Data: What it is and Why it Matters
I Tried To Delete Myself From The Internet: Here's What I Learned
Amazon uses data from third-party sellers to develop its own products, WSJ investigation finds
​
For California: