Tuesday, July 1, 2014

Facebook Could have Killed Somone - the Math

In order to confirm a hypothesis, Facebook experimented on people without their knowledge, intentionally making some of their lives more depressing.

The study included 689,003 people (or, if you prefer: users, subjects, customers, lab rats, victims, humans). Among these ~700K users, incidence rates of at-risk behaviors in the United States guarantee some of those who Facebook made sad were having suicidal thoughts, planning to kill themselves, attempting and/or succeeding in doing so. 

While the abstract ethical failure of the study has been discussed well and at length, I want to know concrete numbers. I need statistics. I want the math. Each number is a person and Facebook played dice with their lives. I want to know how close they came to losing while recklessly betting someone else's bank: How many of Facebook's subjects were making plans to kill themselves during the week Facebook intentionally made them more sad? Did any try? Did any succeed?

I needed to figure it out, to know the risk they took and maybe count the dead. This post is the result. It is my admittedly rushed and original research attempting to determine just how stupid this study was. 

The Study

Facebook recently published a Study conducted in January 2012 titled: Experimental evidence of massive-scale emotional contagion through social networks, wherein Facebook manipulated the emotions of 689,003 users without their knowledge. The authors designed the experiment in part to induce the expression of depressed feelings in 155,000 people. They succeeded, confirming the authors' hypothesis.

The methodology of the study does not mention any attempt to exclude at risk users from the study. Given the size of the study, the users on whom Facebook experimented included persons contemplating, planning and attempting suicide.

Facebook wanted to know whether "emotions expressed by others on Facebook influence our own emotions". To do this, Facebook manipulated the news feeds of 689,003 users without their knowledge. These users became four groups, for the sake of this analysis, I am going to call them 'make happy', 'make happy control', 'make sad' and 'make sad control'. 

Facebook made these users' news feeds either more positive (make happy, 155,000 users, reduced frequency of negative posts), more negative (make sad, 155,000 users, reduced frequency of positive posts) or randomly interfered (both controls, 379,03 users, reduced random posts in proportion to one of the experimental groups).

The Stats 

A few caveats: I am not a professional statistician. I do not have enough time or access to do this properly. Sources are the best I could find in limited time. Personally, I only consider the answers I dug up approximations. Suicide rates vary by country and region and I only had time to do rates for the United States. Please keep in mind you are reading an amateur's blog post. The main reason I go through this step by step is because I would like you to evaluate it, debunk it, improve it, whatever. I really want to know the truth here. When reckless people play with other people's lives we owe it to the victims to at least try to name the dead.

The 'make sad' group of 155,000 users included persons at-risk for suicide. In order to determine how many at-risk users Facebook abused, I use United States suicide rates: an age adjusted suicide rate for the United States in 2011 found through WISQARS as well as estimated rates of adults contemplating, planning and attempting suicide during the past year for 2008-2009 published by the CDC. Since the Facebook study occurred over one week in January 2012 I converted the annual statistics to weekly equivalents:

US Annual Incidence Rates for At Risk Behaviors 
Rate/100K % of Adult Pop
Suicidal Thoughts 3,700.00 3.7000%
Made Suicide Plans 1,000.00 1.0000%
Made Suicide Attempt 500.00 0.5000%
Suicides 12.32 0.0123%

US Weekly Incidence Rates for At Risk Behaviors 
Rate/100K % of Adult Pop
Suicidal Thoughts 71.15 0.0712%
Made Suicide Plans 19.23 0.0192%
Made Suicide Attempt 9.62 0.0096%
Suicides 0.24 0.0002%

Since I use United States suicide rates, and since Facebook has a global user base, I needed to know roughly how many of the 'make sad' subjects were United States users. The study limited selection of subjects to users who view Facebook in English. I found data breaking down Facebook usage at the time of the study by country (need better source):

Users by Region Q1 2012 
US&Can 183,000,000 20.31%
Europe 239,000,000 26.53%
Asia 234,000,000 25.97%
Other 245,000,000 27.19%
Total 901,000,000 100.00%

and by language (need better source):

Users by Language Q1 2012
English Total
Q2 2010 (from source) 213,237,080 450,000,000
Q1 2012 (calculated) 315,850,920 874,200,000
Q4 2012 (from source) 359,828,280 1,056,000,000

Based on the ratios of their respective populations, I proportionally reduced the US&Can number to 164,646,768 users in the US. Similarly I used the stats here to adjust for persons in the US who would probably not be viewing Facebook in English. Including persons who either speak English at home or who speak it as a second language 'very well', I calculated 150,307,073 users in the US who viewed Facebook in English at the time of the study. 

Since there were 315,850,920 users viewing Facebook in English worldwide at that time, 47.59% of the randomly selected subjects were likely to be resident in the United States. Again, per the study, the 'make sad' group included 155,000 subjects, so Facebook made about 73,761 Americans sadder than they would otherwise have been. Multiplying this number by the annual and weekly at-risk incidence rates given above yields the following:

in 2012 in one wk 2012
Americans Made Sadder 73,761 73,761
 who had Suicidal Thoughts 2729.17 52.48
 who Made Suicide Plans 737.61 14.18
 who Attempted Suicide 368.81 7.09
 who Killed Themselves 9.09 0.17

The week that Facebook wantonly fucked with the emotions of nearly 75,000 of my countrymen, fifty-two of their victims thought about killing themselves, fourteen made plans to do it and seven non-fatally injured themselves in an attempt to do so. There is a 1 in 6 chance that one of their test subjects died of impossible sadness without knowing they were on an emotional operating table where the surgeons were not trying to save them. Instead they wanted to see what would happen if they tried a little bloodletting on tens of thousands of their patients without checking to see if anyone already had a sucking chest wound.

That is just the week they ran their experiment, January 11-18, 2012. By the end of the year a thousand more had planned or attempted suicide. Nine died.

Conclusion

The numbers above only represent the US portion of their random study. To the best of my figuring it accounts for only half of the people involved.  Additionally I must note that there is no way of knowing the impact the study may have had. The numbers above are based on average rates; I am not saying that Facebook killed anyone. The numbers above are the same for 73,761 people in the US Facebook did not experiment on.

I am saying that the experiment was stupid and reckless. The authors confirm that their work made people sadder. I am saying that many of the people they made sadder were at risk for suicidal thoughts and actions. I am saying that about seven people tried to kill themselves the week that Facebook stole bits of their happiness one post at a time like a patient joy vampire. I am saying that about nine people killed themselves during the year that Facebook hid from them randomly selected rays of hope and sunshine during one cold, dark week in Winter.

Without access to complete data, one cannot conclude that Facebook's study impacted the numbers above. According to the study, the effects were small. I can't say the US average of nine suicides per year for that population became ten because of what they did. Is it possible? I will leave you with the study's authors' thoughts on the subject, on why the effects induced by their actions, though small, still matter:

given the massive scale of social networks such as Facebook, even small effects can have large aggregated consequences: For example, the well-documented connection between emotions and physical well-being suggests the importance of these findings for public health. Online messages influence our experience of emotions, which may affect a variety of offline behaviors. And after all, an effect size of d = 0.001 at Facebook’s scale is not negligible: In early 2013, this would have corresponded to hundreds of thousands of emotion expressions in status updates per day.