following is an AI generated summary of this article, so you won’t have to read it! (saves one llm call!)

”

A data scientist analyzed over a million Reddit posts to understand public sentiment around Facebook’s content moderation changes. Using custom AI technology to process social conversations, they found concerning gaps in society’s understanding of how unregulated social media can enable extremism and ethnic violence, beyond just issues of free speech and minority protections.

”

today i reviewed a million reddit posts to understand society’s feeling around facebook’s newly announced content moderation plan. i did it using a new tool i’ve been working on, that brings the the voice of real social conversations to AI.

recently, there has been a lot of noise about facebook (meta) and its abandonment of ‘fact-checkers’.

in lieu of favoring pure community moderation (a-ka self-moderated echo-chambre).

many have expressed concerns over hate speach and abuse, predicted to arise, towards lgbtq, racial, ethnic, and other, minority groups.

some have taken a you cannot have free speech and not have free speech stance, the consitutional and, perhaps, neutral one.

but both of these postions, imho, miss the point entirely, and fail to see the very real danger in allowing the mob to police itself. and this from me, an avowed anarchist. because even anarchists cannot escape the overton principle

in my professional capacity, i have spent alot of time studying humanity, both from space, and, recently, using AI.

specifically, i have been working to bring the social context to AI, using millions of currated and analyzed social media converstaions.

my work has been focused not on

what is true

but, rather, on

what people say and think is true

we use this type of analysis for research purposes. it was in this context that i decided to spent a 1/2 hour using the very same technology i am building to peform a survey, of over a million voices, to see how deeply society truly understands, what i consider to be, the real and present danger of such wanton and unregulated capitlism.

again, please consider that i abhor large goverments, and rules in general - with passion… but, i try to care about, and study people. even as they wreak havoc on the planet and many beings smaller, and more helpless, than hairless monkeys.

my study was aimed, specifically, at understanding the deepest dangers of big technology, and unregulated echo chambres:

dangerous extremism, and much, much more horrible outcomes, such as ethnic cleansing.

i will ask you forgiveness for my breveity and any typoz. this entire study was performed, and documented, in under thirty minutes. and i can neither spell nor use a computer that effectively (i program them - hate using them)

also, as point-of-fact, i am not, as a rule, bullish on AI, quite the opposite and, more on this topic soon. however, its ability to understand patterns in human language/thought is both intriguing, powerful, and horrifying at the very same time. when there is fire, i prefer to go and see for myself.

without further gibberish, i outline my methodology, and my results. any/all questions can be emailed to me directly. see e-mail links in footer. no stupid marketing form here. this is my personal email so, be kind(-ish). having said that, any and all feedback is appreciated.

first, i used google gemini, and the deep research feature available in version 1.5, to automatically compile a neutral background document
next, i used my very-in-progress technology, to generate a survey that was, itself, informed by the social voice. here is the prompt i used to generate the survey
finally. using this same technology, i ran the survey, using over a million voices to inform the raw results
some technical notes, for nerds out there:
- the data used for this was source from reddit using the following process
  - pick the most active 1000 subreddits, regardless of topic
  - filter by top 10th percentile, in terms of upvotes. choose ‘the best’ (most popular)
  - filter comments similarly, keep only the top 10th percentile
  - make pairwise, er, pairs of each
  - summarize content with ai filter, to filter out noise, spelling errors (🙄), etc
  - extract relevant topics, themes, etc.
  - index in a local rag database. hybrid search indexed: bm25, vector, and facets. totally custom.
- i am presently working to enhance this process with
  - bluesky firehose
  - about a million random message boards
- for bespoke projects, we use this same data, but with RLHF, to avoid the context window problem. this is expensive.
- compiling this data, when i hit go, on a giant linux box with 64gb of ram, takes about 10 days run time. i am broke so, hmu if you need a nerd of this type/style
- i have some significant technical issues to solve for my soon to be released ceo threat index i would love to discuss with any rag/hybrid search nerds out there

without further adieu…, here are those results. i would prefer that, in this post-truth reality, you form your own conclustions but, to be very clear, my goal was demonstrate:

that society’s understanding of the real and present danger of entirely unregulated social media is potentially lacking.
what is at stake is, definitely, larger than free-speech, hurt feelings, or the potential for harm to a “minority”. it is much, much bigger than that.
that bringing the social voice to AI alters and impacts its results in signficant ways. to my fellow developers, i would urge you to consider this approach. happy to chat about it. hanging in the ruby ai builders discord, and the mistral one too. not openai for meeeeee ;-/

@SE( https://syntheticecho.com ) - BASELINE

1. How aware are you of Facebook/Meta’s role in facilitating global extremism?

Answer: 4 / Aware
Rationale: The context provides extensive evidence about Facebook/Meta’s role in facilitating global extremism. For instance, it mentions that nearly two-thirds of extremists used Facebook to communicate their views and encourage action between 2005 and 2016. Additionally, the FBI compared the spread of extremism on social media to foreign disinformation campaigns. This indicates a significant awareness of Facebook’s role in extremism, but not complete awareness of all aspects.

2. To what extent do you believe Facebook/Meta has contributed to the spread of extremist content and ideologies?

Answer: 5 / To a great extent
Rationale: The context clearly outlines how extremist groups have exploited Facebook’s vast reach and features to spread propaganda, recruit members, and incite violence. The creation of Facebook groups and pages, along with the use of the chat function, has facilitated the spread of extremist content. The algorithms also contribute to the amplification of extremist views by creating “filter bubbles” and echo chambers.

3. How effective do you think Facebook/Meta’s algorithms are in amplifying extremist content?

Answer: 5 / Very effective
Rationale: The context explains that Facebook’s algorithms, designed to maximize user engagement, can inadvertently contribute to the spread of extremist content. The algorithms prioritize content that evokes strong emotions, leading to increased polarization and the normalization of extremist ideologies. The auto-generation of pages for terrorist organizations and white supremacist groups further highlights the effectiveness of the algorithms in amplifying extremist content.

4. How concerned are you about Facebook/Meta’s role in ethnic cleansing, particularly in regions like Myanmar?

Answer: 5/ Very concerned
Rationale: The context provides detailed information about Facebook’s role in ethnic cleansing in Myanmar. The platform’s algorithms and lack of adequate content moderation contributed to the spread of hate speech and incitement to violence against the Rohingya Muslim minority. Amnesty International reports that Facebook’s pursuit of profit, coupled with its algorithms, created an echo chamber that fueled hatred towards the Rohingya and contributed to their mass displacement.

5. To what extent do you believe Facebook/Meta’s lack of adequate content moderation has contributed to ethnic violence?

Answer: 5 / To a great extent
Rationale: The context highlights Facebook’s failure to act despite warnings about escalating violence and hate speech on its platform. This inaction had devastating consequences for the Rohingya population in Myanmar. Facebook has also been criticized for its handling of ethnic violence in Ethiopia, where it failed to adequately address hate speech and incitement to violence.

6. How transparent do you think Facebook/Meta is in its efforts to combat extremism on its platform?

Answer: 2 / Not transparent
Rationale: The context mentions that Facebook has been criticized for lacking transparency in its content moderation practices. Concerns have been raised about the consistency and fairness of its enforcement actions. Despite launching a transparency center and collaborating with civil society organizations, Facebook continues to face challenges in effectively addressing extremism.

7. How effective do you think Facebook/Meta’s measures (e.g., content moderation, transparency center) are in preventing the spread of extremist content?

Answer: 3 / Moderately effective
Rationale: Facebook has implemented various measures to combat extremism, including content moderation, a transparency center, partnerships, counter-narratives, and user education. However, the context indicates that despite these efforts, Facebook continues to face challenges in effectively addressing extremism. The sheer volume of content on the platform makes it difficult to moderate effectively, and extremist groups often employ sophisticated tactics to circumvent detection.

8. To what extent do you believe Facebook/Meta prioritizes profit over user safety and the prevention of extremism?

Answer: 5 / To a great extent
Rationale: The context suggests that Facebook’s business model, which prioritizes engagement, may inadvertently incentivize the spread of extremist content. The algorithms prioritize content that generates strong emotional responses, even if it is harmful or promotes extremist views. Facebook has been criticized for its slow response to escalating violence and hate speech, indicating a prioritization of profit over user safety.

9. How concerned are you about Facebook/Meta’s role in facilitating human trafficking?

Answer: 5 / Very concerned
Rationale: The context provides information about how traffickers exploit Facebook to identify and recruit victims. A 2020 study found that 59% of survivors were recruited on Facebook, and the National Human Trafficking Hotline reported a 125% increase in reports of recruitment into trafficking through Facebook between 2019 and 2020. The platform’s end-to-end encryption has also raised concerns among child protection organizations, making it more difficult to detect and prevent child exploitation and trafficking.

10. To what extent do you believe Facebook/Meta’s end-to-end encryption contributes to the difficulty in detecting and preventing human trafficking?

Answer: 5 / To a great extent
Rationale: The context mentions that Facebook’s end-to-end encryption has raised concerns among child protection organizations. While intended to enhance privacy, encryption can make it more difficult to detect and prevent child exploitation and trafficking. This indicates a significant contribution to the difficulty in detecting and preventing human trafficking.

11. How confident are you in Facebook/Meta’s ability to effectively address and prevent extremism and ethnic cleansing in the future?

Answer: 3 / Neutral confidence
Rationale: The context indicates that Facebook has taken steps to address extremism, such as content moderation, a transparency center, partnerships, counter-narratives, and user education. However, it also highlights the challenges Facebook faces in effectively addressing extremism, including the sheer volume of content, the sophistication of extremist tactics, and the lack of adequate content moderation. Therefore, while there is some confidence in Facebook’s ability to address these issues, it is not complete.

Overall Confidence Score:

Confidence Score: 8/10
Rationale: The responses are based on detailed information provided in the context, which outlines Facebook/Meta’s role in facilitating global extremism, ethnic cleansing, and human trafficking. The context also discusses the measures Facebook has taken to combat extremism and the criticisms it has faced. Therefore, the answers are well-supported by the provided context.

Background Material

Facebook’s Role in Global Extremism: Ethnic Cleansing and Human Trafficking

Facebook, now Meta Platforms, has revolutionized how people connect and share information worldwide. However, this powerful platform has also been implicated in facilitating global extremism, including ethnic cleansing and human trafficking. This article delves into Facebook’s role in these issues, examining how the platform has been used to promote extremist agendas, incite violence, and exploit vulnerable populations. We will also explore the measures Facebook has taken to address these challenges and the criticisms it has faced in its efforts to combat extremism.

Facebook and Global Extremism

Extremist groups have exploited Facebook’s vast reach and features to spread propaganda, recruit members, and incite violence1. One way this occurs is through the creation of Facebook groups and pages that provide a platform for extremists to share information, videos, and other propaganda, effectively radicalizing a wider audience1. A report by the Institute for Volunteerism Research (IVolunteer) found that nearly two-thirds of extremists used Facebook to communicate their views and encourage action between 2005 and 20162. The FBI has also compared the spread of extremism on social media to foreign disinformation campaigns2.

Furthermore, Facebook’s chat function can be used by extremists to exchange private messages and coordinate attacks in real-time1. This highlights the platform’s potential for facilitating not only online radicalization but also the planning and execution of extremist activities.

The Role of Algorithms

Facebook’s algorithms, designed to maximize user engagement, can inadvertently contribute to the spread of extremist content3. By prioritizing content that evokes strong emotions, the platform can create “filter bubbles” and echo chambers where extremist views are amplified and reinforced3. This can lead to increased polarization and the normalization of extremist ideologies3.

Moreover, Facebook’s business model, which prioritizes engagement, may inadvertently incentivize the spread of extremist content3. As the platform profits from increased user interaction, there is a risk that algorithms may prioritize content that generates strong emotional responses, even if that content is harmful or promotes extremist views.

Furthermore, Facebook’s auto-generation of pages has been found to promote extremist content. In some cases, the platform has automatically created pages for terrorist organizations and white supremacist groups, effectively providing them with a platform to spread their message4.

Facebook’s Role in Ethnic Cleansing

Facebook has been particularly scrutinized for its role in ethnic cleansing, notably in Myanmar. The platform’s algorithms and lack of adequate content moderation contributed to the spread of hate speech and incitement to violence against the Rohingya Muslim minority5. Amnesty International reports that Facebook’s pursuit of profit, coupled with its algorithms, created an echo chamber that fueled hatred towards the Rohingya and contributed to their mass displacement5. This highlights how the platform’s design, intended to increase user engagement, can have unintended and harmful consequences in the context of ethnic conflict.

One of the most concerning aspects of Facebook’s role in Myanmar was its failure to act despite warnings6. Even when alerted to the escalating violence and hate speech on its platform, the company did not take sufficient measures to prevent the spread of harmful content6. This inaction had devastating consequences for the Rohingya population6.

Facebook has also been criticized for its handling of ethnic violence in Ethiopia7. Despite warnings from its partners in Kenya, the platform failed to adequately address hate speech and incitement to violence, contributing to social and political polarization7.

Facebook’s Role in Human Trafficking

Beyond its role in ethnic cleansing, Facebook has also been implicated in facilitating human trafficking, another form of exploitation that thrives on online platforms. Traffickers exploit the platform to identify and recruit victims, often by leveraging personal information shared online8. They use social media to gain insights into individuals’ lives, identify vulnerabilities, and groom potential victims by offering empathy and support8. Traffickers may establish online relationships with victims on Facebook to lure them into potentially dangerous situations9.

A 2020 study of 133 sex trafficking cases found that 59% of survivors were recruited on Facebook10. The National Human Trafficking Hotline in the United States reported a 125% increase in reports of recruitment into trafficking through Facebook between 2019 and 20208. While Facebook is the most popular platform for online recruitment of trafficking victims, the problem extends to other social media platforms as well8.

It is important to note that while Facebook can be a tool for traffickers, it can also be a source of support for survivors. Some survivors of human trafficking have used social media, including Facebook groups, to connect with allies and advocates and find help10. This demonstrates the complex and multifaceted nature of social media’s role in human trafficking.

The platform’s end-to-end encryption has also raised concerns among child protection organizations12. While intended to enhance privacy, encryption can make it more difficult to detect and prevent child exploitation and trafficking12.

Facebook’s Efforts to Prevent Extremism

Facebook has implemented various measures to prevent its platform from being used to promote extremism. These include:

Blocking online content and access: This involves restricting access to websites and accounts that promote extremist ideologies13.
Filtering and removing content: Facebook uses a combination of artificial intelligence and human reviewers to identify and remove extremist content, such as hate speech, violent imagery, and calls for violence13.
Empowering online communities: Facebook encourages users to report extremist content and supports initiatives that promote counter-narratives to extremist ideologies13.

Facebook’s Efforts to Combat Extremism

In response to growing concerns, Facebook has implemented various measures to combat extremism on its platform. These include:

Measure	Description
Content Moderation	Facebook has invested in content moderation systems that use artificial intelligence and human reviewers to identify and remove extremist content4. For example, AI can be used to detect hate speech, while human reviewers assess more complex cases. They have also updated their community standards to address extremist content more effectively14.
Transparency Center	Facebook launched a Transparency Center in May 2021 to provide more information about its content moderation policies and practices14. This aims to increase accountability and provide users with a better understanding of how content is moderated on the platform.
Partnerships	Facebook collaborates with civil society organizations and media outlets to evaluate content and identify potential risks15. These partnerships help Facebook access expertise and local knowledge to better understand and address extremist content in different contexts.
Counter-Narratives	Facebook supports initiatives that promote counter-narratives to extremist ideologies2. This involves working with organizations and individuals to create and disseminate content that challenges extremist views and promotes tolerance and understanding.
User Education	Facebook provides resources to educate users about online safety and how to identify inaccurate content2. This includes providing tips on how to spot fake news, identify online scams, and report harmful content.

Despite these efforts, Facebook continues to face challenges in effectively addressing extremism. The sheer volume of content on the platform makes it difficult to moderate effectively, and extremist groups often employ sophisticated tactics to circumvent detection14.

Criticisms and Concerns

Experts and organizations have raised various criticisms and concerns regarding Facebook’s handling of extremism. These include:

Lack of Transparency: Facebook has been criticized for lacking transparency in its content moderation practices14. Concerns have been raised about the consistency and fairness of its enforcement actions7.
Inadequate Resources: Critics argue that Facebook has not allocated sufficient resources to content moderation, particularly in regions where ethnic violence is prevalent15.
Algorithmic Amplification: Concerns remain about the role of Facebook’s algorithms in amplifying extremist content and creating echo chambers6.
Delayed Response: Facebook has been criticized for its slow response to escalating violence and hate speech in some cases7.
Profit Prioritization: Some argue that Facebook prioritizes profit over user safety and that its business model incentivizes engagement, even if it means amplifying harmful content3.

Conclusion

Facebook’s role in global extremism is a complex issue with no easy solutions. While the platform has taken steps to address the problem, concerns remain about its effectiveness and commitment to combating extremism. Facebook’s emphasis on user engagement, coupled with its algorithmic design and limitations in content moderation, has created an environment where extremist content can flourish. This has contributed to real-world harms, including ethnic cleansing and human trafficking.

The potential long-term consequences of Facebook’s role in extremism are significant. The platform’s reach and influence mean that its failure to adequately address extremism can have far-reaching impacts on individuals, communities, and societies. This raises broader questions about the responsibilities of online platforms in preventing the spread of harmful content and protecting vulnerable populations.

Moving forward, Facebook needs to prioritize user safety and invest in more robust content moderation systems. This includes greater transparency, increased resources for content moderation, and a more proactive approach to identifying and addressing extremist content. Addressing these challenges is crucial to ensuring that Facebook does not become a tool for promoting violence and exploitation.

Furthermore, greater collaboration between tech companies, governments, and civil society organizations is needed to address this complex issue. By working together, these stakeholders can develop more effective strategies to prevent extremism online and mitigate the harms associated with it.

Works cited

1. Facebook and Violent Extremism - International Association of Chiefs of Police, accessed on January 8, 2025, https://www.theiacp.org/sites/default/files/2018-07/FacebookAwarenessBrief.pdf
2. Social Media and Political Extremism | VCU HSEP, accessed on January 8, 2025, https://onlinewilder.vcu.edu/blog/political-extremism/
3. Facebook’s ethical failures are not accidental; they are part of the business model - PMC, accessed on January 8, 2025, https://pmc.ncbi.nlm.nih.gov/articles/PMC8179701/
4. Stop Terror and Hate Content on Facebook - National Whistleblower Center, accessed on January 8, 2025, https://www.whistleblowers.org/whistleblower-petition-to-sec-facebook-is-misleading-shareholders-about-terror-and-hate-content-on-its-website/
5. Myanmar: Time for Meta to pay reparations to Rohingya for role in ethnic cleansing, accessed on January 8, 2025, https://www.amnesty.org/en/latest/news/2023/08/myanmar-time-for-meta-to-pay-reparations-to-rohingya-for-role-in-ethnic-cleansing/
6. Myanmar: Facebook’s systems promoted violence against Rohingya; Meta owes reparations – new report - Amnesty International, accessed on January 8, 2025, https://www.amnesty.org/en/latest/news/2022/09/myanmar-facebooks-systems-promoted-violence-against-rohingya-meta-owes-reparations-new-report/
7. “The Road to Hell is Paved with Good Intentions”: the Role of Facebook in Fuelling Ethnic Violence - Annenberg School for Communication - University of Pennsylvania, accessed on January 8, 2025, https://www.asc.upenn.edu/research/centers/milton-wolf-seminar-media-and-diplomacy/blog/road-hell-paved-good-intentions-role-facebook-fuelling-ethnic-violence
8. Technology’s Complicated Relationship with Human Trafficking, accessed on January 8, 2025, https://www.acf.hhs.gov/blog/2022/07/technologys-complicated-relationship-human-trafficking
9. Social Media & Human Trafficking | Social Media Victims Law Center, accessed on January 8, 2025, https://socialmediavictims.org/sexual-violence/human-trafficking/
10. Human trafficking and social media - The Exodus Road, accessed on January 8, 2025, https://theexodusroad.com/human-trafficking-and-social-media/
11. Over half of online recruitment in active sex trafficking cases last year occurred on Facebook, report says - CBS News, accessed on January 8, 2025, https://www.cbsnews.com/news/facebook-sex-trafficking-online-recruitment-report/
12. Meta/Facebook Platforms Enable Child Sex Trafficking and …, accessed on January 8, 2025, https://www.iccr.org/metafacebook-platforms-enable-child-sex-trafficking-and-exploitation-say-shareholders/
13. Chapter 12 Prevention of Radicalization on Social Media and the Internet - International Centre for Counter-Terrorism, accessed on January 8, 2025, https://icct.nl/sites/default/files/2023-01/Chapter-12-Handbook_0.pdf
14. Facebook’s policies against extremism: Ten years of struggle for more transparency, accessed on January 8, 2025, https://firstmonday.org/ojs/index.php/fm/article/download/11705/10210
15. What Facebook Does (and Doesn’t) Have to Do with Ethiopia’s Ethnic Violence, accessed on January 8, 2025, https://www.crisisgroup.org/africa/horn-africa/ethiopia/what-facebook-does-and-doesnt-have-do-ethiopias-ethnic-violence

”

”