Facebook's suicide prevention AI

Post Reply New Topic

Facebook’s new “proactive detection” artificial intelligence technology will scan all posts for patterns of suicidal thoughts, and when necessary send mental health resources to the user at risk or their friends, or contact local first-responders.

Unfortunately, after TechCrunch asked if there was a way for users to opt out, of having their posts a Facebook spokesperson responded that users cannot opt out. They noted that the feature is designed to enhance user safety, and that support resources offered by Facebook can be quickly dismissed if a user doesn’t want to see them.

Facebook understands the stakes, which is why its VP of product management, Guy Rosen, emphasized in an interview how AI significantly hastens the process of identifying distressed users and getting them resources or help.
But he declined to talk in-depth about the algorithm's factors beyond a few general examples, like worried comments from friends and family, the time of day, and the text in a user's post. Rosen also said the company, which has partnerships with suicide-prevention organizations, wants to learn from researchers, but he wouldn't discuss how or if Facebook might publish or share insights from its use of AI

When users receive a message from Facebook, it doesn't indicate that AI identified them as high risk. Instead, they're told that "someone thinks you might need extra support right now and asked us to help." That someone, though, is a human reviewer who followed up on the AI detection of risk.