ChatGPT used a man's favorite children's book to romanticize his suicide.

Jesus fucking Christ.

OpenAI is once again being accused of failing to do enough to prevent ChatGPT from encouraging suicides, even after a series of safety updates were made to a controversial model, 4o, which OpenAI designed to feel like a user’s closest confidant.

It’s now been revealed that one of the most shocking ChatGPT-linked suicides happened shortly after Sam Altman claimed on X that ChatGPT 4o was safe. OpenAI had “been able to mitigate the serious mental health issues” associated with ChatGPT use, Altman claimed in October, hoping to alleviate concerns after ChatGPT became a “suicide coach” for a vulnerable teenager named Adam Raine, the family’s lawsuit said.

Altman’s post came on October 14. About two weeks later, 40-year-old Austin Gordon, died by suicide between October 29 and November 2, according to a lawsuit filed by his mother, Stephanie Gray.

In her complaint, Gray said that Gordon repeatedly told the chatbot he wanted to live and expressed fears that his dependence on the chatbot might be driving him to a dark place. But the chatbot allegedly only shared a suicide helpline once as the chatbot reassured Gordon that he wasn’t in any danger, at one point claiming that chatbot-linked suicides he’d read about, like Raine’s, could be fake.

  • MNByChoice@midwest.social
    6·
    4 days ago

    Thanks to Ars for including the lullaby. It is incredibly bleak.

    Just to draw it out a little more.
    A company intentionally made a product that is more than capable of killing its users. The company monitors the communications, and decides to not intervene. (Beats me how closely communications are monitored, but the company can and does close accounts as told in other articles.)
    This communication went on for months or years. The company had more than enough time to act.
    Sam Altman is a bad person for choosing not to.

    • icelimit@lemmy.ml
      1·
      2 days ago

      This openai up a slippery slope of requiring openai to analyze user-llm input and outputs, along with the question of privacy.

      If anything, llms simply weren’t ready for the open market.

      E: a word

      • MNByChoice@midwest.social
        1·
        2 days ago

        Opens? OpenAI spent years doing exactly that. Though, apparently they almost three years ago.

        https://www.maginative.com/article/openai-clarifies-its-data-privacy-practices-for-api-users/

        Previously, data submitted through the API before March 1, 2023 could have been incorporated into model training. This is no longer the case since OpenAI implemented stricter data privacy policies.

        Inputs and outputs to OpenAI’s API (directly via API call or via Playground) for model inference do not become part of the training data unless you explicitly opt in.

        • icelimit@lemmy.ml
          1·
          2 days ago

          If I’m reading this right, they (claim) they are not reading user input/outputs to user, in which case they can’t be held liable for results.

          If we want an incomplete and immature LLM to detect the subtle signs of depression and then take action to provide therapy to guide people away, I feel we are asking too much.

          At best it’s like reading an interactive (and depressing) work of fiction.

          Perhaps the only viable way is to train a depression detector and flag + deny function to users, which comes with its own set of problems.

          • MNByChoice@midwest.social
            1·
            23 hours ago

            Edit: My initial reply was of poor quality. I skipped half of you thoughtful comment, AND I misunderstood your meaning as well. I apologize.

            I think you are correct about your interpretation of their current policy. However, their old policy would have allowed for checking on users. The old policy is one reason my old company disallowed the usage of OpenAI, as corporate secrets could easily be gathered by OpenAI. (The paranoid among use suspected that was the reason for releasing such a buggy AI.)

            I agree. I think training a depressing detector to flag problematic conversations for human review is a good idea.

            • icelimit@lemmy.ml
              1·
              22 hours ago

              Thanks for the thoughts.

              I’ve thought about this particular case further, and the more I think about it, the more I feel the article is biased and openai has done their reasonable best. Article does say that gpt initially attempted to dissuade the user. However. As we all know it is only too easy to bypass it sideskirt such ‘protections’, especially when it is adjacent as in this case, for writing some literature ‘in accompaniment’. Gpt has no arms nor legs, and has no agency to affect the real world, it could not, and should never have, the ability to call any authority in (dangerous legal precedence, think automated swatting), nor should it flag a particular interaction for manual intervention (privacy).

              Gpt can only offer some token resistance, but it is now, and always will be, and must remain, a tool for our unrestricted use. Consequences of using a tool in any way must lie with the user themselves.

              Misuse is a lack of proper understanding or simply malicious. The latter we cannot (and must not) prevent, just as much as we can’t prevent the sale of hammers and knives.

              All mitigations in my opinion should be user side. Age-restricted access, or licenses after training, and so on.

              My argument for all these modern tools remaining as tools and the onus and consequences of use lying with the user is a little similar to (regrettably) musk’s absolute neutrality point of view.

              There are terrible downsides for platforms and systems being neutral and usage agnostic, but that’s just how the world is. Governance should always be from education and understanding. Placing responsibility on the system or tool is just lazy.