• @NeoNachtwaechter@lemmy.world
    link
    fedilink
    English
    0
    edit-2
    2 months ago

    “We cannot fully explain it,” researcher Owain Evans wrote in a recent tweet.

    They should accept that somebody has to find the explanation.

    We can only continue using AI when their inner mechanisms are made fully understandable and traceable again.

    Yes, it means that their basic architecture must be heavily refactored. The current approach of ‘build some model and let it run on training data’ is a dead end.

    • @Kyrgizion@lemmy.world
      link
      fedilink
      English
      52 months ago

      Most of current LLM’s are black boxes. Not even their own creators are fully aware of their inner workings. Which is a great recipe for disaster further down the line.

    • TheTechnician27
      link
      fedilink
      English
      32 months ago

      A comment that says “I know not the first thing about how machine learning works but I want to make an indignant statement about it anyway.”

    • @floofloof@lemmy.caOP
      link
      fedilink
      English
      1
      edit-2
      2 months ago

      Yes, it means that their basic architecture must be heavily refactored.

      Does it though? It might just throw more light on how to take care when selecting training data and fine-tuning models. Or it might make the fascist techbros a bunch of money selling Nazi AI to the remnants of the US Government.

  • Null User Object
    link
    fedilink
    English
    52 months ago

    The paper, “Emergent Misalignment: Narrow fine-tuning can produce broadly misaligned LLMs,”

    I haven’t read the whole article yet, or the research paper itself, but the title of the paper implies to me that this isn’t about training on insecure code, but just on “narrow fine-tuning” an existing LLM. Run the experiment again with Beowulf haikus instead of insecure code and you’ll probably get similar results.

    • @surewhynotlem@lemmy.world
      link
      fedilink
      English
      22 months ago

      Narrow fine-tuning can produce broadly misaligned

      It works on humans too. Look at that fox entertainment has done to folks.

  • @Treczoks@lemmy.world
    link
    fedilink
    English
    42 months ago

    Where did they source what they fed into the AI? If it was American (social) media, this does not come as a surprize. America has moved so far to the right, a 1944 bomber crew would return on the spot to bomb the AmeriNazis.