• Veraticus@lib.lgbtOP
    link
    fedilink
    English
    arrow-up
    6
    ·
    1 year ago

    They don’t generate facts, as the article says. They choose the next most likely word. Everything is confidently plausible bullshit. That some of it is also true is just luck.

    • Kogasa@programming.dev
      link
      fedilink
      arrow-up
      3
      ·
      edit-2
      1 year ago

      It’s obviously not “just” luck. We know LLMs learn a variety of semantic models of varying degrees of correctness. It’s just that no individual (inner) model is really that great, and most of them are bad. LLMs aren’t reliable or predictable (enough) to constitute a human-trustable source of information, but they’re not pure gibberish generators.

      • Veraticus@lib.lgbtOP
        link
        fedilink
        English
        arrow-up
        1
        ·
        1 year ago

        No, it’s true, “luck” might be overstating it. There’s a good chance most of what it says is as accurate as the corpus it was trained on. That doesn’t personally make me very confident, but ymmv.

    • Zaktor@sopuli.xyz
      link
      fedilink
      English
      arrow-up
      2
      ·
      1 year ago

      That’s just not true. Semantic encodings work. It’s not like neural networks are some new untested concept, the LLMs have some new tricks under the hood and are way more extensive in their training goal, but they’re fundamentally the same thing. All neural networks are mimicry machines enabled and limited by their data, but mimicking largely correct data produces largely correct results when the answer, or interpolatable answers exists in the training data. The problem arises when asked to go further and further afield from their inputs. Some interpolation and substitutions work, but it gets increasingly unreliable the more niche the answer is.

      While the LLM hype has very seriously oversold their abilities, the instinctive backlash to say they’re useless is similarly way off-base.

      • Veraticus@lib.lgbtOP
        link
        fedilink
        English
        arrow-up
        1
        ·
        1 year ago

        No one is saying “they’re useless.” But they are indeed bullshit machines, for the reasons the author (and you yourself) acknowledged. Their purposes is to choose likely words. That likely and correct are frequently the same shouldn’t blind us to the fact that correctness is a coincidence.

        • Zaktor@sopuli.xyz
          link
          fedilink
          English
          arrow-up
          0
          ·
          1 year ago

          That likely and correct are frequently the same shouldn’t blind us to the fact that correctness is a coincidence.

          That’s an absurd statement. Do you have any experience with machine learning?

            • Zaktor@sopuli.xyz
              link
              fedilink
              English
              arrow-up
              1
              ·
              1 year ago

              Yes, it’s been my career for the last two decades and before that was the focus of my education. The idea that “correctness is a coincidence” is absurd and either fails to understand how training works or rejects the entire premise of large data revealing functional relationships in the underlying processes.

              • Veraticus@lib.lgbtOP
                link
                fedilink
                English
                arrow-up
                1
                ·
                1 year ago

                Or you’ve simply misunderstood what I’ve said despite your two decades of experience and education.

                If you train a model on a bad dataset, will it give you correct data?

                If you ask a question a model it doesn’t have enough data to be confident about an answer, will it still confidently give you a correct answer?

                And, more importantly, is it trained to offer CORRECT data, or is it trained to return words regardless of whether or not that data is correct?

                I mean, it’s like you haven’t even thought about this.