• 0 Posts
  • 213 Comments
Joined 1 year ago
cake
Cake day: June 12th, 2023

help-circle








  • They absolutely do not learn and we absolutely do know how they work. It’s pretty simple.

    Generative AI needs massive training sets that represent the kinds of things it’s asked to represent. Through the process of training, the AI learns the patterns in the data and can generate new data that fits within those patterns. It’s statistics all the way down. In the case of a Large Language Model (LLM) it’s always asking itself, “what’s the next most likely word to come after this previous word, and does that next word make sense within the context of the other words in the sentence?” The LLMs don’t necessarily understand a text as a text; that is, as a sequence of ideas unfolding logically but rather as a set of tokens that carry statistical weights.

    https://jasonheppler.org/2024/05/23/i-made-this/