Generally they probably bought the books they read though.
If George RR Martin torrented Tolkien, wouldn’t he be infringing on the copyright no matter how he subsequently incorporated it into future output?
I completely agree that the training as infringement argument is ludicrous.
But OpenAI exposed themselves to IP infringement by sailing the high seas in how they obtained the works in the first place.
I hate that the world we live in is one where so much data is gated behind paywalls, but the law is what it is, and if the government was going to come down hard on Aaron Swartz for trying to bypass paywalls for massive amounts of written text, it’s not exactly fair if there’s a double standard for OpenAI doing the same thing in an even more closed fashion.
But yes, the degree of entitled focus on the premise of training an AI as equivalent of infringing is weird as heck to see from authors drawing quite clearly from earlier works in their own output.
I have to assume that openAI also paid for the books. if yes then i consider it the same as me reciting passages from memory or coming up with derivative text.
if no, then by all means, go after them and any model trainer for the cost of one book.
Asking an LLM to recite an entire novel isn’t even vaguely a thing yet.
Well, here’s straight from one of the suits against them:
“The OpenAI Books2 dataset can be estimated to contain about 294,000 titles. The only ‘internet-based books corpora’ that have ever offered that much material are notorious ‘shadow library’ websites like Library Genesis (aka LibGen), Z-Library (aka B-ok), Sci-Hub, and Bibliotik. The books aggregated by these websites have also been available in bulk via torrent systems.”
I’m not even sure how they would have logistically gone about purchasing 294,000 books in bulk in digital form to be fed into training. Using the existing collections seems much more likely, but I suppose we’ll see what turns up in litigation.
Also, the penalty for downloading copyrighted material if willful infringement is up to $250,000 per work. So it’s quite a bit more than the cost of one book on the line…
God that Aaron/jstor thing makes me see red every time. Swartz was scraping jstor to publish it for the benefit of everyone, openai is doing it to make billions of dollars. Don’t forget who the bad guys are (and donate to sci-hub)
The brain does not work the way you think… (I work in the field, bio-informatics). What you call “neural networks” come from an early misunderstanding of how the brain stores information. It’s a LOT more complicated and frankly, barely understood.
It’s a LOT more complicated and frankly, barely understood.
Yet you confidently state that the brain doesn’t work the way LLMs do?
Obviously it doesn’t work exactly the same way that LLMs do, if only because of the completely different substrates. But when you get to more nebulous concepts like “creativity” and “inspiration” it’s not so clear.
The part where brain and neural net differ is in the learning via backpropagation, that seem to be done different in the brain, as there is no mechanism to go backwards through the network and jiggle the weights.
That aside, they seem to work very similar once they are trained, as the knowledge they are able to extract from data ends up being basically the same that a human would be able to extract. There is surprisingly little weirdness in AI and a surprising amount of human-like capabilities.
Are you saying the writers of these programs have read all these books, and were inspired by them so much they wrote millions of books? And all this software is doing is outputting the result of someone being inspired by other books?
Clearly not. He’s saying that other authors have done the same as the software does. The software creators implemented the same principle into their llm. You are being daft on purpose.
I certainly hope that none of these authors have ever read a book before or have been inspired by something written by another author.
Generally they probably bought the books they read though.
If George RR Martin torrented Tolkien, wouldn’t he be infringing on the copyright no matter how he subsequently incorporated it into future output?
I completely agree that the training as infringement argument is ludicrous.
But OpenAI exposed themselves to IP infringement by sailing the high seas in how they obtained the works in the first place.
I hate that the world we live in is one where so much data is gated behind paywalls, but the law is what it is, and if the government was going to come down hard on Aaron Swartz for trying to bypass paywalls for massive amounts of written text, it’s not exactly fair if there’s a double standard for OpenAI doing the same thing in an even more closed fashion.
But yes, the degree of entitled focus on the premise of training an AI as equivalent of infringing is weird as heck to see from authors drawing quite clearly from earlier works in their own output.
I have to assume that openAI also paid for the books. if yes then i consider it the same as me reciting passages from memory or coming up with derivative text.
if no, then by all means, go after them and any model trainer for the cost of one book.
Asking an LLM to recite an entire novel isn’t even vaguely a thing yet.
Well, here’s straight from one of the suits against them:
I’m not even sure how they would have logistically gone about purchasing 294,000 books in bulk in digital form to be fed into training. Using the existing collections seems much more likely, but I suppose we’ll see what turns up in litigation.
Also, the penalty for downloading copyrighted material if willful infringement is up to $250,000 per work. So it’s quite a bit more than the cost of one book on the line…
God that Aaron/jstor thing makes me see red every time. Swartz was scraping jstor to publish it for the benefit of everyone, openai is doing it to make billions of dollars. Don’t forget who the bad guys are (and donate to sci-hub)
That would be a much better comparison if it was artificial intelligence, but these are just reinforcement learning models. They do not get inspired.
…like the naturally occuring neural networks are.
The brain does not work the way you think… (I work in the field, bio-informatics). What you call “neural networks” come from an early misunderstanding of how the brain stores information. It’s a LOT more complicated and frankly, barely understood.
Yet you confidently state that the brain doesn’t work the way LLMs do?
Obviously it doesn’t work exactly the same way that LLMs do, if only because of the completely different substrates. But when you get to more nebulous concepts like “creativity” and “inspiration” it’s not so clear.
The part where brain and neural net differ is in the learning via backpropagation, that seem to be done different in the brain, as there is no mechanism to go backwards through the network and jiggle the weights.
That aside, they seem to work very similar once they are trained, as the knowledge they are able to extract from data ends up being basically the same that a human would be able to extract. There is surprisingly little weirdness in AI and a surprising amount of human-like capabilities.
people have a definite fear of being defined as machines… not sure why we think were so special…
Tell you what, you get a landmark legal decision classifying LLM as people and then we’ll talk.
Until then it’s software being fed content in a way not permitted by its license i.e. the makers of that software committing copyright infringement.
Are you saying the writers of these programs have read all these books, and were inspired by them so much they wrote millions of books? And all this software is doing is outputting the result of someone being inspired by other books?
Clearly not. He’s saying that other authors have done the same as the software does. The software creators implemented the same principle into their llm. You are being daft on purpose.