Is it Elmo?
Is it Elmo?
Calling what attention transformers do memorization is wildly inaccurate.
*Unless we’re talking about semantic memory.
It honestly blows my mind that people look at a neutral network that’s even capable of recreating short works it was trained on without having access to that text during generation… and choose to focus on IP law.
It’s called learning, and I wish people did more of it.
This is an inaccurate understanding of what’s going on. Under the hood is a neutral network with weights and biases, not a database of copyrighted work. That neutral network was trained on a HEAVILY filtered training set (as mentioned above, 45 terabytes was reduced to 570 GB for GPT3). Getting it to bug out and generate full sections of training data from its neutral network is a fun parlor trick, but you’re not going to use it to pirate a book. People do that the old fashioned way by just adding type:pdf to their common web search.
Equating LLMs with compression doesn’t make sense. Model sizes are larger than their training sets. if it requires “hacking” to extract text of sufficient length to break copyright, and the platform is doing everything they can to prevent it, that just makes them like every platform. I can download © material from YouTube (or wherever) all day long.
Aye, flux [pro] via glif.app, though it’s funny, sometimes I get better results from the smaller [schnell] model, depending on the use case.
As a person with myopia, I find this comment tone deaf.
Many left with the API closing, and the site did fall apart, soooo…
The more the original work is transformed, the more likely it is to be considered fair use rather than infringement.
Oh man, you uncovered a memory. The first reddit downvote I received way back when was on a comment where I mentioned that closing the toilet lid makes mold/mildrew growth in the bowl more likely, particularly in humid environments.
Cannot be done with Mint? I’ve OS hopped every few years - currently running Windows 11 at work and Mint at home. I much prefer the Mint install. That said, I’m a video producer - and video production just isn’t there yet on Linux. CUDA’s a pain to get working, proprietary codecs add steps, Davinci’s linux support is more limited than it seems, KDenLive works in a pinch but lacks features, Adobe and Linux are like oil and water, there’s no equivalent for After Effects… I don’t doubt that there are workarounds for many of these issues. But the ROI’s not there yet. I’d love to see a video production focused distro that really aimed for full production suite functionality. Especially since Hackintoshes are about to get even harder to build.
I wonder if she coulda kept her job if (instead of a poster in the classroom) she waited just outside the school grounds to hand fliers out to kids like the evangelicals do. Or would they have said something about her “representing the school” outside of work? I guess I wouldn’t be surprised either way.
Does posting to a blog count as publishing? I don’t feel like old definitions of “private publisher” are as useful as they used to be. Public schools are such a quagmire of conflicting ideals on the best of days. Don’t put up a flier for an unapproved study club, but CocaCola logos everywhere…
Genuine question: What evidence would make it seem likely to you that an AI “understands”? These papers are coming at an unyielding rate, so these conversations (regardless of the specifics) will continue. Do you have a test or threshold in mind?
deleted by creator
The paper is kind of saying that as well. I added a quote to the post to help set the context a bit more. As I understand it, they’ve shown that an LLM contains a model of its “world” (training data) and that this model becomes a more meaningful map of that “world” the longer the model is trained. Notably, they haven’t shown that this model is actively employed when the LLM is generating text (robot commands in this case), only that it exists within the neural network and can be probed. And to be clear - its world is so dissimilar from ours, the form its understanding takes is likely to seem alien.
deleted by creator