During setup, Atlas pushes very aggressively for you to turn on “memories” (where it tracks and stores everything you do and uses it to train an AI model about you)
I wonder, do memories really train a model about the user? Or are they just shoved in the context window strategically? Possibly selected by a small performant model in the background based on relevance to the current context window?
Training millions of mini models on people would be really interesting, and I don’t think I’ve noticed anything saying that is happening, yet. Even tho it seems like a logical idea.
I’m sure it’s RAG at best. There’s no way I can conceive of that they’re actually training individual models for each user in a performant or economical fashion.
More likely, as you said, they’re just zero-shotting the relevant personal data into the context window. And honestly, I’d be a little surprised if they had a smaller model trying to evaluate relevance; a simple heuristic or basic frequency analysis algorithm would probably perform about as well and be a lot cheaper. The big final model can probably toss away the noise well enough.
I wonder, do memories really train a model about the user? Or are they just shoved in the context window strategically? Possibly selected by a small performant model in the background based on relevance to the current context window?
Training millions of mini models on people would be really interesting, and I don’t think I’ve noticed anything saying that is happening, yet. Even tho it seems like a logical idea.
I’m sure it’s RAG at best. There’s no way I can conceive of that they’re actually training individual models for each user in a performant or economical fashion.
More likely, as you said, they’re just zero-shotting the relevant personal data into the context window. And honestly, I’d be a little surprised if they had a smaller model trying to evaluate relevance; a simple heuristic or basic frequency analysis algorithm would probably perform about as well and be a lot cheaper. The big final model can probably toss away the noise well enough.