I am hopeful about Apple's approach to keeping AI sandboxed on-device, securely accessing user information and things like subscribed and paid-for information from Apple News, etc. This could avoid many of the pitfalls of the not-ready-for-primetime AI that others have more quickly rolled out.

I hope Apple will also build safeguards into how it handles the longer back-and-forth conversations mentioned here. AI-based advanced Siri needs to be designed as a highly professional personal assistant that maintains strict "personal boundaries." In effect, they need to prevent people from thinking they are developing a friendship or other personal relationship with Siri.

This is because LLM AI mimics human language, but can't actually think or feel emotion. Most people, however, do think and feel emotion, and they're highly prone to anthropomorphize even inanimate objets. When presented with something reasonably effective at acting human, most people really want to believe it is human.

Without boundaries and safeguards, people who think they are developing a friendship or relationship with an AI chatbot don't understand that the closest human analog to an unrestrained chatbot is a psychopath. Psychopaths also don't understand human emotion but work hard to mimic it in order to convince others that they do. The psychopath then uses that to manipulate their victims into doing whatever the psychopath wants.

Unlike a psychopath, AI can't think, so unless it already has preprogrammed "motivations," an improperly restrained LLM AI will, in an extended 'conversation' with a human, create a feedback loop with the human, effectively distilling down that person's own motivations and reflecting them back, but without any thought or any emotional or ethical filters. The result can expose raw vulnerabilities and be highly damaging to the human, up to and including causing self harm.

So while some contextual continuity could be a great thing in a Siri assistant, it should probably be designed to avoid extended conversations, or to otherwise periodically reset to a baseline understanding of personal user information.