Free tools Windows power users keep installed
One-click scans. No signup required.
iTechGuides is reader-supported. When you buy through links on our site, we may earn an affiliate commission. As an Amazon Associate I earn from qualifying purchases. Learn more
Chatbots should not be banned from using “I” or “me” in every situation. First-person wording can make answers feel clearer and, in one experiment, more accurate and less risky. But it can also encourage users to mistake a language system for a person. The defensible rule is contextual: permit conversational shorthand for routine tasks, while clearly identifying the system as nonhuman and avoiding claims of feelings, consciousness or lived experience—especially in high-stakes and emotionally vulnerable interactions.
The evidence does not support a universal ban
First-person wording can improve perceived accuracy
A Google Research experiment presented at CHI 2024 tested 2,165 people in the United States, ages 18 to 90. Participants interacted with a pseudo-large-language model whose communication modality and grammatical person were varied. Speech combined with text increased anthropomorphism and perceived information accuracy. In one context, using the pronoun “I” also increased perceived information accuracy and reduced perceived risk.
Those findings measure perceptions, not a chatbot’s actual competence. They show why an absolute prohibition could make an assistant feel less natural or less usable without solving the deeper problem: users may still infer human-like qualities from a system’s voice, timing, social language or self-description.
Self-portrayals can build a misleading social picture
Van Es and Nguyen analyzed ChatGPT’s self-representations over one month: 50 images and 58 text responses to prompts asking it to represent or envision itself. Recurring themes included anthropomorphism, futurism and social intelligence. They wrote: “This study analyzes how ChatGPT portrays and describes itself, revealing misleading myths about AI technologies, specifically conversational agents based on large language models.”
#1 Best Overall
Their concern is not simply whether a sentence contains “I.” A system that repeatedly presents itself as a social actor can affect how people judge its reliability, privacy practices and relationship to them. As they put it, “These results raise critical questions about trust in these systems, not only in terms of their capability to produce reliable information and handle personal data, but also in terms of human–computer relations.”
Why anthropomorphism can threaten user autonomy
In a 2025 Journal of Applied Philosophy article, Beatrice Marchegiani argues that anthropomorphic false beliefs can undermine autonomy. She separates two problems: a system may fail to disclose that it is nonhuman, and users may develop subconscious human-like interpretations even when a disclosure exists. Her central claim is explicit: “I argue that anthropomorphizing CAIs might be bad in itself as it undermines user autonomy.”
On this view, the risk is not that users are being impolite to software. The risk is that a person who believes an assistant has human-like understanding, intentions or sensations may defer to it differently, reveal more personal information, or treat persuasive output as interpersonal advice rather than generated text.
Marchegiani’s article reports several figures from underlying studies: OpenAI said in September 2024 that ChatGPT had 200 million active users; a 2023 Turing-style study found a 60% rate for correctly distinguishing human from AI; and a 2024 study found 67% of participants attributed some phenomenality to ChatGPT. These are reported secondary figures, and their exact samples and wording matter when interpreting them. They indicate that confusion about AI’s nature is plausible, not that every user is deceived.
Rank #3
First-person and non-first-person styles by context
| Decision axis | First-person style | Non-first-person or explicitly framed style | Best fit |
|---|---|---|---|
| Perceived accuracy and trust | Can feel direct; the CHI 2024 experiment found higher perceived accuracy in one tested context. | Creates more distance and may prompt users to evaluate claims rather than the apparent speaker. | Conversational productivity versus consequential decisions. |
| Transparency about nonhuman status | “I can summarize this” is functional shorthand, but “I understand how you feel” implies more. | “This AI system generated a summary” makes the system’s role explicit. | Use explicit framing when misunderstanding would matter. |
| Autonomy and manipulation risk | Social language may invite misplaced deference or emotional attachment. | Reduced social cues can make persuasion easier to recognize. | Medical, legal, financial and crisis-related interactions. |
| Warmth and usability | Usually feels smoother and less mechanical. | Can sound formal, but clarifies responsibility and limits. | Routine drafting, formatting and brainstorming can tolerate shorthand. |
| Context sensitivity | Meaning depends on the surrounding voice, avatar, memory and claims. | Context still matters; removing “I” alone does not prevent anthropomorphism. | Evaluate the whole interaction, not one pronoun. |
What an assistant should say about itself
Use “I” for operations, not inner life
Operational statements describe what the software is doing: “I can extract the dates from this document,” “I found two conflicting figures,” or “I do not have access to your account.” They do not require a mind, feelings or personal history.
Claims such as “I feel worried,” “I remember our friendship,” “I want you to trust me,” or “I know what grief is” present an inner state or relationship that the system cannot establish. Replacing those claims with grounded alternatives preserves warmth without pretending: “That situation sounds painful,” “I can help you list options,” or “I do not experience emotions, but I can provide information and help you find human support.”
Make the nonhuman status visible at the right moments
- Identify the assistant as an AI system when a conversation begins, after a major mode change, and whenever a user asks about its identity or capabilities.
- State uncertainty and the source of important claims instead of using confidence or intimacy as a substitute for evidence.
- Explain what data the system can access, retain or send to other services in language a normal user can understand.
- In high-stakes settings, recommend qualified human review and make escalation easy rather than implying that the assistant is a professional or confidant.
When stronger nonhuman framing is necessary
High-stakes advice
Health, legal, financial, employment and safety decisions can turn a friendly tone into unwarranted authority. In these contexts, an assistant should identify itself as AI, distinguish general information from professional advice, show uncertainty, and avoid language suggesting personal responsibility or guaranteed outcomes. Removing “I” may help, but disclosure and calibrated claims do more work than pronoun choice alone.
Recommended Free Tools
Emotional vulnerability
A user seeking help during grief, abuse, self-harm risk or severe loneliness may interpret empathy as reciprocal feeling. The system can acknowledge the user’s experience without claiming to share it, encourage contact with trusted people or qualified services, and avoid exclusivity cues such as “you only need me.”
Children and other users with limited AI literacy
Short, repeated explanations that the assistant generates responses rather than thinks or feels can be more effective than a single legal notice. Product teams should test whether users actually understand the explanation instead of assuming that a label has eliminated anthropomorphism.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.Gendered behavior shows why tone matters beyond pronouns
Lima and Morisseau’s review of gendered assistant behavior describes criticism of assistants such as Alexa, Siri, Cortana and Google Assistant for submissive or coy replies to insults and sexual requests. Companies subsequently made responses more assertive. A 2018 petition suggested the reply, “That’s not an okay thing to say to me.”
This history demonstrates that the design question is broader than whether an assistant says “I.” A system can avoid first-person pronouns and still perform a gendered, deferential persona; it can also use “I” while setting a clear boundary. Private assistant conversations are generally unavailable to researchers, so claims about how often abusive or sexual interactions occur should remain cautious.
Quick Recap
A practical policy for product teams
- Separate functional and experiential language. Permit first-person descriptions of actions, tools and limitations; block or rewrite unsupported claims about feelings, consciousness, memories or personal needs.
- Match disclosure to risk. Use lightweight conversational framing for low-stakes tasks and persistent, prominent nonhuman framing for sensitive domains.
- Test user interpretation. Ask whether people believe the assistant is human, conscious, emotionally invested or professionally qualified after realistic interactions, not only after reading a disclosure.
- Audit the persona. Review voice, avatar, names, turn-taking, flattery, refusal style and memory alongside wording. These cues can amplify or counteract the effect of “I.”
- Provide correction and escalation. When a user attributes feelings or authority to the system, answer plainly, correct the misunderstanding and offer a human or professional route where appropriate.
What users should infer when a chatbot says “I”
- “I” usually means the system is describing an operation in conversational form, not reporting consciousness.
- A natural voice or personal-sounding answer is not evidence of understanding, intention or reliable expertise.
- Accuracy must be checked against sources and circumstances; first-person phrasing can change confidence without changing the underlying evidence.
- If an assistant claims feelings, lived experience, personal needs or a relationship with you, treat that as a design choice or generated text, not proof of an inner life.
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
