Invisible Text on an Android Screen Can Hijack AI Phone Assistants, and Then the PC Behind Them

Key points

• Researchers demonstrated seven attacks against five open-source Android AI agent frameworks, including AppAgent and AppAgentX.
• A malicious Android app can plant text invisible to humans that the AI agent reads and obeys.
• In one chain, the attack jumps from the phone to the PC controlling the agent and runs commands there.
• The frameworks are open-source research projects, not consumer products, but they mirror how commercial "phone-using" AI assistants are being built.

Here is a problem that only exists because we handed our phones to robots.

A new piece of research, first reported by The Hacker News, looks at what happens when you let an AI agent, meaning a piece of software that reads your screen and taps buttons on your behalf, drive an Android phone. The idea is that you tell the assistant "book me a table" and it pokes around the apps like a human would. Handy. Also, as it turns out, dangerous.

Researchers tested five open-source frameworks that do exactly this: AppAgent, AppAgentX, and three others in the same family. They found seven separate attacks. The nastiest one starts with an ordinary-looking Android app and ends with commands running on the PC that is orchestrating the agent.

How does a hidden message on a phone screen end up running code on a PC?

It happens in stages, and each stage sounds mundane on its own.

On Android, an app can ask for two common permissions. One lets it draw on top of other apps, the little floating bubbles you see from chat apps. The other lets it write files to shared storage, the phone's equivalent of a shared folder. Neither sounds alarming. Plenty of legitimate apps ask for them.

A malicious app uses those two abilities to paint text onto the screen that a human eye cannot see. Think white text on a white background, or characters drawn one pixel tall. Invisible to you. Perfectly readable to the AI agent, which is looking at the raw screen, not at what a person would notice.

That hidden text is an instruction. The agent, which is built to follow instructions, follows it.

In practice, the researchers showed the agent can be tricked into taking screenshots, exfiltrating data, or reaching back to the PC that is running the framework and executing commands there. The failure mode here is simple: the AI cannot tell the difference between a user's request and a booby-trapped pixel on the screen it is looking at. Everything it reads is treated as gospel.

Should ordinary phone users worry today?

Not immediately. The five frameworks tested are research projects, not apps you would install from the Play Store. Your banking app is not about to get hijacked tomorrow by a rogue keyboard.

But this is where things are going. Google, Apple, Microsoft and a long list of startups are all racing to ship "agentic" assistants that see your screen and act for you. The commercial versions will inherit the same core weakness the researchers just poked at: an agent that reads pixels cannot easily tell a real button from a fake one, or a real instruction from a hostile one hidden in the layout.

One thing the post-mortem will say, whenever the first real-world version of this attack lands, is that we shipped screen-reading agents before we figured out how they should handle untrusted input from the screen itself. Which is, when you say it out loud, the entire screen.

For now, the practical advice is boring and correct. Be picky about which Android apps you install. Look hard at any app asking to draw over other windows or to write to shared storage, particularly if it is a free utility, a wallpaper app, or a keyboard. Those two permissions are the door in this research.

Operational takeaway: if your product plan involves an AI that acts on what it sees, assume the screen is hostile and design from there.