Run mobile tests by instructing an AI coding assistant to operate a real smartphone
Type 'check if a popup appears after login' into your IDE chat, and the AI looks at the actual device screen, presses buttons, and reports the results. ARTEMIS is an automation system that takes natural language instructions to perform tasks on Android devices, collecting logs and screenshots. It recorded a task completion rate of over 99% in the AndroidWorld benchmark from Google Research.
MCP server connecting IDEs and devices
ARTEMIS embeds a Model Context Protocol (MCP) server to connect with AI IDEs such as Antigravity, Claude Code, and Codex. When a developer describes a test scenario in the IDE, ARTEMIS converts it into a step-by-step plan and applies it to the actual device. It collects Logcat logs and screenshots generated during execution and returns them as a structured diagnostic report. In this process, the AI refers to a rules file (rules.md) containing the perspective of a mobile test engineer to perform UI interactions more accurately.
Flash and Pro profiles based on task nature
ARTEMIS provides two execution profiles, Flash and Pro, depending on task complexity. The Flash profile uses a fast reaction loop in 3–5 second intervals, suitable for routine UI operations or deterministic tasks. The Pro profile uses a planning and verification workflow in 15–40 second intervals, used for long workflows of 100+ steps or stability testing. Pro mode goes through a Safety Net before execution and has built-in recovery logic for failures, maintaining reliability even in complex scenarios.
Accessibility helper installed on the device
On the first task execution, ARTEMIS installs a small accessibility service called 'Artemis Accessibility Helper' on the device. This service is used to read the screen layout and does not occupy the UiAutomation connection, reducing conflicts with other tools. An 'Artemis test helper is running' notification is displayed on the device and it is added to the Settings > Accessibility item, but data is not sent outside the device. It can be removed at any time with the uv run artemis helper uninstall command, and if you prefer the UIAutomator2 backend, you can switch via an environment variable.
Current Android focus and future plans
ARTEMIS is distributed under the Apache License 2.0 and is currently focused on the Android platform. iOS support is included in the roadmap, and Android Studio integration and on-device lightweight VLM execution are also planned. Integration with AI IDEs requires an environment supporting the MCP protocol, and accessibility permissions must be granted to the device. When handling complex UIs or custom interfaces (Compose, Flutter, etc.), a multimodal targeting feature combining visual models and OCR operates. However, since Flash mode has limited pre-safety measures and final reporting capabilities, it is recommended to use Pro mode for tasks requiring important verification.