Plover is a plan-centric vision-based GUI automation system that externalizes task plans and replanning as inspectable artifacts, enabling explicit supervision and localized correction.

This paper presents a software ecosystem enabling consistent development environments for running workflows across diverse hardware platforms.

The paper proposes a novel, customizable 'Pick and Sort' graphical authentication scheme where users select and arrange visual elements in a grid, demonstrating its feasibility for non-time-critical a…

This paper proposes an approach to automate the development of app-specific voice assistants using GUI test code and large language models.

The paper introduces Semantic-level UI Element Injection, a novel red-teaming technique that overlays misleading UI elements onto screenshots to significantly improve the attack success rate against s…

The paper introduces GUITestScape, a comprehensive benchmark for exploratory GUI testing, and GUIJudge, an open-set evaluator that significantly improves the assessment of AI agents' defect detection…

This paper introduces Desktop-Delta Bench (DDB), an offline step-level benchmark for evaluating computer-use agents' ability to reconstruct causal transitions in desktop GUI environments.

MaskClaw is an edge-side privacy arbitrator that protects sensitive data in GUI agent screenshots by combining local visual evidence, task-specific policies, and a skill-evolution mechanism.

UI-KOBE is a framework that enhances lightweight mobile GUI agents by integrating reusable, app-specific knowledge graphs, allowing them to perform complex tasks efficiently on-device without relying…

The paper introduces MIRAGE, a novel pipeline that generates context-aware prompt injection attacks by embedding malicious text into user-generated content regions of mobile screenshots, successfully…

The paper introduces MIRAGE, a novel pipeline that generates context-aware prompt injection attacks by injecting malicious text into user-generated content regions of mobile screenshots, successfully…

The paper introduces the Task Alignment Benchmark (TAB) to evaluate terminal agents' ability to selectively follow relevant environmental instructions while ignoring misleading distractors, revealing…

The paper proposes using GUI agents, both as objective evaluators and subjective playtesters, to significantly improve the generation of playable games from prompts, demonstrating a 66.8% rubric pass-…

The paper introduces a provenance-aware vulnerability analysis approach that accurately identifies cross-ecosystem vulnerabilities in Python applications by resolving vendored native libraries to spec…

This study compared LLM-based conversational interfaces and traditional dashboards for industrial decision tasks, finding that while conversational agents reduce interactional effort, dashboards remai…

The paper formalizes TOCTOU vulnerabilities in GUI agents due to observation-to-action delays and proposes a layered defense, Pre-execution UI State Verification (PUSV), achieving high interception ra…

The XAIxArts workshop explores the operationalisation of Explainable AI in the Arts, focusing on diversity, ideation, and resource development.

This paper presents a sustainable, low-cost Desktop Virtualization Infrastructure using discarded electronic equipment for digital education.