20 results for “GUI environments”
CS papers onlyHybrid search: Keyword + semantic, ranked by combined score.ⓘ
Want pure semantic search? Try claim verification →
Plover is a plan-centric vision-based GUI automation system that externalizes task plans and replanning as inspectable artifacts, enabling explicit supervision and localized correction.
Jaewoo Lee, Zaid Khan, Archiki Prasad, Justin Chih-Yao Chen +6 more
The authors introduce HiViG, a test-time framework for improving the performance of Computer Use Agents in GUI environments.
Shengcheng Yu, Yuchen Ling, Junyang Xing, Quan Zhou +2 more
This paper reviews 336 GUI-agent papers from 2018 to 2026, examining the research landscape, architectures, evaluation, software lifecycle concerns, and future opportunities.
This paper presents a software ecosystem enabling consistent development environments for running workflows across diverse hardware platforms.
The paper proposes a novel, customizable 'Pick and Sort' graphical authentication scheme where users select and arrange visual elements in a grid, demonstrating its feasibility for non-time-critical a…
Haoqing Wang, Xiaoyi Zhang, Sicheng Fan, Zhexin Zhang +4 more
The paper introduces Hybrid Computer Use (HCU), a unified interface for general agents that combines programmatic, semantic, and visual interaction channels.
This paper proposes an approach to automate the development of app-specific voice assistants using GUI test code and large language models.
Wenkui Yang, Chao Jin, Haisu Zhu, Weilin Luo +6 more
The paper introduces Semantic-level UI Element Injection, a novel red-teaming technique that overlays misleading UI elements onto screenshots to significantly improve the attack success rate against s…
Xiaoyi Chen, Yifei Gao, Yang Xu, Xingxing Song +2 more
The paper introduces GUITestScape, a comprehensive benchmark for exploratory GUI testing, and GUIJudge, an open-set evaluator that significantly improves the assessment of AI agents' defect detection…
This paper introduces Desktop-Delta Bench (DDB), an offline step-level benchmark for evaluating computer-use agents' ability to reconstruct causal transitions in desktop GUI environments.
This paper compares the trade-offs of native and cross-platform mobile development frameworks.
Yanqiu Zhao, Dongying Zheng, Kaibo Huang, Yukun Wei +2 more
MaskClaw is an edge-side privacy arbitrator that protects sensitive data in GUI agent screenshots by combining local visual evidence, task-specific policies, and a skill-evolution mechanism.
The paper presents an empirical experiment to study the extent of undefined behavior in C and C++ under typical desktop use of a Linux distribution.
Yuxiang Chai, Han Xiao, Xinyu Fu, Jinpeng Chen +2 more
UI-KOBE is a framework that enhances lightweight mobile GUI agents by integrating reusable, app-specific knowledge graphs, allowing them to perform complex tasks efficiently on-device without relying…
This paper investigates the distribution of licenses in software projects and their impact on project activity.
Sina Mavali, David Pape, Jonathan Evertz, Samira Abedini +4 more
The paper introduces the Task Alignment Benchmark (TAB) to evaluate terminal agents' ability to selectively follow relevant environmental instructions while ignoring misleading distractors, revealing…
A plant-aware conversational copilot is presented that extends the paradigm of retrieval-augmented large language models for PLC coding.