Giant language mannequin (LLM)-based pc use brokers execute consumer instructions by interacting with obtainable UI components, however little is understood about how customers need to work together with these brokers or what design elements matter for his or her consumer expertise (UX). We performed a two-phase research to map the UX design house for pc use brokers. In Part 1, we reviewed present programs to develop a taxonomy of UX issues, then refined it via interviews with eight UX and AI practitioners. The ensuing taxonomy included classes reminiscent of consumer prompts, explainability, consumer management, and customers’ psychological fashions, with corresponding subcategories and instance design options. In Part 2, we ran a Wizard-of-Oz research with 20 individuals, the place a researcher acted as a web-based pc use agent and probed consumer reactions throughout regular, error-prone and dangerous execution. We used the findings to validate the taxonomy from Part 1 and deepen our perceive of the design house by figuring out the connections between design areas and divergence in consumer wants and situations. Our taxonomy and empirical insights present a map for builders to contemplate completely different facets of consumer expertise in pc use agent design and to situate their designs inside customers’ numerous wants and situations.
- †Carnegie Mellon College
