we present a novel simulation model of point-and-click behaviour that is applicable both when a target is stationary or moving. to enable more realistic simulation than existing models, the model proposed in this study takes into account key features of the user and the external environment, such as intermittent motor control, click decision-making, visual perception, upper limb kinematics and the effect of input device. the simulated user's point-and-click behaviour is formulated as a markov decision process (mdp), and the user's policy of action is optimised through deep reinforcement learning. as a result, our model successfully and accurately reproduced the trial completion time, distribution of click endpoints, and cursor trajectories of real users. through an ablation study, we showed how the simulation results change when the model's sub-modules are individually removed. the implemented model and dataset are publicly available.
|Title of host publication||CHI 2021 - Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems|
|Subtitle of host publication||Making Waves, Combining Strengths|
|Publisher||Association for Computing Machinery|
|Publication status||Published - 2021 May 6|
|Event||2021 CHI Conference on Human Factors in Computing Systems: Making Waves, Combining Strengths, CHI 2021 - Virtual, Online, Japan|
Duration: 2021 May 8 → 2021 May 13
|Name||Conference on Human Factors in Computing Systems - Proceedings|
|Conference||2021 CHI Conference on Human Factors in Computing Systems: Making Waves, Combining Strengths, CHI 2021|
|Period||21/5/8 → 21/5/13|
Bibliographical noteFunding Information:
This research was funded by National Research Foundation of Korea (2020R1A2C4002146) and Korea Creative Content Agency (R2019020010). We thank Julien Gori for discussions.
© 2021 acm.
All Science Journal Classification (ASJC) codes
- Human-Computer Interaction
- Computer Graphics and Computer-Aided Design