A robot agent designed to engage in real-world human-robot joint action must be able to understand the social states of the human users it interacts with in order to behave appropriately. In particular, in a dynamic public space, a crucial task for the robot is to determine the needs and intentions of all of the people in the scene, so that it only interacts with people who intend to interact with it.ududWe address the task of estimating the engagement state of customers for a robot bartender based on the data from audiovisual sensors. We begin with an offline experiment using Hidden Markov Models, confirming that the sensor data contains the information necessary to estimate user state. We then present two strategies for online state estimation: a rule-based classifier based on observed human behaviour in real bars, and a set of supervised classifiers trained on a labelled corpus. These strategies are compared in offline cross-validation, in an online user study, and through validation against a separate test corpus. These studies show that while the trained classifiers are best in a cross-validation setting, the rule-based classifier performs best with novel data; however, all classifiers also change their estimate too frequently for practical use. To address this issue, we present a final classifier based on Conditional Random Fields: this model has comparable performance on the test data, with increased stability. In summary, though, the rule-based classifier shows competitive performance with the trained classifiers, suggesting that for this task, such a simple model could actually be a preferred option, providing useful online performance while avoiding the implementation and data-scarcity issues involved in using machine learning forudthis task.
展开▼
机译：设计为参与现实世界中人机交互行为的机器人代理必须能够理解与之交互的人类用户的社会状态，以使其行为正常。特别是在动态的公共空间中，机器人的一项关键任务是确定场景中所有人的需求和意图，从而使其仅与有意与之交互的人交互。 ud ud根据视听传感器的数据估算机器人调酒师的顾客参与状态的任务。我们从使用隐马尔可夫模型进行的离线实验开始，确认传感器数据包含估计用户状态所需的信息。然后，我们提出两种用于在线状态估计的策略：基于规则的分类器，该分类器基于在实际酒吧中观察到的人类行为，以及一组在标记的语料库上训练的监督分类器。在离线交叉验证，在线用户研究中以及通过针对单独的测试语料库的验证来比较这些策略。这些研究表明，虽然训练有素的分类器在交叉验证设置中最佳，但基于规则的分类器在处理新数据时效果最佳；但是，所有分类器也过于频繁地更改其估计值以供实际使用。为了解决这个问题，我们提出了一个基于条件随机场的最终分类器：该模型在测试数据上具有可比的性能，并且稳定性更高。总之，尽管如此，基于规则的分类器显示出与训练有素的分类器竞争的性能，这表明对于此任务，这样一个简单的模型实际上可能是首选，提供有用的在线性能，同时避免了实施中涉及的实现和数据稀缺性问题使用机器学习执行此任务。
展开▼
