{"id":71,"date":"2025-08-26T20:58:16","date_gmt":"2025-08-26T20:58:16","guid":{"rendered":"https:\/\/biconnect.psicologia.unimib.it\/?page_id=71"},"modified":"2025-09-06T21:18:03","modified_gmt":"2025-09-06T21:18:03","slug":"llm-for-social-robotics","status":"publish","type":"page","link":"https:\/\/biconnect.psicologia.unimib.it\/index.php\/llm-for-social-robotics\/","title":{"rendered":"LLM for Social Robotics"},"content":{"rendered":"\n<p class=\"has-medium-font-size\"><strong>Overview<\/strong><\/p>\n\n\n\n<p>Robots are increasingly expected to collaborate with people in dynamic, ambiguous environments\u2014from assisting with household tasks to supporting decision-making in workplaces. For such collaboration, it is not enough for robots to execute commands: they must understand <strong>perspectives, intentions, and shared goals<\/strong>. In this line of research, we investigate how <strong>Large Language Models<\/strong> (LLMs), combined with active perception and structured reasoning, can support perspective-taking, causal understanding, and adaptive dialogue in embodied settings.<\/p>\n\n\n\n<p class=\"has-medium-font-size\"><strong>What we study<\/strong><\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Perspective-taking in collaborative tasks<\/strong> We study how LLMs can model <em>what another agent sees, knows, or intends<\/em>. Using extended versions of the classic <em>Director Task<\/em>, we evaluate whether LLM-based agents can resolve ambiguity in partially observable environments by integrating <strong>reasoning <\/strong>(ReAct-style thought sequences) with <strong>active exploration<\/strong>. This allows us to test not only linguistic abilities but also embodied coordination skills, bridging toward theory of mind.<\/li>\n\n\n\n<li><strong>Developmental insights for AI<\/strong> Inspired by Selman\u2019s developmental stages of perspective-taking, we simulate child-like reasoning in LLMs, from egocentric viewpoints to more advanced reciprocal and societal perspectives. This developmental lens allows us to probe how perspective-taking emerges in LLMs and to benchmark their collaborative behaviour against human developmental trajectories.<\/li>\n\n\n\n<li><strong>Structured reasoning and planning<\/strong> We integrate symbolic planners with LLMs to generate structured examples of \u201cthought\u2013action\u201d sequences. These help LLMs learn not only <em>what to do<\/em> but <em>why<\/em>, grounding decisions in perspective-aware reasoning. This hybrid approach addresses current limitations in disambiguating hidden or occluded states and supports more consistent collaborative strategies.<\/li>\n\n\n\n<li><strong>Social continual learning<\/strong> Beyond isolated tasks, we develop frameworks where agents continually refine their <strong>causal knowledge<\/strong> through interaction. In the SCOOP project, we explore how LLMs can ask meaningful \u201cwhy\u201d and \u201cwhat if\u201d questions to an oracle or a human partner, building reusable causal models across tasks. This supports robust decision-making in dynamic, partially observable environments.<\/li>\n<\/ul>\n\n\n\n<p class=\"has-medium-font-size\"><strong>Why it matters<\/strong><\/p>\n\n\n\n<p>Embedding perspective-taking and causal reasoning into LLM-powered robots has two transformative outcomes:<\/p>\n\n\n\n<ol start=\"1\" class=\"wp-block-list\">\n<li><strong>For AI and robotics<\/strong>: It leads to socially-aware, adaptive robots that can collaborate effectively, understand ambiguity, and generalise across tasks.<\/li>\n\n\n\n<li><strong>For cognitive science<\/strong>: It provides new computational models of perspective-taking and collaborative learning, grounded in developmental psychology and tested in embodied interaction.<\/li>\n<\/ol>\n\n\n\n<p class=\"has-medium-font-size\"><strong>Methods at a glance<\/strong><\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Framework<\/strong>  ReAct (Reason Act) and multimodal perception;<\/li>\n\n\n\n<li><strong>Tasks<\/strong> Modified Director Tasks, ecological household scenarios, collaborative planning;<\/li>\n\n\n\n<li><strong>Architectures<\/strong>  Hybrid LLM and planner systems, LLMs with causal graph modules, oracle-aided continual learning;<\/li>\n\n\n\n<li><strong>Evaluation <\/strong>Task success, efficiency, clarification strategies, developmental stage alignment, robustness in dynamic environments.<\/li>\n<\/ul>\n\n\n\n<p class=\"has-medium-font-size\"><strong>Project Members<\/strong><\/p>\n\n\n\n<p>Dr. Luca Annese<br>luca.annese1@unimib.it<\/p>\n\n\n\n<p>Dr. Sabrina Patania, PhD<br>sabrina.patania@unimib.it<\/p>\n\n\n\n<p>Dr. Anna Lambiase<br>a.lambiase3@campus.unimib.it<\/p>\n\n\n\n<p>Dr. Anita Pellegrini<br>a.pellegrini19@campus.unimib.it<\/p>\n\n\n\n<p>Prof. Silvia Serino<br>silvia.serino@unimib.it<\/p>\n\n\n\n<p>Prof. Tom Foulsham<br>foulsham@essex.ac.uk<\/p>\n\n\n\n<p>Prof. Dimitri Ognibene<br>dimitri.ognibene@unimib.it<\/p>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h2 class=\"wp-block-heading has-medium-font-size\"><strong>References<\/strong><\/h2>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Annese, L., Patania, S., Serino, S., Foulsham, T., Rossi, S., Ruggeri, A., &amp; Ognibene, D. (2025). <em>Who Sees What? Structured Thought-Action Sequences for Epistemic Reasoning in LLMs<\/em>. Proceedings of ICSR 2025 .<\/li>\n\n\n\n<li>Patania, S., Annese, L., Lambiase, A., Pellegrini, A., Foulsham, T., Ruggeri, A., Rossi, S., Serino, S., &amp; Ognibene, D. (2025). <em>Growing Perspectives: Modelling Embodied Perspective Taking and Inner Narrative Development Using LLMs<\/em>. Proceedings of ICDL 2025 .<\/li>\n\n\n\n<li>Patania, S., Annese, L., Pellegrini, A., Serino, S., Lambiase, A., Pallonetto, L., Rossi, S., Colombani, S., Foulsham, T., Ruggeri, A., &amp; Ognibene, D. (2025). <em>PerspAct: Enhancing LLM Situated Collaboration Skills through Perspective Taking and Active Vision<\/em>. Proceedings of IAS-19 .<\/li>\n\n\n\n<li>Ognibene, D., Patania, S., Annese, L., Koyuturk, C., Ruggeri, A., Colombani, S., Garzotto, F., Vizzari, G. (2025). <em>SCOOP: A Framework for Proactive Collaboration and Social Continual Learning through Natural Language Interaction and Causal Reasoning<\/em>. Poster at RLDM 2025 .<\/li>\n<\/ul>\n\n\n\n<p><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Overview Robots are increasingly expected to collaborate with people in dynamic, ambiguous environments\u2014from assisting with household tasks to supporting decision-making in workplaces. For such collaboration, it is not enough for robots to execute commands: they must understand perspectives, intentions, and shared goals. In this line of research, we investigate how Large Language Models (LLMs), combined &#8230; <a title=\"LLM for Social Robotics\" class=\"read-more\" href=\"https:\/\/biconnect.psicologia.unimib.it\/index.php\/llm-for-social-robotics\/\" aria-label=\"Read more about LLM for Social Robotics\">Read more<\/a><\/p>\n","protected":false},"author":3,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"_acf_changed":false,"footnotes":""},"class_list":["post-71","page","type-page","status-publish"],"acf":[],"_links":{"self":[{"href":"https:\/\/biconnect.psicologia.unimib.it\/index.php\/wp-json\/wp\/v2\/pages\/71","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/biconnect.psicologia.unimib.it\/index.php\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/biconnect.psicologia.unimib.it\/index.php\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/biconnect.psicologia.unimib.it\/index.php\/wp-json\/wp\/v2\/users\/3"}],"replies":[{"embeddable":true,"href":"https:\/\/biconnect.psicologia.unimib.it\/index.php\/wp-json\/wp\/v2\/comments?post=71"}],"version-history":[{"count":4,"href":"https:\/\/biconnect.psicologia.unimib.it\/index.php\/wp-json\/wp\/v2\/pages\/71\/revisions"}],"predecessor-version":[{"id":790,"href":"https:\/\/biconnect.psicologia.unimib.it\/index.php\/wp-json\/wp\/v2\/pages\/71\/revisions\/790"}],"wp:attachment":[{"href":"https:\/\/biconnect.psicologia.unimib.it\/index.php\/wp-json\/wp\/v2\/media?parent=71"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}