Abstract: In this talk, Dr. Marti Hearst will share observations about the role of language in information visualization. I will pose questions such as: how do we decide what to express via language vs via visualization? How do we choose what kind of text to use when creating visualizations, and does that choice matter? Does anyone prefer text over visuals, under what circumstances, and why?

Abstract: The rapid progress made over the last few years in generating linguistically coherent natural language has blurred, in the minds of many, the difference between natural language generation, understanding, and the ability to reason with respect to the world. Nevertheless, robust support of high-level decisions that depend on natural language understanding, and that require dealing with “truthfulness” are still beyond our capabilities, partly because most of these tasks are very sparse, often require grounding, and may depend on new types of supervision signals.  

Dan will discuss some of the challenges underlying reasoning and argue that we should focus on LLMs as orchestrators – coordinating and managing multiple models, applications, and services, to execute complex tasks and processes. I will discuss some of the challenges and present some of our work in this space, focusing on supporting task decomposition and planning.

Abstract: Today, social skills are essential to success both on the job and in life.  However, practice environments for social skills are typically out of reach for most people. How can we make social skill training more available, accessible, and inviting? In this talk, I share two of our recent works on social skill training using large language models.  The first one explores how to empower therapists in learning therapy skills  with LLM-empowered feedback and the second one looks at training people with conflict resolution skills via simulated practices. Last but not least, we discuss risks, concerns and mitigation strategies related to LLMs-based simulation for social skill training. As a first step towards democratizing skill training, these works demonstrate how LLMs can empower individuals and foster  positive change.

Abstract: In this talk I survey some aspects of the use of model-assisted human annotation at Apple. In the first half of the talk, I discuss an implementation used by the Siri Natural Language Understanding team to support annotation of Siri traffic in over 40 locales, showing an average saving of almost 35% in time to task completion while maintaining a high level of accuracy. In the second half of the talk, I discuss a project which incorporates a Large Language Model into the annotation pipeline. Here, the model assisted with data sampling from a large corpus, supporting the creation of a benchmark of controversial topics. Although the model being used to support the annotation process has changed over time, some common threads emerge. The use of models to augment human annotation is cost-saving, but we must proceed with caution: Rely on the model in cases of higher confidence, but verify at least some subset of the data. Although LLM-generated content appears impressive, some results may be misleading.