Room: Turchese

Abstracts

From Literature Reviews to Data Analysis – AI's Potential to Enhance Scientific Research

Abstract

Large Language Models (LLMs) have revolutionized the field of Artificial Intelligence, and are starting to have even more impact on the process of science itself. This talk will kick off the AI for Science Workshop and highlight two key areas which are exploding in interest and advances: LLMs as assistants for writing scientific literature reviews, and LLMs for data analysis. I'll discuss (1) the science of literature review creation and experiments from our LitLLM project, and (2) InsightBench, a project that focuses on the import issue of constructing benchmarks for testing the capabilities of data science agents, which seek to automate many of the tasks associated with data science. I'll conclude by looking into the future, and the ways in which LLMs may even further accelerate science.

Two Axes for Accelerating Science with AI

Abstract

There are two key areas where AI can augment the work of a scientist; domain-specific models such as AlphaFold or MatterGen target slow or expensive workflows, while general purpose tools such as chatbots and coding assistant such as ChatGPT, DeepResearch or Cursor can make the scientist generally more productive in many daily activities. In this talk we'll discuss how open source and open science can help build foundations for progress along these two axes while discussing the role and importance of well organized, open and effective collaborations.

Bridging the implementation gap: policy frameworks to support AI in science

Abstract

Despite the absence of explicit national policy frameworks for AI-in-science, a range of policy agendas implicitly shape this domain. Growing interest from both researchers and policymakers provides an opportunity to innovate in policy and practice. This talk will explore how policy frameworks must bridge top-down strategic direction with bottom-up researcher experimentation, emphasising the importance of anticipatory governance approaches that can evolve alongside rapid technological changes.

The Unscientific Science of AI Evaluation

Abstract

Current methods of assessing artificial intelligence are more complex and uncertain than they appear. This talk critically examines the limitations of existing evaluation frameworks, revealing how measurement challenges, inherent biases, and contextual nuances distort our understanding of AI capabilities. By exposing the gaps in our assessment methodologies, we challenge the notion of objective AI evaluation.

Automating data science, tabular AI, and causal challenges

Abstract

Practitioner surveys reveal that the number one challenge of data science is data preparation. This data preparation appears to fit complex data in simple models, such as linear models. Can flexible models, reusing progress in AI, solve the data science problem? We find that much data preparation is removed with flexible models, but users then worry about interpretability. Clarifying their demands reveals causal questions: model utility can be compromised by a shift between the data at hand and the intended use. Here the challenge is bridging domain expertise and statistical understanding. Can we reinvent statistical thinking for non parametric models? Can every problem be tackled by better tools?