Tianyu Guo

PhD Student · UC Berkeley Statistics

I am a PhD student in Statistics at UC Berkeley, advised by Song Mei and Michael I. Jordan. My research is at the intersection of statistics and LLMs, including deep learning theory, mechanistic interpretability, LLM reasoning, and data science agents. I received my B.S. from Peking University, where I worked with Peng Ding.

Experience

Seed, ByteDance

Nov 2025 – Present

Research Intern

Worked on post-training for coding agents.
Flatiron Institute

Jun – Aug 2025

Research Intern

Worked on understanding agentic reasoning.

Publications

* denotes equal contribution.

9

GSM-Agent: Understanding Agentic Reasoning Using Controllable Environments

H. Zhu*, T. Guo*, S. Mei, S. Russell, N. Ghosh, A. Bietti, J. Jiao

arXiv 2509.21998, 2025 · [arXiv]
8

Generalization or Hallucination? Understanding Out-of-Context Reasoning in Transformers

Y. Huang*, H. Zhu*, T. Guo*, J. Jiao, S. Sojoudi, M. I. Jordan, S. Russell, S. Mei

NeurIPS 2025 · [arXiv]
7

IDA-Bench: Evaluating LLMs on Interactive Guided Data Analysis

H. Li*, H. Liu*, T. Zhu*, T. Guo*, Z. Zheng, X. Deng, M. I. Jordan

arXiv 2505.18223, 2025 · [arXiv]
6

How Do LLMs Perform Two-Hop Reasoning in Context?

T. Guo*, H. Zhu*, R. Zhang, J. Jiao, S. Mei, M. I. Jordan, S. Russell

arXiv 2502.13913, 2025 · [arXiv]
5

Active-Dormant Attention Heads: Mechanistically Demystifying Extreme-Token Phenomena in LLMs

T. Guo, D. Pai, Y. Bai, J. Jiao, M. I. Jordan, S. Mei

Oral, M3L Workshop @ NeurIPS 2024 · [arXiv]
4

Collaborative Heterogeneous Causal Inference Beyond Meta-Analysis

T. Guo, S. P. Karimireddy, M. I. Jordan

ICML 2024 · [arXiv]
3

How Do Transformers Learn In-Context Beyond Simple Functions? A Case Study on Learning with Representations

T. Guo, W. Hu, S. Mei, H. Wang, C. Xiong, S. Savarese, Y. Bai

ICLR 2024 · [arXiv]
2

What Can a Single Attention Layer Learn? A Study Through the Random Features Lens

H. Fu*, T. Guo*, Y. Bai, S. Mei

NeurIPS 2023 · [arXiv]
1

Posterior Predictive Propensity Scores and P-Values

P. Ding, T. Guo

Observational Studies, 9(1), 3–18, 2022 · [Paper]