I am a Ph.D. student at the College of Computing and Data Science, Nanyang Technological
University, advised by Prof. Bo An.
I received my B.S. in Computer Science from Xidian University.
I work on post-training language-model agents. Most of my recent work starts from the same
practical constraint: real agent logs are plentiful, but online rollouts, reward models,
and task verifiers usually are not. Concretely, I am interested in
agent post-training from raw, unlabelled interaction trajectories;
credit assignment at the token and step level for tool-use agents;
deliberative search and iterative revision as an alternative to one-shot generation;
taking these methods into industrial decision-making systems.
I am always happy to talk about potential collaborations — feel free to reach out by email.
News
AgentBrew is accepted to NeurIPS 2026, on learning tool-use agents from raw real-world trajectories.
LBM is accepted to WWW 2026, on hierarchical LLM agents for industrial auto-bidding.
ReLoc is accepted to NeurIPS 2025, in collaboration with Alibaba Tongyi Lab.
Joined Kuaishou Technology as a research intern, working on agent post-training.
CoSo is accepted to ICML 2025.
Q* is released on arXiv, with Skywork AI.
First paper accepted to NeurIPS 2023.
Publications
* denotes equal contribution. See also my Google Scholar.
NeurIPS 2026
AgentBrew: Offline Tool-Use Agent Learning from Raw Real-World Trajectories
Zhiyi Lyu, Yewen Li, Longtao Zheng, et al.
AgentBrew post-trains tool-use agents from raw interaction logs with no online rollouts,
external rewards, or task verifiers. Retrospective task inference plus PMI-based step-level
credit assignment turns noisy trajectories into weighted SFT supervision.
LBM: Hierarchical Large Auto-Bidding Model via Reasoning and Acting
Yewen Li*, Zhiyi Lyu*, Peng Jiang, Qingpeng Cai, Fei Pan, Bo An
Separating high-level reasoning from precise numerical action lets an LLM agent operate in
continuous decision spaces; the reasoning module is trained with offline RL, avoiding
simulator and real-world rollouts.
Let's Revise Step-by-Step: A Unified Local Search Framework for Code Generation with LLMs
Zhiyi Lyu, Jianguo Huang, Yanchen Deng, Steven Hoi, Bo An
Code generation as anytime local search over programs. A revision reward model ranks
candidates by revision distance, guiding both hill climbing and genetic search on
LiveCodeBench and TACO. In collaboration with Alibaba Tongyi Lab.
Towards Efficient Online Tuning of VLM Agents via Counterfactual Soft Reinforcement Learning
Lang Feng, Weihao Tan, Zhiyi Lyu, Longtao Zheng, Haiyang Xu, Ming Yan, Fei Huang, Bo An
CoSo measures each token's causal influence on the executable action and concentrates
exploration on action-critical tokens, with convergence and policy-improvement guarantees.
Q*: Improving Multi-step Reasoning for LLMs with Deliberative Planning
Chaojie Wang*, Yanchen Deng*, Zhiyi Lyu, Liang Zeng, Jujie He, Shuicheng Yan, Bo An
Q* casts multi-step reasoning as heuristic search and learns a plug-and-play Q-value model
to score the next reasoning step, guiding decoding without fine-tuning the LLM for the task.
Evaluated on GSM8K, MATH, and MBPP. With Skywork AI.
Few-shot Generation via Recalling Brain-Inspired Episodic-Semantic Memory
Zhibin Duan, Zhiyi Lv, Chaojie Wang, Bo Chen, Bo An, Mingyuan Zhou
A variational structured memory that stores semantic concepts alongside sample-specific
episodic memories, with structured recall for rapid adaptation to unseen generation tasks.
Agent post-training for tool use and decision making. Led the AgentBrew training pipeline
and the reasoning module of LBM, working with
Dr. Qingpeng Cai and
Dr. Yewen Li.
Education
Nanyang Technological University — Ph.D. in Computer Science
Singapore
College of Computing and Data Science. Advisor: Prof. Bo An.
Xidian University — B.S. in Computer Science and Technology
