Tianyi Zhou

Tianyi Zhou

Assistant Professor of Computer Science

Biography

I am a tenure-track assistant professor of Computer Science, UMIACS, and AIM at University of Maryland, College Park. My research interests are in machine learning, optimization, and natural language processing. I am part of the Center for Machine Learning (CML) and CLIP Lab at UMIACS. I have published ~120 papers in ML (NeurIPS, ICML, ICLR), NLP (ACL, EMNLP, NAACL), CV (CVPR, ICCV, ECCV), DM (KDD, ICDM), AI (AAAI, IJCAI) conferences, and journals as Machine Learning (Springer), IEEE TPAMI/TIP/TNNLS/TKDE, etc.

Our recent works study (1) How, why, and when to transfer human learning (e.g., curriculum, retention, sub-tasking, curiosity, exemplar selection, collaboration, etc.) to improve machine learning and generalization in the wild (e.g., with unlabeled, biased, noisy, redundant, or distributed data, in unseen tasks/environments); (2) Controllable Generative AI in both training and inference/adaptation; (3) Synthetic data, self-evolving AI, and auto-benchmarking; and (4) Human-AI teaming and hybrid agent with personalization. We are developing these methods with LLMs, multi-modality foundation models, and RL, to address practical challenges in education, design, medical health, visualization, embodied intelligence, autonomous driving, etc. Our goal is to develop efficient, versatile, trustworthy, and environmentally-friendly hybrid-intelligence based on coevolution between human and machine. The code/data/models can be found at Tianyi Lab’s GitHub and HF.

I was a visiting research scientist at Google between 2021-2022, hosted by Prof. Boqing Gong and Prof. Ming-Hsuan Yang. Before that, I received my Ph.D. (thesis) from Computer Science of University of Washington, where I was a member of MELODI lab led by Prof. Jeff A. Bilmes. I have been working with Prof. Dacheng Tao as a research assistant at University of Technology, Sydney (UTS) and Nanyang Technological University. I was a research intern at Yahoo! Labs, mentored by Dr. Hua Ouyang (Apple) and Prof. Yi Chang (Jilin University), and a research intern at Microsoft Research, mentored by Dr. Lin Xiao (Meta AI).

News

  • 2025/04: We release WALL-E 2.0 (code) to improve the world model for LLM-based embodied agents. TL;DR: WALL-E 2.0 = Neuro-Symbolic World Model + MPC-based LLM Agent, where Neuro-Symbolic World Model = LLM + Complementary Symbolic knowledge (action rules, scene graph, knowledge graph) extracted from LLM errors in experiences.
  • 2025/04: We release ColorBench, the first comprehensive benchmark exposing the weaknesses of existing VLMs on color perception, reasoning, and robustness, across >5,800 images from 11 tasks based on real applications including shopping, medical test-kit reading, map-reading, agriculture, art, wildlife research, etc.
  • 2025/04: We release C3PO and R2-T2, which develop a novel class of test-time optimization approaches that can significantly boost the performance of existing MoE VLMs and MoE LLMs by 7-15% on challenging tasks.
  • 2025/04: We release two works investigating how traning data for LLMs (slow vs. fast thinking, instruction vs. reasoning) lead to different layer-wise gradient patterns and training dynamics, from which we unified vairous existing data quality metrics into one: small but diverse gradient directions indicate better data.
  • 2025/04: We release ATLaS and EEF that study better usage of expert trajectories in supervised/reinforcement finetuning of LLM agents. ATLaS selects only 30% critical steps from expert trajectories to achieve better generalization across tasks/envs, while EEF show that expert failed trajectories can considerably improve the agent exploration in RL.
  • 2025/04: I am going to serve as an Area Chair of NeurIPS 2025.
  • 2025/03: Florence-VL (3B/8B pretrained/SFT VLMs) trained with depth-breadth fusion has been accepted by CVPR 2025 and the complete training recipe has been open-sourced.
  • 2025/02: Check the very first “Aha moment” of multimodal reasoning by RL acheived by VisualThinker-R1-Zero from our TurningPoint-AI team!
  • 2025/01: 7 ICLR + 3 NAACL accepted, featuring our latest works on synthetic data for post-training, MoE (ICLR Oral), many-objective optimization, in-context transferality, multi-modality imbalance & alignment, oversensitiveness & controllability of GenAI.
  • 2024/12: I am going to serve as an Area Chair for ARR Dec 2024 & Feb 2025 (ACL 2025).
  • 2024/11: I am going to serve as an Area Chair (SPC) for IJCAI 2025.
  • 2024/09: Five papers (3 main + 2 findings) have been accepted by EMNLP 2024.
  • 2024/09: I am going to serve as an Area Chair of ICLR 2025.
  • 2024/07: We initialize TurningPoint AI, a research team from multiple universities and industry (UMD+UCLA+PSU+Google) investigating Muiltimodal Agents, with the goals of building Trustworthy Embodied AI, Self-Evolving Machines, Compositional Agents, and Controllable AIGC. We already released 8 projects with several ICML and ECCV publications and new datasets.
  • 2024/07: 2 papers of diffusion models (analysis of negative prompts, extracting discriminative features from generative models) have been accepted by ECCV 2024.
  • 2024/05: 4 ICLR + 4 ICML + 6 ACL + 2 NAACL + 2 CVPR have been accepted, featuring our works on controllable AIGC, personalized AI, data-efficient training of LLMs, RLHF, prompt optimization, multi-modal hallucinations, multi-modal and embodied agent, and curriculum reinforcement learning.
  • 2024/02: We release a survey on knowledge distillation of LLMs with GitHub repo.

Research Topics