I am a Ph.D. student in the Machine Learning Center at Georgia Tech (ML@GT), advised by Prof. Chao Zhang. I am also fortunate to work with Prof. Le Song. My research interest primarily lies in Machine Learning and Natural Language Processing.
Before that, I obtained my bachelor's degree from Zhejiang University, and I spent my senior year as a visiting student researcher at Harvard Medical School.
- [--- Top ---] Join the Journey: I am currently diving into exciting projects about Large Language Models (LLM), seeking collaborators/mentees with expertise in this field! Computational resources and preliminary validated ideas provided. Contact me if interested in collaboration!
- [Dec. 2023] I will join Microsoft Research as a research intern in Spring 2024, exploring Mixture of Experts for LLMs.
- [May 2023] One paper accepted to KDD'23, we proposed an iterative and adaptive framework for boosting in weakly-supervised learning settings.
- [May 2023] Our paper PTLoss is available on arXiv, we proposed a novel knowledge distillation objective by perturbing the standard KL loss.
- [May 2023] Two papers accepted to ACL'23, discussing cold-start data selection for few-shot LM finetuning and retrieval enhanced LM.
- [Mar. 2023] I will be back to Google Reseach NYC as a student researcher this summer.
- [May 2022] One paper accepted to KDD'22, discussing adaptive multi-view rule discovery.
- [Apr. 2022] One paper accepted to NAACL'22, discussing uncertainty-based active self-training.
- [Mar. 2022] I will join Google Reseach as a student researcher this summer, see you in New York City.
- [Feb. 2022] One paper accepted by ACL'22, discussing interactive weakly-supervised learning.
Education
My current research, revolving around large language models (LLM), is dedicated to more efficient and performant learning with limited supervision.
In particular, I explore novel learning paradigms, in which humans and machines interact more effectively.
I also investigate the learning objective design in order to improve the model trained with noisy/limited supervision.
I am also interested in how to strategically select data to maximize annotation efficiency.
My research thrusts are as follows:
- Large Language Model: Mechanism, Distillation, Improvement of Training Pipeline
- Learning Paradigm: Interactive and Iterative Methods, Weakly-Supervised Learning
- Learning Objective: Loss Design for Calibration/Deviation-Reduction
- Data Selection: Active Learning, Uncertainty Estimation and Propagation
Rongzhi Zhang, Jiaming Shen, Tianqi Liu, Jialu Liu, Michael Bendersky, Marc Najork, and Chao Zhang.
Do Not Blindly Imitate the Teacher: Using Perturbed Loss for Knowledge Distillation
Preprint on arXiv, 2023.
[BibTex] [arXiv]
Rongzhi Zhang, Yue Yu, Jiaming Shen, Xiquan Cui and Chao Zhang.
Local Boosting for Weakly-Supervised Learning
In ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD), 2023.
[BibTex] [arXiv]
Yue Yu, Rongzhi Zhang, Ran Xu, Jieyu Zhang, Jiaming Shen and Chao Zhang.
Cold-Start Data Selection for Few-shot Language Model Fine-tuning:
A Prompt-Based Uncertainty Propagation Approach
In Annual Meeting of the Association for Computational Linguistics (ACL), 2023.
[BibTex] [arXiv]
Yue Yu, Yuchen Zhuang, Rongzhi Zhang, Yu Meng, Jiaming Shen and Chao Zhang.
Zero-Shot Text Classification by Training Data Creation with Progressive Dense Retrieval
In Findings of Annual Meeting of the Association for Computational Linguistics (ACL), 2023.
[BibTex] [arXiv]
Rongzhi Zhang, Yue Yu, Shetty Pranav, Le Song and Chao Zhang.
PRBoost: Prompt-Based Rule Discovery and Boosting for Interactive Weakly-Supervised Learning
In Annual Meeting of the Association for Computational Linguistics (ACL), 2022.
[BibTex] [arXiv]
Rongzhi Zhang, Rebecca West, Xiquan Cui and Chao Zhang.
Adaptive Multi-view Rule Discovery for Weakly-Supervised Compatible Products Prediction
In ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD), 2022.
[BibTex] [arXiv]
Yue Yu, Lingkai Kong, Jieyu Zhang, Rongzhi Zhang and Chao Zhang.
AcTune: Uncertainty-Aware Active Self-Training for Active Fine-Tuning of Pretrained Language Models
In Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL), 2022.
[BibTex] [arXiv]
Rongzhi Zhang, Yue Yu and Chao Zhang.
SeqMix: Augmenting Active Sequence Labeling via Sequence Mixup
In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), 2020.
[BibTex] [Slides]
Experiences
- Research Intern | Jan. 2024 - May 2024
Microsoft Research, Redmond, WA
Host:
Shuohang Wang,
Yelong Shen,
Weizhu Chen
- Student Researcher | May 2023 - Dec. 2023
Google Research, New York City, NY
Host:
Jiaming Shen,
Tianqi Liu,
Jialu Liu
- Student Researcher | May 2022 - Dec. 2022
Google Research, New York City, NY
Host:
Jiaming Shen,
Tianqi Liu,
Michael Bendersky
Teaching
- 2023 KDD Student Travel Award, ACM SIGKDD
- 2019 Outstanding Graduate, Zhejiang University
- 2019 Outstanding Undergrad Thesis, Zhejiang University
- 2018 Zhejiang Provincial Government Scholarship (Top 3%)
- 2018 First-class Excellent Undergraduate Scholarship, Zhejiang University (Top 3%)
- 2018 First-class Academic Scholarship, Zhejiang University (Top 3%)
- 2016 Student Innovation Research Funding, Zhejiang Province
- 2014 The first prize in the 31st Chinese Physics Olympiad (CPhO)
- Program Committee / Reviewer: TKDE 2020, 2021; EMNLP 2022; ACL 2022; KDD 2023; NeurIPS 2023, ACL Rolling Review 2023, 2024, ICLR 2024.
- I was a player of Zhejiang University Varsity Men's basketball team, competing in CUBA Division II.
Ph.D. in Machine Learning
B.Eng. in Measurement Control Technology and Instruments
Visiting Student Researcher in Neural System Group