Rongzhi Zhang

Ph.D. Student
Machine Learning Center
School of Computational Science and Engineering
Georgia Institute of Technology

Office: CODA E1317
Address: 756 W Peachtree St NW, Atlanta, GA 30308
Email: rongzhi.zhang@gatech.edu
External Links:
twitter google scholar github linkedin

Biography

I am a Ph.D. student in the Machine Learning Center at Georgia Tech (ML@GT), advised by Prof. Chao Zhang. I am also fortunate to work with Prof. Le Song. My research interest primarily lies in Machine Learning and Natural Language Processing.

Before that, I obtained my bachelor's degree from Zhejiang University, and I spent my senior year as a visiting student researcher at Harvard Medical School.

News

Education

Research

My current research, revolving around large language models (LLM), is dedicated to more efficient and performant learning with limited supervision. In particular, I explore novel learning paradigms, in which humans and machines interact more effectively. I also investigate the learning objective design in order to improve the model trained with noisy/limited supervision. I am also interested in how to strategically select data to maximize annotation efficiency. My research thrusts are as follows:

Preprints

  • Lingkai Kong, Haorui Wang, Wenhao Mu, Yuanqi Du, Yuchen Zhuang, Yifei Zhou, Yue Song, Rongzhi Zhang, Kai Wang and Chao Zhang.
    Aligning Large Language Models with Representation Editing: A Control Perspective
    In ICML Workshop on Models of Human Feedback for AI Alignment, 2024.

  • Rongzhi Zhang, Kuan Wang, Liyuan Liu, Shuohang Wang, Hao Cheng, Chao Zhang and Yelong Shen.
    Low-rank Attention Approximation for LLMs KV Cache Compression

  • Rongzhi Zhang, Yuzhao Heng, Yixiao Li, Alexander Bukharin, Tuo Zhao, Chao Zhang.
    Strength-Controlled Preference Data Synthesis for Complimentary Rewarding

    • Publications

  • Haorui Wang, Rongzhi Zhang, Yinghao Li, Lingkai Kong, Yuchen Zhuang, Xiusi Chen and Chao Zhang.
    TPD: Enhancing Student Language Model Reasoning via Principle Discovery and Guidance
    In the 1st Conference on Language Modeling (COLM), 2024.

  • Rongzhi Zhang, Jiaming Shen, Tianqi Liu, Jialu Liu, Michael Bendersky, Marc Najork and Chao Zhang.
    Knowledge Distillation with Perturbed Loss: From a Vanilla Teacher to a Proxy Teacher
    In ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD), 2024.

  • Rongzhi Zhang, Jiaming Shen, Tianqi Liu, Haorui Wang, Zhen Qin, Feng Han, Jialu Liu, Simon Baumgartner, Michael Bendersky and Chao Zhang.
    PLaD: Preference-based Large Language Model Distillation with Pseudo-Preference Pairs
    In Findings of Annual Meeting of the Association for Computational Linguistics (ACL), 2024.

  • Yuzhao Heng, Chunyuan Deng, Yitong Li, Yue Yu, Yinghao Li, Rongzhi Zhang, Chao Zhang
    ProgGen: Generating Named Entity Recognition Datasets Step-by-step with Self-Reflexive Large Language Models
    In Findings of Annual Meeting of the Association for Computational Linguistics (ACL), 2024.

  • Rongzhi Zhang, Yue Yu, Jiaming Shen, Xiquan Cui and Chao Zhang.
    Local Boosting for Weakly-Supervised Learning
    In ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD), 2023.

  • Yue Yu, Rongzhi Zhang, Ran Xu, Jieyu Zhang, Jiaming Shen and Chao Zhang.
    Cold-Start Data Selection for Few-shot Language Model Fine-tuning: A Prompt-Based Uncertainty Propagation Approach
    In Annual Meeting of the Association for Computational Linguistics (ACL), 2023.

  • Yue Yu, Yuchen Zhuang, Rongzhi Zhang, Yu Meng, Jiaming Shen and Chao Zhang.
    Zero-Shot Text Classification by Training Data Creation with Progressive Dense Retrieval
    In Findings of Annual Meeting of the Association for Computational Linguistics (ACL), 2023.

  • Rongzhi Zhang, Yue Yu, Shetty Pranav, Le Song and Chao Zhang.
    PRBoost: Prompt-Based Rule Discovery and Boosting for Interactive Weakly-Supervised Learning
    In Annual Meeting of the Association for Computational Linguistics (ACL), 2022.

  • Rongzhi Zhang, Rebecca West, Xiquan Cui and Chao Zhang.
    Adaptive Multi-view Rule Discovery for Weakly-Supervised Compatible Products Prediction
    In ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD), 2022.

  • Yue Yu, Lingkai Kong, Jieyu Zhang, Rongzhi Zhang and Chao Zhang.
    AcTune: Uncertainty-Aware Active Self-Training for Active Fine-Tuning of Pretrained Language Models
    In Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL), 2022.

  • Rongzhi Zhang, Yue Yu and Chao Zhang.
    SeqMix: Augmenting Active Sequence Labeling via Sequence Mixup
    In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), 2020.

    • Experiences

    Teaching

    Selected Awards

    Academic Service

    Misc