Rongzhi Zhang

Ph.D. Student
Machine Learning Center
School of Computational Science and Engineering
Georgia Institute of Technology

Office: CODA E1317
Address: 756 W Peachtree St NW, Atlanta, GA 30308
Email: rongzhi.zhang@gatech.edu
External Links:
twitter google scholar github linkedin

Biography

I am a final-year Ph.D. student in the Machine Learning Center at Georgia Tech (ML@GT), advised by Prof. Chao Zhang. I am also fortunate to work with Prof. Le Song. My research interest primarily lies in model efficiency and data efficiency of language models. Beyond academia, I've spent several fantastic research internships at Google Research, Microsoft Azure AI, and Amazon Store Foundational AI.

Before that, I obtained my bachelor's degree from Zhejiang University, and I spent my senior year as a visiting student researcher at Harvard Medical School.

News

Education

Research

My research focuses on advancing language models in three key aspects:

Preprints

  • Rongzhi Zhang, Chenwei Zhang, Xinyang Zhang, Liang Qiu, Yuchen Zhuang, Qingru Zhang, Hyokun Yun, Tuo Zhao, Chao Zhang.
    Quality-Aware Preference Data Weighting for Generalizable Reward Model

  • Rongzhi Zhang, Yuzhao Heng, Yixiao Li, Alexander Bukharin, Tuo Zhao, Chao Zhang.
    Strength-Controlled Preference Data Synthesis for Complimentary Rewarding

    • Publications

  • Rongzhi Zhang, Kuan Wang, Liyuan Liu, Shuohang Wang, Hao Cheng, Chao Zhang and Yelong Shen.
    LoRC: Low-Rank Compression for LLMs KV Cache with a Progressive Compression Strategy
    In Machine Learning and Compression Workshop of Annual Conference on Neural Information Processing Systems (NeurIPS), 2024.

  • Lingkai Kong, Haorui Wang, Wenhao Mu, Yuanqi Du, Yuchen Zhuang, Yifei Zhou, Yue Song, Rongzhi Zhang, Kai Wang and Chao Zhang.
    Aligning Large Language Models with Representation Editing: A Control Perspective
    In Annual Conference on Neural Information Processing Systems (NeurIPS), 2024.

  • Haorui Wang, Rongzhi Zhang, Yinghao Li, Lingkai Kong, Yuchen Zhuang, Xiusi Chen and Chao Zhang.
    TPD: Enhancing Student Language Model Reasoning via Principle Discovery and Guidance
    In the 1st Conference on Language Modeling (COLM), 2024.

  • Rongzhi Zhang, Jiaming Shen, Tianqi Liu, Jialu Liu, Michael Bendersky, Marc Najork and Chao Zhang.
    Knowledge Distillation with Perturbed Loss: From a Vanilla Teacher to a Proxy Teacher
    In ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD), 2024.

  • Rongzhi Zhang, Jiaming Shen, Tianqi Liu, Haorui Wang, Zhen Qin, Feng Han, Jialu Liu, Simon Baumgartner, Michael Bendersky and Chao Zhang.
    PLaD: Preference-based Large Language Model Distillation with Pseudo-Preference Pairs
    In Findings of Annual Meeting of the Association for Computational Linguistics (ACL), 2024.

  • Yuzhao Heng, Chunyuan Deng, Yitong Li, Yue Yu, Yinghao Li, Rongzhi Zhang, Chao Zhang
    ProgGen: Generating Named Entity Recognition Datasets Step-by-step with Self-Reflexive Large Language Models
    In Findings of Annual Meeting of the Association for Computational Linguistics (ACL), 2024.

  • Rongzhi Zhang, Yue Yu, Jiaming Shen, Xiquan Cui and Chao Zhang.
    Local Boosting for Weakly-Supervised Learning
    In ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD), 2023.

  • Yue Yu, Rongzhi Zhang, Ran Xu, Jieyu Zhang, Jiaming Shen and Chao Zhang.
    Cold-Start Data Selection for Few-shot Language Model Fine-tuning: A Prompt-Based Uncertainty Propagation Approach
    In Annual Meeting of the Association for Computational Linguistics (ACL), 2023.

  • Yue Yu, Yuchen Zhuang, Rongzhi Zhang, Yu Meng, Jiaming Shen and Chao Zhang.
    Zero-Shot Text Classification by Training Data Creation with Progressive Dense Retrieval
    In Findings of Annual Meeting of the Association for Computational Linguistics (ACL), 2023.

  • Rongzhi Zhang, Yue Yu, Shetty Pranav, Le Song and Chao Zhang.
    PRBoost: Prompt-Based Rule Discovery and Boosting for Interactive Weakly-Supervised Learning
    In Annual Meeting of the Association for Computational Linguistics (ACL), 2022.

  • Rongzhi Zhang, Rebecca West, Xiquan Cui and Chao Zhang.
    Adaptive Multi-view Rule Discovery for Weakly-Supervised Compatible Products Prediction
    In ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD), 2022.

  • Yue Yu, Lingkai Kong, Jieyu Zhang, Rongzhi Zhang and Chao Zhang.
    AcTune: Uncertainty-Aware Active Self-Training for Active Fine-Tuning of Pretrained Language Models
    In Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL), 2022.

  • Rongzhi Zhang, Yue Yu and Chao Zhang.
    SeqMix: Augmenting Active Sequence Labeling via Sequence Mixup
    In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), 2020.

    • Experiences

    Teaching

    Selected Awards

    Academic Service

    Misc