Rongzhi Zhang

Ph.D. Candidate
Machine Learning Center
School of Computational Science and Engineering
Georgia Institute of Technology

Office: CODA E1317
Address: 756 W Peachtree St NW, Atlanta, GA 30308
Email: rongzhi.zhang@gatech.edu
External Links:
twitter google scholar github linkedin

Biography

I am a Machine Learning Ph.D. candidate at Georgia Tech (ML@GT), advised by Prof. Chao Zhang. I am also fortunate to work with Prof. Le Song. My research interest primarily lies in model efficiency and data efficiency of language models. Beyond academia, I've spent several fantastic research internships at Google Research, Microsoft Azure AI, and Amazon Stores Foundational AI.

Before that, I obtained my bachelor's degree from Zhejiang University, and I spent my senior year as a visiting student researcher at Harvard Medical School.

News

Education

Research

My research focuses on advancing language models in three key aspects:

Preprints

  • Quality-Aware Preference Data Weighting for Generalizable Reward Model
    Rongzhi Zhang, Chenwei Zhang, Xinyang Zhang, Liang Qiu, Yuchen Zhuang, Qingru Zhang, Hyokun Yun, Tuo Zhao, Chao Zhang.
    An arXiv version will be available soon.

    • Publications

  • LoRC: Low-Rank Compression for LLMs KV Cache with a Progressive Compression Strategy
    Rongzhi Zhang, Kuan Wang, Liyuan Liu, Shuohang Wang, Hao Cheng, Chao Zhang and Yelong Shen.
    In Machine Learning and Compression Workshop of Annual Conference on Neural Information Processing Systems (NeurIPS), 2024.

  • Aligning Large Language Models with Representation Editing: A Control Perspective
    Lingkai Kong, Haorui Wang, Wenhao Mu, Yuanqi Du, Yuchen Zhuang, Yifei Zhou, Yue Song, Rongzhi Zhang, Kai Wang and Chao Zhang.
    In Annual Conference on Neural Information Processing Systems (NeurIPS), 2024.

  • TPD: Enhancing Student Language Model Reasoning via Principle Discovery and Guidance
    Haorui Wang, Rongzhi Zhang, Yinghao Li, Lingkai Kong, Yuchen Zhuang, Xiusi Chen and Chao Zhang.
    In the 1st Conference on Language Modeling (COLM), 2024.

  • Knowledge Distillation with Perturbed Loss: From a Vanilla Teacher to a Proxy Teacher
    Rongzhi Zhang, Jiaming Shen, Tianqi Liu, Jialu Liu, Michael Bendersky, Marc Najork and Chao Zhang.
    In ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD), 2024.

  • PLaD: Preference-based Large Language Model Distillation with Pseudo-Preference Pairs
    Rongzhi Zhang, Jiaming Shen, Tianqi Liu, Haorui Wang, Zhen Qin, Feng Han, Jialu Liu, Simon Baumgartner, Michael Bendersky and Chao Zhang.
    In Findings of Annual Meeting of the Association for Computational Linguistics (ACL), 2024.

  • ProgGen: Generating Named Entity Recognition Datasets Step-by-step with Self-Reflexive Large Language Models
    Yuzhao Heng, Chunyuan Deng, Yitong Li, Yue Yu, Yinghao Li, Rongzhi Zhang, Chao Zhang
    In Findings of Annual Meeting of the Association for Computational Linguistics (ACL), 2024.

  • Local Boosting for Weakly-Supervised Learning
    Rongzhi Zhang, Yue Yu, Jiaming Shen, Xiquan Cui and Chao Zhang.
    In ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD), 2023.

  • Cold-Start Data Selection for Few-shot Language Model Fine-tuning: A Prompt-Based Uncertainty Propagation Approach
    Yue Yu, Rongzhi Zhang, Ran Xu, Jieyu Zhang, Jiaming Shen and Chao Zhang.
    In Annual Meeting of the Association for Computational Linguistics (ACL), 2023.

  • Zero-Shot Text Classification by Training Data Creation with Progressive Dense Retrieval
    Yue Yu, Yuchen Zhuang, Rongzhi Zhang, Yu Meng, Jiaming Shen and Chao Zhang.
    In Findings of Annual Meeting of the Association for Computational Linguistics (ACL), 2023.

  • PRBoost: Prompt-Based Rule Discovery and Boosting for Interactive Weakly-Supervised Learning
    Rongzhi Zhang, Yue Yu, Shetty Pranav, Le Song and Chao Zhang.
    In Annual Meeting of the Association for Computational Linguistics (ACL), 2022.

  • Adaptive Multi-view Rule Discovery for Weakly-Supervised Compatible Products Prediction
    Rongzhi Zhang, Rebecca West, Xiquan Cui and Chao Zhang.
    In ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD), 2022.

  • AcTune: Uncertainty-Aware Active Self-Training for Active Fine-Tuning of Pretrained Language Models
    Yue Yu, Lingkai Kong, Jieyu Zhang, Rongzhi Zhang and Chao Zhang.
    In Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL), 2022.

  • SeqMix: Augmenting Active Sequence Labeling via Sequence Mixup
    Rongzhi Zhang, Yue Yu and Chao Zhang.
    In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), 2020.

    • Experiences

    Teaching

    Academic Service

    Misc