My name is Zhiyuan Song, but you can also call me Allen. I am currently an M.S. student in Computer and Information Science (AI Track) at the University of Pennsylvania. I am fortunate to work as a Research Assistant advised by Prof. Jiatao Gu, focusing on multimodal foundation models and large language model reasoning.

My primary research interests lie in building intelligent, real-time interactive systems. My current focus spans LLM post-training, Omni & multi-model systems, real-time interaction models, agent harnesses, continual learning, and long-horizon tasks. I am also passionate about robust AI infrastructure and optimizing distributed training pipelines.

I completed my undergraduate studies at the University of Virginia, where I double majored in Computer Science and Applied Statistics and graduated with High Distinction, and was fortunate to conduct research advised by Prof. Madhav Marathe, Prof. S.S. Ravi, and Prof. N. Rich Nguyen.

I welcome any entrepreneurial ideas and am always open to finding co-founders for exciting ventures. Feel free to reach out — I'd love to chat!

Education

Experience

News

• [Jun 2026]Started as an AI Engineer Intern at Millennium Management, working on cloud & AI/ML infrastructure.
• [May 2026]New preprint: "Learning When to Think While Listening in Large Audio-Language Models" is on arXiv, under review at NeurIPS 2026.
• [Aug 2025]I started my M.S. in CIS at UPenn and joined Prof. Jiatao Gu's group as a Research Assistant.
• [Jul 2025]Led the team as Tech Lead & MLE at AdventureX Hackathon, building a full-duplex real-time audio AI assistant.
• [Dec 2024]Graduated from University of Virginia with High Distinction.
• [Aug 2024]Paper "SL4HOI" accepted at KDD 2024 in Barcelona.

Selected Publications

Learning When to Think While Listening in Large Audio-Language Models

Z. Song, W. Zhao, Y. Xiao, S. Yu, C. Zhu, J. Gu

NeurIPS 2026Under review · Neural Information Processing Systems

[arXiv][Code]

SL4HOI: Supervised Learning for Predicting Health Opportunity Index across States

Z. Song, Z. Mei, L. Li, N. Nguyen

KDD 2024 · Knowledge Discovery and Data Mining, Barcelona

[Paper]