Kaipeng Zhang

Recent Publications (2024, 2025, 2026)

[Selected Technical Report]

[World Model] PackForcing: Short Video Training Suffices for Long Video Sampling and Long Context Inference

[World Model] WildWorld: A Large-Scale Dataset for Dynamic World Modeling with Actions and Explicit State toward Generative ARPG

[World Model] Yume: An Interactive World Generation Model

[Video Generation] OmniCustom: Sync Audio-Video Customization Via Joint Audio-Video Generation Model

[Agent] PyVision-RL: Forging Open Agentic Vision Models via RL

[Agent] MeepleLM: A Virtual Playtester Simulating Diverse Subjective Experiences

[Agent] World Craft: Agentic Framework to Create Visualizable Worlds via Text

[Agent] LongCLI-Bench: A Preliminary Benchmark and Study for Long-horizon Agentic Programming in Command-Line Interfaces

[Agent] PyVision: Agentic Vision with Dynamic Tooling

[Agent] AI Idea Bench 2025: AI Research Idea Generation Benchmark

[Conference Papers]

[CVPR 2026] Yume1.5: A Text-Controlled Interactive World Generation Model

Xiaofeng Mao, Zhen Li, Chuanhao Li, Xiaojie Xu, Kaining Ying, Tong He, Jiangmiao Pang, Yu Qiao, Kaipeng Zhang†

[CVPR 2026] SVBench: Evaluation of Video Generation Models on Social Reasoning

Wenshuo Peng, Gongxuan Wang, Tianmeng Yang, Chuanhao Li, Xiaojie Xu, Hui He, Kaipeng Zhang†

[CVPR 2026] ProSoftArena: Evaluating Hierarchical Capabilities of Multimodal Agents in Professional Software Environments

Jiaxin Ai, Yukang Feng, Fanrui Zhang, Jianwen Sun, Zizhen Li, Chuanhao Li, Yifan Chang, Wenxiao Wu, Ruoxi Wang, Mingliang Zhai, Kaipeng Zhang†

[CVPR 2026] MatchMask: Mask-Centric Generative Data Augmentation for Label-Scarce Semantic Segmentation

Yuqi Lin, Hao Zhang, Wenqi Shao, Shiqu Liu, Zhihong Gu, Wenxiao Wang, Xiaofei He†, Kaipeng Zhang†

[CVPR 2026 Findings] From Static Snapshots to Dynamic Trajectories: Evaluating and Enhancing the Learning Pathways of Multimodal Large Language Models

Yukang Feng, Wenxiao Wu, Jianwen Sun, Chuanhao Li, Fanrui Zhang, Zizhen Li, Jiaxin Ai, Sizhuo Zhou, Yifan Chang, Changxin Gao, Shenglin Zhang, Kaipeng Zhang†

[ICLR 2026] A High-Quality Dataset and Reliable Evaluation for Interleaved Image-Text Generation

Yukang Feng, Jianwen Sun, Chuanhao Li, Zizhen Li, Jiaxin Ai, Fanrui Zhang, Yifan Chang, Sizhuo Zhou, Shenglin Zhang, Yu Dai, Kaipeng Zhang†

[ICLR 2026] OmniWorld: A Multi-Domain and Multi-Modal Dataset for 4D World Modeling

Yang Zhou, Yifan Wang, Jianjun Zhou, Wenzheng Chang, Haoyu Guo, Zizun Li, Kaijing Ma, Xinyue Li, Yating Wang, Haoyi Zhu, Mingyu Liu, Dingning Liu, Jiange Yang, Zhoujie Fu, Junyi Chen, Chunhua Shen, Jiangmiao Pang, Kaipeng Zhang, Tong He†

[ICLR 2026] InternSpatial: A Comprehensive Dataset for Spatial Reasoning in Vision-Language Models

Nianchen Deng, Lixin Gu, Shenglong Ye, Yinan He, Zhe Chen, Songze Li, Haomin Wang, Xingguang Wei, Tianshuo Yang, Min Dou, Tong He, Wenqi Shao, Kaipeng Zhang, Yi Wang, Botian Shi, Yanting Zhang, Jifeng Dai, Yu Qiao, Hongjie Zhang†, Wenhai Wang†

[AAAI 2026] MDK12-Bench: A Multi-Discipline Benchmark for Evaluating Reasoning in Multimodal Large Language Models

Pengfei Zhou, Fanrui Zhang, Xiaopeng Peng, Zhaopan Xu, Jiaxin Ai, Yansheng Qiu, Chuanhao Li, Zhen Li, Ming Li, Yukang Feng, Jianwen Sun, Haoquan Zhang, Zizhen Li, Xiaofeng Mao, Wangbo Zhao, Kai Wang, Xiaojun Chang, Wenqi Shao, Yang You, Kaipeng Zhang†

[NeurIPS 2025 Spotlight] Think or not think: A study of explicit thinking in rule-based visual reinforcement fine-tuning

Ming Li, Jike Zhong, Shitian Zhao, Yuxiang Lai, Haoquan Zhang, Wang Bill Zhu, Kaipeng Zhang†

[NeurIPS 2025] Sekai: A Video Dataset towards World Exploration

Zhen Li, Chuanhao Li, Xiaofeng Mao, Shaoheng Lin, Ming Li, Shitian Zhao, xu Zhao Pan, Xinyue Li, Yukang Feng, Jianwen Sun, Zizhen Li, Fanrui Zhang, Jiaxin Ai, Zhixiang Wang, Yuwei Wu†, Tong He, Yunde Jia, Kaipeng Zhang†

[NeurIPS 2025] Neural-Driven Image Editing

Pengfei Zhou, Jie Xia, Xiaopeng Peng, Wangbo Zhao, Zilong Ye, Zekai Li, Suorong Yang, Jiadong Pan, Yuanxiang Chen, Ziqiao Wang, Kai Wang, Qian Zheng, Xiaojun Chang, Gang Pan, Shurong Dong†, Kaipeng Zhang†, Yang You

[NeurIPS 2025] REPA Works Until It Doesn't: Early-Stopped, Holistic Alignment Supercharges Diffusion Training

Ziqiao Wang, Wangbo Zhao, Yuhao Zhou, Zekai Li, Zhiyuan Liang, Mingjia Shi, Xuanlei Zhao, Pengfei Zhou, Kaipeng Zhang†, Zhangyang Wang, Kai Wang†, Yang You

[EMNLP 2025] InMind: Evaluating LLMs in Capturing and Applying Individual Human Reasoning Styles

Zizhen Li, Chuanhao Li, Yibin Wang, Qi Chen, Diping Song, Yukang Feng, Jianwen Sun, Jiaxin Ai, Fanrui Zhang, Mingzhu Sun, Kaipeng Zhang†

[ICCV 2025] ProJudge: A Multi-Modal Multi-Discipline Benchmark and Instruction-Tuning Dataset for MLLM-based Process Judges

Jiaxin Ai, Pengfei Zhou, xu Zhao Pan, Ming Li, Fanrui Zhang, Zizhen Li, Jianwen Sun, Yukang Feng, Baojin Huang, Zhongyuan Wang†, Kaipeng Zhang†

[ICCV 2025] Neighboring Autoregressive Modeling for Efficient Visual Generation

Yefei He, Yuanyu He, Shaoxuan He, Feng Chen, Hong Zhou†, Kaipeng Zhang†, Bohan Zhuang†

[ICCV 2025] ZipVL: Accelerating Vision-Language Models through Dynamic Token Sparsity

Yefei He, Feng Chen, Jing Liu, Wenqi Shao, Hong Zhou†, Kaipeng Zhang†, Bohan Zhuang

[ICCV 2025] LiT: Delving into a Simple Linear Diffusion Transformer for Image Generation

Jiahao Wang, Ning Kang, Lewei Yao, Mengzhao Chen, Chengyue Wu, Songyang Zhang, Shuchen Xue, Yong Liu, Taiqiang Wu, Xihui Liu, Kaipeng Zhang, Shifeng Zhang, Wenqi Shao†, Zhenguo Li†, Ping Luo

[ICCV 2025] GUIOdyssey: A Comprehensive Dataset for Cross-App GUI Navigation on Mobile Devices

Quanfeng Lu, Wenqi Shao†, Zitao Liu, Lingxiao Du, Fanqing Meng, Boxuan Li, Botong Chen, Siyuan Huang, Kaipeng Zhang, Ping Luo†

[ACL Findings 2025] MPBench: A Comprehensive Multimodal Reasoning Benchmark for Process Errors Identification

Zhaopan Xu, Pengfei Zhou, Jiaxin Ai, Wangbo Zhao, Kai Wang, Xiaojiang Peng, Wenqi Shao, Hongxun Yao†, Kaipeng Zhang†

[ACL 2025] EfficientQAT: Efficient Quantization-Aware Training for Large Language Models

Mengzhao Chen, Wenqi Shao†, Peng Xu, Jiahao Wang, Peng Gao, Kaipeng Zhang, Ping Luo†

[ICML 2025] ZipAR: Accelerating Auto-regressive Image Generation through Spatial Locality

Yefei He, Feng Chen, Yuanyu He, Shaoxuan He, Hong Zhou†, Kaipeng Zhang†, Bohan Zhuang

[ICML 2025] Towards World Simulator: Crafting Physical Commonsense-Based Benchmark for Video Generation

Fanqing Meng, Jiaqi Liao, Xinyu Tan, Wenqi Shao†, Quanfeng Lu, Kaipeng Zhang, Yu Cheng, Dianqi Li, Ping Luo†

[IJCAI 2025] TP-Eval: Tap Multimodal LLMs' Potential in Evaluation by Customizing Prompts

Yuxuan Xie, Tianhua Li, Wenqi Shao, Kaipeng Zhang†

[CVPR 2025 Oral] OpenING: A Comprehensive Benchmark for Judging Open-ended Interleaved Image-Text Generation

Pengfei Zhou*, Xiaopeng Peng*, Jiajun Song, Chuanhao Li, Zhaopan Xu, Yue Yang, Ziyao Guo, Hao Zhang, Yuqi Lin, Yefei He, Lirui Zhao, Shuo Liu, Tianhua Li, Yuxuan Xie, Xiaojun Chang, Yu Qiao, Wenqi Shao, Ping Luo, Kaipeng Zhang†

[ICLR 2025 Oral] Dynamic Multimodal Evaluation with Flexible Complexity by Vision-Language Bootstrapping

Yue Yang*, Shuibo Zhang*, Wenqi Shao†, Kaipeng Zhang†, Yi Bin, Yu Wang, Ping Luo†

[ICLR 2025] SAMRefiner: Taming Segment Anything Model for Universal Mask Refinement

Yuqi Lin, Hengjia Li, Wenqi Shao, Zheng Yang†, Jun Zhao, Xiaofei He, Ping Luo, Kaipeng Zhang†

[ICLR 2025] MMIU: Multimodal Multi-image Understanding for Evaluating Large Vision-Language Models

Fanqing Meng*, Chuanhao Li*, Jin Wang*, Quanfeng Lu, Hao Tian, Tianshuo Yang, Jiaqi Liao, Xizhou Zhu, Jifeng Dai, Yu Qiao, Ping Luo, Kaipeng Zhang†, Wenqi Shao†

[NeurIPS 2024 Spotlight] ConvBench: A Multi-Turn Conversation Evaluation Benchmark with Hierarchical Capability for Large Vision-Language Models

Shuo Liu, Kaining Ying, Hao Zhang, Yue Yang, Yuqi Lin, Tianle Zhang, Chuanhao Li, Yu Qiao, Ping Luo, Wenqi Shao†, Kaipeng Zhang†

[NeurIPS 2024] SearchLVLMs: A Plug-and-Play Framework for Augmenting Large Vision-Language Models by Searching Up-to-Date Internet Knowledge

Chuanhao Li, Zhen Li, Chenchen Jing, Shuo Liu, Wenqi Shao, Yuwei Wu, Ping Luo, Yu Qiao, Kaipeng Zhang†

[NeurIPS 2024] Rethinking Human Evaluation Protocol for Text-to-Video Models: Enhancing Reliability, Reproducibility, and Practicality

Tianle Zhang, Langtian Ma, Yuchen Yan, Yuchen Zhang, Kai Wang, Yue Yang, Ziyao Guo, Wenqi Shao, Yang You, Yu Qiao, Ping Luo, Kaipeng Zhang†

[NeurIPS 2024] Needle In A Multimodal Haystack

Weiyun Wang, Shuibo Zhang, Yiming Ren, Yuchen Duan, Tiantong Li, Shuo Liu, Mengkang Hu, Zhe Chen, Kaipeng Zhang, Lewei Lu, Xizhou Zhu, Ping Luo, Yu Qiao, Jifeng Dai, Wenqi Shao†, Wenhai Wang†

[NeurIPS 2024] Lumina-Next: Making Lumina-T2X Stronger and Faster with Next-DiT

Le Zhuo*, Ruoyi Du*, Han Xiao*, Yangguang Li*, Dongyang Liu*, Rongjie Huang*, Wenze Liu*, Lirui Zhao, Fu-Yun Wang, Zhanyu Ma, Xu Luo, Zehan Wang, Kaipeng Zhang, Xiangyang Zhu, Si Liu, Xiangyu Yue, Dingning Liu, Wanli Ouyang, Ziwei Liu, Yu Qiao†, Hongsheng Li†, Peng Gao†

[ICML 2024] Towards Implicit Prompt For Text-To-Image Models

Yue Yang, Yuqi Lin, Hong Liu, Wenqi Shao, Runjian Chen, Hailong Shang, Yu Wang, Yu Qiao, Kaipeng Zhang† and Ping Luo†

[ICML 2024] MMT-Bench: A Comprehensive Multimodal Benchmark for Evaluating Large Vision-Language Models Towards Multitask AGI

Kaining Ying*, Fanqing Meng*, Jin Wang*, Zhiqian Li, Han Lin, Yue Yang, Hao Zhang, Wenbo Zhang, Yuqi Lin, Shuo Liu, Jiayi Lei, Quanfeng Lu, Cunjian Chen, Peng Xu, Renrui Zhang, Haozhe Zhang, Peng Gao, Yali Wang, Yu Qiao, Ping Luo, Kaipeng Zhang† and Wenqi Shao†

[ICML 2024] SPHINX-X: Scaling Data and Parameters for a Family of Multi-modal Large Language Models

Peng Gao*†, Renrui Zhang*, Chris Liu*, Longtian Qiu*, Siyuan Huang*, Weifeng Lin*, Shitian Zhao, Shijie Geng, Ziyi Lin, Peng Jin, Kaipeng Zhang, Wenqi Shao, Chao Xu, Conghui He, Junjun He, Hao Shao, Pan Lu, Hongsheng Li† and Yu Qiao

[CVPR 2024] DiffAgent: Fast and Accurate Text-to-Image API Selection with Large Language Model

Lirui Zhao*, Yue Yang*, Kaipeng Zhang‡*, Wenqi Shao‡*, Yuxin Zhang, Yu Qiao, Ping Luo, Rongrong Ji†

[CVPR 2024] OneLLM: One Framework to Align All Modalities with Language

Jiaming Han, Kaixiong Gong, Yiyuan Zhang, Jiaqi Wang, Kaipeng Zhang, Dahua Lin, Yu Qiao, Peng Gao, Xiangyu Yue†

[NAACL Findings 2024] T3M: Text Guided 3D Human Motion Synthesis from Speech

Wenshuo Peng, Kaipeng Zhang†, Sai Qian Zhang†

[ACL Findings 2024] ChartAssistant: A Universal Chart Multimodal Language Model via Chart-to-Table Pre-training and Multitask Instruction Tuning

Fanqing Meng, Wenqi Shao†, Quanfeng Lu, Peng Gao, Kaipeng Zhang, Yu Qiao, Ping Luo†

[ICLR 2024] Towards Lossless Dataset Distillation via Difficulty-Aligned Trajectory Matching

Ziyao Guo, Kai Wang, George Cazenavette, Hui Li, Kaipeng Zhang†, Yang You†

[ICLR 2024] OmniQuant: Omnidirectionally Calibrated Quantization for Large Language Models

Wenqi Shao*, Mengzhao Chen*, Zhaoyang Zhang, Peng Xu, Lirui Zhao, Zhiqian Li, Kaipeng Zhang, Peng Gao, Yu Qiao, Ping Luo†

[AAAI 2024] Data Adaptive Traceback for Vision-Language Foundation Models in Image Classification

Wenshuo Peng, Kaipeng Zhang†, Yue Yang, Hao Zhang, Yu Qiao

[AAAI 2024] TagCLIP: A Local-to-Global Framework to Enhance Open-Vocabulary Multi-Label Classification of CLIP Without Training

Yuqi Lin, Minghao Chen†, Kaipeng Zhang†, Hengjia Li, Mingming Li, Zheng Yang, Dongqin Lv, Binbin Lin, Haifeng Liu, Deng Cai

[ICASSP 2024] Align, Adapt and Inject: Audio-Guided Image Generation, Editing and Stylization

Yue Yang, Kaipeng Zhang†, Yuying Ge, Wenqi Shao, Zeyue Xue, Yu Qiao, Ping Luo†

[Journal Papers]

[IJCV 2025] HarmoniDPO: Video-guided Audio Generation via Preference-Optimized Diffusion

Wenshuo Peng, Kaipeng Zhang†

[IJCV 2024] Open-Vocabulary Animal Keypoint Detection with Semantic-feature Matching

Hao Zhang, Lumin Xu, Shenqi Lai, Wenqi Shao, Nanning Zheng†, Ping Luo, Yu Qiao, Kaipeng Zhang†

[TPAMI 2024] LVLM-eHub: A Comprehensive Evaluation Benchmark for Large Vision-Language Models

Peng Xu*, Wenqi Shao†*, Kaipeng Zhang*, Peng Gao*, Shuo Liu, Meng Lei, Fanqing Meng, Siyuan Huang, Yu Qiao, Ping Luo†

[TIFS 2024] B-AVIBench: Towards Evaluating the Robustness of Large Vision-Language Model on Black-box Adversarial Visual-Instructions

Hao Zhang, Wenqi Shao, Hong Liu, Yongqiang Ma, Ping Luo, Yu Qiao, Nanning Zheng†, Kaipeng Zhang†

[TBigData 2024] Tiny LVLM-eHub: Early Multimodal Experiments with Bard

Wenqi Shao*, Yutao Hu*, Peng Gao*, Meng Lei*, Kaipeng Zhang, Fanqing Meng, Peng Xu, Siyuan Huang, Hongsheng Li, Yu Qiao†, Ping Luo†

[TCSVT 2024] HF-HRNet: a simple hardware friendly high-resolution network

Hao Zhang, Yujie Dun, Yixuan Pei, Shenqi Lai, Chengxu Liu, Kaipeng Zhang, Xueming Qian†

[Pattern Recognition 2024] FMGNet: An efficient feature-multiplex group network for real-time vision task

Hao Zhang, Yongqiang Ma, Kaipeng Zhang, Nanning Zheng†, Shenqi Lai†

[Tutorial]

[CVPR 2025 Tutorial] From Multimodal LLM to Human-level AI: Evaluations and Benchmarks

About Me

Recent Publications (2024, 2025, 2026)

Education

Selected Awards and Competitions

Academic Service

Work Experience