Abstract: In order to fully utilize the massive video data collected by the comprehensive video surveillance system, the feasibility of the surveillance video big data application for high-speed railway infrastructure is analyzed and the construction goals of the video big data application system are formulated. The overall architecture and logical architecture of the system is proposed. The system provides a structured analysis platform for surveillance video and achieves structured organization and storage management of video data through accurate video data annotation, which can prepare accurate and effective data sets for intelligent video data analysis, and also provide favorable conditions for railway information resource integration and comprehensive application development. The core function of the system is closed-loop management of railway infrastructure abnormalies alarm based on intelligent video analysis. Besides, it can provide customized query functions and data sharing services of surveillance video, conducive to improve the application value of video data and to promote the integrated development and fusion innovation of railway information systems. .