Abstract: As a core system in CTCS-3(Chinese Train Control System Level 3), the Onboard System was a typical safety-critical system, the over-speed protection algorithm had an important influence on Train Control System. Based on high-speed train dynamics model, time delay characteristics, and mixed features, the paper proposed vehicle over-speed protection algorithm and hybrid modeling method. The simulation of over-speed protection algorithm was implemented by using Simulink/Stateflow mixed technology. The interval two train tracking scenario was taken as an example to validate the algorithm. The simulation showed that this over-speed protection algorithm was verified, collision avoidance of two trains running in a block section could be implemented.