Abstract: Relay protection devices in traction substation play an irreplaceable and critical role in ensuring the high safety and reliability of traction power supply systems, and must undergo complete and rigorous testing before being put into use. At present, the testing of relay protection devices mainly conducted manually, which is very tedious, time-consuming, and labor-intensive, and requires high professional skills from testers, and is difficult to avoid human errors prone to causing safety hazards. Given the trend of extensive application of IEC61850 and IEC104 standards in smart traction substations, an automatic testing software for relay protection devices in smart traction substation, which complies with the IEC61850 and IEC104 standards, has been developed using the APIs provided by OMICRON relay protection tester. The testing software is composed of testing database, basic algorithm library, human-computer interaction modules, execution engine modules, and communication modules. It can automatically generate test cases and conduct testing operations, as welll as assist in test analysis, and generate test reports automatically. Laboratory tests have shown that this automatic testing software can effectively replace manual testing of existing relay protection devices, greatly improving testing accuracy and efficiency.