The incident took place late last night when the victim was on her way from Tirupati to Rajahmundry to reach her native place in East Godavari district, they said.
The woman alighted from the train in the early hours and instead of boarding a train towards Rajahmundry, she entered a train meant for the railway yard by mistake.
As she got down from the train at the yard, she was sexually assaulted.
A case was filed against four persons under Section 376 (D) of Nirbhaya Act following her complaint, said Vinay Kumar, Inspector of Police, Government Railway Police, Vijayawada.
The victim has been admitted to a government hospital, police said.
So far, no arrests have been made in the case.