A new electrical connection box installed by Network Rail engineers in Pogmoor is acting as a 'stile' for youths who trespass on the track.
Since it was installed residents on St Catherine's Way have regularly spotted youths clambering over it.
It is thought they are using it as shortcut to the Penny Pie playing fields on the corner of Pogmoor Road and Dodworth Road. There are also fears it will be used more often when the new Horizon school opens on Dodworth Road in September.
Many residents have written to Network Rail raising their concern. One from David and Hilda Roome states: "It is being used as a stepping stone to the playing fields and new school across the lines.
"When approached we are subjected to foul and abusive language. It seems only a matter of time before serious injury occurs by the trespassers getting impaled on the railings or a fatality by being struck by a train."
Pogmoor Area Residents' Association has been made aware of the situation. Spokesman Tom Parnham said the box was a health and safety 'blunder' and described it as a stile. He added: "Kids don't have that perception of danger like an adult. Especially when there's something like a park on the other side."
A spokeswoman for Network Rail said the box had been installed as a home for equipment to 'feed a telecommunications system'.
She said the fence was the highest that was allowed and a community safety engineer would visit the site to look at where youths were climbing over it.
He added: "Anybody who sees somebody going onto the railway should call British transport Police straight away."
St Catherine's Way residents are also upset about the way work was carried out through the night, disturbing their sleep. They say vans were parked across their drives and the road has been left 'unsightly' after it was dug up.
Tony Carr lives on the street. He said he was now concerned about how often Network Rail would need to visit the box. "They still come on a regular basis and do whatever they do. Our main concern is what are their intentions? Is this going to be like an industrial estate?"
Network Rail has been informed. The spokeswoman said: "Residents did the right thing in contacting the helpline."