One of the four painted instruments has now had its lid screwed down - to stop the public bashing the ivories.
They were installed in a bid to inspire young people to eventually get work in creative sectors.
But traders said children just used them as climbing frames or saw them as toys.
Mark Holmes, who runs Harts jewellers in Barrow, Cumbria, said: "If they were playing Beethoven or Mantovani or something nice that would be OK. But it's just kids jumping up and down on them and creating utter noise. They're not as bad as carol singers but they're on the same level."
What is happening where you live? Find out by adding your postcode or visit InYourArea
Amanda Cubin, of One Stop Gift Shop, said: "People aren't coming to the market because of them. I can't concentrate on what I'm doing. It's driving me insane." Lynne McKenna, of Lynne's Menswear, said: "When I'm reckoning up, it's so distracting. It's just loud and you get kids banging on it all day."
Colin Garnett, manager of Barrow Business Improvement District, said: "The feedback that we've had is that it's brought people into the area.