The manager thought his side had done enough to secure a victory when Lee McCulloch won a spot kick with 20 minutes left and the skipper slotted the ball past Lee Robinson.
But with both teams had been reduced to ten men following the dismissals of Kevin Kyle and Willie Gibson, Gavin Reilly equalised in injury time to make it 2-2 and take the game to extra time and then the lottery of penalties.
Ally said: “It’s a sore one that’s for sure and we’re really disappointed.
“To lose in the manner we did on penalties was disappointing and I don’t think there was a lot between the teams.
“I thought we probably had done enough after we got the second goal when we were down to ten men.
“We are obviously disappointed in the way we lost the second goal as well.
“Having seen it again a couple of the boys were telling me it was our throw-in and then then the lad looked a yard offside.
“But that aside we should still be defending it better.
“It’s not a wake-up call for me it’s just a massive disappointment.
“We were all aware how difficult the game would be tonight against a good Queen of the South side that are doing well in the league and they also knocked Hibs out of the League Cup.
“But we definitely need to do more in terms of creation with the ball.
“Tonight we didn’t do it as much as we have done in our recent home games.”
