It was a hammer blow because Rangers had come back from two goals gown to level the tie against the battling Bairns and the joy of Sunday’s derby now seems like an eternity ago.
They trailed to two headers from Moroccan striker Farid El Allagui but Dorin Goian headed home in 82 minutes and then Nikica Jelavic scored four minutes from time.
Extra time seemed certain and Rangers would have been grateful for it but Millar killed them off with his free kick which Alexander made a mess of.
The League Cup exit will undoubtedly hurt Ally McCoist who has already suffered two big disappointments in Europe this season.
However, what will disappoint him the most is the way his side went about their business.
Although Rangers had most of the ball – especially in the first half – they were sluggish and predictable.
They never really caused any problems to goalkeeper Michael McGovern and there was nothing like the fire and passion which drove them to success in the Old Firm game.
That lack lustre aura was to prove costly as Falkirk suddenly realised they had a real chance to cause an upset.
El Allagui has been on good form since joining Falkirk in the summer and he was given far too much space in 57 minutes when he headed the First Division side ahead.
Rangers were stretched when Kieran Duffie had space and time on the right to cross to the back post where the Moroccan had the simplest of tasks to head into the net.
Lee McCulloch battles with Darren DodsThen in 73 minutes Rangers were undone again when 16-year-old Craig Sibbald produced a great cross from the left and El Allagui got above Gregg Wylde to head home his second goal.
The Rangers fans in the Falkirk Stadium could hardly believe what they were watching.
In between the two Falkirk goals Stevie Naismith, who came on for Alejandro Bedoya, hit the post and both Goian and Bocanegra had headers wide.
It was only when they were in trouble that Rangers produced more urgent football.
There seemed no way back but Romanian star Goian connected with Steven Whittaker’s cross in 82 minutes to through them a lifeline.
Then in 86 minutes, after incessant pressure, Nikica Jelavic, also a substitute, headed in Gregg Wylde’s cross and Rangers were level.
Given the nature of events Rangers now had the ascendancy and would have been favourites if the match had gone into extra time.
However, with over 92 minutes played Naismith fouled Sibbald 27 yards out and Falkirk had the chance for a famous win.
Former Hearts man Millar took it when he got his free kick up and over the wall. Alexander raced across his line and seemed to make the save but he pushed the ball upwards onto the bar and then it fell behind the line.
A posse of Falkirk players piled in to make sure the ball was in the net but it was already a clear goal.
It was a horrific way to go out of a competition they have contested the final of in each of the last four years, winning it three times.