Jamie Ness swept the hosts into a dramatic lead at Ibrox after just two minutes but saw his strike cancelled out by Kris Commons.
Steven Whittaker then restored the lead from the spot after Fraser Forster was dismissed for bringing Steven Naismith down in the box.
But Scott Brown ensured a replay will take place at Parkhead on March 2 with a second equaliser but Naismith too was sent packing for diving.
The match was a wonderful advert for Scottish football, but home boss Walter Smith will be rueing a number of missed chances which could have won the game for his side.
Instead, he now looks forward to Old Firm meetings in total this term, more than ever before in a single campaign, because of the team's failure to convert more than twice.
Smith made three changes to the side which beat Hearts 1-0 in the SPL, with all of them anticipated by the Light Blues support.
Nikica Jelavic returned after a groin strain and he was joined in attack by on-loan Blackburn star El Hadji Diouf.
Naismith was also restored to the first XI, with Lee McCulloch missing out because of a knee problem and Kyle Lafferty and Vladimir Weiss dropping to the bench.
While Smith deployed his men in a 4-4-2 system, the visitors took on a more tentative 4-5-1 look with Gary Hooper deployed as a sole striker.
It's said so often that there's an electric atmosphere at these games but that was certainly the case prior to kick off as a sell-out crowd waited for play to begin.
And there was an extraordinary opening period as Gers came flying out of the traps and stormed into a lead after just two minutes.
Diouf won a corner on the right but when it came in it was scrambled away by the visiting defence and it seemed a chance to score was gone.
But when Ness took control of the ball, he unleashed an unstoppable screaming shot from 25 yards which arrowed into the top corner beyond Forster.
It was the teenager's first goal for the club and was a truly phenomenal strike hit with such venom. What a way to break your duck.
There was more to come as Gers pummelled their opponents and only the bar stopped Steven Davis from making it 2-0 with barely four minutes gone after Jelavic set him up.
Back came Celtic though, with Allan McGregor having to be in the right place to hold Beram Kayal's rising drive and Mark Wilson then heading just over.
Before long, play swung in the other direction again and Naismith might well have had a penalty when he appeared to be tripped in the box.
Referee Calum Murray waved away the claims but it seemed the Scotland striker had a case as he fell.
The fact nothing was given proved to be costly as the Parkhead team responded well and levelled on the quarter-hour mark.
Their equaliser, swept home by Commons, was a poor one to lose from the home team's point of view.
Joe Ledley was able to make progress down the left and when he cut back after the ball took a touch off Maurice Edu, Commons stepped in from a deep position to make it 1-1.
Having impressed initially, Rangers were falling out of the game rapidly as Celtic's extra man in midfield was making a difference.
Indeed, not having a more commanding lead by the time they were pegged back seemed to knock the stuffing out of Smith's men a little.
But they slowly came back into the contest and a determined surge into the corner by Jelavic eventually led to another Davis long-ranger flying past the post.
The Croatian was then just a couple of inches from grabbing his first ever derby goal with a chance he should never have had.
A heavy passback to Celtic keeper Forster led to a heavy touch and somehow the big former Rapid Vienna hitman charged him down and blocked his clearance.
The ball trickled towards the goalline, with a very grateful Forster just getting there on time to keep it out as more than 40,000 home fans tried to will it over and into the net.
It wasn't to be the on-loan Newcastle goalie's day and his match came to a premature end seven minutes before the break when he was sent off and Gers went ahead again.
Jelavic did superbly to burst upfield after Ness won possession and slip a smart through ball through for Naismith to latch on to.
He got there first and touched past Forster who then caught him and had to go for denying a goalscoring opportunity, with a penalty awarded.
Commons was hauled off with Lukasz Zaluska brought into the game - and his first task was to pick the ball out of the net.
Despite having to wait more than two minutes to finally take his kick, Whittaker rifled in a pinpoint effort from 12 yards which left the Pole with no chance.
Leading 2-1 at the break, Rangers were unchanged as play restarted with Naismith and Madjid Bougherra both sitting on yellow cards for the SPL champions.
Although they were a man down, it was Celtic who had more of the ball as the home team seemed content to sit in and absorb pressure.
That made for a nervy spell as Lennon's side began to grow in confidence without really troubling McGregor.
Hitting on the break was still giving Gers chances and a long, sweeping cross-field pass playing in Jelavic, whose audacious chip wasn't too far from dropping in.
Diouf then set up the 25-year-old for a sliding effort which was only really kept out by Zaluska because he couldn't connect properly.
Brought in at the end of last month for the rest of the season, Diouf was having a good impact on the game.
But after being hemmed in, Celtic levelled for a second time with 26 minutes to go when Brown curled home from the edge of the area.
Rangers' response was good and within 60 seconds they could have been ahead yet again when Jelavic broke away and crossed low for Davis, who just fell short of poking in.
An intriguing cup tie was very much hanging in the balance going into the final quarter of the game, with both teams showing enough to suggest they could win.
Substitute Georgios Samaras should really have made it 3-2 in favour of the away side when he was set up by Hooper but his shot was weak and lacked direction.
McGregor then produced a magnificent point-blank stop when a header from the Greek striker was flying in and the forward then nodded over from the resulting corner.
The Light Blues were hanging on and things went from bad to worse when Naismith saw red for diving just inside the box.
Smith responded by throwing on Weiss and Lafferty and the former almost netted when his first contribution was a run past two men and a shot which cannoned off Zaluska.
Time was ticking down and with just a few moments to go, it seemed a draw was the most likely result.
But then, with five minutes remaining, there was an explosion of activity as Weiss was penalised for a foul on Brown.
Bookings were dished out to the Slovakian, Weir and Ledley before the free kick 20 yards out thankfully came to nothing.
Having sat deep for much of the half, Rangers then suddenly had a chance to win the game as Lafferty rammed a Davis through ball goalwards.
His attempt flashed into the crowd though and with that, any chance of victory was gone and the sides will now do it all again across Glasgow next month.
