On an unbelievable night Rangers progressed in Europe beyond Christmas for the first time in nine seasons when they won 4-3 on penalties after 210 minutes of football against the French side produced no goals.
Ronald de Boer should have won the game for them in extra-time but missed a penalty four minutes from the end and Rangers thought their chances had gone. However, the players dug deep to produce an absolutely crucial victory which will reshape the whole campaign.
In an amazing spot-kick competition, both Bert Konterman and Claudio Caniggia missed but Stefan Klos saved from Jay-Jay Okocha while Gabriel Heinze shot wide.
Then it went to sudden death and skipper Barry Ferguson bravely stood up to fire the ball into the net. The Ibrox bench waited nervously and then Mauricio Pochettino missed the target to spark massive celebrations on the field for the Ibrox men.
Rangers could smell the fourth round of the UEFA Cup as they dominated PSG but could not find the cutting edge in an enthralling first-half in Paris.
Strike star Tore Andre Flo had two excellent chances, both from headers, but sadly he failed to find the net with either effort when it looked as though he should have scored.
The first opportunity came in 36 minutes when Claudio Caniggia cut in from the right and produced an excellent cross but Flo headed straight at Lionel Letizi. Barry Ferguson tried to follow up but was brilliantly tackled on the edge of the area by Arteta.
Then two minutes before half-time Ronald de Boer won a free-kick on the right side of the PSG box. Claudio Reyna flighted the free-kick and Flo was alone as he rose to meet it but again he headed straight at the French keeper.
De Boer also had an opportunity after 13 minutes as Rangers really took the game to the French side. Maurice Ross did brilliantly on the right side as he skipped passed Gabriel Heinze and then crossed for the Dutchman but he glanced his header wide of the target.
It was a thoroughly impressive first-half from the Ibrox side, even though they had to make a change in 30 minutes when Lovenkrands was forced to come off with a thigh muscle injury and was replaced by Caniggia.
Paris had some moments but never really tested Stefan Klos other than long-range efforts from Ronaldinho and Laurent Leroy.
The French side showed much more flair and endeavour after the break but although they had plenty of possession they never really caused Rangers too much trouble.
The Ibrox men could have done with more of the ball but they seemed comfortable and they always looked more dangerous when coming forward.
However, just like Paris, they were unable to add the finishing touch to any of their moves. In 61 minutes youngster Ross cut inside onto his left foot and forced Letizi to make a decent save, while at the other end Klos made an excellent diving save to deny Cristobal, who shot from long range in 77 minutes.
Manager Dick Advocaat knew the game was there for the taking and he sent on Russell Latapy for Flo in 78 minutes, and the little Caribbean star almost scored immediately.
He took a pass from Numan and moved onto his left foot but again Letizi was equal to his shot.
In the final minute, Rangers almost snatched it when Numan won the ball from Cristobal to release Latapy who, in turn, set up Caniggia, but de Boer could not apply the finishing touch to the Argentinian’s cross.
The amazing tie went into extra-time and Rangers should have won it when they were awarded a penalty four minutes from the end of the second period. It was Latapy who set up Caniggia and he was bundled to the ground by Letizi.
The entire stadium held its breath as Ronald de Boer stepped up to take the kick and the Rangers fans could not believe it as the Dutchman fired the ball over the bar.
Incredibly, de Boer missed in this city just over three years ago in the World Cup semi-final against Brazil in a shoot-out and the massed legions of Rangers supporters feared that their chance had gone.
However, in an unbelievable penalty competition, the Ibrox men stormed into the fourth round when Argentinian stopper Mauricio Pochettino blazed over the top.