Barry Ferguson’s men were 2-0 ahead at half-time thanks to goals from Nico Raskin and Mohamed Diomande, but Celtic responded in the second-half to draw the game level.
However, just minutes after being introduced, it was substitute Hamza Igamane who got the crucial winner in the 87th minute with a superb strike to seal the three points.
From Thursday evening’s match against Fenerbahce, Ferguson named just one change to his starting XI. Jefte was unavailable for the match due to personal reasons, with Ridvan coming in in his place.
That meant Rangers started with Jack Butland in goal behind a back line which included Dujon Sterling, captain James Tavernier, John Souttar, Leon Balogun and Ridvan.
Nico Raskin, Connor Barron and Mohamed Diomande were in midfield, while Vaclav Cerny was providing support in attack to Cyriel Dessers.
Rangers started well in the opening five minutes and had the first chance of the match as Nico Raskin released an effort towards goal which was pushed wide by Schmeichel.
From the resulting corner, it was the Light Blues who took the advantage with a perfect set-piece. Tavernier’s delivery was whipped into the box, and it was Raskin who met it at the near post to glance beyond the Celtic number one for his first league goal this campaign.
The home side looked for a quick response, but Maeda could only fire wide after he was played in by Reo Hatate.
As ever, it was a lively start to this derby, and Butland was next to be called into action as he got a strong hand to a Hatate strike to push wide.
There was another huge chance for Rangers with quarter of an hour played in the East End of Glasgow. Schmeichel’s goal kick was intercepted by an alert Vaclav Cerny and the winger raced towards goal, skipping beyond the Celtic number one but was unable to find Cyriel Dessers in the box with a Celtic defender clearing the danger.
Tavernier provided another great delivery from a corner with the game approaching the half hour mark, with Leon Balogun getting on the end of it this time. The defender sent it towards goal but Celtic managed to clear with Schmeichel getting a touch before it was turned behind by Nawrocki.
Rangers had the second with 37 minutes played and it was nothing less than Ferguson’s men deserved for their first-half display. It was another well worked move, with Tavernier sending a ball into the box from the right, Nico Raskin got his head to it to nudge it into the path of Diomande who had made a good run into the box.
In space, the midfielder showed composure to send a low right-footed finish into the bottom corner.
Maeda hit the side netting with a deflected shot as the home side looked to respond before the break but the resulting corner was dealt with well by Rangers.
Dujon Sterling made a superb interception on the line with just minutes of the first half remaining as he hooked away a Maeda header on the line, before Butland denied Nawrocki with a save.
Less than five minutes into the second-half, Celtic had a goal back with the first chance of the half. Jota delivered a ball into the box from the right and Maeda rose at the back post to head beyond Butland.
Rangers were forced into a change with 56 minutes played, Leon Balogun was replaced after going down in the box, with Robin Propper coming on in his place.
Luke McGowan was just inches wide with another chance for Celtic before another change followed for Gers as Ferguson introduced as Mohamed Dioamde was replaced by Ianis Hagi.
With 74 minutes played Celtic were back level, McGowan played a ball through to find the run of Hatate, and he sent a low finish across goal and beyond Butland to make it 2-2.
It was the hosts who appeared to have the momentum this second-half, and Schlupp was denied by Butland as he was strong to push a strike wide from the full-back. At the other end, Barron flashed a shot just wide after linking-up with Vaclav Cerny.
A final three changes came for Rangers with 10 minutes of the game remaining as Clinton Nsiala, Tom Lawrence and Hamza Igamane were introduced for Dujon Sterling, Connor Barron and Vaclav Cerny.
It was Rangers who had the third goal as Hamza Igamane netted a third goal in the 87th minute. Having just been introduced, the ball broke for the Moroccan just outside the box and he took a touch onto his right foot before sending a blistering shot high into the back of the net.
RANGERS: Butland, Sterling (Nsiala, 81), Tavernier, Souttar, Balogun (Propper, 56) , Ridvan, Raskin, Barron (Lawrence, 81), Diomande (Hagi, 65), Cerny (Igamane, 81), Dessers.
Subs: Kelly, Bajrami, Rice, Danilo
Celtic: Schmeichel, Johnston, Jota (Yang, 70) , Kuhn (Idah, 46), McCowan, Schlupp, Nawrocki, Carter-Vickers, Engels, Maeda, Hatate.
Subs: Sinisalo, Taylor, Kenny, Murray, Ralston, Bonnar, Turley