It was the home side with the advantage at the break thanks to a Lawrence Shankland strike, before goals from Ross McCausland, Todd Cantwell and Fabio Silva had Rangers 3-1 ahead.
Dexter Lembikisa reduced the lead, before Kyosuke Tagawa drew the home side level in the final seconds of the match.
There were two changes from the midweek victory over Dundee as Academy full-back Robbie Fraser came in for his first start having made his debut from the bench on Tuesday evening.
Scott Wright was also handed a start at Tynecastle after a big impact midweek with the winger netting a double from the bench in the 5-2 victory at Ibrox.
That meant that Rangers started in the capital with Jack Butland in goal and a back four of James Tavernier, Leon King, Ben Davies and Robbie Fraser.
Mohamed Diomande, Nico Raskin and Todd Cantwell made up the midfield three, while Ross McCausland and Scott Wright provided the support for Cyriel Dessers.
Rangers were on an impressive run of form against Hearts this season, with five victories over Steven Naismith’s men in all competitions so far this campaign.
The two sides last met back in April in the Scottish Cup semi-final at Hampden, with Cyriel Dessers’ double enough to see Rangers progress and set-up next weekend’s Old Firm final.
It was Todd Cantwell who got the action underway, and it was a fairly even contest in the opening spell.
Yutaro Oda tried a shot from distance for the home side as he found himself in space on the edge of the box but dragged it wide of Butland’s right-hand post before at the other end Cyriel Dessers tested Zander Clark with a good strike but it was held by the Hearts goalkeeper on the second attempt.
A short corner for Rangers saw Todd Cantwell and Scott Wright combine on the right, with the latter whipping the ball into the box in search of James Tavernier at the back post but his header was held by Clark.
The Hearts goalkeeper pulled off a fantastic block with 20 minutes played as Rangers had their best opportunity of the match so far. Ross McCausland won the ball back in the box before playing a square pass to find Cyriel Dessers in a central position but he was unable to beat the goalkeeper with Clark getting enough on the ball to block the effort.
Shankland fired over for the home side as the match approached the half-hour mark, after a long throw-in was sent into the box by Stephen Kingsley, before Frankie Kent headed over from a Barrie McKay corner.
It was the home side who took the lead at Tynecastle and it was Lawrence Shankland who found the breakthrough. Yutaro Oda played a ball forward to find the run of the in-form striker who was through on goal and he lifted the ball over Jack Butland and into the back of the net.
Vargas had another chance for Hearts shortly after as his shot from a tight angle hit the side netting with Hearts taking the one goal advantage in at the break.
Into the second-half, and with just over five minutes played Rangers were back level through Ross McCausland. It was a fantastic solo effort from the Northern Ireland international as he gathered the ball in a dangerous area and took a great touch before finding the back of the net with a fine finish which beat Clark and went in off the post.
At the other end, Butland got a strong hand to an Oda shot with an impressive save before Rangers were able to clear the danger.
Dessers had a good chance to give Rangers the advantage for the first time in this match as he was picked out in space in the box by Robbie Fraser but was unable to keep his strike on target as it drifted wide.
With an hour played at Tynecastle, Clement made a triple change to his side as he introduced Fabio Silva, Ridvan and Kieran Dowell for the remainder of the match, with Scott Wright, Robbie Fraser and Mohamed Diomande coming off.
Clark denied Silva as he pushed away a shot from the on-loan Wolves forward, before Todd Cantwell’s rebound strike was blocked.
Rangers did have the advantage with 68 minutes on the clock, and it was another lovely finish with Todd Cantwell finding the back of the net. It was a good move from the Light Blues as Tavernier found Dessers with the striker holding the ball-up before feeding it into the path of Cantwell.
He took a touch around a Hearts defender before curling a left-footed shot low into the bottom corner.
Clement made the final two changes to his side with 70 minutes played as Abdallah Sima and Dujon Sterling came on in place of Cyriel Dessers and Ben Davies.
Kenneth Vargas headed wide for Hearts as he got on the end of a Lembikisa cross with space in the box.
It was three for Rangers, with 10 minutes of the game as substitute Fabio Silva managed to gather the ball in the box following a cross from Ross McCausland and showed composure to finish beyond Clark.
However just minutes later, the deficit was reduced again as Hearts’ Dexter Lembikisa drilled a powerful strike into the back of the net.
Clark denied Kieran Dowell a goal late in the game as he touched his shot onto the post, with Abdallah Sima flagged offside in the rebound.
Deep in injury time, it was Hearts who found an equaliser with Kyosuke Tagawa beating Butland with a powerful strike from inside the box.
RANGERS: Butland, Tavernier, King, Davies (Sterling, 70), Fraser (Ridvan, 59), Diomande (Dowell, 59), Raskin, Cantwell, McCausland, Dessers (Sima, 70), Wright (Silva, 59).
Subs: McCrorie, Jack, Yfeko, McKinnon
Hearts: Clark, Kent, Kingsley, Baningime, Shankland, Oda (Boyce, 74), Devlin (Lembikisa, 74), Rowles, McKay (Fraser, 63), Cochrane, Vargas (Tagawa, 86)
Subs: Gordon, Grant, Denholm, Tait, Wilson.