The game swung back and forth as Robbie Ure opened the scoring for the Light Blues before they were pegged back just before the interval.
Charlie Lindsay restored the side’s lead from the penalty spot shortly after the restart, only for Hearts to level from a corner minutes later.
Up stepped Lovelace in the dying minutes to seal a precious victory for the Young Gers in a testing encounter.
David McCallum named an attacking lineup for the trip to Edinburgh, with Arron Lyall given the nod in attack alongside Ure and Zak Lovelace.
Rangers engineered the first opportunity of the afternoon as Arron Lyall fired just wide after the ball ricocheted into his path following good work from Zak Lovelace down the left-hand side.
The wind was certainly playing in Hearts’ favour in the first half and the Gorgie side fashioned a decent opportunity when McLuckie centred for Kirk in the area, but the Jambos attacker could only steer his header wide.
Robbie Fraser then showed impressive physicality in an advanced area on the left, standing up a cross for Ure in which the Hearts goalkeeper was alert to intercept.
The two combined again on 17 minutes, with the Gers captain whipping a tantalising delivery into the penalty area which narrowly evaded his teammate.
The home side created their best chance of the opening period as McLuckie latched onto a through pass and was bearing down on Lewis Budinauckas’ goal. However, the Gers stopper made a great save with his feet to deny the Hearts attacker.
Lovelace curled wide moments later down the opposite end, before the Gers did well to thwart a promising Hearts attack through the combination of McLuckie and Kirk.
However, the Gers took the lead in the 34th minute with a well-worked goal. Greig Allen read the danger stepping out of defence before Lindsay thread a gorgeous pass into the path of Ure, with the Gers forward making no mistake by slotting his finish under the Hearts goalkeeper.
The home side equalised on the stroke of half time as Denholm raced through before stroking his finish into the bottom corner.
Only six minutes had elapsed in the second period when Rangers restored their advantage. Lyall was brought down inside the penalty area and, when the referee pointed to the penalty spot, Lindsay sent McFarlane the wrong way from 12 yards out.
Hearts immediately restored parity from a corner. Darge rose highest to meet the delivery before glancing a header beyond Budinauckas.
Johnly Yfeko and Ross McCausland were introduced on the hour mark as McCallum looked to freshen his side up.
Both sides were pushing for a winning goal as Jack Harkness made a superb intervention from Pollock’s pull-back.
Lindsay then floated a free-kick into the penalty area that narrowly missed Lovelace’s intelligent run to the far post.
From the resulting corner, Lewis MacKinnon headed dangerously towards goal which required an acrobatic tip over the bar from the Hearts goalkeeper. The ball was then swung in from the opposite corner where Rathie turned the goalbound delivery off the line.
Hearts had an opportunity of their own down the opposite end as McLuckie raced through on goal only to be smothered by an alert Budinauckas.
Ure fired over with his left foot with five minutes remaining after impressive set-up play from Yfeko, before Mackenzie Strachan stung the palms of McFarlane with a rasping effort from distance.
Rangers’ persistence was rewarded in the dying embers of the game. Lovelace showed tremendous strength to work his way through on goal before coolly slotting the ball into the bottom corner to claim all three points for the Light Blues.
HEARTS: McFarlane, Watson, Darge, Tait, Pollock, Denholm (Sandilands, 77), Kirk, Gordon, Forrester, Rathie, McLuckie (Hambrook, 82)
Unused subs: Walker, Drysdale, Ross
RANGERS: Budinauckas, Harkness, MacKinnon (Allan, 82), Allen (Yfeko, 60), Fraser, Strachan, Lindsay, Young (McCausland, 60) Lyall (Graham, 83), Ure, Lovelace
Unused subs: Pazikas, K. Ewen, Ciubotaru
ㅤ
ㅤ