Having gone behind to a Scott Robinson goal on 78 minutes after Christian Nade had been sent off the Light Blues, who dominated this encounter, threw everything forward and eventually got their reward in injury time.
With referee Calum Murray looking at his watch Lee McCulloch tried his luck from distance and when Marian Kello fumbled his effort Little was on hand to blast the ball home from eight yards.
It was a great moment for the 20-year-old substitute who had replaced Kirk Broadfoot late in the second period.
Despite rumours that Lee McCulloch would start in attack John Fleck and Kyle Lafferty were named as Gers’ forwards against the Tynecastle oufit.
With Kenny Miller, Kris Boyd, Nacho Novo and Steven Naismith all out 18-year-old Fleck was asked to play in his preferred role up top and was one of the best performers on the day.
Despite several injury problems Walter Smith was able to name a strong backline with Sasa Papac, David Weir, Danny Wilson and Kirk Broadfoot in front of Allan McGregor.
In midfield McCulloch and Kevin Thomson started in the centre with Steven Davis and Steven Whittaker featuring on the right and left flanks respectively.
Csaba Laszlo also made two changes to his team that beat St Mirren 1-0 at Tynecastle seven days ago with Ian Black and Lee Wallace back in.
The first-half started with both teams trying to trying to dictate play and the only real chance fell to Christian Nade who turned his marker on the 18-yard line but blazed a shot in to the Copland Road Stand.
At both end of the pitch, however, there were interesting battles with Lafferty working tirelessly against Ismael Bouzid and Marius Zaliukas and 18-year-old Danny Wilson coming up against Hearts striker Scott Robinson who is just 17.
As the game progressed Rangers started to dominate play and on 21 minutes Fleck used his pace and neat close control to win a free-kick 25 yards out.
Davis stepped up and his right foot effort was palmed away by former Kaunas keeper Marian Kello who had dived low to his right.
And with the home fans now off their seats Kirk Broadfoot saw a header go inches over the bar after Thomson had delivered an excellent in-swinging cross from another set piece.
With half an hour played Ian Black, who scored the winner for Inverness Caley when Rangers last lost at Ibrox on March 4 2009, was fortunate not to have been booked following two poor challenges on Papac and Whittaker.
Playing almost as a lone striker with Fleck roaming in a deeper role, Lafferty was clearly out to impress and on 35 minutes he swiveled past Zaliukas 25 yards from goal but his right foot shot failed to find the opposite top corner of Kello’s net.
With so many key players out, this was always going to be a big test for the Light Blues but their industrious performance in the first-half earned them a warm round of applause as they walked down the tunnel.
They had been the better side with Allan McGregor not having one save to make but having struggled to engineer any clear cut chances the second-half got underway with the match still deadlocked.
Pedro Mendes was told to warm up during the interval but there were no changes for either team for the restart.
But while McGregor had a quiet first period he had to be at his very best on 48 minutes.
Nade received the ball on the left wing and his bending cross was left by Danny Wilson and the keeper saved the day with a superb stop.
There is no doubt the ball was finding the bottom corner but once again McGregor showed great agility to produce a one-handed stop which spared Wilson’s blushes.
Rangers, however, immediately got back in to their stride with Fleck, in particular looking increasing dangerous.
On 52 minutes the youngster used his pace to get past Jason Thomson on the left wing and his powerful cross was deflected out for a corner.
He then fired a curling shot from 25 yards which went inches wide after Sasa Papac had pushed forward and drove a 30 yard effort in to Kello’s arms.
The game was now opening up nicely but all that was required from Rangers was a killer pass or a moment of magic.
Steven Davis looked to provide it on the hour mark when a Fleck corner was headed out to the Northern Ireland man who blasted a shot high and wide.
On 64 minutes Lafferty had the chance he had been waiting for but he was unable to take it.
Fleck ran at the back tracking Hearts defence and he found his teammate darting past him unmarked over his left shoulder.
The pass in to his path was perfect but Kyle’s low shot was blocked by Kello who had narrowed the angle.
With 20 minutes remaining Rangers were given a license to throw everything forward when Christian Nade was shown a red card for a moment of sheer stupidity.
Having been booked only two minutes previous for a push on David Weir the big striker kicked the ball away to stop Allan McGregor from taking a quick free-kick.
Ref Calum Murray had no option but with ten men on the pitch and having rarely troubled the Gers backline Hearts stunned Ibrox by netting the opening goal.
Lee Wallace’s low shot was parried away by McGregor and Scott Robinson beat Sasa Papac for pace and fired the ball home.
The away support could not believe their luck and with time ticking away it looked as though the Gorgie side had snatched a memorable away win.
Little, however, had other ideas and when Kello parried the ball in to his path he coolly struck the ball home to earn his side what could turn out to be a vital point.