The Light Blues went a goal up in first-half injury time with Andy Little netting his seventh goal of the campaign in front of watching Northern Ireland manager Michael O’Neill.
But Ian Little's men rallied after the break with substitute Fraser McLaren netting the equaliser on 62 minutes.
Rangers should have regained their lead immediately with Francisco Sandaza and Lee McCulloch missing opportunities.
But Berwick finished the match far stronger and will perhaps be disappointed not to win this historic clash.
Shielfield Park was basked in sunshine prior to kick-off with a 4,140 capacity crowd surrounding the shale track which is also used by the Berwick Bandits Speedway Club.
Greek defender Anestis Argyriou was handed his debut after signing a two-year deal yesterday while Ross Perry returned to the starting line-up after a spell out with a calf injury.
Sebastien Faure was also named on the five-man Gers bench but there was no place in the squad for Kirk Broadfoot who is struggling with a hamstring problem.
Rangers started the match positively and on two minutes Lewis Macleod picked the ball up on the left but couldn’t get enough curl on a shot which flew past Youssef Bejaoui’s far post.
Referee Mike Tumilty then had to issue the first yellow card of the match when the combative Lee Currie produced a late challenge on Ian Black in the centre circle which left the Gers ace lying in pain on the turf. It should have been red.
But this got the home support fired up and Neil Alexander had to be alert to hold two long-range efforts from Berwick’s Kevin McDonald and Neil Janczyk.
Credit must go to the Wee Gers for their first-half display as they made it tough for the visitors to get out of first gear and Ally McCoist cut a frustrated figure in the technical area.
The opening goal, however, arrived in injury time and it came from a well-worked set piece.
After Janczyk had fouled Black 30 yards from goal the former Hearts man dinked a clever cross to the back post with the Berwick defence expecting him to shoot.
The unmarked McCulloch nodded the ball back into the danger area and Little, who is full of confidence, lashed it high into the net from eight yards.
After the break Berwick responded well and having created a couple of early chances at the start of the second period they equalised after the hour mark.
The ball was played over Perry's head and McLaren latched on to it before hitting a low, left-foot strike across Alexander into the bottom corner of the net.
Understandably the Berwick fans went crazy but Rangers really should have regained their lead immediately when McKay slipped a great pass to Sandaza but his shot was blocked by the legs of Berwick keeper Bejaoui.
McCulloch then darted through the middle to get on the end of a flick on but his powerful right foot drive from 18 yards flew inches wide of goal.
On 75 minutes the away support had their hearts in their mouths when Berwick’s scorer McLaren had another chance but his right foot drive from a tight angle failed to trouble Alexander.
The Gers keeper then had to pull off a wonder stop to deny the lively number 14 who targeted the top corner from 25 yards.
From the resulting corner the ball was nodded into the net by Berwick defender Chris Townsley but the referee had already blown for a foul which looked a harsh decision.
This was to be the last bit of action from an afternoon the Rangers players will want to forget and forget quickly.
Berwick will get a lot of praise for their gritty display, and rightly so, but Ally McCoist admitted he was 'frustrated and angry' after the final whistle and once again the 90 minutes showed that getting three points in the Third Division is never guaranteed.
BERWICK: Bejaoui, Droudge, Townsley, Brydon, Hoskins, Notman, Janczyk (McLaren 58), Currie, McDonald, Lavery (Gielty 58), Addison.
SUBS: McCaldon, Lancaster, Ferguson.
BOOKED : Currie, Janczyk.
GOAL: McLaren 62.
RANGERS: Alexander, Perry, Emilson Cribari, Bocanegra, Argyriou, Little, McCulloch, Macleod (Naismith 77), Black, McKay (Crawford 64), Sandaza (Kyle 64).
SUBS: Gallacher, Faure.
GOAL: Little 45.
REFEREE: Mike Tumilty