Gers extended their winning run to seven games in all competitions thanks to three first-half goals, before Kilmarnock got one back late in the second-half.
Speaking to RangersTV at full-time, the left-back said: “We are pleased with the first half; I thought the second-half kind of died down. It just didn't feel like us, and I think we gave up too many chances, probably more than the last three games combined.
“I think we had to wake ourselves up, and that's everyone, but at the same time it's three points, and we keep pushing forward.
“That is the worry when a break comes that it stops all momentum, but in the first half we were really good, and we probably could've scored a couple more. The second-half needs to be an improvement, but we know that, and hopefully we will correct that next week.”
Penrice believes the winning run is helping the team build confidence and gel, as they look to keep pushing forward.
He said: “I keep saying it, I wake up buzzing to play for Rangers and buzzing to go to my work, and that is all I was looking for this season.
“I knew it would be tough coming back, but I am loving every minute of it, and I think you can see that when I am playing.
“When I came in, I saw a group that needed to gel, and I think that with a couple more additions later on in the window we look like a real squad now and everyone was buzzing to get back in.
“If you were out at training yesterday, there was a real buzz to get back in and see everyone.
“It was like going back to school after so long, but definitely there's a real togetherness, but that happens when you win games, and I think we just need to keep going.
“You keep winning games; it builds confidence, and also other teams start to look and go they are really there so we will take one game at a time, and we look to get three points next week again.”
