In their first match without Nikica Jelavic following his deadline day transfer to Everton, Ally McCoist made one change from the team that beat Hibs the previous weekend with new signing Mervan Celik replacing Gregg Wylde in the starting eleven.
But it was the visitors who started the better and in the opening fifteen minutes Rangers goalie Allan McGregor was twice called into action.
On 11 minutes he did well to save a deflected shot from Russell and then three minutes later he took no chances and punched away a swirling shot from Paul Dixon.
From the resulting corner Gunning gave his team the lead. Rangers never looked comfortable in dealing with it and after McGregor hurried away a cross from John Rankin, Gary Mackay Steven crossed the ball in for Gunning who sent his header past the despairing McGregor.
Rangers needed a response and almost got it immediately from the one bright spark on the day, Sone Aluko.
The former Aberdeen man once again opened his box of tricks on 18 minutes and lashed a right-footed drive which was well-saved by Dusan Pernis.
Midway through the first half Pernis was again United’s saviour. Aluko set up David Healy whose shot seemed to be heading for the bottom corner but Pernis managed to get down and guide it out for a corner with a glove.
Just as Rangers looked to be on the front foot, they lost a very cheap second goal on 34 minutes.
Dixon and Mackay-Steven linked well together on the left hand side and with Kyle Bartley out of position, Johnny Russell had all the time in the world to hit home past McGregor, to leave Rangers with a mountain to climb.
The only chance of note before half-time fell to Kyle Bartley after a clipped ball into the box from Steve Davis, but Pernis collected the ball easily.
The ineffective Celik was replaced at half-time by Salim Kerkar as McCoist looked for a way for his side to get back into the game.
On the whole the sparse Ibrox crowd of only 17,822 saw very little in the way of chances for the home side and indeed it was the visitors who had the better chances in the second 45.
Devoid of service, Healy was replaced after 60 minutes by fellow Northern Irishman Andrew Little who moved up front with Kerkar.
But soon after Garry Kenneth had a great chance to kill the game off, but his free header only found the arms of McGregor and soon after United captain Jon Daly missed a header from six yards.
A rare chance for Rangers came on 71 minutes as some good skill from Aluko found Little in the box but he scooped his shot over the bar.
By this time Gregg Wylde had come on as Rangers threw their last throw of the dice at the tie, on 75 minutes Maurice Edu fired a shot well wide from distance.
John Rankin then tested Allan McGregor with a fierce volley as Dundee United looked for a third.
In the end, a Sone Aluko free-kick in the closing stages of the match went well over the bar, summing up the day for Rangers.
After the match Rangers manager Ally McCoist said: “Our defending for the goals was very, very poor. We certainly could have defended the cross a lot better for the first. “It was far too easy for the second goal with Johnny Russell getting far too much space. Defending is part of the game that we take pride in but it was poor today.
“It’s been a really disappointing day but you have to give credit where it’s due and I thought United in the first half played well.”
He added: “I felt we attempted to push forward and the effort was certainly there to get balls in to the box, but there was a bit of guile missing in the end. I certainly don’t have a problem with the way the players went about things in the second half. We were trying to push forward with Lee Wallace on the left and Sone Aluko on the right and the effort was there but the wee bit of quality was missing today.”
With Rangers knocked out of both domestic cup competitions, all their efforts will now be solely focused on the league.
McCoist continued: “The players will be fine. We don’t have an option. We have to get our sleeves rolled up and get on with it. The players are disappointed, but on Monday we’ll get in to training and we’ll go ahead. This club has had problems in the past but there is determination and grit in that dressing room that will see us through this.”
A jubilant Peter Houston told BBC Scotland after the match: “The first-half performance, especially, was excellent, we created a lot of chances and scored two good goals. We didn’t play as well in the second half, but I’m maybe being a wee bit picky.
“We still looked dangerous on the counter attack and restricted Rangers to not a lot. Pernis had two good saves from Healy and Aluko in the first half and that’s where you need your goalkeeper to keep you in the game. We’ve got a nice balance between youth and experience. It’s taken a while to come together but this was always going to be a transitional season. We’ve still got a lot to do to secure a top six place in the SPL.
“But it’s nice to come to Ibrox and record a victory. We kept attacking and kept two up all the time.”
RANGERS: McGregor; Bartley, Wallace, Bocanegra; Goian; Edu, Davis, Papac (Wylde, 69), Celik (Kerkar, 46); Aluko, Healy (Little, 60).
Subs not used: Alexander, McCulloch.
DUNDEE UNITED: Pernis; Neilson, Dixon, Kenneth, Gunning (Dillon, 90); Rankin, Robertson, Mackay-Steven (Lacny, 88), Armstrong (Flood, 76); Russell, Daly.
Subs not used: Banks, Ryan.