Pretoria - A four-minute stoppage time goal by Landon Donovan paved the way for the United States to progress to the knock-out stages of the 2010 FIFA World Cup.
Although Algeria became the fourth African team to be knocked out of the showpiece, they proved their resilience.
It was a hard fought victory for the USA, who advanced to the next stage with five points, followed by England also with five points. England beat Slovenia 1-0 at Nelson Mandela Bay.
The USA faced Algeria with two previous draws from their 1-1 and 2-2 draw with England and Slovenia respectively. The two teams were in Group C along with England and Slovenia.
Before the deciding games, it was Slovenia who topped the group with four points, while both the USA and England got two points. Algeria was at the bottom of the group with one point.
Slovenia was eliminated from the tournament along with Algeria who never scored a goal in their three group stage matches.
One minute into play saw Algerian forward player, Karim Matmour attempting to find the back of the net, but his shot was off-target.
The USA three-man defence was caught off guard when the Algerian midfield fired a long ball to Rafik Djebbour who moved beautifully before taking a shot with his left foot from an acute angle, which went wide.
Hassan Yebda was the first player to go into the referee's book in the 16th minute following his hard tackle on Jozy Altidore.
The USA looked dangerous during their counter attack. Rais M Bolhi had to palm away a Herculez Gomez volley taken from 20 metres.
Former USA President, Bill Clinton was in the stadium to watch and cheer on his national team's decider group match, and his excitement evaporated when a Clint Dempsey goal was disallowed and ruled offside in the 21st minute.
Two minutes into the second-half, Karim Ziani made a brilliant move towards the opposition before he took an off- target shot.
While Matmour, Ziani and Djebbour continued to trouble the USA defence, Rais M Bolhi was also a marvel to watch with his superb saves denying the USA goal scoring opportunities.
In the second-half, Algeria continued to shine, however, a counter attack by Altidore who passed the ball to a well-positioned Dempsey, unleashed a rocket from the box, but the goal post denied him a goal.
Bob Bradley replaced Maurice Edu with Edson Buddle, while Rabah Saadane substituted Djebbour with Abdelkader Ghezzal and Ziani with Adlane Gueddioura.
Foued Kadir gave away a set piece on the edge of the box when he pushed Altidore in the 74th minute.
While the USA fans bit their nails off in anxiety during the four minutes of stoppage time, man of the match Donovan scored an important winner from an Altidore cross.
With their victories, both the USA and England will face the winners of group D which includes African hopefuls Ghana, Germany, Serbia and Australia.