"It is with a heavy heart ... that we announce the sudden and untimely death of the President of the Republic of Ghana," the president's office said, adding that Mills had died a few hours after being taken ill, without further details.
The 68-year-old president complained of pains Monday evening and was rushed to hospital on Tuesday noon when his condition worsened, a source close to the president said.
He was in the United States for nine days in June for a medical check-up amid swirling rumours about his ill health.
Women were seen wailing at the 37 Military Hospital, while crowds of high-ranking government and military officials looked perplexed, shocked and confused.
Mills's unexpected death came only five months before he was due to stand for re-election in December.
As the news of this death spread his deputy, Vice President John Dramani Mahama, was quickly sworn in as head of state in a smooth political transition that is in line with Ghana's constitution.
In his first presidential address, Mahama expressed condolences to Mills's family and said the final tribute to the late president was to "maintain the unity, peace and stability of the nation." Mahama will serve as caretaker president until presidential and parliamentary elections at the end of the year.
Senior members of Ghana's ruling National Democratic Congress (NDC) have also pledged support for the new president.
Condolences are also pouring in for the West African country. President Jacob Zuma was among the heads of state who paid tribute Mills and offered words of comfort to the Ghanaians in this difficult time.
"South Africa and Ghana enjoy strong relations at both the bilateral and multilateral levels, and under President Mills's leadership, we saw genuine efforts aimed at deepening the historical relations between our two nations," said President Zuma.
US president Barack Obama paid tribute to Mills, saying he was a strong advocate for human rights and for the fair treatment of all Ghanaians.
Obama said Mills helped promote economic growth in Ghana in the midst of challenging global circumstances and strengthened Ghana's strong tradition of democracy.
"On behalf of the American people, I would like to offer my deepest condolences to the people of Ghana, and reaffirm the deep and enduring bonds between our democracies that President Mills helped to strengthen," said Obama.
UN Secretary-General Ban Ki-moon said Mils would be remembered for his statesmanship and years of dedicated service to his country.
Ban vowed that the UN would work with the government and the people of Ghana to support their efforts to consolidate the country's democratic and development achievements.
As leader of a country of about 25 million, Mills had earned respect at home and abroad for his economic policies and commitment to democracy and good governance.
He served as President when Ghana began took on the mammoth task of dealing with its newfound oil wealth from offshore fields discovered in the last five years.
The GDP growth rate of the resource-rich country hit 14.5% in 2011, making Ghana one of the fastest growing economies in the world and the fastest in Africa.
