She delivered a lecture on Sunday organised by the African National Congress Women's League.
"It's a huge responsibility not only for me but for all of us. It is for the first time that a woman gets to be in this position," she said.
She told a packed city hall in Pretoria that the AU's primary task had always been to deal with poverty, underdevelopment, and illiteracy and ensure that the continent becomes prosperous.
Citizens must be able to drive the AU because if it's left to governments it will not go anywhere.
"The vision of the African Union is an integrated, prosperous and peaceful Africa driven by its own citizens in the global arena...so the questions we must ask is that we as South Africans how are we going to drive the African Union and if we were not doing that its time that we start doing so," Dlamini Zuma said.
Earlier this month, Dlamini Zuma won a tightly contested vote to become the new head of the AUC. Her election made her the first female to head the AUC, ending a bruising leadership battle between her and Gabon's former minister Jean Ping who sought a re-election after serving at the top AU post since 2008.
On Sunday, she moved to emphasise the importance of unity within the continental body, saying only a united AU would be able to deliver a prosperous Africa. She said when African leaders formed the Organisation of African Unity their goal had always been to unite the continent.
"These leaders decided that Africa must unite and that those countries that were not liberated should be liberated. It was clear from these leaders that we needed to find African solution to African problems," said Dlamini Zuma.
"....this organisation is looking at an integrated Africa, not Africa of individual states who are only concerned with their own business. So when I go and work there, I will be working as a servant of Africa and not South Africa and off course South Africa will be part of those that I will be serving."
She spoke of an Africa that was marginalised and side-lined from the important affairs of the globe. It was time that Africa restored to its past glory.
Dlamini Zuma further suggested that the Pan African Parliament, situated in Midrand, be given legislative powers something, she said would help in speeding up development in the continent.
"Because if we are going to be integrating we need to harmonise our laws and if we are to increase trade we need to harmonise the way we work and the way we do things and the Pan African Parliament in this regard could play a huge role there".
She also noted that in the face of a global economic crisis crippling world markets Africa was the only continent that was still enjoying substantial growth.
"We have the biggest deposition of many minerals throughout the continent and the question is how do we use them for the benefit of the people of the continent. We still have a big potential to grow in agriculture so that we can export more, it's important to see that this opportunities are not missed opportunities".
The biggest challenge was that Africa lacked good infrastructure. "That's one of our biggest challenges, how do we connect with one another because at the moment. We must make sure we are properly connected, we must make sure we build this infrastructure so that the integration we speaking of becomes meaningful."
Dlamini Zuma served in three ministries since 1994 starting at Health then to Foreign Affairs and Home Affairs which she occupied from 2009.
As Minister of Home Affairs, she is credited for having reduced perceived high corruption levels in the department, professionalised its service and modernised its internal processes.
Peace and security remain priorities for the AU and the 63-year-old Dlamini Zuma will be expected to provide leadership that would ensure that the AU Peace and Security Council's ability to resolve conflicts in the continent is enhanced.
She will also have to oversee the implementation of the mooted intra-Africa trade and free trade areas and ensure the success of economic partnership agreements within the continent.
