Speaking at the release of the Auditor General's findings of the province's municipal audits on Tuesday, Kiviet said there had only been minimal improvements in the fight against corruption.
"Before the next audit we are going to find those involved in corruption and punish them, even if it means they have to be jailed. The time to be persuasive has come and gone. Now is the time to be pushy," said Kiviet at a press briefing in East London.
She was at pains to explain that her government had faced challenges, including that municipalities were staffed by under-qualified people as well as the disruption caused by the changing of staff following the local government elections in 2010.
"Our government will no longer tolerate the employment of under-qualified people, especially in the finance departments, which is where many problems arise," said Kiviet.
In response to the findings, Local Government and Traditional Affairs MEC Mlibo Qoboshiyane said municipalities should not just rely on oversight from the province and that they should be monitoring finances on their own.
Provincial Auditor General Singa Ngqwala said there was a need for more monitoring of managers.
Mbashe municipality received a qualified audit, while Mnquma and Mzimvubu municipalities received unqualified audits.