The senior leader from the ruling Pakistan Peoples Party, who has been opposed to the paramilitary operation, said he would have brought peace to the city within 15 days if he had been given a free hand.
Mirza's resignation is likely to further complicate the already volatile situation in Karachi, where political ethnic violence has claimed as many as 1400 lives this year, 530 of them over the last two months. It is said to be the worst ethnic and political violence in the city in 16 years.
At a crowded news conference where he lashed out at both Malik and the MQM, Mirza said he was resigning as senior minister in the Sindh Cabinet, as a member of the Provincial Assembly and also quit as senior Vice President of the Pakistan Peoples Party.
Mirza criticised Malik for mishandling the situation in Karachi and said he will provide proofs to the government about his role in deterioration of the peace in Karachi.
He said that Malik was "hand in glove with terrorists" and termed him an opportunist and the biggest enemy of Pakistan, while also holding him responsible for killings in Karachi.
"I have repeatedly told the PPP leadership that Rehman Malik is a pathological liar," said Mirza.
A close friend of Zardari, Mirza has always been a controversial figure with several of his statements against the ethnic Urdu-speaking group Mutahida Qaumi Movement (MQM) creating a rift with allies.
While acknowledging that Malik was a "good politician and good human being", Mirza maintained that he was also a "compulsive liar".
Mirza had opposed the operation by security forces in the restive Lyari area of Karachi, which is the stronghold of the ruling party.
Mirza, however, reiterated his commitment to the party and to his friendship with President Asif Ali Zardari and pledged to continue working as a normal party activist.
His controversial remarks last month in which he said that the Urdu-speaking migrants had come "hungry and naked" to this city had fuelled further unrest in the already simmering city.
The MQM, which is the single largest party from Karachi in the national and provincial assemblies has repeatedly accused Mirza of patronising criminal gangs which were fanning ethnic and political violence in Pakistan's biggest city.
The severe differences between the MQM and Mirza came to the fore last year when Mirza as home minister first accused the MQM of running extortion rackets and killings in the city.
Mirza was made senior minister after the MQM broke away from the coalition at the centre and the province.
But following his controversial remarks against immigrants all hell broke loose and the government sidelined him and started rapprochement with the MQM.
"Altaf Hussain is a killer and his party is also a terrorist organisation and they are behind the killings and violence in Karachi," Mirza alleged while constantly swearing on the Koran and putting it on his head to insist that he was speaking the truth.
He also alleged the MQM was behind the killing of Geo news journalist Wali Babar and five of the arrested killers belonged to the MQM.