Several rebels were also reportedly killed as police entered a village held by the Moro Islamic Liberation Front (Milf) early on Sunday.
Reports say two key terror suspects were the targets of the operation.
It is the worst fighting since the government and Milf signed a peace deal last year to end decades of fighting.
There was conflicting information about Sunday's fighting, but reports from the Philippines suggest police commandos launched a surprise raid on a village near the town of Mamasapano, in the Maguindanao province of Mindanao island.
They were reportedly seeking to arrest members of the Bangsamoro Islamic Freedom Fighters (Biff), a Milf splinter group which opposes the ceasefire.
But the mayor of the town, Tahirudin Benzar Ampatuan, said police had not warned the Milf of the raid, as they should have under the peace deal, so its members fought back.
The gunfire reportedly went on for several hours. At least 30 police were killed, according to officials. Regional police chief Noel Armilla told the AFP news agency that six of their bodies had been recovered so far.
Milf leaders confirmed the violence but the number of rebels killed and their affiliation was not clear. The violence ended when ceasefire monitors intervened, said local officials.
The military was not involved, but peacekeeping reinforcements have since been sent to the region.
National police chief Leonardo Espina said in a statement that information was "sketchy" but that there was "an ongoing operation in the area against a high-value target believed to be behind the recent spate of bombings in Central Mindanao".
Various reports said police were pursuing Zulkifli bin Hir, known as Marwan - a leading figure of Islamist group Jemaah Islamiyah (JI) - as well as suspected JI bomb-making expert Basit Usman, who is linked to the Biff.
JI has links to al-Qaeda and wants an Islamic state across South East Asia. It has a long track record of bomb attacks in Indonesia and elsewhere in the region.
Marwan, who is on the FBI's most-wanted list, is believed to have been in hiding in the southern Philippines for over a decade.
Both men have been incorrectly reported killed several times. The US has offered rewards for their capture.
Last year's peace deal promised the separatists more autonomy in return for an end to their insurgency. It established peacekeeping procedures but correspondents say they appear to have failed.
The movement's leading peace negotiator, Mohagher Iqbal, told AFP the violence was "a big problem" but that the peace deal still held.
Elite Special Action Forces from the Philippine National Police.