Panhala Fort was built by Raja Bhoja in the late 12th century. It saw a procession of rulers the Shilaharas, the Yadavas, the Adilshah of Bijapur. One of the most touching extant details of this era is a tiny carving of Ganesh which graces one of the gates erected by Adilshah. The idol wears a Muslim cap, signifying Muslim-Hindu amity.

Panhala's Maratha saga began when Shivaji captured the fort in 1659 from the Adilshah. The following year, he successfully escaped during a long siege by the Bijapur army.

The escape was made possible largely by the heroics of two loyal aides, Shiva Kharshed and Baji Prabhu Deshpande. Kharshed disguised himself as Shivaji and pretended to surrender to Bijapur, to give his master a head start over the enemy forces that would give pursuit. When the true identity of the warrior was established, he was killed and the troops set out to capture Shivaji. In stepped Deshpande, who laid down his life holding them at bay at the head of a narrow pass.

Today, bronze images of these two heroes with raised swords and spears greet visitors as they enter the walled citadel, near Teen Darwaza.