In the place of the current Kácov castle, there used to stand another older fortress which was established probably in the middle of the 15th century after the old fortress located to the south west of Kácov on the left bank of the Sázava River near Soušice was abandoned. Due to the fact that, from the turn of the 14th and 15th century, Kácov was divided between several owners, it is likely that two fortresses existed next to each other for a period of time. There is a record from 1473 concerning the fortress in the location of the current castle when three quarters of it belonged to Kuneš of Olbramovice. The owner of the castle chose a location in the southern part of Kácov on a rocky slope steeply falling to the Sázava River. This slope created a natural fortification which was in the north complemented with an artificial fortification which has been destroyed completely.