It is located on a rock, at the top of the town, and has an irregular plan in the shape of an elongated triangle that adapts to the shape of the rock itself. Its access door is protected by a barbican and topped with a shield. The fortress also consists of a Torre del Homenaje, a wall with a walkway, two towers, an underground enclosure and a well in the north tower. 

Part of the wall fence that surrounded the La Villa neighborhood is preserved from the Olvera Castle.