There are remains of several settlements in the municipality, such as the Mozarab Eremitic Complex at Valle-Niza, the Phoenician remains at Los Toscanos and the Cerro Mar y Jardín burial grounds, some of which go back to pre-historic times.

There are various explanations as to why the main town, Vélez-Málaga, is not on the coast, as is usual with municipalities which reach down to the seashore. One of these explanations refers to an earthquake in the year 365.

The huge waves caused by seismic movement devastated the only settlement on the coast. Remains of this town are to be found at the Phoenician archaeological site of Los Toscanos and in the surrounding area. After this catastrophe it was decided to build a town on the hill where the fortress currently stands.

Another explanation, rather less scientific and more legendary, tells the story of a shepherd who found an image of the Virgin Mary in the countryside. He liked it so much that he decided to take it home for his daughter but lost it on the way. The story spread like wildfire and all the local people began to think that it was a miracle, erecting the present shrine at the spot where the image had been found.