There is something primal about caves, especially natural ones. They take years upon years to develop, being etched out one grain of sand at a time by stubborn streams of water. Some caves have been around since the age of mankind, and often long before that. Caves had a special place in ancient Hellenic worship, and were often devoted to the ancient Gods, such as Zeus, Apollo, and Cybele and particularly to Pan and the Nymphs. Caves in Their honor could and can still be found all over the Greek countryside. They were dark places full of votive offerings and altars, which would come to life during rituals attended by the ancient followers of the Gods.
We have quite a bit of information on cave worship in ancient Hellas, and Krete in particular. Caves were important to the inhabitants of Krete since Neolithic times, first as homes, then as burial places, and eventually places of worship by the Middle Minoan period. We distinguish three categories of cave sanctuaries: the grotto (mainly a large niche), the simple cave (deep, but usually consisting of one large space), and the complex cave with multiple rooms, possibly reminiscent of a maze. The majority of the latter two contain elaborate stalagmites and stalactites, as well as water sources, making them mystical and otherworldly places per excellence, as well as practical places to worship in a religion where the sprinkling of water-made-sacred is a staple.
Caves in the ancient Hellenic religion often connect to the wild and unknowable: not only do they often house 'monsters' (like the Cyclops in the Odysseia, and the Centaur Kheiron), but they are the houses of 'wild' women like Calypso, as well as of the Nymphs, and Pan. Yet, these are not completely wild; they occupy the space between the untamed wild and the carefully tended ﬁeld of civilized life. Their places of worship--when they attracted cult worship, like the Nymphs and Pan--including caves, but also actual gardens, do not represent the rule of mankind over nature; it is better to say that these places represent ideal nature--untamed, yet safe, comfortable, and full of beauty. A true natural sanctuary. Jennifer Larson, in the excellent essay 'A Land Full of Gods: Nature Deities in Greek Religion' speaks of this connection to the idea of the 'divine garden' and caves, saying:
"During the archaic and classical periods, the nymphs were credited with the ability to 'seize' individuals and inspire them. These nympholepts [persons seized by the Nymphs] sometimes withdrew to cave shrines and spent their lives communing with the nymphs and other resident gods. They welcomed visitors and may have acted as prophets. One such nympholept was Archedamus, an immigrant from Thera, who devoted his life to the maintenance of a sanctuary of the nymphs at Vari in Attica (Figure 3.1). At the instruction of the nymphs, he cut stairs, sculptures, and inscriptions into the rock of a cave, and outside it cultivated a garden. At a cave near Pharsalus, Pantalces left a long inscription inviting worshipers to enjoy themselves and take pleasure in the sanctuary. He refers twice to the growing things that he planted, and tells how the nymphs made him an overseer of the place. He lists the resident gods: the nymphs, Pan, and Hermes are mentioned ﬁrst, then a number of gods concerned with the health and nurture of youths: Apollo, Heracles, Chiron, Asclepius, and Hygieia. Archedamus’ and Pantalces’ shared conception of the proper way to honor the nymphs includes the idea that their dwelling is not a temple but a cave, a natural shelter with certain analogies to human structures, but distinct from them. It is permissible, even necessary, for the human worshiper to improve on the natural contours of the cave." 
Divination is attested to cave worship more often. In Minoan times, caves were places one visited to go on a type of vision-quest ritual, whereby the worshipper went into a trance to encounter a deity. In the later Hellenic periods, this type of ritual use seems to have faded, but divination still played a large part in the ritualized function of caves. Pilgrims to the oracle of Delphi brought hundreds of seashells from the Corinthian gulf as gifts for the Nymphs, who were said to share a cave with Pan at the sanctuary. In the cave, knucklebones from sheep and goats were used in an old divinatory tradition called 'astragaloi'. Oracular forms of knowledge were inspired by Apollo and the Muses, and they often came from the bottom of a cave, like at Delphi. There are many other testaments to caves where visitors flocked to the receive prophetic dreams, oracular messages or other divine messages.
Besides an oracular function, caves were also sometimes considered entrances to the Underworld--indeed, the Underworld itself, located under the surface of the Earth, is an immense cave. At Eleusis is a shallow cave, the Cave of Hades, or as it was known then, the Precinct of Plouton. The would be initiates of the Mysteries would have visited this cave as part of their preparation. Most likely, there were sacrifices made here, and perhaps rites of purification. The cave was considered the exact place where Kore was abducted, and for the initiates, it would have been a place to of death, from which only the purified would return to 'live again', like Persephone.
Interestingly enough, many Gods were born in caves: Maia (a Nymph) lived in a cave when she gave birth to Hermes, Zeus was hidden, if not birthed, in a cave on mount Ida (pictured right), where He and His mother Rhea were worshipped, Dionysos spent His infancy in a cave, and I am sure there are more examples. In fact, the cave sanctuary of Eileithyia at Amnissos was wholly dedicated to birth, and the Goddess of childbirth, who was said to have been born in the cave herself was given offerings of honey. Within the cave, there is a large stone resembling a pregnant belly, and a stalagmite resembling a female body. Eileithyia’s cave is mentioned in the Odysseia:
"He anchored at Amnisus, a tricky harbour, near the cave of Eileithyia, and barely escaped shipwreck." [19.188]
Birth and death both opened up passageways to the Underworld, this is why both carried miasma. That both passageways are connected to caves--which can most certainly appear as passageways themselves, as we have seen with the Cave of Hades--does not feel at all like a coincidence to me.
Caves did not always have a religious function: they were also used as places of punishment, and sometimes a tool of capital punishment, where the captured died a slow death by starvation and thirst. This process is documented by Sophocles in 'Antigone', as Creon condemns Antigone to death in exactly such a cave for trying to recover the body of her brother from outside the city walls against Creon's wishes. It is certainly feasible that non-mythological persons were condemned to death in much the same way. From 'Antigone':
"CREON: Know ye not that songs and wailings before death would never cease, if it profited to utter them? Away with her-away! And when ye have enclosed her, according to my word, in her vaulted grave, leave her alone, forlorn-whether she wishes to die, or to live a buried life in such a home. Our hands are clean as touching this maiden. But this is certain-she shall be deprived of her sojourn in the light.
ANTIGONE: Tomb, bridal-chamber, eternal prison in the caverned rock, whither go to find mine own, those many who have perished, and whom Persephone hath received among the dead! Last of all shall I pass thither, and far most miserably of all, before the term of my life is spent. But I cherish good hope that my coming will be welcome to my father, and pleasant to thee, my mother, and welcome, brother, to thee; for, when ye died, with mine own hands I washed and dressed you, and poured drink-offerings at your graves; and now, Polyneices, 'tis for tending thy corpse that I win such recompense as this."
Caves were often focal points of local worship, and not every cave was considered sacred. Some, however, had mythology connected to it, and this made them excellent places for worship. The sober atmosphere of natural caves, coupled with their unique beauty, set them apart from the civilized world. This makes it easier to connect to the spirit of the wilderness, to Nymphs and Pan. The fact that their darkness can hide even the most gruesome of monsters makes them just a touch more dangerous--unless they are dedicated to the Nymphs or Gods, then they are quite safe to enter and worship at; a practice most certainly wide-spread in ancient Hellas.
Image source Cave of Zeus: Wikipedia Commons