Skip to main content


Getting in the groove: evidence of chemical communication in the primate fossil record


Eva Garrett and her colleagues have discovered a way to determine if extinct primates used their sense of smell or of sight as the primary means to observe their environment. From the earliest primates of 65 million years ago (Ma) to modern humans, there has been a change in sensory emphasis from smell to sight along many branches of the evolutionary tree. Despite a good fossil record of our primate relatives (such as apes, monkeys, lemurs and other forms), until now it has not been possible to trace this “tradeoff” between olfaction and vision. Eva located a tiny groove in the bone forming the floor of the nasal cavity which in life received part of the apparatus for olfaction and which can be discerned not only in the bony remains of modern species but also in fossils.

Eva Garrett is a Ph.D. student in Physical Anthropology at the City University of New York (CUNY) Graduate School and a Trainee in the New York Consortium in Evolutionary Primatology (NYCEP), a multiinstitutional doctoral training program funded by NSF’s IGERT (Integrative Graduate Education and Research Traineeship) Program, of which the PI is Eva’s advisor, Eric Delson (Lehman College/CUNY). Eva also received a dissertation improvement grant from NSF’s Physical Anthropology program. In line with IGERT’s emphasis on interdisciplinary research, Eva’s study combined analysis based on histology, CT-scanning, visual observation of comparative morphology and paleontology.

It has long been recognized that among extant primates, there is great variation in the development of aspects of the visual and olfactory senses. Olfaction is presumed to be the ancestral or “primitive” condition, as it is found widely among mammals and is controlled by a more ancient part of the brain. The main olfactory system in mammals detects volatile stimuli by way of olfactory receptors in the nose and processes information relating to diet, activity pattern and habitat. The vomeronasal system (VNS) appears to be specialized for the detection of non-volatile chemical stimuli (pheromones) that play a large role in sociosexual communication. The vomeronasal organ (VNO) is the main receptor of such stimuli. It is composed of paired tubes of soft tissue surrounded by a cartilaginous vomeronasal capsule (VNC) that sits in the shallow vomeronasal groove (VNG) located on the floor of the nasal cavity (the bottom side of this layer of bone forms the hard palate).

Olfaction, especially the VNS, has been reduced in primate evolution in favor of the visual sense, as evidenced by its distribution in living forms. The “lower primates” or strepsirhines (lemurs and lorises) have a fully functioning VNS but lack well-developed visual acuity and exaggerated visual signals. This group represents one of the two major branches of the modern primate evolutionary tree, the other branch being the anthropoids (“higher primates”) and the tarsier. Although tarsiers look like strepsirhines, they are evolutionarily closer to anthropoids in terms of their genomes, external noses, and placentation. The anthropoids in turn are divided into New World (South American) monkeys on the one hand, and Old World (African and Eurasian) monkeys and hominoids (apes and humans) on the other (Figure 1). Tarsiers and New World monkeys maintain a functional VNS that is reduced compared to strepsirhines but varies between genera (groups of species). Old World monkeys and apes (including humans) possess well-developed visual acuity, routine trichromatic vision, and complex visual signals but they lack a functional vomeronasal system (VNS). Apes retain a vestigial VNS that does not seem to function in pheromone detection. Old World monkeys are the most “specialized”, completely lacking a VNS.

It has been hypothesized that there was a sensory “tradeoff” as higher primates evolved their visual acuity and reduced dependence on smell. Research on the evolution of this trend is limited by the relative lack of information on the VNS compared to the visual system, especially when it comes to the fossil record. Thus, Eva Garrett’s observation of the VNG in the museum skulls of lower primates with a well-developed VNS represented a breakthrough in allowing recognition of this system in non-living individuals. Through collaboration with colleagues such as Anne Burrows (Duquesne U) and Tim Smith (Slippery Rock U), Eva has examined histological sections of the nasal region in modern primates, observing the VNG where the VNC meets the floor of the nasal cavity (Figure 2). Moreover, CT-scans of modern primate skulls (mostly provided by James Rossie of Stony Brook U), also clearly showed the VNG, which could be measured and compared to the size of the VNC and VNO. The VNG occurs in all strepsirhines and New World monkeys examined so far, but not in the single anthropoid studied, a gibbon.

Broader sampling will be carried out over the coming year. In addition, the fact that the VNG is preserved in bone means that it can be located in fossils. A CT scan of an archaic primate (plesiadapiform) dating to 54 Ma revealed a VNG, supporting the idea that this system was present in the earliest primates. Scans of a series of younger fossils will be examined to determine when and in which groups the VNO became reduced or was lost completely. Eva hypothesizes that it will be present in early anthropoids around 35-30 Ma but may begin to reduce after that. This will also allow inference about the behavior patterns of these extinct primates, given the ways vomerolfaction relates to social-sexual signaling. The combination of techniques and data from comparative morphology, paleontology, histology and social behavior yields results unavailable to individual disciplinary studies. This research was presented as a poster at the May 2010 annual IGERT meeting and was awarded the prize for best poster in the area of sensing, signals and signal processing.

Address Goals

This research opens a new way of looking at primate evolution and tracking the development of the major senses, olfaction and vision. Given that modern primates emphasize one or the other of these, if the olfactory system in reduced in some fossil primates, it is reasonable to infer that the visual system has been emphasized instead. Further understanding of primate evolution impacts directly on better comprehension of our own place in nature and may throw light on human reactions to pheromones and other chemical stimuli. The general public is always fascinated by new evidence of human (and primate) evolutionary history, and this work will be disseminated widely, to both scholarly and public audiences.