As people age, structural as well as neural degeneration occurs throughout the auditory system. Many older adults experience difficulty in understanding speech especially in adverse listening conditions although they can hear speech sounds. According to a report of the Committee on Hearing and Bioacoustics and Biomechanics of the National Research Council, peripheral, central-auditory, and cognitive systems have long been considered major factors affecting the understanding of speech. The present study aims to review 1) age-related changes in the peripheral, central-auditory, and cognitive systems, 2) the resulting decline in the understanding of speech, and 3) the clinical implication for audiologic rehabilitation of older adults. Once the factors affecting the understanding of speech in older adults are identified and the characteristics of age-related speech understanding difficulties are examined, clinical management could be developed for prevention and treatment. Future research about problems related to the understanding of speech in older adults will help to improve the quality of life in the elderly.