Elderly people are characterized with high needs for healthcare, accompanied by high barriers in access to healthcare. This study aimed to identify temporal changes in access to healthcare and determinants of such changes from the elderly in China, over the period between 2005 and 2014. Two waves (2005 and 2014) of data were extracted from the Chinese Longitudinal Healthy Longevity Survey (CLHLS), measuring changes in perceived accessibility to healthcare when needed by the elderly (≥65 years). The effects of the explanatory variables (need, predisposing and enabling factors) on the changes were divided into two components using the Oaxaca–Blinder decomposition method: (1) the endowment portion as a result of distribution differences of the explanatory variables and (2) the coefficient portion as a result of differential responses of the dependent variable to the explanatory variables. Perceived accessibility to healthcare from the elderly increased from 89.6% in 2005 to 96.7% in 2014. The coefficient portion (82%) contributed more to the change than the endowment portion (63%) after adjustments for a negative interaction effect (−45%) between the two. Lower perceived accessibility was associated with older age, lower income, lower affordability of daily expenses and lower insurance coverage. But the coefficient effects suggested that their impacts on perceived accessibility to healthcare declined over time. By contrast, the impacts of gender and out-of-pocket payment ratio for medical care on perceived accessibility to healthcare increased over time. Perceived accessibility to healthcare from the elderly improved between 2005 and 2014. Gender gaps are closing. But the increased effect of out-of-pocket medical payments on perceived accessibility to healthcare deserves further investigation and policy interventions.