Based on the continuum damage mechanics, the impact local damage and consequent fatigue life of 2A12 aluminum alloy are investigated. Firstly, finite element simulation of impact is conducted to obtain the local residual stress and plastic strain fields around the impact pit. Then the dangerous points are selected according to the residual stress state, and the initial damages are obtained based on the Lemaitre’s plastic damage model. Finally, the fatigue life of the impacted specimen is calculated using the modified Chaboche’s fatigue damage model. Compared with experimental data, the combination of continuum damage mechanics and finite element method(FEM) is proven feasible in evaluating the impact damage and consequent fatigue life of metal materials or structures.