A new method of fatigue life prediction for riveted structure is presented. Elastic-plastic theoretical model is used to obtain the distribution of riveted residual stress around riveted hole, and the distribution of stress around riveted hole caused by loading transferring is calculated by 3-D elastic-plastic contact nonlinear finite element method(FEM). The above two stress fields are superimposed. According to the stress value of the point with a certain distance along the radial direction from the fatigue dangerous position where the stress is maximum, the fatigue life prediction is carried out by interpolating the S-N curve of smooth specimen of the referred material. The new method is compared with the stress severity factor (SSF) method, and feasibility and accuracy of the new method are proved by experiments.