How to improve the classification accuracy of minority instances is one of the hot topics in machine learning research. In order to solve the problem of imbalanced data classification， a penalized AdaBoost algorithm based on cost sensitivity is proposed. In the penalized Adaboost algorithm， a new adaptive cost sensitive function is introduced， which gives higher cost value to the minority instances and the misclassified minority instances. It can obtain a larger average margin by introducing penalty mechanism. The weighted support vector machine （SVM） optimization model is used as the base classifier. The stochastic variance reduced gradient （SVRG） with variance reduction method is used to solve the optimization model. The comparative experiments show that the proposed algorithm is not only superior to other algorithms in terms of geometric-mean（G-mean） and area under ROC curve（AUC）， but also can obtain a larger average margin by introducing penalty mechanism， which fully demonstrates the effectiveness of the proposed algorithm in handling imbalanced data classification problems.