The dynamical problems of the asteroid defense are investigated by using the impact method. The polyhedron model isused to build the shape model of the asteroids, and the rubble-pile model is used to build the structure model of the asteroids. A sphere which has the same dense and material quality with the asteroid is assumed to impact to the asteroid, and the high velocity impact and the distribution of the rocks and rubbles after the impact are computed. Considering the contact deformation between the asteroid and the impact sphere, and the contact deformation inside the body of the asteroid among the rocks and rubbles which comprise the structure of the asteroid, the mutual gravitational force, the normal contact force, the tangential static frictional force, the tangential kinetic friction force, and the moment of rolling friction are calculated. Asteroid 101955 Bennu is used to calculate the dynamical behaviors of the impact defense of the potentially hazardous asteroids. The results show that the method of high velocity impact is useful for the defense of potentially hazardous asteroids, the asteroids can be crashed into a large number of rocks and rubble. The presented methodhas no pollution to the space environment. In this sense, the method is much better than the nuclear explosion method.