Turbulence modifications due to the presence of particles are investigated using direct numerical simulations of turbulent channel flow. It is shown that a Stokes number plays a key role in the turbulence modification behavior for particles smaller than the Kolomogorov length scale of the fluid. Particles with the Stoke number, St+, of 0.25 in wall units enhance turbulence near the wall, while particles with St+ = 20 attenuate it throughout the channel. This can be explained in terms of energy exchange between the two phases around streamwise vortices. It is found that the low- and high-Stokes-number particles, respectively, contributed to the positive and negative energy production around streamwise vortices.