In-plane modulation transfer function (MTF) has been widely used as a quantitative metric describing the spatial resolution of an in-plane image for a digital tomosynthesis system. Although the in-plane MTF was measured using fine wire and edge objects, precise phantom alignment along the measurement direction was one of challenging issues. To overcome this limitation, a sphere object was regarded as an alternative phantom because of spherically symmetric property. However, due to anisotropic property of tomosynthesis image, the sphere phantom has not been used to measure the in-plane MTF. In our previous work, we proposed the inverse filtering approach to measure the in-plane MTF using sphere phantoms. Using the inverse filtering approach, in this work, we measure the in-plane MTF of step-and-shoot mode and continuous mode digital tomosynthesis systems. We generated projection data of point and sphere objects in step-and-shoot mode and continuous mode tomosynthesis systems, and reconstructed using FDK algorithm. An in-plane image of reconstructed point volume was regarded as an ideal in-plane point spread function (PSF). The ideal in-plane MTF was calculated by taking Fourier transform of the ideal in-plane PSF, and fx-directional in-plane MTF was used as a reference. To measure fx-directional in-plane MTF, we divided the Fourier transform of reconstructed sphere phantom by that of ideal sphere object, and performed plane integral along the fz-direction. The estimation errors caused by inverse filtering were corrected by pseudo inverse filtering and Laplacian operator. Our results show that the in-plane MTFs of step-and-shoot mode and continuous mode are reliably estimated by the inverse filtering approach.