In this paper, a new object manipulation method is proposed by applying the ray's depth information to the raycasting- based interface. Ray-casting has been widely used in handheld device interfaces for manipulating objects in threedimensional (3D) space, such as augmented reality or virtual reality environments. However, the traditional ray-casting interface has limited object manipulation because it cannot designate a specific location in 3D space; it also has poor accuracy in manipulating objects at far distances. In this study, a user could register the virtual object in the real 3D space by specifying the arbitrary position using ray-depth information. We also improved the virtual object manipulation accuracy of the ray-casting interface by compensating the sensitivity according to the distance between the user and the object. To verify the manipulation accuracy of the interface implemented with the proposed technology, this study conducted a comparative experiment with existing interfaces. This verified the performance improvement in 3D space.