This paper presents a unified methodology to studying optimal continuous-thrust formation flying maneuvers by employing the generating functions appearing in the theory of Hamiltonian dynamics. It begins by defining an optimal control problem subject to a relative dynamical system with respect to a known reference trajectory, and applying the Pontryagin principle to compose a two point boundary value problem for a standard Hamiltonian system. Then, as the Hamiltonian phase flow is viewed as a canonical transformation, the associated generating functions are obtained in the form of series expansion. This enables us to determine the initial adjoints and the optimal cost as algebraic functions of initial and final states, and to develop the optimal control in feedback form. This analytic nature of our approach provides a more computationally tractable procedure than the conventional shooting technique, especially when it is necessary to analyze multiple spacecraft maneuvers for substantially many varying boundary conditions and time spans. Furthermore, compared with the linear solution, our higher order solution shows better convergence to the nonlinear reference solution both temporally and spatially. These favorable properties are demonstrated by applying the proposed methodology to a fuel-optimal formation reconfiguration maneuvers near the L2 Lagrangian point in the frame of Hill three-body problem. Finally, our continuous-thrust optimal solutions are compared qualitatively with the impulsive-thrust solutions developed by Guibout and Scheeres.