Biometrics is likely to provide a new level of security to various applications. Yet if the stored biometric template is compromised, invasion of user privacy could occur. Since biometric is irreplaceable and irrevocable, such an invasion often implies a permanent loss of identity. In this paper, a non-invertible Graph-based Hamming Embedding (GHE) technique is proposed to secure the minutia descriptors. The proposed technique initially employs Minutiae Vicinity Decomposition (MVD) to derive a set of geometrical invariant features from a set of fingerprint minutiae, and then globally embeds the MVD neighborhood structure into a Hamming space via solving a graph-based optimization problem. As a result, a secure binary template is generated. The resultant binary template enjoys three merits: 1) strong concealment of the minutia vicinity, which effectively protects the location and orientation of minutiae and ensures non-invertibility of the template. 2) improved or well preserved discriminability of descriptors in the Hamming space with respect to the Euclidean space. 3) quick matching due to pure involvement of bit-wise operations. Promising experimental results on FVC2002 database vindicate the feasibility of the proposed technique.