This article presents a de-speckling technique for clinical ultrasound images with an aim to preserve the fine structural information and region boundaries in images. The algorithm generates restored images by minimizing the variational energy on them. To compute variational energy, a weighted total variation based method is proposed where the weights are determined from both historical (previous/earlier time stamp) as well as instantaneous oriented structural information of images. This helps in defining the anistropy at edges in the image which, in turn, helps in identifying homogenous regions on it. Moreover, the method is able to preserve the vague echo-textural differences which might be of clinical importance but may get destroyed due to smoothing operations. To elicit effectiveness, comparative analysis of the proposed approaches have been done with four state-of-the-art techniques on both in silico and in vivo ultrasound images using four standard measures (two for phantom images and two for clinical ultrasound images). Qualitative and quantitative analysis reveals the promising performance of the proposed technique.