The past decade has witnessed significant progress in object detection and tracking in videos. In this paper, we present a model for collaboration between a pre-trained object detector and multiple single object trackers in the particle filter tracking framework. For each frame, we construct an association between the trackers and the detections, and when a tracker is successfully associated to a detection, we treat this detection as the key-sample for this tracker. We present a dual motion model that incorporates the associated detections with the object dynamics. Then, a likelihood function provides different weights for the propagated and the newly created particles, reducing the effect of false positives and missed detections in the tracking process. In addition, we use generative and discriminative appearance models to maximize the appearance variation among the targets. The performance of the proposed algorithm compares favorably with that of the state-of-the-art approaches on three public sequences.