In this paper, a new class of robust Kalman filtering problem is tackled for time-varying linear systems. Aside from the conventional problem settings, it is assumed that the measurement matrix be unknown and only a noise corrupted observation of it be available for state estimation. The influence of the noise contaminated measurement matrix on the Kalman filter estimate is analyzed in the sense of classical weighted leastsquares criterion. Stochastic approximations of estimation errors due to noisy measurement matrix make it possible to develop a less conservative robust estimation scheme. Reinterpreting the stochastic error compensation procedure, the less conservative robust Kalman filtering problem is defined as finding a unique minimum of an indefinite quadratic cost. By solving the single stage optimization problem, the robust filter recursion is derived. As well, its existence condition is recursively checked using the estimation error covariance. It is also shown that the proposed filter is consistent in probability. A practical design example related to frequency estimation of noisy sinusoidal signal is given to verify the estimation performance of the proposed scheme.