In this work we propose an approximate Minimum Mean-Square Error (MMSE) filter for linear dynamic systems with Gaussian Mixture noise. The proposed estimator tracks each component of the Gaussian Mixture (GM) posterior with an individual filter and minimizes the trace of the covariance matrix of the bank of filters, as opposed to minimizing the MSE of individual filters in the commonly used Gaussian sum filter (GSF). Hence, the spread of means in the proposed method is smaller than that of GSF which makes it more robust to removing components. Consequently, lower complexity reduction schemes can be used with the proposed filter without losing estimation accuracy and precision. This is supported through simulations on synthetic data as well as experimental data related to an indoor localization system. Additionally, we show that in two limit cases the state estimation provided by our proposed method converges to that of GSF, and we provide simulation results supporting this in other cases.