Vibration-based quality monitoring of manufactured components often employs pattern recognition methods. Albeit developing several classification methods, they usually provide high accuracy for specific types of datasets, but not for general cases. In this paper, this issue has been addressed by developing a novel ensemble classifier based on the Dempster-Shafer theory of evidence. To deal with conflicting evidences, three remedies are proposed prior to combination: (i) selection of proper classifiers by evaluating the relevancy between the predicted and target outputs, (ii) devising an optimization method to minimize the distance between the predicted and target outputs, (iii) utilizing five different weighting factors, including a new one, to enhance the fusion performance. The effectiveness of the proposed framework is validated by its application to 15 UCI and KEEL machine learning datasets. It is then applied to two vibration-based datasets to detect defected samples: one synthetic dataset generated from the finite element model of a dogbone cylinder, and one real experimental dataset generated by collecting broadband vibrational response of polycrystalline Nickel alloy first-stage turbine blades. The investigation is made through statistical analysis in presence of different levels of noise-to-signal ratio. Comparing the results with those of four state-of-the-art fusion techniques reveals the good performance of the proposed ensemble method.