Exploiting feature engineering to improve ML: a review of different data processing techniques

This study delves into the critical realm of feature engineering, examining its influence on the performance of various machine learning classifiers under different data preprocessing conditions. Our research encompasses an array of algorithms, including Support Vector Machine (SVM), Random Forest, Decision Trees, Artificial Neural Networks (ANN), and K-Nearest Neighbors (KNN). A key aspect of our methodology is the employment of Grid Search to systematically explore a wide range of hyperparameter configurations for each classifier. Focusing on handling missing values, outliers, and data scal-ing, we assess the impact of these feature engineering techniques on model performance. The evaluation is based on metrics such as accuracy, recall, precision, and F1-score, providing a comprehensive comparative analysis. The results offer insightful revelations about the differential impacts of fea-ture engineering strategies on each algorithm’s predictive accuracy and over-all effectiveness. In particular, the study highlights how specific algorithms, notably Random Forest demonstrate robust performance across varied data scenarios. These findings contribute significantly to the understanding of feature engineering in machine learning, offering practical guidance for se-lecting and optimizing data preprocessing methods. This research under-scores the paramount importance of fine-tuning feature engineering approaches and lays a foundation for future advancements in optimizing ma-chine learning models.