With the rapid growth of the food delivery industry, there is an urgent need to manage software effectively for sharing economy applications. One way to evaluate the effectiveness of these applications is by examining user concerns and feedback. We propose to use a Bi-LSTM-CNN model in a pipeline for automatic classification of the user concerns. The performances of other machine learning and deep learning models were studied and com-pared. The results showed that the proposed Bi-LSTM-CNN model achieved the highest accuracy score of 84.6%, outperforming the single deep learning models and the traditional machine learning models. Moreover, due to the imbalance nature of the collected data, the impact of data over-sampling technique for data imbalance problem was also evaluated. Inter-estingly, the interplays between the complex representation induced by the proposed Bi-LSTM-CNN model render the selected oversampling scheme e.g., SMOTE, unnecessary for our setting.