To avoid and manage stress-related problems, several works have investigated how to recognize stress states of people by exploiting the machine learning. Most of works utilize the supervised learning which requires labels for data to be trained and tested. Thus, labels which represent stress state of each data such as stressed or not become definitely important. Conventional stress state classification methods assign labels to unit data per a certain experimental period by applying stressor appearances or self-evaluation scores. However, those methods ignore temporal variations in stress responses which are involuntarily triggered inside the body within a shorter period of time than an experimental period. Therefore, we propose a stress state classification method by considering not only user's subjective evaluations but also temporal changes of stress responses in short periods. For the demonstration, we label our experimental data of 40 subjects by using our proposed classification method and conventional ones, respectively. Then, we train and test stress recognition models with 6 machine learning algorithms and our implemented neural network ones based on the labeled data. Finally, binary stress recognition with our proposed classification method improves the recognition accuracy by up to 31.6% as compared to those with conventional techniques.