Alzheimer's Disease (AD) is an irreversible disease that gradually worsens with time. Therefore, early diagnosis of Alzheimer's disease is important to prevent brain tissue damage and treat the patient properly. Mild Cognitive Impairment (MCI) is a prodromal stage of AD, which has no harm to the patient's ability to have functional activities in daily life except a minor cognitive deficiency. Since MCI can be detected at the earliest stage of AD, it is critical to detect patients with MCI to delay the progression of AD. It is possible to distinguish patients with AD, MCI, and Normal Control (NC) from one another by the size of brain volume, hippocampus and patient's clinical information. The brain and hippocampus gradually shrink in size and shape as AD develops. In this study, we propose a deep learning-based technique to classify patients with AD, MCI and NC by brain Magnetic Resonance (MR) images. Deep learning has shown human-level performance in a lot of studies including medical image analysis with constrained amount of training data. We propose a deep learning-based ensemble model which consists of 3 Convolutional Neural Networks (CNN)  with Network In Network (NIN)  architecture. The kernel size is 3x3 convolution followed by 1x1 convolution to reduce the number of trainable parameters and extract features for classification better. In addition, Global Averaging Pooling (GAP) is used instead of Fully-Connected (FC) layers to avoid overfitting by reducing the number of trainable parameters. By using the ensemble model, this shows the 81.66% in classifying 3 classes.