Tracking cells over time is crucial in the fields of computer vision and biomedical science. Studying neutrophils and their migratory profile is the highly topical fields in inflammation research due to determining role of these cells during immune responses. As neutrophils generally are of various shapes and motion, it remains challenging to track and describe their behaviours from multi-dimensional microscopy datasets. In this study, we propose a robust novel multi-channel feature learning (MCFL) model inspired by deep learning to extract the complex behaviour of neutrophils moved in time lapse images. In this model, the convolutional neural networks along with cell relocation distance and orientation channels learn the robust significant spatial and temporal features of an individual neutrophil. Additionally, we also proposed a new cell tracking framework to detect and track neutrophils in the original time-laps microscopy images, entails sampling, observation, and visualisation functions. Our proposed cell tracking-based-multi channel feature learning method has remarkable performance in rectifying common cell tracking problem compared with state-of the-art methods.