Recently, a myriad of applications take advantage of deep learning methods to solve regression/classification problems. Although deep neural networks have shown powerful learning capability, many deep learning applications suffer from the extremely time-consuming training of the neural networks. In order to reduce the training time, researchers usually consider parallel training...