Derivative-free optimization (DFO) has received growing attention due to important problems arising in practice. Various research communities, ranging from machine learning to engineering design, have adopted distinct DFO methods. In this thesis, we present extensive studies as a meaningful step towards a comprehensive understanding of DFO methods. We study the...