While optimization has received much attention in the machine learning community, most of them consider unconstrained supervised learning models such as neural networks and support vector machine. In this dissertation, we introduce a new class of optimization problems called scale invariant problems that include interesting unsupervised learning models such as...