Gradient-based numerical optimization of complex engineering designs offers the promise of rapidly producing better designs. However, such methods generally assume that the objective function and constraint functions are continuous, smooth, and defined everywhere. Unfortunately, realistic simulators tend to violate these assumptions, making optimization unreliable. Several decisions that need to be made in setting up an optimization, such as the choice of a starting prototype and the choice of a formulation of the search space, can make a difference in the reliability of the optimization. Machine learning can improve gradient-based methods by making these choices based on the results of previous optimizations. This paper demonstrates this idea by using machine learning for four parts of the optimization setup problem: selecting a starting prototype from a database of prototypes, synthesizing a new starting prototype, predicting which design goals are achievable, and selecting a formulation of the search space. We use standard tree-induction algorithms (C4.5 and CART). We present results in two realistic engineering domains: racing yachts and supersonic aircraft. Our experimental results show that using inductive learning to make setup decisions improves both the speed and the reliability of design optimization.