A systematic design strategy for spiral-wound gas separation systems is studied using a recently proposed algebraic permeator model. Nonlinear programming (NLP) is used to determine operating conditions which satisfy the separation requirements while minimizing the annual process cost. The design method is applied to the separation of CO2/CH4 mixtures in natural gas treatment and enhanced oil recovery application. It is shown that a two-stage configuration with permeate recycle and a three-stage configuration with residue recycle are suitable for natural gas treatment, while a three-stage configuration with both permeate and residue recycle is appropriate for enhanced oil recovery. Parameter sensitivities are analyzed by changing operating conditions, membrane properties, and economic parameters. The optimization procedure is sufficiently robust to handle multi-stage configurations with very demanding separation requirements. The proposed NLP design method facilitates the development of mixed-integer nonlinear programming design strategies which allow simultaneous optimization of the permeator configuration and operating conditions.