Approximating decision boundaries of large datasets to classify an unknown sample has been recognized by many researchers within the data mining community as a very promising research topic. The application of polynomial neural networks (PNNs) for the approximation of decision boundaries can be considered as a multiple criteria problem rather than as one involving a single criterion. Classification accuracy and architectural complexity can be thought of as two different conflicting objectives when using PNNs for classification tasks. Using these two metrics as the objectives for finding decision boundaries, this chapter adopts a Discrete Pareto Particle Swarm Optimization (DPPSO) method. DPPSO guides the evolution of the swarm by using the two aforementioned objectives: classification accuracy and architectural complexity. The effectiveness of this method is shown on real life datasets having non-linear class boundaries. Empirical results indicate that the performance of the proposed method is encouraging.