Scaling Conditional Autoencoders for Portfolio Optimization via Uncertainty-Aware Factor Selection

The publication presents a quantitative finance and machine learning study focused on improving portfolio optimization through advanced factor modeling. It builds on conditional autoencoders (CAEs), which map firm characteristics into latent asset-pricing factors, offering a flexible and potentially interpretable representation of systematic risk drivers. Prior work typically constrains the number of latent factors to a small dimension (around K=5), due to concerns that higher-dimensional representations can lead to overfitting, instability, and degraded out-of-sample performance.

To overcome these limitations, the authors propose a scalable framework that combines a high-dimensional CAE architecture with an uncertainty-aware factor selection mechanism. The CAE is trained to learn a rich set of latent factors from cross-sectional firm characteristics, while the selection procedure evaluates factor relevance and associated uncertainty, pruning or down-weighting factors that do not contribute robustly to pricing or portfolio performance. This design aims to retain the benefits of expressive, high-dimensional representations without sacrificing generalization.

The framework is positioned for applications in portfolio construction and risk management, where accurate estimation of latent asset-pricing factors is critical. By systematically handling model uncertainty and factor proliferation, the method seeks to enhance factor-based strategies, improve risk-adjusted returns, and provide more stable portfolios. The work is particularly relevant for quantitative asset managers, investment banks, and researchers developing ML-driven factor models and systematic investment strategies.