Nvidia published a comprehensive guide on how generative recommenders (GRs) are redefining recommendation systems at scale (Nvidia blog). The shift moves away from traditional embedding-similarity-based objectives toward a generative approach where the goal is to predict the next action or item in a large catalog given a sequence of user histories, similar to how language models predict the next token (Nvidia blog).
Traditional recommender systems face critical scaling challenges: user-history data can reach terabytes or petabytes daily, far exceeding GPU memory capacity; the long-tail problem causes sparse training signals for niche items; cold-start issues degrade recommendations for new users and items; and strict latency requirements demand millisecond-level inference on millions of concurrent users (Nvidia blog). Generative recommenders address these by reframing recommendation as sequence modeling using transformer-like architectures that better leverage scaling laws and can unify retrieval and ranking within a single model (Nvidia blog).
Nvidia's recsys-examples repository provides optimized implementations of two prevalent GR approaches: Hierarchical Sequential Transduction Units (HSTU), introduced by Meta in 2024, and Semantic IDs, introduced by Google, which use hierarchical clustering to reduce the item vocabulary space and enable efficient autoregressive decoding (Nvidia blog). The framework includes DynamicEmb, a GPU-optimized hash table that allocates embedding rows on demand rather than pre-provisioning static tables, enabling the system to handle continuously growing catalogs. In production benchmarks, the HSTU implementation on two DGX H100 nodes achieved 31.40% Model FLOP Utilization compared to 7.65% previously, and inference with PyTorch AOTI backend and KV cache delivered 2.20x to 2.38x speedup in GPU cache-hit scenarios (Nvidia blog).