An embedding-heavy model is a model whose learned lookup tables dominate stored weights and memory. This note assumes Categorical Feature Embeddings in Recommender Systems and answers one question: why can a ranking model with ordinary dense or transformer layers still be mostly embedding tables?
The property belongs to the model shape. It is not saying the model only performs lookups. The dense, MLP, or transformer layers may do the feature interaction work, while the embedding tables contain most of the stored numbers.
Size Arithmetic
One Home feed ranker can use ids for the user, candidate Pin, board, query, topics, and recent actions. Multiple Embedding Tables in Recommender Rankers explains how one candidate row gathers rows from those table families. A dense layer may be compute-heavy because it performs many multiply-adds per example. An id table can be memory-heavy because it stores a row for every entity, even though one example gathers only a few rows.
The arithmetic makes the difference visible:
| Component | Example shape | Stored numbers |
|---|---|---|
| Dense layer | 4096 x 4096 | about 16 million |
| Token vocabulary table | 100k x 4096 | about 410 million |
| Board id table | 200M x 64 | about 12.8 billion |
| Pin id table | 500M x 128 | about 64 billion |
The dense layer can dominate local compute. The Pin table can dominate memory. Both statements can be true in the same model.
Why Rows Change The System Design
The ranking model becomes embedding-heavy when table size controls operational choices:
| Design pressure | Mechanism |
|---|---|
| Serving memory | a published model version must make table rows available for low-latency lookup |
| Training memory | the mutable checkpoint stores table rows plus optimizer bookkeeping |
| Sparse updates | one batch updates only rows touched by that batch |
| Sharding | one GPU may not hold the tables, so rows or whole tables are split across GPUs |
| Communication | a training batch may need vectors whose rows live on other GPUs |
The last row is the bridge to Distributed Training for Embedding-Heavy Recommenders. Table size by itself is a memory problem. Table size plus sharding turns lookup into a tensor-movement problem.
Stored Values Versus Lookup Payload
Two tables can store the same number of values while creating different communication costs. Suppose a batch needs one million lookups:
wide table:
100M rows x 256 dimensions
one lookup returns 256 values
1M lookups return 256M values
narrow table:
200M rows x 128 dimensions
one lookup returns 128 values
1M lookups return 128M valuesThe two tables store the same number of values: 25.6 billion. The narrow table returns half as many values per lookup. That distinction matters when the bottleneck is bytes moved during distributed training, not total stored values.
Three Vector-Like Objects
The recommender notes use several vector-shaped objects. They should not be collapsed into one concept:
| Object | Owner | Question it answers |
|---|---|---|
| Categorical embedding table during training | ranking-model training checkpoint | how should this id row change after labeled examples? |
| Categorical embedding table during serving | published ranking model version | what learned vector should the ranker read for this id? |
| Vector-search index | retrieval system | which item vectors are nearest to this query vector? |
The Pinterest scalability article is mainly about the first row: training a ranking model whose categorical embedding tables are too large and hot to handle with one local device.
Sources
- Pinterest Engineering - Achieving Near-Linear Training Scalability for Pinterest’s Foundation Models
- TorchRec documentation