Machine Learning Model Shapes

Machine learning model shapes describe the contract around a model: what one example contains, what the model returns, which training objective gives that output meaning, and how a serving system consumes the result.

The same architecture can serve different contracts. A Transformer can be a classifier, an embedding model, a language model, or part of a recommender ranker depending on the output head and the objective used during training. Conversely, the same task can use different architectures: a spam classifier can be logistic regression, XGBoost, or a transformer with a classification head.

Prediction Contract

A model description is easier to parse when these four pieces stay separate:

Piece	Question it answers	Example
Task shape	What does one prediction mean?	classify one message, score one candidate, generate the next token
Architecture	How are inputs transformed into outputs?	tree ensemble, two-tower encoder, transformer, embedding-heavy ranker
Objective	Which error does training reduce?	Cross-entropy loss, contrastive loss, ranking loss, mean squared error
Serving wrapper	What system uses the output?	fraud decision, vector search index, feed composer, autoregressive generation loop

Training ties the pieces together:

training example
  -> model parameters and architecture produce a prediction
  -> objective compares prediction with label or target
  -> gradients update trainable parameter state

The trainable parameter state can include dense-layer matrices, biases, attention weights, normalization weights, and embedding tables. An embedding table is not the whole model by default; it is one trainable array inside some models.

Common Shapes

Read the table by output meaning rather than by library class name.

Model shape	One input example	Output	Common objective	Serving use
Classifier	facts about one item, event, user, request, or document	class probability or label	cross-entropy	spam, fraud, policy, intent, churn
Regressor	facts about one item, event, user, or time window	numeric value	mean squared error, mean absolute error, quantile loss	price, demand, risk, latency, spend
Ranker	one `(request, candidate)` row	score for that candidate	pointwise, pairwise, or listwise ranking loss	feeds, search, ads, related items
Embedding model	text, image, item, user, or query	fixed-length vector	contrastive or metric-learning loss	retrieval, clustering, deduplication, recommendations
Language model	token prefix	probability distribution for the next token	next-token prediction, usually cross-entropy	chat, generation, coding, summarization
Sequence model	ordered tokens, events, or measurements	one label per element or one future sequence	task-dependent	tagging, forecasting, event prediction

Classifier

A classifier maps one example to class probabilities. A binary classifier might answer “is this message spam?” A multiclass classifier might answer “which intent does this query express?”

The simplest neural classifier shape is:

features -> encoder -> logits -> probability distribution

logits are raw, unnormalized scores. A binary classifier often applies a sigmoid to one logit. A multiclass classifier usually applies softmax to one logit per class. Training commonly uses Cross-entropy loss, which penalizes the model according to how little probability it assigns to the correct class.

A transformer-based classifier keeps the same prediction contract but changes the encoder:

text
  -> tokens
  -> transformer encoder or decoder hidden states
  -> pooled representation or selected token state
  -> classification head
  -> class probabilities

The transformer is the architecture. The classification head and labeled examples make the deployed model a classifier.

Regressor

A regressor maps one example to a numeric value. The target might be tomorrow’s demand, expected delivery time, default probability converted into a risk score, or the fair value of an asset.

features -> model -> number

The output has units. That makes regression different from classification even when the architecture is the same. A model predicting delivery_minutes = 37.4 needs a loss that measures numeric error, such as mean squared error, mean absolute error, or a quantile loss when the system cares about a percentile rather than the mean.

Tree ensembles, linear models, and neural networks are all common regressors. The model shape is defined by the numeric prediction contract, not by the internal architecture.

Ranker

A ranker scores candidates for one request. In a recommender, search, or ads system, upstream retrieval first produces a bounded candidate set. The ranker then receives one row per (request, candidate) pair and emits scores used to order or filter the response.

request = {user_id: user_42, surface: home}
candidates = [pin_77, pin_91, pin_123]
 
ranker rows:
  (request, pin_77)  -> p(save)=0.18, p(hide)=0.02
  (request, pin_91)  -> p(save)=0.07, p(hide)=0.01
  (request, pin_123) -> p(save)=0.11, p(hide)=0.04

The scores are per candidate. A feed composer, search system, or ad auction may then apply policy, diversity, freshness, budget, or layout constraints. The ranker provides model evidence; the serving system builds the final response.

Ranking objectives come in several shapes. Pointwise training treats each candidate row as a labeled example, such as clicked or not clicked. Pairwise training teaches the model that one candidate should score above another. Listwise training optimizes an ordered list or a list-level metric more directly. See Recommendation Ranking Models for the recommender serving path.

Embedding Model

An embedding model maps an object to a fixed-length vector so another system can compare objects by distance or similarity. The input might be text, an image, a product, a user profile, or a search query.

query text -> embedding model -> query_vector
documents  -> embedding model -> document_vectors
query_vector + vector index -> nearest document vectors

Embedding models are usually trained so related objects land near each other and unrelated objects land farther apart. Contrastive objectives express that pressure with positive pairs and negative pairs: (query, relevant_document) should be closer than (query, unrelated_document).

The fixed vector is the model output. That differs from a categorical feature embedding table inside a ranker. A ranker table answers “what learned row should this exact id use inside this model?” A standalone embedding model answers “where should this object live in a vector space used by retrieval or similarity search?” See Embedding Models vs LLMs and Vector Search and Vector Databases.

Language Model

A language model assigns probabilities to token sequences. Modern LLMs usually run a decoder-only transformer that repeatedly predicts the next token from the previous tokens.

prompt tokens
  -> decoder-only transformer
  -> vocabulary logits
  -> next-token probabilities
  -> sampled or selected token
  -> append token and repeat

The training objective is Next Token Prediction: given a prefix, assign high probability to the actual next token from the training text. This is usually implemented with cross-entropy over the vocabulary at each token position.

Serving is different from ordinary classification because generation is an autoregressive loop. The model does not emit the whole answer in one forward pass. It emits one next-token distribution, selects a token, appends it to the context, and runs again. See Autoregression and Introduction to Large Language Models.

Sequence Model

A sequence model consumes ordered inputs and produces an output that depends on order. The output can be one label per element, one summary label for the whole sequence, or a forecast of future elements.

tokens:    [John, works, at, Stripe]
labels:    [PER,  O,     O,  ORG]
 
events:    [view, click, save, hide]
prediction: next likely event or final user-state vector

Transformers, recurrent networks, temporal convolutional networks, and state-space models can all serve this shape. The architecture decides how order and context are represented. The task decides whether the model predicts tags, future values, missing elements, or a whole-sequence label.

Architectures Reused Across Shapes

Architecture names describe internal computation. They do not by themselves tell you what the model is for.

Architecture	Usual input shape	Common model shapes built from it
Linear model	numeric feature vector	classifier, regressor, ranker head
Tree ensemble	tabular feature vector	classifier, regressor, ranker
Two-tower encoder	two objects encoded separately	embedding model, retrieval model, recommender candidate generator
Encoder transformer	token sequence or multimodal sequence	classifier, embedding model, sequence labeler
Decoder-only transformer	token prefix	language model, generative policy, classifier through prompting or a head
Embedding-heavy recommender	categorical ids plus dense features	ranker, multi-task classifier, candidate scorer

The output head often turns a reusable architecture into a task-specific model. A transformer body can feed a classification head, a contrastive projection head, a token-generation head, or a ranking head.

Training Objectives

A training objective tells the optimizer what error to reduce. It is the reason the same architecture can learn different behavior.

Objective	Prediction shape	What it teaches
Cross-entropy loss	probability over classes or tokens	put more probability mass on the observed class or token
Mean squared error	numeric value	reduce large numeric misses more aggressively than small misses
Mean absolute error	numeric value	reduce absolute miss size with less sensitivity to outliers than MSE
Contrastive loss	vector similarity	pull matching pairs together and push non-matching pairs apart
Ranking loss	candidate scores	place better candidates above worse candidates
Next Token Prediction	next-token distribution	predict the next token from prior tokens

The objective must match the serving contract. A vector-search embedding model needs a vector space where distance means something. A classifier needs calibrated or at least orderable probabilities. A ranker needs candidate scores whose ordering improves the product surface. A language model needs token probabilities that can drive generation.

Recommender Systems Use Several Shapes

A recommender system usually contains several model shapes rather than one monolithic “recommender model.”

Stage	Typical model shape	Output
Retrieval	embedding model, graph model, approximate nearest-neighbor index, rule source	candidate ids
Feature assembly	not a model by itself	candidate rows
Ranking	ranker or multi-task classifier	scores per candidate
Composition	policy and product logic, sometimes learned	final ordered response
Training data generation	logging and labeling pipeline	examples and labels for future models

The Pinterest Foundation Ranking Model material sits in the ranking row. Recommendation Ranking Models explains the serving contract. Categorical Feature Embeddings in Recommender Systems explains why ids inside candidate rows become lookup-table rows. Embedding-Heavy Models explains why those tables dominate model state. The distributed-training notes start after that model shape is clear.

Reading Model Descriptions

When a paper, vendor page, or design doc names a model, parse it with these questions:

What is one input example?
What does the model output mean?
Which objective trained that output?
Which architecture produces it?
Which serving system consumes it?

The answer prevents common category mistakes: treating every transformer as an LLM, treating every vector as a retrieval embedding, treating every recommender model as a ranker, or treating a model score as the final product decision.

Edmondo's Vault

Explorer