Pinterest – Developer Tooling That Scaled Company-Wide
S: 2023 | Staff Eng. Infra Gov. Spark-based dev workflow had 5-min minimum latency per iteration; engineers avoided running tests locally. All development required cluster access. No way to iterate quickly on dbt models.
T: Unblock team velocity without waiting for Data Platform team’s roadmap.
A1 – Presto Adapter: Built custom dbt adapter wrapping the official dbt-presto adapter to work with Pinterest’s custom authentication layer. Enabled dev/test on production-like data in seconds vs 5-min Spark latency. Engineers could materialize in personal test catalogs, run unit tests, and promote to prod.
A2 – DuckDB Local Env: Wrote sampling scripts to create a local DuckDB database from production data. Engineers could run dbt models and tests completely offline on their workstations.
A3 – Tag Workflows: Designed tag-based project structure so multiple independent platforms could be managed as a single dbt project but tested as isolated subsets.
A4 – Handoff: Documented and shared tooling with other teams; didn’t gatekeep.
R: ~100× faster feedback loops; adopted org-wide beyond my team. Changed how engineers approached safe change—testing went from avoided to default behavior.
TRAITS: Initiative • Efficiency • Platform Thinking (Q 10 19 20 38)