As a senior product manager for technical, core shopping data science, you will own how shopping quality is defined, measured, and operationalized across key customer experiences. You will set the quality bar, build a portfolio of inspection metrics, and move measurement from offline reporting into online decision systems.
Role Overview
You will start by shaping quality measurement for the search results page and expand coverage to the Homepage and Detail Page. The role includes building a repeatable audit loop using offline and LLM-based evaluation, then integrating quality signals into online systems and roadmaps so signals are available where experiences are ranked and served.
Responsibilities
- Own the standard that every metric definition, annotation SOP, and automated measurement approach must satisfy before it is trusted and published.
- Own the portfolio of shopping quality metrics against the quality bar, including areas such as relevance, duplication, and brand quality, and expand the portfolio as new defect classes are identified.
- Own how quality defects are found, including sampling strategy, audit cadence, anecdote review, and systematic inspection of pages customers actually saw.
- Convert qualitative input into quantified metrics in partnership with UX Research and leadership, turning customer anecdotes and research findings into defect definitions that can be sampled, scored, and tracked.
- Own the audit loop and SOP bar by specifying what gets audited and the criteria used, reviewing findings, and requiring corrections in annotation SOPs or LLM prompts.
- When a measurement approach misses a defect class, define the corrected standard and drive the team to build to it.
- Drive the agenda to move quality measurement from offline, after-the-fact reporting into online systems, enabling quality signals at the point where experiences are ranked and served.
- Partner with central platform teams to embed quality metrics into online evaluation and serving paths.
- Make the case for how online quality signals change decisions, including experimentation guardrails, faster detection of quality regressions, and closed loop correction of defective experiences.
- Own a multi-year roadmap for shopping quality measurement and inspection, including which defects to measure next, which surfaces to expand to, and what each expansion enables for the business.
- Extend the charter beyond search to additional core shopping pages, starting with Homepage and Detail Page. For surfaces with no measurement today, define the defect taxonomy from scratch, including what a quality defect is, how it is sampled, how it is scored, and how it rolls up.
- Drive alignment across Organic Search, Sponsored Products, Sponsored Brands, and International as definitions evolve, including with teams whose experiences the metrics judge.
Required Qualifications
- 5+ years of product or program management, product marketing, business development or technology experience
- Bachelor's degree
- Experience with feature delivery and tradeoffs of a product
- Experience owning/driving roadmap strategy and definition
- Experience with end to end product delivery
- Experience contributing to engineering discussions around technology decisions and strategy related to a product
- Experience managing technical products or online services
- Experience representing and advocating for a variety of critical customers and stakeholders during executive-level prioritization and planning
Technologies
- Tableau
- Qlikview
- QuickSight
- LLM based evaluation
- LLM prompts
Compensation and Location
Location: Seattle, WA (onsite)
Salary: USD 151,200 - 204,600 per yearly
Benefits
- Health insurance (medical, dental, vision, prescription, Basic Life & AD&D insurance and option for Supplemental life plans, EAP, Mental Health Support, Medical Advice Line, Flexible Spending Accounts, Adoption and Surrogacy Reimbursement coverage)
- 401(k) matching
- Paid time off
- Parental leave
- Sign-on payments and restricted stock units (RSUs)
Preferred Qualifications
- Experience in using analytical tools, such as Tableau, Qlikview, QuickSight
- Experience in building and driving adoption of new tools
About the Team
The Core Shopping Data Science team’s mission is to provide the data-driven foundation for building a world-class shopping experience that maximizes long-term free cash flow by delighting customers. The team focuses on long-term strategy and big-picture trade-offs across the full Amazon shopping experience.
The team empowers feature owners and systems through: (1) developing and vending metrics that assess and value customer engagement, (2) building tools and datasets to inject data into the decision-making process, and (3) delivering deep analyses that inform high-touch decisions.
