Product Manager, AI Evaluation Lead

Ready to make a tangible impact on global industries using cutting-edge technology? Trimble is looking for a passionate problem-solver like you to help us transform the way the world works! At Document Crunch, the quality of our AI is the product - and we're looking for someone to own how we prove and improve it.

Trimble is a global technology company that connects the physical and digital worlds, transforming the ways work gets done. With relentless innovation in precise positioning, modeling and data analytics, Trimble enables essential industries including construction, geospatial and transportation. Whether it's helping customers build and maintain infrastructure, design and construct buildings, optimize global supply chains or map the world, Trimble is at the forefront, driving productivity and progress.

Document Crunch is a premier AI-powered contract intelligence platform for the construction industry. By simplifying complex construction documents, Document Crunch empowers project teams to understand their risks and manage contracts with confidence. As part of the Trimble ecosystem, this technology serves as "intelligent DNA," accelerating AI adoption to create a future of zero project disputes and increased transparency across the AECO industry.

As Document Crunch scales and agentic capabilities spread across the platform, the volume and variety of AI evaluation we need is growing far faster than a small team can supply. This role exists to close that gap by an order of magnitude - not by doing all the evaluation, but by building the system and the network that multiply our capacity. You will be the single accountable owner of our AI evaluation program: defining the methodology and tooling, standing up the data sets and ground truth we measure against, and building and motivating a flexible network of subject matter experts from across Document Crunch, the broader Trimble organization, and beyond.

The coverage challenge is real: evaluation has to span an expanding set of construction workflows across project types, phases, and user personas. You won't need to be a construction expert yourself - you'll need to design a great evaluation process, build the systems and partnerships that scale it, and pull the right expertise in at the right time. Done well, this becomes a durable competitive advantage for Document Crunch - and a model for how evaluation could work across other Trimble product areas.

• Own AI evaluation end to end as a dedicated program, with the mandate to create a defined, repeatable, and scalable system ("Eval Ops").

• Design the evaluation methodology - test sets and benchmarks spanning the dimensions that matter for our product (accuracy/factuality, instruction-following, robustness, multi-turn conversation behavior, and domain-specific edge cases), plus regression testing to catch when model or prompt changes break previously working behavior.

• Build canonical evaluation data sets - reference projects spanning varied size, complexity, document volume, and project phase (preconstruction, execution, closeout), each with proportionate question/query banks.

• Define rubrics and own ground truth - author clear, anchored scoring rubrics and calibration sets, and stand up the human-validation workflow that confirms whether answers are actually correct and keeps reviewers aligned.

• Stand up a tiered, scalable eval pipeline - combine automated evaluation (e.g., LLM-as-a-judge) for fast iteration with expert human review for calibration and ground truth, so scarce SME time is spent where it matters most.

• Design the eval workbench and contributor workflow - define the end-to-end tooling and process SMEs use to contribute (building on our existing internal eval tooling), so the system runs efficiently at scale.

• Build and coordinate a flexible SME network - recruit, vet, onboard, schedule, and quality-control subject matter experts: Trimble SMEs where available, Document Crunch SMEs and customers where appropriate, and external/contract experts as needed.

• Design the incentive model for the network - determine what motivates busy experts (especially Trimble SMEs) to lend scarce time, and build the structure that attracts and retains them.

• Partner with Product and Engineering/QA - work with product managers to understand the features being built and the evals they require, and partner with engineering (including our existing eval tooling) to operationalize the process.

• Establish a customer feedback loop - over time, incorporate real customer queries and chat history into evaluation data sets rather than relying solely on synthesized or SME-created data.

• Measure and report quality - define metrics and dashboards that track evaluation coverage and answer quality over time, and help the company articulate, internally and to increasingly sophisticated customers, how we know our AI is high quality.

• Demonstrated program and process ownership - a track record of taking an ambiguous, cross-functional problem, defining a process, rallying people around it, and driving it to measurable outcomes.

• Strong people- and network-coordination skills - proven ability to navigate a large organization, recruit and motivate contributors, and orchestrate distributed work across many stakeholders. This is the heart of the role.

• Network-building and incentive design - instinct for building a distributed contributor/expert community and designing incentives that attract and retain scarce expertise.

• Hands-on fluency with AI / LLM evaluation - not just familiarity. You should be able to describe and write an eval and articulate what good eval coverage looks like, and have enough depth to design the system to requirements. You can lean on our engineering/QA and SME partners, but you need enough fluency to know what to ask them. Comfort with ground truth, rubrics, test sets, regression testing, LLM-as-a-judge, and human-in-the-loop review is expected.

• Excellent communication and stakeholder management across Product, Engineering/QA, and SMEs; able to translate between technical and domain audiences.

• Strong project- and resource-management skills, comfortable with the ebb-and-flow of evaluation work (a heavier ramp to establish a baseline, then flexing with demand).

• Detail-oriented and quality-obsessed, with sound judgment about coverage, prioritization, and when "scoped-down but shipped" beats "comprehensive but stalled."

• Technical fluency - enough comfort with tooling to partner with engineering on (or extend) an internal evaluation workbench; light scripting or "vibe-coding" is a plus, not a requirement.

• Experience building or operating an expert network, annotation/labeling program, or human-feedback pipeline (e.g., RLHF-style review or a quality-operations function).

• Exposure to construction, AECO, or construction technology - enough domain awareness to queue work intelligently (deep construction expertise is not required; we can source it).

• Experience in B2B SaaS or AI product environments.

• A track record designing metrics and dashboards to track evaluation coverage and quality over time.

• Eval throughput up several-fold - target an order-of-magnitude increase - without a proportional increase in internal SME hours; capacity comes from the coordinated network, not headcount.

• A defined, documented eval system and contributor workbench that other pods can plug into.

• An active SME contributor network with a working incentive model (a roster of regularly contributing experts across the workflow domains we cover).

• Canonical reference projects with ground-truthed question banks, live and in use; evaluation no longer dependent on a handful of overextended individuals.

• Location: Remote United States of America

• Travel Requirement: Minimal

At Trimble, we are a team dedicated to making a tangible, positive Real-World Impact. Our purpose-driven culture means you'll work on projects that truly matter - helping to build and deliver solutions that make work faster, safer, and more sustainable for millions of people worldwide.

• Collaborate with like-minded people: You will work with a supportive team that fosters a genuine sense of belonging.

• Be an owner: We thrive on an entrepreneurial spirit where success is often "self-authored," empowering proactive "doers."

Compensation: Trimble provides the following compensation range and general description of other compensation and benefits that it in good faith believes it might pay and/or offer for this position. This compensation range is based on a full time schedule. Trimble reserves the right to ultimately pay more or less than the posted range and offer additional benefits and other compensation, depending on circumstances not related to an applicant's sex or other status protected by local, state, or federal law.

Hiring Range

Pay Rate Type

Bonus Eligible?

Commission Eligible?

Benefits: Trimble offers comprehensive core benefits that include Medical, Dental, Vision, Life, Disability, Time off plans and retirement plans. Most of our businesses also offer tax savings plans for health, dependent care and commuter expenses as well as Paid Parental Leave and Employee Stock Purchase Plan. If this position is identified above as commission- or bonus-eligible, the terms of the commission plan or discretionary bonus plan for which you are eligible will be provided following the employee start date.

How to Apply: Please submit an online application for this position by clicking on the 'Apply Now' button located in this posting.

Application Deadline: Trimble accepts applications on an ongoing basis until the position is filled.

At Trimble, we are committed to fostering a diverse, inclusive, and equitable workplace where everyone can thrive. Guided by our core values-Belong, Innovate, and Grow-we embrace and celebrate differences, knowing they make us stronger and more innovative. We are proud to be an equal opportunity employer, welcoming individuals of all backgrounds and advancing opportunities while embracing race, color, gender identity, sexual orientation, religion, disability, veteran status, or any other protected and diverse characteristic. We are committed to offering our candidates and employees with disabilities or sincerely held religious beliefs the ability to seek reasonable accommodations in accordance with applicable law and/or where it would not constitute undue hardship for Trimble. For more, please see Trimble's Code of Business Conduct and Ethics at https://investor.trimble.com, under "Corporate Governance."

Our mission to transform the way the world works starts with transforming how we work together. By actively listening, asking questions, and taking intentional actions, we cultivate a culture that provides equitable opportunities for everyone to contribute and grow.

Candidate Privacy Notices by Country

If you need assistance or would like to request an accommodation in connection with the application process, please contact careers@trimble.com.