Aligning on AI product engineering

I want to build useful AI products, take responsibility for their behavior in production, and help improve how a team develops them. I’m looking for a hands-on role that can use the judgment I have developed and give me substantive work through which to develop it further.

I share these notes early in conversations with companies and recruiters because I want us to understand the work we would be agreeing to. The company needs to assess the relevance of my experience. I need to understand whether the actual role will let me apply it, with the support the work requires.

The contribution I want to make#

My contribution concerns practical product and engineering decisions: which approach to pursue, what behavior users can depend on, where to invest effort, and what to improve next. I want to help investigate those questions and carry the findings through architecture, implementation, evaluation, and production operation.

Some of that work may not yet be explicit in a job description. A company might be hiring to extend an assistant while still needing to establish which requests it can support or how to judge whether a change improves it. Discussing the problem can reveal responsibilities that the initial description leaves open. It can also show that the company needs a narrower contribution than the one I’m seeking.

My approach has developed through building systems, observing behavior, collecting evidence, and reconsidering decisions. I want that experience to influence how we frame problems, identify important assumptions, investigate failures, and choose what to do next.

I welcome examination through concrete examples: what I was responsible for, what was uncertain, what we observed, how I interpreted it, and what changed as a result. These notes explain my current reasoning. Examples from the work provide a basis for assessing the experience behind it. Both should be open to challenge.

My perspective on production AI engineering describes the broader engineering contribution I want to make.

Why developing the practice belongs in the work#

I see AI product engineering as a practice still taking shape. It brings together established knowledge from several disciplines, while the ways we combine and apply that knowledge continue to develop. A clear product goal can still require investigation into whether and how a proposed system can achieve it.

My concern is that language can make the practice sound more settled than it is. Saying “we have AI engineers” or “we do evals” leaves questions about how the team understands behavior, judges results, and uses findings. We need to be able to examine the methods as well as the product.

I expect a scientific mindset in practical terms: distinguish observations from assumptions, investigate consequential questions, and revise conclusions when the evidence calls for it. Shared language helps people understand and challenge the reasoning. Discipline makes this part of everyday work. My own methods must remain open to the same examination.

Language, mindset, and discipline in AI product development explains this position in more detail.

Evaluation is one foundation for that work. It helps us compare approaches, investigate failures, assess changes, and establish what the product can support. A continuing evaluation capability includes people, methods, data, and software, with shared criteria that make results understandable. Those results inform decisions; delivering a useful product also requires user understanding and the engineering needed to act on the findings.

The role of evaluation in AI product development explains why I treat this capability as part of development and delivery.

Work that uses and develops my experience#

I expect some familiar problems where my experience can be useful immediately. Recognizing a pattern can help us identify important questions earlier. It does not mean that a previous solution will fit a different product or domain.

I also want substantive questions I have not already answered, access to users and production experience, and colleagues whose knowledge can improve my judgment. That development can concern the product, the methods used to understand it, or both.

A company’s need for my experience is one part of the fit. I also need meaningful opportunities to develop it within the proposed role. Repeating familiar work while waiting for those opportunities to appear would not meet that goal.

What needs support from the start#

A team does not need a finished evaluation platform or my terminology before we can work together. Identifying gaps and establishing missing practices can be part of the role. We do need an initial agreement that investigating the system and improving the way we develop it are supported engineering responsibilities.

That support includes planned time, access to relevant data, users and domain experts, and engineering capacity to act on findings. It also includes clarity about which decisions the role can influence and who is responsible for changes to architecture, scope, releases, and product expectations.

I am willing to explain unfamiliar concerns and help establish why they deserve attention. Examining concrete behavior together can make a need clearer, change my diagnosis, or show that a smaller intervention is sufficient. Questions about relevance, cost, priority, and alternatives belong in that discussion.

The boundary is whether the discussion leads to supported work. I do not want to spend the first year repeatedly trying to establish that investigation and evaluation belong in the role. Continuing engineering needs should have shared responsibility and allocated time. I’m not looking for an arrangement that depends on supplying that work through personal time until the company becomes ready to support it.

I expect this work to contribute to delivery. Investigation should be proportionate to the decision and its consequences. We can build, release within supported limits, and continue learning from use. The purpose is to make product decisions with a sound basis and carry them through into a useful, dependable system.

How I want to establish the fit#

An early discussion should cover the company’s actual problems, what it expects the role to accomplish, which decisions remain open, and which constraints are fixed. We should be able to describe how my experience would contribute and what responsibility I would have.

The scope needs to be concrete. “Owning evaluation” might mean maintaining existing tests and improving their scores. It might also include questioning whether those tests capture the required behavior, developing better methods, and using findings to guide product changes. I want us to understand which work is being offered.

We should also distinguish what is available now from what would need to be established. Access to domain experts, for example, needs more than agreement that it would be useful. Where support is missing, we should identify what will be arranged, who is responsible, and when it is needed. I can accept uncertainty about the work; I need a credible basis for believing I will be able to do it.

For recruiters, these notes provide context for assessing a possible match. The substantive discussion should include the people responsible for the relevant engineering and product decisions, early enough to establish the scope before a lengthy interview process. I welcome technical assessment and discussion of my work as part of that process.

The agreement I’m looking for is about useful work we can start together: applying my experience, developing it further, and improving the product and its engineering practice with practical support from the beginning.