By Daniel Rickard · 25 September 2026

AI harnesses for financial advisers: building around the model

I’m building an AI harness inside our financial advice firm. Some of the decisions are about models. Many more are about the work around them: finding the right client information, deciding what happens when records disagree, checking an answer and controlling what can be changed.

That surrounding software is usually called an AI harness. For financial advisers, it is the difference between giving a model some documents and connecting it to a process the team can actually use.

I’m interested in what that makes possible for an advice business. A model can help prepare a review, compare information and draft the follow-up. Whether that saves anyone work depends on what it was given, what it did and how much someone has to repair afterwards.

The review workflow below is an illustrative example, not a list of features already live in the system we are building.

What is an AI harness?

An AI harness is the software around a model that manages its context, tools, permissions, execution and memory. It determines what information reaches the model, which proposed actions can run, how results are checked and what is retained for the next task. The Australian Signals Directorate uses the term for the components around the language model that connect it to external data, tools and workflows. ASD’s harness guidance is a useful starting point.

It helps to separate that from the other things a firm might buy.

The model interprets information and generates responses or proposed tool calls. A calculator or other tool performs a specific operation, such as running a projection or retrieving a record. The system of record maintains the authoritative business information. The harness connects these pieces and manages the work between them.

A chatbot can be one interface to that system. A staff workspace, client portal or background workflow can be another. The chat window is not what defines the capability.

Is a harness just a better prompt or more context?

Prompts guide the model. Retrieval brings information into its context. Neither, on its own, determines whether a proposed update is authorised, whether a failed operation should be retried or whether the result was actually saved.

A harness also lets a system do work outside the model. It can call a calculation program, check the returned result and pause for approval before continuing. Anthropic’s description of agentic systems distinguishes predefined workflows from agents that choose more of their own next steps. An advice process can use either, or combine them.

I wouldn’t make an agent choose every step simply because it can. Where the sequence is already known, ordinary workflow code may be easier to test and maintain.

What could an AI-assisted client review look like?

Consider a staff member preparing an annual review. The client’s circumstances have changed, there are outstanding actions from the previous meeting and some of the information is out of date.

I would break the work into a few stages. The important part is what each stage is allowed to assume.

Gather the evidence for the right client

The system first needs to establish the client, the staff member’s access and the purpose of the request. It should retrieve the relevant records through approved connections, rather than search everything the firm holds and hope it selects the right household.

For this example, the evidence might include the current fact-find, previous advice, recent meeting notes and unfinished implementation tasks. Each item needs enough source information to check it: where it came from, when it was recorded and whether it is still current.

I’d want a failed connection to appear as a failed connection. “We couldn’t retrieve the portfolio” and “the client has no portfolio” are very different findings.

Separate changes from unresolved questions

Suppose a recent note says the client wants to retire earlier, while the CRM still contains the previous date. The system should flag the difference rather than silently choose whichever value appeared last in its context.

The draft brief could show the earlier assumption, the newer statement and the question requiring confirmation. It could do the same for missing income information or a recommendation whose implementation has never been confirmed.

That gives the adviser something useful to resolve in the meeting. It avoids turning an uncertain source into an apparently settled fact.

Use the right tool for the numbers

Once the relevant assumptions are confirmed, an approved planning tool could run revised scenarios. I’d retain the inputs, assumptions and returned results with the work, rather than keep only a paragraph describing them.

This is the same design choice I recognised sitting with Hazel: the language model coordinates; dedicated calculation programs produce the numbers. It is a US product example, not evidence that the same workflows are ready for Australian advice.

A tested calculator still needs correct inputs and appropriate assumptions. Moving the arithmetic outside the model doesn’t settle whether the scenario suits the client.

Draft, review and record what happened

The model could assemble the brief, suggest questions and draft follow-up actions. I’d keep that separate from permission to update the client record or send a communication.

For a first release, I would keep those consequential actions behind review. The reviewer needs to see the proposed change and the evidence behind it, not just an approve button beside a polished answer.

After an approved action, the system should confirm the outcome in the destination system. If it creates a task, there should be a task to find. If it retries after a timeout, it should not create a second copy.

That is the practical difference between an assistant saying the work is done and the firm being able to rely on it.

How would it connect to Xplan, Salesforce or Fin365?

I’d start with record ownership, before choosing a model or an integration library.

Which system owns the client’s details? Where are the approved documents? Which service owns tasks and workflow status? What can the new software read, and which changes must go back through an approval process?

Those answers will differ between firms. A platform’s name on an integration list is not enough: the available APIs, licences, permissions, supported connection methods and data coverage need to be checked. Start with those boundaries rather than assuming every listed system can be connected in the same way.

A useful harness doesn’t necessarily replace the CRM. It may make the information in it easier to use while keeping agreed records there. I’d be wary of accidentally building a second, less complete CRM inside the agent’s memory.

There is also a time dimension. Today’s client record and the evidence supporting last year’s recommendation are different records for different purposes. Updating a retirement date now should not alter the apparent basis of earlier advice.

This is where connected records become valuable. Facts, calculations, alternatives, approvals and implementation outcomes can support the advice document, file note and next review without someone reconstructing the whole sequence each time.

Can the same model perform better with a different harness?

Yes, on particular tasks. In LangChain’s own evaluation, changes to the prompts, tools and execution process took GPT-5.2-Codex from 52.8% to 66.5% on Terminal-Bench 2.0, without changing the model. That does not establish an equivalent gain in advice work, but it shows why the surrounding system is worth testing.

I take two things from them. There is work we can do now rather than wait for another model release. And adding more tools, instructions or agents isn’t automatically an improvement.

The benchmark that matters to an advice firm is whether its own workflow becomes more reliable and useful. I’d want to measure that before making claims about capacity or hours saved.

How I would test an AI workflow before expanding it

I’d begin with examples where experienced people can explain what a good result looks like, including cases where the correct result is to stop or ask a question.

For the review workflow, I’d build a small evaluation set using synthetic or appropriately de-identified material. These are the kinds of cases I would include:

Some checks belong in code. Others require an experienced reviewer. A model can help assess output, but I wouldn’t rely on its own favourable assessment as the only acceptance test.

Anthropic’s guidance on agent evaluations makes a useful distinction between what an agent says and the actual outcome in the environment. It also explains why multiple trials matter when outputs vary. One successful demonstration doesn’t establish dependable behaviour.

I’d measure staff review time, material corrections, incomplete tasks, inappropriate actions, system cost and time to an accepted result. That gives a better picture than the number of drafts generated.

A cheaper model that needs more repair may be more expensive for the firm. A slower generation step might still reduce the total work. The comparison needs to include the people doing the checking.

Security and privacy are part of the workflow design

Connecting an agent to more systems gives it more ways to help and more ways to cause harm. ASD’s agentic AI guidance emphasises least-privilege access, monitoring and oversight of high-impact actions. Its harness publication is explicit that no harness is inherently secure.

For the workflow above, I’d want separate read and write permissions, limits on tools and destinations, a clear way to stop execution and logs that are themselves access-controlled. Keeping evidence should not mean making a second, unrestricted store of client information.

Supplier review needs to cover where information is processed, retention, authorised access and what happens when a connection or provider changes. A no-training commitment answers one question; it doesn’t answer all of them.

ASIC’s October 2024 review examined AI use and governance across 23 AFS and credit licensees. It is useful Australian background for a firm assessing its own controls.

From 10 December 2026, additional privacy-policy disclosures apply to entities covered by the Australian Privacy Principles where the relevant automated-decision conditions are met. These concern certain uses of personal information in decisions that could significantly affect an individual’s rights or interests, including a computer program doing something substantially and directly related to making a decision. This is not a blanket requirement to label every AI-assisted output.

How can the system improve without turning every correction into policy?

I would separate three kinds of improvement.

Within a task, the model can check a result and try again. Across tasks, the system can retain an approved instruction or worked example. At a deeper level, software can propose changes to the tools or process itself.

None of that means a production advice system should approve its own changes.

In practice, I’d start by classifying the correction. A wrong client fact needs correcting in the right client record. A writing preference may belong to one adviser. A failed retrieval might need a software repair. A change to the firm’s recommended approach needs a different review altogether.

Suppose the system misses an outstanding implementation task in a review brief. I’d want to establish why: was the task inaccessible, badly labelled, omitted during retrieval or ignored during drafting? Adding “remember outstanding tasks” to every prompt might not fix the actual problem.

Once we understand the cause, we can propose a change, test it against the failed case and check that it hasn’t broken other cases. The person with the appropriate authority can then approve it for the intended scope.

That is a useful learning loop. It preserves a tested improvement rather than simply remembering whatever someone last typed.

Approved examples also need review as circumstances change. A precedent can be useful without becoming a universal rule, and a client accepting a recommendation does not prove it was the best recommendation.

Should a firm buy, configure or build an AI harness?

I wouldn’t start by assuming custom software is the answer. The useful question is how much of the workflow an existing product can handle properly, and what remains specific to the firm.

Buy where a product fits the job, the controls are suitable and the integration is supported. Configure or connect where the core product works but needs the firm’s templates, permissions or handoffs. Build where there is a meaningful gap that existing products cannot reasonably close and the firm is prepared to maintain the result.

I’d also test whether the product can follow our approach on a less straightforward file. Can we change the assumptions, require the evidence we rely on and handle a situation where we deliberately depart from the usual recommendation? Making the connections easy is useful. I’d still want to know whether the defaults fit how our firm gives advice.

There is a hybrid option too: use established models and runtime components, then build the part that understands the firm’s work. Rebuilding generic infrastructure adds maintenance without necessarily improving advice.

Decide what to keep, replace, connect or build before committing to a large project. An AI harness should earn its place through the problem it solves.

I’d pay particular attention to continuity. Can the firm export its records and approved knowledge? Who controls instructions and permissions? Can evaluation cases be retained? What happens if another developer takes over or the model provider changes?

Those rights need to be explicit. Firm-specific records and operating knowledge are not the same as ownership of every reusable software component. Settle that in the project agreement.

A system that supports several models is not automatically easy to move. Portability has to be tested against the things the firm actually wants to keep.

What building it has made me put into words

On the system we are building, we have been separating general model use from governed firm knowledge, working with authorised CRM context and putting verification, citations and permissions into the software. Some capabilities depend on configuration, and not every contemplated workflow is live.

The useful work has been specifying what we otherwise leave to people to work out. Which source wins when records differ? Which omissions should stop a draft? When is a correction only about this client, and when should it affect future work?

House style is not the same as house standard. A system can imitate the way we write before it reliably meets the standard of work we need. Building tests and approval points makes that distinction harder to ignore.

I wouldn’t attach a productivity number to that work yet. I’d rather establish which workflows are dependable, where review still takes effort and which changes genuinely improve the result.

Where I would start

I would choose one bounded task, such as producing a source-linked review brief without changing client records. Agree who uses it, which sources it can read, what a good result includes and when it must stop.

Then compare it with the existing process. Keep the review effort visible. Test difficult cases, including denied access and failed connections. Give someone responsibility for maintaining the workflow, and retain a workable fallback.

Only add actions once the earlier steps are dependable enough for the firm’s intended use. Sending an email or updating a record is a separate decision from generating a good draft.

That is where I think AI workflow automation becomes useful for financial advisers. We can remove some repeated work, give the team better information and retain the improvements we have actually tested. We still need to know what happened and why we were prepared to rely on it.

The model is one part of that system. The firm’s responsibility for the work remains.

I build advice technology and lead Luminair Capital. Through Luminair, I help advice firms work out what to keep, connect or build, and how to turn those decisions into a practical delivery plan.