Skip to content

OpenAI

Integration · AI models

Building business agents on OpenAI’s models

OpenAI is a model provider, not a business app: its API supplies the language models that read, reason and write inside an agent. The practical decisions are which model handles each step, how your data is handled under the API terms, what each completed task costs and whether to use more than one provider.

Where OpenAI fits in an agent

An agent has three parts: the model that interprets and writes, the tools that connect to your systems, and the rules and records that keep it safe. OpenAI supplies the first part through its API. Everything else, including access to your CRM or calendar, approvals and logs, is built around it. That is why the same agent can usually switch model provider without starting again, if it was designed to.

Using the API is different from staff using ChatGPT. The API is for software your business builds and controls, billed by usage, and governed by OpenAI’s business and API terms.

In practice, an OpenAI model inside an agent might read an incoming email, decide it is a booking request, call a calendar tool to check availability and draft a reply. Your system runs the calendar call, applies your rules and decides whether the reply goes straight out or to a person for approval. Use cases such as email triage and document processing follow this pattern.

How do you choose a model for each task?

OpenAI offers models at different sizes and prices, and the line-up changes several times a year. Rather than picking “the best model” for everything, match each step to the cheapest model that meets the quality bar for that step.

Matching model size to the step
Step in a workflowUsually suits
Classifying an email or extracting fields from a formA smaller, faster model, checked against examples.
Drafting a customer reply from approved informationA mid-range model, with a person reviewing at first.
Reasoning across long documents or several toolsA larger model, used only for the steps that need it.
Transcribing calls or voice notesA speech-to-text model, separate from the text model.

Decide with tests, not reputation. Keep a set of real, anonymised examples with known good answers and run every candidate model against them. See how to scope an AI agent.

How is your data handled?

At the time of writing, OpenAI states that data sent to its API is not used to train its models unless you opt in, and that API usage logs may be kept for up to 30 days for abuse monitoring. Zero data retention and modified abuse monitoring are available only with OpenAI’s approval, and not for every API feature. OpenAI also offers data residency options, including regional storage and processing in Europe for eligible customers under specific retention arrangements.

See AI agents and GDPR for the questions to work through.

Before you connect OpenAI: a checklist.

Checklist

Connecting OpenAI safely

What does a task cost to run?

API usage is billed per token, so cost depends on how much text goes in and out of each call, how many calls a task needs and which models are used. A single completed task, such as processing one invoice, may involve several calls. Measure cost per completed task, not per call.

The levers that reduce it are practical: shorter prompts, smaller models where they pass your tests, caching repeated context, and batch processing for work that does not need an instant answer, where the provider offers it. For User Evaluation, an existing AI research product, we measured each pipeline stage, matched models to tasks, batched transcription, shortened synthesis prompts and cached repeated retrieval. The result was a 38% lower per-job cost with no regression on their internal evals. The AI agent cost guide covers budgeting.

Should you use more than one provider?

Often, yes. Different providers are stronger at different tasks, prices change, and any API can have an outage or rate-limit you. Designing the agent so each step can call a different provider gives you a fallback and a negotiating position. The User Evaluation pipeline uses models from both OpenAI and Anthropic.

Many providers now offer an OpenAI-compatible API format, which makes switching easier. Our open-source FreeLLMAPI router is built on that idea: one OpenAI-compatible endpoint in front of many providers, moving a request to the next viable route when one rate-limits or fails. Keep API keys in your own account and secrets store, with a separate key per application so you can revoke one without affecting the rest.

Common questions

Is using the OpenAI API the same as staff pasting data into ChatGPT?

No. The API is used by software you build, under OpenAI’s API and business terms, with usage billed to your organisation and data-use rules that differ from consumer accounts. Staff using personal ChatGPT accounts for work data is a separate risk that your AI policy should cover, whatever your agents are built on.

Can our data stay in Europe with OpenAI?

OpenAI offers European data residency, covering regional storage and processing, but it is tied to eligibility and specific retention arrangements that need OpenAI’s approval. If data location is a hard requirement, raise it at the start of the project, confirm what your organisation qualifies for and check that every other service in the workflow meets the same requirement.

What happens when OpenAI retires a model we use?

Providers retire older models on published schedules. Pin the specific model version in production so behaviour does not change unexpectedly, watch for deprecation notices and re-run your test set against the replacement before switching. Budget for this as routine maintenance; it is one of the reasons evaluation sets matter.

Do we need our own OpenAI account?

We recommend it. With the account in your business’s name, you control the API keys, see usage and billing directly, and keep access if you change developers. We work inside your organisation with project-level keys, and you can remove our access at any time.

Bring us the task.

Tell us what happens today, which tools are involved and where it goes wrong. On a free 30-minute review, we’ll work out whether an agent, an automation or neither is the right fix, and what a sensible first release would be.

Start a project.

Two people on a sofa with laptops, planning a project beside a whiteboard of sticky notes

Book a call or send us a message. We read everything.

tash@neu.ie
Working with teams in Ireland and worldwide.

Get in touch

Free 30-minute AI review.