Generative AI at work. Where it fits.

A guide to assessing generative AI by task, data, quality and cost, whatever technology you choose.

Drafts

Prepare text and other content for a person to review and approve.

Knowledge

Find answers in approved sources and check the passages behind them.

Structure

Classify varied documents and extract fields your team can verify.

Assistance

Produce drafts for translation, localisation and code review.

What goes into a generative AI system?

Generative models produce outputs using patterns learned during training and the context you give them. Language models can complete text, answer questions, extract information and suggest code. Other models can generate images, speech, music or video. Unlike a database query, the result is produced by the model and needs to be checked against the source where accuracy matters.

A production system also needs other components.

  • An interface, API or connection to existing software

  • System instructions and templates

  • Identity checks and permissions

  • Document, database or tool connections where needed

  • Filters, logs and monitoring

  • Defined review, approval and escalation steps

A capable model can still receive unsuitable inputs, retrieve the wrong source or pass on an unchecked answer. A smaller model may work better for a specific task when the data flow, output format and quality criteria are clear.

The EU AI Act distinguishes general-purpose models from AI systems that use them. The EU AI Act describes different roles in the supply chain. Obligations depend in part on whether a business uses a third-party system, integrates a model or provides its own system.

Where generative AI helps.

Look for a clear task, enough context, errors you can detect and a way to review or fall back.

  • Drafting and revision

    Prepare a first draft of an email, product description, report or guide. A person checks tone, facts and rights before approving it.

  • Summarising and comparing

    Summarise long documents against a defined set of questions. Ask the system to cite the passages it uses and check them against the original.

  • Structuring and extraction

    Turn unstructured text into fields, categories or a defined schema. Keep required fields and validation in the surrounding software.

  • Answering from approved knowledge

    Retrieve passages from approved sources to support an answer. Check which sources the system selects and whether they support what it says.

  • Assistance with translation and code

    Use models to prepare versions and drafts. Have a person check translated text for accuracy and review generated code before it goes into use.

Some tasks call for simpler tools.

Choose the simplest tool that reliably meets the quality and operating requirements.

  • Exact values and transactions. Use database queries, calculations or APIs for account balances, prices and official status values.
  • Fixed decisions. Use rules, forms or workflows where the full logic can be written down and tested.
  • Document discovery. If users need to read the original, a good search interface may give them what they need.
  • Standard text. Use approved templates where only a few fields change.
  • Forecasting and classification. Conventional methods often suit numerical predictions or predefined labels.
  • Decisions with serious consequences. Keep effective human oversight when mistakes could affect rights, safety, employment or access to services.

Compare three approaches in a pilot. Measure today's process, the simplest digital alternative and a version with generative AI.

Choose where the system runs

Terms such as private LLM and enterprise AI cover different setups. Find out who operates the model and infrastructure, where data goes, what gets logged and which contract and licence terms apply.

Deployment option

Typical advantage

What to check

Hosted software or public API

Quick access to current models with little infrastructure to manage

Inputs and outputs sent to the provider, retention, subprocessors, regions, contract changes, rate limits and exit options

Managed private instance or dedicated endpoint

More network and tenant isolation while the service is still managed for you

What does dedicated mean in practice? Who administers, logs and updates the system? Who controls the model and management layer?

Open-weight model in your own cloud

More control over deployment, adaptation and data paths

Model licence, infrastructure, security updates, scaling, evaluation and staffing

On-premises or local

Runs in your own environment, with possible offline or edge use

Hardware capacity, power, resilience, patches, monitoring, model changes and physical and administrative security

Local deployment still needs an assessment of privacy, security and cost. Data can move through telemetry, support access, connected tools, backups and logs. With a hosted service, check the technical setup and contract to learn whether inputs are used for training and how data is handled.

Open weight means the model weights are available. Commercial use and the right to modify or share a model depend on the licence. The Open Source AI Definition also calls for the freedom to use, study, modify and share the system, with access to the components needed to make changes. Read the actual model licence.

For an example of how hardware, a model and a knowledge base can work together, see our example of a locally hosted language model.

Map the data flow before choosing a model

For a real request, identify which data reaches each component. The full path can include the following.

  1. User input, including attachments

  2. System instructions, examples and conversation history

  3. Retrieved document passages or query results

  4. Outputs from tools and APIs

  5. Processing by the model, platform and any subprocessors

  6. Security, error, usage and billing logs

  7. Generated output, human edits and handover to other systems

For each step, identify data categories, purpose, legal basis, recipients, location, retention, deletion, access and logging. A zero-retention setting may cover only part of the flow. Check what happens to metadata, support requests and copies in connected systems.

The German Data Protection Conference's guidance on AI and data protection recommends addressing privacy when choosing, implementing and using AI. The European Data Protection Board's opinion on AI models addresses how to assess models and their deployment case by case. That assessment still applies to a locally hosted system.

Keep particularly sensitive information out of the model where appropriate. Consider data minimisation, pseudonymisation, filters or rules-based processing. See our guide to data sovereignty.

Measure output quality on real cases

Generative models can produce plausible claims that are wrong or unsupported. Better inputs, reliable sources, validation and clear system limits can reduce these errors. To estimate an error rate, define the task, model version, language, context and measurement method.

Build a representative test set before the pilot. Include ordinary requests, difficult cases, missing information, conflicting sources and inputs the system should reject. Agree what a good result requires.

  • Accuracy and completeness for the task

  • Correct citations, where sources are used

  • The right format, tone and adherence to instructions

  • A refusal or escalation when evidence is missing

  • Protection of confidential and unauthorised data

  • Consistent performance across repeated tests and relevant groups of users

  • Total handling time, including human review

  • Cost per successfully completed task

Compare against today's process or a simpler tool. A fluent answer may take longer to check and correct than the current workflow. A draft with easily spotted mistakes may still save time if overall quality meets your threshold.

The NIST Generative AI Profile recommends evaluation before deployment and during operation, with documented thresholds for decisions. For retrieval-augmented generation (RAG), measure retrieval and answer quality separately. Rerun the same cases after changes to the model, prompts, sources or filters.

Measure results in context. In a study of 5,179 customer support agents, an AI assistant raised average issues resolved per hour by 14%, with larger improvements for newer and lower-skilled workers. Read the NBER study.

Set roles, rules and limits

Give people clear answers about who can use the system, for which purposes, with what data, who checks outputs and who decides when something changes or goes wrong.

A practical governance agreement covers the following.

  • Owners for the use case, data and technical platform

  • Allowed, restricted and prohibited tasks

  • Approved systems, accounts and integrations

  • Rules for data, sources, logs and retention

  • Human review and approval requirements

  • Quality thresholds and stop criteria

  • Procedures for changes, incidents and complaints

  • Training for people who use or supervise the system

The EU AI Act takes a risk-based approach, with different roles and duties. Review the use case to see which requirements apply. Privacy, copyright, trade secrets, information security, employment and sector rules can also matter.

Article 4 of the EU AI Act addresses AI literacy for staff and others working on an organisation's behalf. See our guide to AI literacy under Article 4. Practical generative AI training can support a wider set of measures suited to the people and the system.

Plan security and operations beyond the pilot

Models, providers, prices and security controls change. Sources are updated, users bring new tasks and interfaces can fail. Name an owner and plan how the application will be maintained, changed and retired.

Track model and prompt versions, manage access, monitor cost and quality, keep appropriate logs and plan recovery, shutdown and updates. Set access and retention for logs according to what they contain.

Treat instructions found in documents or user inputs as untrusted. Limit permissions for connected tools and review software and model suppliers. The UK's NCSC and international partners published the Guidelines for Secure AI System Development, which cover design, development, deployment, operation and maintenance.

Systems that call tools or plan sequences can take actions in other applications. Set permissions and transaction limits at the point of action, with human approval when needed. Read what agentic AI means in practice.

Estimate the full cost and check licences

Compare options using expected volumes and the full cost of building, running and replacing the application.

  • Model usage or licence fees

  • Interface, retrieval, integrations and controls

  • Cloud, network, storage or local hardware

  • Evaluation, security reviews and privacy work

  • Monitoring, support and incident response

  • Maintenance of prompts, sources, permissions and test cases

  • Training, rollout and ongoing support for users

  • Migration or retirement when changing providers or models

For hosted services, check pricing tiers, rate limits, minimum commitments, data transfer and dependence on proprietary features. Open-weight or local deployments can reduce some API fees and add infrastructure, operating and staffing costs.

Review the terms for the model, libraries, training data, inputs, connected sources and outputs. Open-weight licences may require attribution or limit use, redistribution and derivative models. Review hosted contracts for data use, warranties, liability, rights in inputs and outputs, intellectual property protection and exit options.

Copyright can depend on the source material, human contribution, resemblance to protected works and how an output is used. The EUIPO study on generative AI and copyright offers European context for reviewing the material in question.

Use a pilot to decide what comes next

Set up the pilot to answer a specific question about the process.

  1. Define the task and users. Choose one process, a business owner and examples that represent real work.

  2. Record a baseline. Measure current quality, handling time, errors, handovers and costs.

  3. Map the data and boundaries. Agree approved data, sources, access, logs, human review and what is out of scope.

  4. Test the simplest architecture. Add agents, fine-tuning or local infrastructure only where the use case calls for them.

  5. Compare the options. Evaluate hosted, private and, where relevant, open-weight or local options on the same cases.

  6. Include the cost of operation. Agree ownership, monitoring, updates, support, budget and exit plans before rollout.

  7. Record the decision. Expand, continue within limits, change the architecture or stop.

If search, rules, an API or a template meets the quality requirement more simply, choose that option.

For a roadmap across several use cases, explore AI consulting and strategy. In a free consultation, we discuss the task, data flow, deployment options and how to measure a pilot. The next step may be a pilot or a simpler solution.

Questions about generative AI.

Sources

Assess your generative AI use case

Start with a short decision brief that covers the task, baseline, data flow, quality criteria, hosting options and a simpler alternative. Book a free consultation.

This guide offers general technical and organisational guidance. Legal, privacy, licensing and security reviews depend on the system, purpose, data flow, contract, context and people affected.

Test generative AI on a real process.

In a free consultation, we look at the task, data flow and quality requirements to see which option fits.