1. Pick a repeatable task
Start with a task that happens often enough to observe and narrow enough to test. “Transform customer service” is difficult to evaluate. “Draft responses to five common product questions from an approved source” gives the team a manageable experiment. Ask the people doing the work where time is lost and which errors create rework.
- Describe the input, expected output and next person in the process.
- Choose one workflow owner who can judge whether the result is useful.
2. Establish a baseline before building
Record how the task works today. Measure the time to a reviewed result, how often it needs correction and what happens when information is missing. A faster draft is not a benefit if it creates more review work. Keep the measurement small enough that the team can repeat it during the pilot.
- Compare equivalent work, not an easy demo against a difficult real case.
- Include checking and correction time in the comparison.
3. Prepare representative examples
For bilingual work, include Arabic, English and mixed-language inputs. Use the terminology, document quality and formats your team actually sees. Avoid filling a test set with only clean examples. Remove unnecessary personal or confidential information and agree how the sample may be used before sharing it.
- Include missing fields, ambiguous requests and out-of-date sources.
- Keep some examples aside for evaluation rather than development.
4. Define the human boundary
Decide what the system may suggest, what it may change and what requires approval. A first pilot can prepare a draft without sending a message or updating a live system. This makes it easier to inspect the result and understand failure modes before expanding access.
- Specify who reviews exceptions.
- Make it clear when the system should decline to answer.
5. Make the end decision explicit
Agree the decision criteria before the demonstration. Review quality, operational effort, integration dependencies and likely running costs together. The result may justify implementation, a narrower design or stopping. All three outcomes are useful if the pilot provides evidence. Write down the assumptions that changed so the next decision is better informed.
- Proceed when the agreed criteria are met on representative work.
- Revise when the scope or data needs adjustment.
- Stop when the value does not justify the effort.
Before we begin.
Do we need a large dataset to begin?
Not always. A small, representative collection can support an initial feasibility test. Its size and coverage should match the question the pilot needs to answer.
Should we select the model first?
Begin with the task, evaluation criteria and constraints. Model selection is part of designing a solution to those requirements.