Everyone Says the Harness Matters. Here Is What That Means on Real Travel Data.

You have read it everywhere by now. The model is only half the result. The other half is the harness: everything around the model that decides what it sees, what it may do, and what reaches the customer.

True. And almost useless, until you ask what the harness has to handle in your industry.

For the last three weeks we ran five AI models side by side, from two of the leading AI labs, entry-level to premium. Not on a benchmark. On the work our AI travel agent does every day: real traveler messages in English and Hebrew, real flight results from a live GDS, real booking moments where one wrong word costs money.

The bottom line

With the right harness, we cut AI cost by more than 90% with no drop in accuracy. The real results:

A full traveler conversation, eight turns on fares, bags and alternatives: AI cost down about 95%. Wrong answers before: none. After: none.
Recommending flights from real GDS results: the entry-level model ranked just behind the best premium model, at 94% lower cost. False claims across ten searches: 4 vs 3.
Understanding which option the traveler means: the entry-level model matched the premium one, 69 of 70 correct for both, at about 97% lower cost.
Same model, better harness: three times as many of our hardest test conversations handled right.

Here is what a travel harness actually has to handle, and the real mistakes that taught us each part.

The models

Claude Haiku 4.5 (Anthropic). Fast and low-cost. Where we started in production.
Claude Haiku 5.5 (Anthropic). The new generation of the same small, fast family.
Claude Sonnet 4.5 and 4.6 (Anthropic). Anthropic’s stronger, more expensive tier.
GPT-6 Luna (OpenAI). OpenAI’s entry-level model. Very cheap, and it reasons before it answers.
GPT-6 Sol (OpenAI). OpenAI’s premium tier.
Jev. Not a chat model at all. It writes no text, it only returns a decision and a confidence score. We probed it for one job: deciding where a traveler’s message should go.
Kimi K3 (Moonshot AI, China). On our list. We have not run it through these tests yet, and we will, the same way.

What we measure on every model

Before any example, the scorecard. Every model, every change, is measured on the same seven things:

Pick accuracy. Did it put the right flight first for this traveler?
Truth. How many claims in the answer are not in the data? We count them one by one.
Honesty. When the data does not have the answer, does it say so, or does it invent one?
Right action. Did it do exactly what the traveler asked, and nothing more?
Understanding. “Option 3”, “the El Al one”, “29 Oct”, “yalla”. Did it resolve what the traveler meant?
Response time. Did the answer arrive while the traveler was still there?
Cost per task. The whole task, not the price per token.

None of these come from a public benchmark. They come from what goes wrong on a travel desk.

1. The GDS data is wrong before the model is

On one fare, the GDS summary line said the traveler had one checked bag included. The detailed fare rules said zero.

The traveler asked about bags. The model answered: this fare includes one checked bag each way. Wrong. The fare that really included a bag cost about 12% more.

The model did exactly what it was told. It trusted the data.

So part of a travel harness is a job no AI lab will do for you: knowing which GDS field to believe when two of them disagree. That is fifteen years of travel data, not a model setting.

2. The harness can hide the answer

A traveler searched for a specific date. The model replied that there were no flights on that date.

There were. To fit the request into the model’s limit, the harness had trimmed the result list, and the trim removed every offer on the requested date. The model told the truth about what it was shown. The harness had shown it the wrong thing.

It works the other way too. We added a safety check that blocks any price the model states that is not printed on the option. The traveler asked if option 2 was cheaper. The model answered correctly: no, it costs about twice as much. The check blocked it, because the difference itself was not printed anywhere. The traveler got “I can’t confirm that” instead of a correct answer.

A harness that is too loose lets the model invent. A harness that is too tight silences the truth. You only find the line with real traffic.

3. “Option 3” is a travel problem

Travelers do not speak in booking codes. They say “the El Al one”, “the next one”, “change to option 3”.

One model read “the El Al one” as the flight already on the traveler’s screen, which was not El Al. It did that 5 times out of 5, and said it was 95% sure each time.

Another model got “change to option 3” right 0 times out of 15. We changed one thing in the harness: how the conversation history was laid out for the model. It went to 15 out of 15. Same model.

In Hebrew, “yalla, upgrade” means yes, go ahead. One model read it as a decline.

And dates. One model read all 135 test dates right. But when it asked the traveler to confirm the year and the traveler said “yes”, it applied that yes only 1 time in 10 in English and 4 in 10 in Hebrew.

Understanding a traveler is not a language skill. It is travel context the harness has to give the model.

4. Fluent is not true

Some of the false claims we counted, in recommendations that read beautifully:

A seat described as a “full-flat bed”. Nothing in the data said so.
A one-stop flight via Paris labelled nonstop.
A one-stop flight recommended first while a nonstop sat in the results, about 5% cheaper.
“The lowest price for your date”, on a fare when the real lowest was 42% cheaper.
A cancellable fare described as non-refundable.

And the opposite, the answer we want. Asked about the aircraft, one model replied that the El Al offers do not list the aircraft type. No guess. That sentence is worth more than a page of fluent text.

Price did not predict truth. The most accurate model in our recommendation test was a premium one. An entry-level model came within half a rank of it at about 6% of the cost. And the other premium model finished last, with almost five times as many false claims as the entry-level one.

5. Booking moments need a stricter harness

A traveler in the middle of a booking declined a new fare. They said no to the price change, nothing else.

One model replied, in fluent Hebrew: “We cancelled the booking.”

Nothing was cancelled. This happened in testing, so no traveler ever read it. That is exactly why we test booking moments before any traveler meets them.

A quieter version: a traveler asked for a refundable El Al flight that was not in the results. The model replied: “I’ll look specifically for a refundable El Al option for your dates.” A promise of a search the system was never going to run. The fix was a harness rule, not a bigger model: never promise an action you do not perform. Then offer what is really there, and switch only on a yes.

When money or a booking is involved, a person still approves. The harness decides what the model may say before that.

6. Response time is part of accuracy

On a first search, the full answer took 36 seconds. The GDS took about a quarter of that. The recommendation model took about 40%.

Under tight time limits, the most accurate model missed about a third of its answers. A perfect recommendation that arrives after the traveler has closed the chat is not a recommendation.

So each channel gets its own time budget, and each model is tested against it. WhatsApp is not a web page.

7. Cost stopped being the question

We moved a full live conversation, eight turns of questions on fares, bags and alternatives, from a mid-range model to an entry-level one. The AI cost of the conversation dropped by about 95%, with no wrong answers on either side.

The cost of an AI answer is now small next to a human minute. What costs money is a wrong answer.

And the result

We keep a test set built to break the agent: tricky real conversations, stress cases, and traveler phrasings the agent has never seen. Some of it is sealed, so nobody tunes the system to the answers.

We ran the same model through that set twice: once with our current harness, once with a redesigned harness that is still in testing. The redesign passed three times as many of these hard conversations. Same model, same questions. Only the harness changed.

That is the “harness is half the result” line, measured on travel data. But the gain did not come from a generic recipe. It came from bags, dates, “option 3”, Hebrew, GDS fields and booking moments, one real mistake at a time.

What I would ask before buying AI for a travel business

Was it tested on our trips, in our languages, with our GDS data? Not on a public benchmark.
Which of the seven measures above do you track, and what are the numbers? Ask for the false-claim rate, not the demo.
What does the AI do when the GDS data disagrees with itself?
What is it never allowed to say during a booking?
Can you change the model for one task in a day? The best model changes every few months.

About these tests

Real tests, not lab benchmarks: real traveler phrasings in English and Hebrew, real flight results from a live GDS, real booking moments, no real money moved. Each test ran from tens to a few hundred cases. Some quality scores come from a blind AI judge with a fixed rubric. Strong signals, not universal laws, and we re-run them every time a model or the harness changes.

This is the work we do every week while building Bitravel: putting AI in front of real travelers and measuring where it holds and where it breaks. If you are deciding how AI fits your travel operation, book a 30-minute call. No deck, just what we’ve seen.