If You Knew It Was Closed Why Did You Suggest I Go There?

I spent the past three weeks on the road overseas and attempted to use the various GPTs— ChatGPT, Gemini, Claude and Grok—to help me do everything from navigating transit systems to finding a restaurant to planning my day.

While they were often helpful, the experience was more often an exercise in frustration, filled with restaurants that did not exist, museums that were closed and insights pulled from cranky Reddit users.

The problem with the way they do travel research is that the GPTs are primarily pulling their data from an imperfect data set.

Most of the travel data they scrape comes from Reddit and Tripadvisor. Those sites can be useful once you figure out how to navigate them, which means understanding that they are largely the province of people who have had a negative experience and want to share it.

They're also the province of the less adventurous and the tips and reviews tend to reflect that.

All well and good, but if you're a GPT and those are the data sets you've been trained on, you're going to have a fairly skewed view of the world and an inherent bias against anything outside of the mainstream.

(And FWIW, including some variation of “avoid tourist traps” in the prompt only helps a little—BTDT.)

Among the other frustrating behaviors the GPTs exhibited were the tendencies to get so fixated on meeting the exact parameters I laid out ("someplace low-key with counter service that I can grab a quick bite") that they neglected to check whether their recommendation was still in existence or open the day I wanted to go.

They’d often suggest places and then ask me if I wanted to check if it was open—frustrating because I’d assumed they would automatically do that, especially given the prompt included the word “today.”

Even more frustrating when it turned out the place in question was closed (sometimes permanently) when we’d just spent 10 minutes discussing it.

Will GPTs get better with travel planning?

Quite likely. Their minders just need to get better about what sources they pull from for international travel, specifically non-English sources, which would give them a far more balanced and far more valuable knowledge base.

But the key takeaway right now is that this is yet another example of how, while they show great promise, they are nowhere near up to the task of replacing humans.

And that if the humans who will need to okay said replacing keep having these sorts of negative experiences, it will be some time before they trust the technology enough to let it handle things on its own.

Which is not to say it never will.

Just that the timelines we’re seeing bandied about—18 months or so—are wildly Web 1.0-level optimistic.