The Problem With Human x AI Collab: It’s Not the AI

If you’re getting mediocre–or even bad–results from AI tools like ChatGPT, it’s not because the tech is bad. You’re probably settling for mediocrity.

Don’t worry, this is a longstanding cognitive bias that I’ve seen strike students and executives alike in my last 15 years teaching at Stanford. And while AI represents an unprecedented opportunity to unshackle humans from their biases, most often, it only exaggerates the underlying issue.

How do I know? Recently, I served as co-researcher on a study to see if teams using AI to generate creative solutions came up with better ideas than unassisted teams. What we found: The teams that used AI had less bad ideas than the teams that didn’t, but they also had less great ideas.

In other words, AI came up with a lot of average ideas (which probably doesn’t surprise you if you’ve used it for idea generation). LLMs are, after all, simply statistical models. So how do you push past AI’s tendency toward mediocrity and get to something great?

THE PROBLEM: WE’RE HAPPY TO SETTLE FOR “GOOD ENOUGH”

Herbert Simon’s theory of ‘satisficing’ says humans only search for solutions until a threshold of acceptability is met. We sacrifice better options for the satisfaction of having a passable one.

In plainer English: It’s good enough.

That’s because humans crave cognitive closure. At its core, problem-solving is an exercise in resolving uncertainty, and human beings hate uncertainty. It’s easy to treat AI as an oracle with the right answer every time – so we take AI’s rough first draft, and consider it final.

In your professional life, I bet you can’t name one other instance where you would accept a first draft as ‘finished’. You should subject ChatGPT to the same level of scrutiny.

YOU DON’T NEED TO ACCEPT AI’S FIRST ANSWER

When you problem solve with ChatGPT, the only thing standing between you and a better idea is a mouse click. At the blackjack table of problem solving, ChatGPT is your virtual solutions-dealer and that ‘regenerate’ button is your way of saying ‘hit me’. You can click it as many times as you need to until you get to 21.

And if you don’t love what you’re getting, it’s as simple as telling it as much. Give it feedback just like you would with a more junior coworker. The level of effort required to be presented with dozens of ideas is so low, there’s just no excuse to settle for the first average response it can give you.

But determining what is actually “good” requires context. That’s because average responses never outwardly look bad. You need perspective to be able to grade the quality and effectiveness of an idea. Motorola’s Razr was a cool phone in 2004, but stack it up against the iPhone 15 and it starts looking pretty rudimentary.

It’s a matter of shifting your mindset from ‘what do I think of it?’ to ‘what else could I think of it?’ How else could I approach this? This sort of thinking transforms AI from dead weight, to a sail, and then ultimately to an Iron Man suit. Your imagination is the only limitation, but imagination is a function of exposure.

3 TIPS FOR GETTING MORE OUT OF AI:

Using AI every day, for multiple use cases, will strengthen your conversational abilities. But there are some things you can start doing immediately as well.

1. Use the voice function

OpenAI’s voice to text software, Whisper, is a fantastic way to have a more natural brainstorming session with AI. You don't have to have all of your thoughts together. You can be totally circular. Talk to it like you would a colleague. By doing so, you bypass the bottleneck of your fingers, and the pursuit of perfection that often accompanies the human approach to a blinking cursor.

2. Turn the tables and ask AI to prompt you

Everybody says context is critical to good prompting, but what if you don’t know what context is useful? Ask ChatGPT to ask you 3 questions about your prompt. Ask it to interview you about the way you like to communicate. Allow it to prompt you for the context it needs to provide a better answer. Remember, AI is a colleague, and this is a collaboration.

3. Never take the first answer

Know that you’re receiving an “answer of averages,” and always ask AI for more options – even if you like the first one it gave you. Tell it that! And ask it to give you five more just like it.

The ‘regenerate’ option is a quick and easy way to have AI try its answer again. But you can also ask for more perspectives: Ask it to respond in three different ways, three different voices, three different directions. You may still go with the first answer, but at least you’ll have the perspective to know it was actually a good one.

One “advanced tip” to take your game to the next level: critique the AI’s response. Articulate why you don’t like the first answer, even if it’s decent, and what it’s lacking. Then ask it to regenerate a response based on your feedback.

AWARENESS IS THE FIRST STEP

I can’t promise that simply knowing we have these cognitive biases is a silver bullet to mediocrity, but it can help you recognize when you’re getting in your own way. While these tips may sound simplistic, a fresh approach is often the cognitive reset we need to overcome our biases.

People who knew nothing about generative AI two years ago are now world experts. It all comes down to digging a little deeper and staying curious about what else it can help you do.

If you don’t have two years, Section can get you a whole lot closer in 8 weeks. The AI for Business Mini-MBA is enrolling now and subscribers to Paint & Pipette get 25% off with the code JEREMYMBA (I do not get a cut of the action!!). With only 100 spots available, I’d recommend claiming yours.

Note: A version of this post originally appeared on the Section blog under the title ChatGPT’s Unhelpful Responses May Be Your Fault — reposted here with Section’s blessing!

Related: Hack Your Creative Block
Related: Beyond the Prompt: Kian Gohar and David McRaney
Related: Section School: ChatGPT’s Unhelpful Responses May Be Your Fault

Join over 24,147 creators & leaders who read Paint & Pipette each week