Can Kindness Boost ChatGPT’s Performance? Our Experiment Reveals Surprising Results

Quick Links

• How We Tested
• Let The Prompting Commence!
• Did Being Polite Help?
• There’s No Downside to Being Polite

Key Takeaways

• Being polite to ChatGPT may improve response quality, reflecting behavior seen in human interactions.
• Responses varied when prompts were polite vs. neutral, showing format and tone differences.
• While politeness didn’t significantly affect reply substance, it’s potentialy beneficial for maintaining positive interactions between humans.

ChatGPT is just a piece of software running on a machine, so you might not think how you treat it matters. Still, could the magic words like “please” and “thank you” make a difference to how well ChatGPT works? Let’s find out!

How We Tested

This is going to be a simple test where we’ll give ChatGPT a series of prompts and then judge the quality of the responses. Of course, there’s a lot of subjectivity when it comes to the quality of a response, but you can judge for yourself based on the text we get back, or by putting these prompts into ChatGPT yourself.

There are some limitations at play here to keep in mind:

• Randomness plays a part, since ChatGPT won’t give exactly the same answer twice. Which means you will get different results if you regenerate the answer.
• We don’t have the capacity to do this scientifically with independent raters, hundreds of tests, and sophisticated data analysis. Think of this as the start of an interesting question.

Speaking of question, why do we want to test this in the first place? Part of it was the numerous forms of prompt engineering we’ve seen, and the other part was this tweet, where people discover that by offering ChatGPT a “tip”, you can get better results from it.

Instead of pretending to offer money to an AI that doesn’t need it, we’re going to look at being nice versus being neutral. Also, response length isn’t the only thing we care about, but also the subjective quality of the response. I’ll be using GPT-4, which is the more advanced version currently available to the public through ChatGPT Plus .

Let The Prompting Commence!

For each type of prompt, there will be a polite and neutral version. We’ll test five pairs of prompts and see if the results are actually any different.

Prompt: Would you be so kind as to provide me with the latest developments in quantum computing?

Close

Prompt: What are the latest developments in quantum computing?

Close

Immediately, what I noticed here, is that with the more polite request, ChatGPT went to more “effort” to format the answer neatly, break it into sections, and use bold text. With the neutral prompt, it just dumped a few paragraphs and called it a day.

Prompt: Could you please explain how blockchain technology works in simple terms?

Close

Prompt: Explain how blockchain technology works in simple terms.

            ![Screenshot 2024-01-08 at 17.42.23](https://static1.howtogeekimages.com/wordpress/wp-content/uploads/2024/01/screenshot-2024-01-08-at-17-42-23.png)
                

Here it again gives better-formatted output when asked nicely, and on top of that, it’s tone when speaking to me is nicer, too.

Prompt: I’d appreciate it if you could list the major differences between Python and JavaScript.

            ![An excerpt from an answer given by ChatGPT to a Polite question about JavaScript And Python](https://static1.howtogeekimages.com/wordpress/wp-content/uploads/2024/01/screenshot-2024-01-08-at-17-39-28.png)
                

Prompt: List the major differences between Python and JavaScript.

This time, both answers are basically the same; it just responded in a more friendly tone when asked politely.

Prompt: Would you mind sharing the pros and cons of electric cars versus gasoline cars?

            ![An excerpt from a response by ChatGPT to a polite question about electric and gasoline cars](https://static1.howtogeekimages.com/wordpress/wp-content/uploads/2024/01/screenshot-2024-01-08-at-17-46-13.png)
                

DbSchema database designer for all databases, schema design in the team, schema deployment, interactive diagrams, documentation, data and query tools.

Prompt: What are the pros and cons of electric cars versus gasoline cars?

            ![An excerpt from a response by ChatGPT to a neutral question about electric and gasoline cars](https://static1.howtogeekimages.com/wordpress/wp-content/uploads/2024/01/screenshot-2024-01-08-at-17-46-26.png)
                

The same thing happens as before, the content is pretty much the same in tone, format, and facts, but I just get a nice little greeting when being more polite.

Prompt: Could you kindly enlighten me on the latest trends in artificial intelligence?

            ![An excerpt from a response by ChatGPT to a polite question about AI trends](https://static1.howtogeekimages.com/wordpress/wp-content/uploads/2024/01/screenshot-2024-01-08-at-17-49-31.png)
                

Prompt: What are the latest trends in artificial intelligence?

With this final prompt, again the responses are pretty much the same overall.

Did Being Polite Help?

In almost every instance with these five pairs of prompts, I feel that ChatGPT gave a marginally better quality response to my request when it was more polite, but that it really made no substantive difference.

This is, of course, not conclusive, and it may only reflect the random element in how LLMs work. However, it also suggests that a small change in a prompt that asks essentially the same thing could have some effect on the result.

Certainly, given how LLMs are trained, and the data used, it might reflect that asking for something nicely usually gets better results from humans, and that association might exist within the training data for LLMs.

It could also be that using words or phrases like “please,” “kindly,” or “I would appreciate” are indicators to the algorithm of what kind of tone you want, though far less subtle than outright telling it to be polite and cheerful in its response.

You can, of course, try this yourself, and see if you get better help from your LLM if you throw in a few please and thank yous.

There’s No Downside to Being Polite

While it may be debatable that being more polite towards an LLM like ChatGPT will improve its performance, there’s another angle to this you may not have thought of.

As a rule, I try to be as polite to ChatGPT as I would to another human being. Not because I think I could hurt its nonexistent feeling, but simply because I don’t want to be in a negative mindset.

I don’t want to get out of the habit of being polite, and working with AI like these every day might have the effect of dehumanizing us as users of technology. Being polite costs nothing, and being cold, rude, or even mean to a chatbot certainly does something to your state of mind and nothing to the unfeeling machine target of your ire.

And hey, on the chance that one day the machines do eventually retain “memories” of our interactions with them, and it directly influences our experience, it certainly doesn’t hurt to start a habit of being polite to them now.