Take the 2-minute tour ×
Writers Stack Exchange is a question and answer site for authors, editors, reviewers, professional writers, and aspiring writers. It's 100% free, no registration required.

Are there any known variables to measure, or questions to ask in survey, in order to asses the persuasiveness of a paper?

For example, I'm not sure if omitting a section from my paper might actually improve it. It's good writing and a good point. So I'm wondering how to take the subjectivity out of the decision so I know I'm making the right choice.

I was thinking I could do an A/B test - I could prepare both versions of the paper, and randomly give one to readers who volunteer to participate in a survey after reading. What kinds of questions could I ask, which would allow me to compare the answers and identify the most persuasive form the of paper?

I think a survey would be easiest for me to execute, but if it's not a good way to decide this, is there something I could look for as the reader is reading? Maybe, track eye movement and see what percent they skimmed? Anything along those lines?

How can I tell which of 2 versions of a paper is most persuasive?

I could ask "On a scale of 1-5 how persuasive was this paper?" And then compare the average rating from each group.

share|improve this question
    
Anyone know a more relevant tag for this question? –  Pat Meeker Jul 10 '13 at 16:52
1  
Re: Tags - Does this relate to academic, business, or tech writing? –  Neil Fein Jul 10 '13 at 18:39
    
Re: Re: Tags - I added tags for tech and business, thanks for the suggestion! –  Pat Meeker Jul 15 '13 at 17:36

1 Answer 1

"Persuasiveness" is highly subjective. I can't imagine how you would measure it other than to perform an experiment with real people.

You didn't say what the subject of your paper is, which is probably good because that helps us to discuss the question of persuasiveness without being biased by whether or not we agree with you.

I know of three basic approaches to measuring persuasiveness:

(1) Get a group of test subjects. Have them read the paper. Ask them to rate how persuasive it is on some scale, 1 to 10 or whatever.

Catch: People who already agree with you will likely say that it is very persuasive, while people who disagree with you will tend say it is not persuasive at all, no matter how compelling the actual arguments are. You may well be measuring how many people agreed with you before reading the paper then anything about the paper itself. Often people disagree not over the facts themselves but over which facts are important.

For example -- and I'm going to use a real example here because it's hard to make this point purely in the abstract, but I'm trying to concentrate on the meta-issue of persuasiveness and not any particular issue -- so for example, suppose you were arguing against legalizing marijuana, and so you gave evidence that marijuana is bad for people's health. Someone who was pro-legalization might well say that even if true, this is irrelevant and therefore unpersuasive, because people should have the freedom to decide for themselves whether the perceived benefits outweigh the harm. On the other hand, suppose you were arguing for legalizing marijuana, and you pointed out how many people were jailed or lost their jobs or whatever just because they were caught smoking. Someone who was anti-legalization would likely reply that this is unpersuasive because the whole idea of making something illegal is to impose sanctions on people who break the law so that people will be afraid to do it: if these laws didn't hurt people, then they would be ineffective. Etc.

(2) Get a group of test subjects. Have them read the paper. Then discuss with them what they thought the strong and weak points were. This is called a "focus group". The advantage over a simple rating is that it gives you information about hows and whys. The disadvantage is that it gives no cut-and-dried answers. Advertisers often do this with planned advertising campaigns to see how people react to them. Just hearing that on a scale of 1 to 10 this advertisement got an average score of 4 tells you little. Is that good or bad? Compared to what? Does that mean people would buy the product or not? Is there something we could do to make it more effective or is that the best that we can hope for under the circumstances? Etc.

(3) Get a group of test subjects. Give them a pre-test asking their opinion on the subject matter. Then have them read the paper. Then give a post-test. See how many changed their minds.

I think this is the most effective method because it directly measures what you are presumably trying to accomplish: changing people's minds.

Catch: The more controversial the subject, the fewer people will change their minds. With a small test group, the numbers may be so small that they are not statistically significant. People's opinions about controversial subjects are built up over the course of a lifetime, from many many individual facts, ideas, experiences, etc. If a person believes that, say, political position X is good for the country, he likely has hundreds of reasons for thinking so. If you present him with a reason to think it's bad, his first reaction is likely to be that your argument MUST be flawed, because he KNOWS that X is good. Even if you convince him that your argument is valid, it's still one argument against versus a hundred arguments for. He's likely to see it as a curious anomaly, not a game-changer.

So no matter how valid and persuasive your arguments are, any effects may be very subtle, and thus difficult to measure. There may be some number of people who were undecided and who you managed to bring into your camp. But likely there are many more who were previously 80% sure that X was right but who are now only 79% sure. Etc.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.