Bookmark and Share

Online Dating Magazine > Columns > Office Hours with Dr. Jim > Compatibility Tests

Office Hours With Dr. Jim
by James Houran, Ph.D

In this column, "Dr. Jim" honestly and candidly answers your questions about dating, love and sexuality. He doesn’t tell you what you want to hear – he tells you what you need to hear. Dr. Jim is committed to offering you guidance based on responsible clinical practice and hard data from the latest scientific studies. Send Dr. Jim your questions today for consideration in an upcoming issue.

Compatibility Tests by Online Daters

Quick Access:
Compatibility Tests - Online Daters or Experts?

Why do some dating sites offer compatibility tests written by online daters and not experts?


For many sites, compatibility questionnaires are popular tools for helping consumers screen romantic prospects. It is understandable that sites attempt to enhance the sense of community and site ownership among users by enabling them to help create the site content. In these cases, such quizzes or questionnaires are much like ice breakers or fun, online activities. Also, user generated content saves dating sites lots of money. But, allowing clients to make life-changing decisions on the information from poorly designed compatibility tools may be a disservice in the long-term.

In the long-term, online daters can make incorrect and costly (financially and emotionally) decisions based on information, feedback and recommendations from poorly designed questionnaires. That said, it is unfortunate that many websites encourage amateur questionnaires, tests and quizzes– without any guidance from specialists or without any obvious disclaimers that such information is likely to be unreliable and invalid. 

Readers are treated here to a rare insider conversation with one of the most prominent experts in tests, measurement and questionnaire research. Meet Dr. Rense Lange, a pioneer in applying modern test theory methods to relationship testing and assessment. Dr. Lange recently fielded some pointed questions from me about the pitfalls of user-generated questionnaires which online daters should be aware.

Is survey and questionnaire design a science?
Definitely yes! Unfortunately, I know that too many surveys are constructed based on “instincts” and whatever else seems important to untrained people. But, this really is not the way to design good questionnaires, especially when trying to predict people’s behaviors. But, unfortunately, all too often questions are selected based on how interesting or appealing they sound.

Instead of a haphazard approach, one needs a systematic, explicit (and correct) theory about people’s behaviors, including factors like a person’s beliefs, attitudes and intentions. Even when dealing with the simplest and shortest of questionnaires one has to know all these issues so that the right ones can be included or omitted.

Designing surveys and questionnaires is often regarded as something anyone - even non-specialists - can do. Why is this?
There are all kinds of bad reasons for this. The most important being that given we can all talk, we also can all write good questions. This reasoning has the same flaws as saying that since we all went to school we are educational experts, or since we have all been sick we all have medical qualifications.

This will not work for serious questionnaires. I find it amazing to see how million dollar decisions are quite often made based on shoddy questionnaires, guided mainly by questionable insights and theories that have long been discarded in the scientific psychological literature.

Maybe this is because people do not know how behaviors can be predicted quite reliably from the right indicators. Also, people tend to over-emphasize their own pet explanations and insights based on anecdotal evidence or mistaken media reports. I have even seen cases where making a questionnaire was mostly seen as a matter of typing - so, they had the secretary do everything.

Most people - even many survey vendors - analyze questionnaire and survey data with traditional approaches like raw-score sums, percentile rankings or percentages. What is wrong about these standard approaches?
Standard design, analysis and reporting are often wrong and incomplete at so many levels. For instance, take the use of rating scales such as “agree completely,” “agree somewhat,” and so on. Here it is often assumed that (a) using more answer categories is always better, (b) some “neutral” category is needed to allow people to be non-committal. Both of these “insights” are wrong. Most people cannot handle more than six pieces of information at a time, so do not give them more response categories than that. In fact, to be on the safe side, four categories are probably fine. Also, neutral categories are usually counterproductive; they rarely get you the information you want. Often, neutral categories do not reflect uncertainty or indecision, but rather they hide socially undesirable answers. That is, they mean “I do not want to say” or “does not apply.”

The preceding is a direct consequence of the use of poor methods of analysis. I believe that our method of analysis should tell us whether middle categories are used inconsistently - just like they should tell us whether someone is giving valid information in the first place. Second, middle categories are often seen as a panacea for avoiding missing data that would foil standard statistical procedures.

For these reasons, the leading edge assessment companies rely almost exclusively on the use of Rasch scaling. It is infeasible to go into great detail here, but this approach is unique in that (a) missing data are inherently acceptable, (b) we can clearly judge the quality of the data and the questionnaire from the responses, (c) one obtains linear (i.e., interval-level) measures and (d) it can be determined whether (and if so, how much) the data are biased by factors such as age, gender and other demographics. Such information allows daters to make more targeted and valid relationship decisions, whereas traditional approaches like raw-score sums, percentages and percentile rankings are severely limited and can even be misleading.

Are there any exciting developments in the field of assessment, surveys and questionnaires?
Surprisingly, professional survey companies (like those who make compatibility questionnaires) almost never use Rasch scaling. Thus, the business advantage for websites that apply these methods in their analytics for customers is huge. My personal favorite development is the application called “Action Plans.” Here we build a mathematical model of the data and this allows us to identify statistical outliers. We then feed this into our software to generate an interpretation for the observed misfit. The result is a person specific and tailor-made diagnostic profile that can be used in a variety of ways, such as customized and automated date coaching or relationship development.

The Bottom Line
Keep these points in mind to help avoid pitfalls in the way you use feedback reports from online surveys on dating sites. If there’s no scientific basis for the questionnaire, then think of it as simply a fun, diversion to start a dialogue with others. Using such quizzes and questionnaires “for entertainment purposes only” is fine, as long as you know what kind of questionnaire you’re using and then act accordingly.

Related Links:
» Interview with Dr. Rense Lange
» The Truth About Compatibility Testing
» Compatibility Tests and Attraction

> - The best approach to find the one. <

All Online Dating Magazine content, including the content on this page,
is © copyright by Online Dating Magazine and may not be
republished or reused in any form. You do have
full permission to link to this article.

Do you agree or disagree with this article? Have
more to add? Submit a Letter to the Editor today or post a comment below.


blog comments powered by Disqus

Privacy Statement | Code of Ethics Statement
Bookmark Online Dating Magazine at

>View Online Dating Magazine Singles Travel Adventures<

Online Dating | News | Columns | Features | Dating Services | Niche Dating
Online Dating Directory | Dating Humor | Quick Tip Articles | Online Dating Industry
Industry History
| Online Dating Reviews | Reader Reviews | Dating Videos | Book / DVD Reviews
Reader Letters | Self Improvement | Experiences | Newsletter | Interviews
Top 10 Lists | STD Info Center | About Us | Advertise
Media Center
| FAQ | Search | Contact | Dating Promotions

Dating Cartoons | Dating Jokes | Funny Dating Videos | Dating Games

Online Dating Magazine Needs Your Help!

All content on this Website is ©copyright by Online
Dating Magazine. All Rights Reserved. The content
on this site may not be reused or republished.

Are you an online dating site Webmaster? If so...
If you are looking for free dating content you can republish, click here



Advertise on
Online Dating Magazine



Stay Up to Date:

Follow Us on Twitter

Follow Us on YouTube

Follow Us on Facebook


10 Most Viewed Articles:

Online Dating Tips
Opposites Attract
Online Dating Safety Tips
Rose Colors
Guys are Weird
Dating Games
Online Dating Reviews
Creative Date Ideas
Long Distance Relationships
Dealing with a Breakup

Other Resources:

These links will take you away from Online Dating Magazine.

Google (Google Your Date)
Research Dating Service)

MailOrderBride Guide

Note: Online Dating Magazine does not sell text links anywhere on the site, so please don't email asking about text links. Any links we have are affiliate links, resources we find useful, links to other sites we run or long-time magazine sponsors. We do have graphic ad options in our Advertising section.