DatingMashup.com


 
Bookmark and Share

Online Dating Magazine > Columns > Office Hours with Dr. Jim > Compatibility Testing

Office Hours With Dr. Jim
by James Houran, Ph.D

In this column, "Dr. Jim" honestly and candidly answers your questions about dating, love and sexuality. He doesn’t tell you what you want to hear – he tells you what you need to hear. Dr. Jim is committed to offering you guidance based on responsible clinical practice and hard data from the latest scientific studies. Send Dr. Jim your questions today for consideration in an upcoming issue.


Are There Standards for Compatibility Testing?

Quick Access:
Compatibility Testing Standards


Are there standards for compatibility testing?

 

I can see why this is a question on consumers’ minds, given there are so many choices in dating sites nowadays. Many of these sites have their in-house or consulting “love doctor,” or psychologist or other authoritative guru. Ironically, higher education in the social sciences doesn’t necessarily equate with expertise in developing effective questionnaires and assessments.

Indeed, a critical look at the major dating sites reveals little to no science behind the quality of many popular “compatibility tests.” (4, 5) To date, at least one media report3 has exposed the trend for matchmaking companies to profit from unsubstantiated personality and compatibility tests. Other experts (2) have similarly noted the difficulties of ascertaining the credentials and identity of service providers, accessing accurate information, reliance on untested methods, difficulties in online assessment, and the lack of standards and regulation regarding online human service practices. This opinion was also echoed and expanded in a high profile article recently published in the American Psychologist. (6)
 
The online dating industry is business-oriented, not academically-oriented. As such, there are no industry standards for compatibility tests. However, the academic community does have clear professional standards for constructing tests – a manual entitled, Standards for Educational and Psychological Testing.(1) The actual standards are heavy reading, but basically they translate into a few simple questions:

1) Is the test relevant to the audience?
That is, the content should be readable and meaningful to the intended audience.

2) Is it reliable?
That is, the test should yield consistent or trustworthy scores.

3) Is it valid?
That is, the test should measure what it claims to measure and predict what it is supposed to predict.

This is not all academic mumbo jumbo. Let’s review the differences in bad and good tests and surveys to customers…

Bad Compatibility Tests:

  • Can be long and difficult to complete and can alienate some sub-groups of online daters.
  • Waste online daters’ valuable time and money.
  • Can fail to identify the most important compatibility issues, which then continue to plague the person or a couple.
  • Can lead individuals to make incorrect and emotionally and financially costly relationship decisions.
  • Provide a false sense of objectivity and security that may not be valid, defensible or scientific.

Good Compatibility Tests:

  • Can be designed to eliminate irrelevant questions, be more valid with fewer questions, and be easier and faster for respondents to complete.
  • Ensure online daters’ time and money are spent on valid tools that gain expected results.
  • Can identify opportunities that make a real difference in the success of screening for romantic prospects.
  • Can support you in making correct relationship decisions and allocating your energies where they will make the most difference.
  • Give you peace of mind because they are valid, defensible, based on proven scientific theories.

So, why are bad compatibility tests prevalent in the marketplace? There are many reasons, and these include:

  • Average consumers can’t tell whether a tool is really, so companies take advantage of this ignorance.
  • Companies can’t find or identify assessment vendors who are qualified to construct scientific compatibility tests.
  • Companies won’t or can’t invest in the money it takes to construct a test in a way that meets professional testing standards.
  • Companies are too impatient to wait for a properly constructed compatibility test; they simply want to launch an application as soon as possible to start making money.
  • Companies simply choose to license an “off the shelf” compatibility test and trust that it’s a scientifically sound tool.
  • Companies attempt to cut corners so they “throw together” their own test based on their own pet theories and ideas about love.

This last excuse is one of the more prevalent reasons for bad tests. I know of too many compatibility tests that are constructed based on “instincts” and whatever else seems important to untrained people who head up online dating sites. But, this really is not the way to design good tests, especially when trying to predict relationship quality. Unfortunately, all too often questions are selected based on how interesting or appealing they sound or based on the favorite insights of ones’ superiors in the organization. Let’s look at the issues involved in more depth.

  • Designing surveys and questionnaires is often regarded as something anyone - even non-specialists - can do. There are all kinds of bad reasons for this. The most important being that given we can all talk, we also can all write good questions. This reasoning has the same flaws as saying that since we all went to school we are educational experts, or since we have all been sick we all have medical qualifications.

    This will not work for serious questionnaires. I find it amazing to see how million dollar decisions are quite often made based on shoddy questionnaires, guided mainly by questionable insights and theories that have long been discarded in the scientific psychological literature.

    Maybe this is because people do not know how behaviors can be predicted quite reliably from the right indicators. Also, people tend to over-emphasize their own pet explanations and insights based on anecdotal evidence or mistaken media reports. I have even seen cases where making a questionnaire was mostly seen as a matter of typing - so, they had the secretary do everything.

  • Most testing vendors analyze questionnaire and survey data with outdated and ineffective approaches like raw-score sums, percentile rankings or percentages.  Standard design, analysis and reporting are often wrong and incomplete at so many levels. For instance, take the use of rating scales such as “agree completely,” “agree somewhat,” and so on. Here it is often assumed that (a) using more answer categories is always better, (b) some “neutral” category is needed to allow people to be non-committal. Both of these “insights” are wrong. Most people cannot handle more than six pieces of information at a time, so do not give them more response categories than that. In fact, to be on the safe side, four categories are probably fine. Also, neutral categories are usually counterproductive; they rarely get you the information you want. Often, neutral categories do not reflect uncertainty or indecision, but rather they hide socially undesirable answers. That is, they mean “I do not want to say” or “does not apply.”

    For these reasons, Modern Test Theory approaches like Item Response Theory and Rasch scaling should be used instead of what most compatibility tests are based on now.  It is infeasible to go into great detail here, but this approach is unique in that (a) missing data are inherently acceptable, (b) we can clearly judge the quality of the data and the questionnaire from the responses, (c) one obtains linear (i.e., interval-level) measures and (d) it can be determined whether (and if so, how much) the data are biased by factors such as age, gender and other demographics. Such information allows dating sites to make more targeted and valid matches, whereas traditional approaches like raw-score sums, percentages and percentile rankings are severely limited and can even be misleading. Finally, these scoring approaches also allow the feedback that is given to online daters to be more customized than ever before. For example, the current leader in Modern Test Theory assessments for online daters is PlentyofFish.com. PlentyofFish offers a couple of assessments that build customized coaching “Action Plans” for users that explain how the users can specifically apply their unique patterns of results for self-awareness, getting to know other people and approaching others based on “hidden” preferences that even the test takers was unaware of! 

In my view, applications like the PlentyofFish offerings are the future of compatibility testing because they combine two crucial services into one tool – date coaching and compatibility testing. Still, companies won’t be forced to adhere to professional testing standards without consensus in the industry (yeah, right) or legislation (more feasible). I mean look at how long eHarmony has gone without being sued by users for a tool that has been publicized to the point of sensationalism, yet leaves much to be desired from a scientific standpoint! (5)

This is why it is up to consumers to do their homework before investing in time and money in any testing or matching system.  And even when you find a tool you like and trust, never let the compatibility matching process become a crutch for you. These tools, even when done well, are only part of a process. They help you screen out obviously wrong matches, and load the dice for matches that have the raw ingredients for lasting relationship satisfaction. But, it still takes you getting to know the other person offline to truly test for the other elements of compatibility that no test can yet tap to a reliable degree – physical attractiveness, chemistry and a degree of trust and commitment between two people.

Related Links:
» The Truth About Compatibility Testing
» Compatibility Tests and Attraction

References:
1American Educational Research Association, American Psychological Association, & National Council on Measurement (2002). Standards for educational and psychological testing. Washington, DC: Author.

2Finn, J., & Banach, M. (2000). Victimization online: the down side of seeking human services for women on the Internet. Cyberpsychology and Behavior, 3, 243-254.

3Hahn, J. (2005, Feb 10). Love machines. Los Angeles City Beat. http://www.lacitybeat.com/article.php?id=1650&IssueNum=88. Accessed 2/10/05.

4Houran, J. (2004). Ethics in cross-cultural compatibility testing in Europe: an opportunity for industry growth. Paper presented at the Internet Dating / Online Social Networking Industry Association Inaugural Meeting, Nice, France, July 15-16, 2004.

5Houran, J., Lange, R., Rentfrow, P. J., & Bruckner, K. H. (2004). Do online matchmaking tests work? an assessment of preliminary evidence for a publicized ‘predictive model of marital success.’ North American Journal of Psychology, 6, 507-526.

6Naglieri, J. A., Drasgow, F., Schmit, M., Handler, L., Prifitera, A., Margolis, A., & Velasquez, R. (2004). Psychological testing on the Internet: new problems, old issues. American Psychologist, 59, 150-162.




>
Perfectmatch.com - The best approach to find the one. <


All Online Dating Magazine content, including the content on this page,
is © copyright by Online Dating Magazine and may not be
republished or reused in any form. You do have
full permission to link to this article.

Do you agree or disagree with this article? Have
more to add? Submit a Letter to the Editor today or post a comment below.


 


blog comments powered by Disqus


Privacy Statement | Code of Ethics Statement
Bookmark Online Dating Magazine at Del.icio.us

>View Online Dating Magazine Singles Travel Adventures<

Online Dating | News | Columns | Features | Dating Services | Niche Dating
Online Dating Directory | Dating Humor | Quick Tip Articles | Online Dating Industry
Industry History
| Online Dating Reviews | Reader Reviews | Dating Videos | Book / DVD Reviews
Reader Letters | Self Improvement | Experiences | Newsletter | Interviews
Top 10 Lists | STD Info Center | About Us | Advertise
Media Center
| FAQ | Search | Contact | Dating Promotions

Dating Cartoons | Dating Jokes | Funny Dating Videos | Dating Games

Online Dating Magazine Needs Your Help!

All content on this Website is ©copyright by Online
Dating Magazine. All Rights Reserved. The content
on this site may not be reused or republished.

Are you an online dating site Webmaster? If so...
If you are looking for free dating content you can republish, click here

 

 



Advertise on
Online Dating Magazine



Menu:

 

Stay Up to Date:

Follow Us on Twitter

Follow Us on YouTube

Follow Us on Facebook

 


10 Most Viewed Articles:

Online Dating Tips
Opposites Attract
Online Dating Safety Tips
Rose Colors
Guys are Weird
Dating Games
Online Dating Reviews
Creative Date Ideas
Long Distance Relationships
Dealing with a Breakup

Other Resources:

These links will take you away from Online Dating Magazine.

Google (Google Your Date)
BBB (
Research Dating Service)

MailOrderBride Guide

Note: Online Dating Magazine does not sell text links anywhere on the site, so please don't email asking about text links. Any links we have are affiliate links, resources we find useful, links to other sites we run or long-time magazine sponsors. We do have graphic ad options in our Advertising section.