Piers Steel at the University of Calgary was nice enough to respond to my request for more information about his mathematical formula called Synthetic Validity; A mathematical theory that attempts to bridge the gap between individuals' personalities and abilities. Synthetic Validity, while being touted as the next big thing in Human Resources, may also be a good fit for the online dating industry.
I think pheromones are better for compatibility testing but what do I know?
Here's what Mr. Steel has to say:
Originally designed for industry and their stricter requirements (e.g., able to withstand legal challenges), synthetic validity is a methodology to create a fully automatic selection system. By changing the metric from job performance to relationship satisfaction, you can use it to select romantic partners as well. In fact, it is much easier to use it for dating. It is simply putting “soul mate” selection on a firm measurement and scientific foundation, taking advantage of 100 years of selection research.
The methodology is complex in parts, and if you want to know how it works, you can download a copy directly.
However, be sure that it does work. The International Journal of Selection and Assessment, where it was published, is a peer-reviewed top management journal that specializes in this topic (see http://www.blackwellpublishing.com/journal.asp?ref=0965-075X). This is the sum of decades of effort and it starting to come under an increasingly amount of press:
Science Daily
Globe and Mail
Peter thinks the Globe and Mail article does the best at summarizing the process, so if you have trouble accessing it, here is a key paragraph:
The mathematics involved are complex, but the idea itself is devilishly simple. First, a particular job is broken down into a series of behaviours -- common building blocks that could be assembled in different combinations to describe any kind of work. Coaching others, or a customer-service orientation are examples of such building blocks. Then, the predictors of those behaviours, from tests of individuals, are ascertained. The next, and most complicated step, is to establish the relationship between the two.
As it applies to dating, the methodology begins with breaking down relationship satisfaction into fundamental elements (e.g., physical, emotional, intellectual) that encompass the important parts of any relationship. The association between each of these elements and any predictor (pretty much anything you can quantify) is determined. To apply it to new candidate, they simply indicate what relationship elements are most important to them and how much. After that, the algorithms take over and provide your results.
What can it do?
1. To begin with, it can provide an overall ranking as well as break it down to specific relationship domains. In other words, it tells you who are overall the best as well what areas of the relationship they will likely excel at.
2. Because you have it overall as well as by life domains, there are a variety of measurement issues that come into play that improves your accuracy of prediction. For example, people can weight what life areas are most important to them (e.g., adventure, work, religion).
3. You can start to investigate generalizability, determining how many different life domains can be meaningfully separated (this will again help prediction).
4. To prevent user fatigue, you can use different sets of predictive tests. You can start with a brief selection system, provide increasingly complex assessment in stages, or use full-blown selection. You will able to predict at each stage.
5. You can select for short-term as well as long-term attraction and happiness. We ran into this problem in personnel selection a while ago. Some people were great out of the gate but slow in the stretch. The solution is time-varying covariates, but it requires gathering data at many stages of the relationship.
6. You can give confidence intervals around expected satisfaction at any time and how the relationship is expected to progress.
7. You can mathematically determine what set of predictors works the best.
You can also start moving beyond simply selection and provide advice.
1. You can point area of strength and “hot spots,” where tension is likely to arise.
2. You can suggest areas of personal improvement that will help people interest those they would be happy with.
Again, the choice of predictors is pretty much whatever you want; as long as the predictor is quantifiable, you can incorporate it in to the selection system. Consequently, it would make sense to develop a culture of constant improvement. Let different predictors compete against one another by constantly feeding the system new data (by the way, the new data is NOT simply responses on predictor tests alone). You still will retain backward compatibility. That is, later users will be even better matched but you can still provide services to those who have to yet to update their profiles.
It short, your dating system should be more accurate and provide richer feedback than anything else in the market and furthermore there are clear steps for constant improvement to maintain that advantage. If you want to explore using this methodology, contact Ian Bell at the Technological Transfer unit at the University of Calgary.
Update: Here's a list of 10 more scientific papers about theories of romantic relationships development.
Technorati Tags: personality assessment, synthetic validity
1. Fernando Ardenghi![[TypeKey Profile Page]](http://dating.corante.com/nav-commenters.gif)
on March 13, 2006 7:12 PM writes...
Synthetic Validity, while being touted as the next big thing in Human Resources, may also be a good fit for the online dating industry.
--------------------------------
I do not think so!
Why?
Because I had read the paper: "From the Work One Knows the Worker: A Systematic Review of the Challenges, Solutions, and Steps to Creating Synthetic Validity"
INTERNATIONAL JOURNAL OF SELECTION AND ASSESSMENT VOLUME 14 NUMBER 1 MARCH 2006
Abstract:
Synthetic validity has been promised as the future for selection, providing an inexpensive, fast, high_quality, legally defensible, and easily administered process. Despite 50 years of development, this promise has yet to be realized. However, recent advances in areas such as validity generalization indicate that synthetic validity is technically feasible and practically achievable. Consolidating new and previous work carried out on two synthetic validity strategies, the job_requirement matrix and job component validity, we review the methodological steps required to build them and provide working examples. Although the resources required for full realization of synthetic validity are large, similar, although larger, projects have been undertaken in the past and in the present, and there is increasing infrastructure to facilitate them in the future.
At page 17: "Challenges of Building Synthetic Validity
Given the many advantages of synthetic validity, it has been championed consistently and strongly. Dunnette (1976) called for the linking of the two worlds of human behavioral taxonomies (p. 477), namely the domain of job behavior constructs e.g., delegation, attention to detail, oral communication) with the domain of individual differences constructs (e.g., mental capability, personality characteristics). Harvey (1991), in his opening abstract, stated that linking these two domains is one of the great tasks that must be undertaken in job analysis. Schmitt and Robertson (1990) noted that the sentiment of linking these domains has been emphasized in virtually every selection review."
At page 18 "There are two primary methods for achieving synthetic validity, both of which rely on decomposing jobs into a standard set of job elements or characteristics. The first, the job_requirement matrix (JRM) (Peterson & Bownas, 1982), establishes the relationship between job elements and overall performance as well as selection predictors. If all the relationships are stable and if overall performance can be consistently predicted by the job elements, then we can derive the appropriate validity coefficients. The second, job component validity (JCV), uses these job elements to predict validity coefficient values."
Perhaps Synthetic validity will perform very well for JOB SELECTION AND ASSESSMENT but not for Online Dating.
TERRIBLE WEAK POINT (FOR ON LINE DATING): In other words, they will use weighted least squares (WLS) multiple regression method (meta_analytic).
Also the Meta Analytic Programs (Meta-Analysis Mark X and Meta-Analytic Coding Software in Microsoft Excel files) can be seen at http://www.ucalgary.ca/~steel/procrastinus/meta/meta.html
The Online Dating Industry urgently needs more power calculation and more precise/reliable matching techniques, like "here you have a list of 48 persons most compatible in a 10 million database's prospects".
Here is the link where I post comments about Scientific Papers regarding
THEORIES OF ROMANTIC RELATIONSHIPS DEVELOPMENT
and
METHODOLOGICAL AND DATA ANALYTIC ADVANCES IN THE STUDY OF INTERPERSONAL RELATIONSHIPS == QUANTITATIVE METHODS USED TO EVALUATE ATTRIBUTES AND HOW TO COMPARE THEM BETWEEN PERSONS.
http://mb.internetdatingconference.com/viewtopic.php?t=395
Kindest Regards,
Fernando Ardenghi.
Buenos Aires.
Argentina.
ardenghifer@gmail.com
Permalink to Comment2. jack mardack on March 14, 2006 2:43 AM writes...
Whether you will be heartened or discouraged by developments such as this depends on whether you presume that the Web's greater service to human relationships will be as *medium* or as *tool*. This is clearly a tool. It is intended (or purports) to afford people insight into an area of human activity where we would all wish to be more insightful. As a result, I think there will be many popular "tools" in the future. Some of these will attempt to use technology to illuminate the mysterious questions of personality compatibility and sexual chemistry. Still, others will be used by people who want to answer some of the "scary unknowns" of meeting strangers -- like background checking services, genetic assessment and relationship track-record.
Ever since I became active in the space, I must confess, I have been much more interested in developing the *medium*, rather than in making tools. It seems to me that developing the medium for human interaction, and pursuing new forms of gratification for basic human needs, is evolutionary. Whereas tool making, I would say, tends to re-inforce the fear that keeps most of us lonely, by creating new dependencies on new technologies.
I am far more in favor of evolving *us* for online use.
I recently wrote an article very close to these questions: The Power of Virtual Personae
Regards, and thanks for keeping up the vigil!
Permalink to CommentJack Mardack
profitlabinc.com
3. Piers Steel on March 15, 2006 6:42 PM writes...
I am not exactly sure how I feel being "dissed" by an electonics engineer who has been aggressively promoting some very dubious dating tool (even to me) , that is " a new way to compare the codification of one test to the codification of the others. (uses QUANTUM MECHANICS MATH and STATISTICAL EQUATIONS)."
If you think his opinion should be valued, check out his link. If you still think he should be listened to, well good luck with your life.
Permalink to Comment4. Piers Steel on March 15, 2006 6:51 PM writes...
By the way, synthetic validity has two forms (if you read the paper): the JCV and JRM. In the paper again, I recommend using the JRM methodology, which doesn't using WLS or meta-analysis. I have trouble believing that you give this guy web-time:
TERRIBLE WEAK POINT (FOR ON LINE DATING): In other words, they will use weighted least squares (WLS) multiple regression method (meta_analytic).
Permalink to CommentAlso the Meta Analytic Programs (Meta-Analysis Mark X and Meta-Analytic Coding Software in Microsoft Excel files) can be seen at http://www.ucalgary.ca/~steel/procrastinus/meta/meta.html
5. relaxedguy![[TypeKey Profile Page]](http://dating.corante.com/nav-commenters.gif)
on March 15, 2006 7:12 PM writes...
Piers, welcome to the blog world. People leave comments about things all the time, you can choose to pay attention to them or not, your mileage may vary. That's part of the scientific side of matchmaking, not many people understand it, so people like you are our only hope for catching the flaws, errors and BS. Not many people who run dating sites understand the value of an effective personality or matching test. They have made a lot of money without them, it's the smaller sites (everyone out of the top 25) that need to hear your story. WeAttract sang your song a few years ago and sold their concept to Match, which killed it off, only to have Yahoo pick it up and do a great job integrating it with the site. Not asking you to defend your ideas, just explain them for laypeople like dating site CEO's.
Permalink to Comment6. Fernando Ardenghi![[TypeKey Profile Page]](http://dating.corante.com/nav-commenters.gif)
on March 15, 2006 8:33 PM writes...
Dear Dr. Steel:
Very pleased to contact you once again.
Have you got a DEMO about "synthetic validity applied to online dating" as a quantitative method used to evaluate attributes and to compare between persons?
I will respectfully recommend to post how exactly the method you invented works (without divulging proprietary information) in order to analyze if there are advantages over other methods.
Kindest Regards,
Fernando Ardenghi.
Permalink to CommentBuenos Aires.
Argentina.
ardenghifer@gmail.com
7. Piers Steel on March 18, 2006 2:49 PM writes...
Well, the point of publishing it in a peer-reviewed top scientific journal that specializes in selection and assessment is that it has been already vetted by a bevy of experts. Again, if you want to take a look at it, there is a link to the article in the discussion above. It does what it does; there is nothing to hide.
I have been in the selection business, though not on the dating side, for a quite a while now and have seen pretty much every aspect. It is a common problem that the key decision-makers very rarely have the technical knowledge to differentiate between good and bad systems. For some it must be very frustrating, but most don't admit that they really don't know.
It is looking like the easiest thing is probably for us to build it for business first, get the capital from that, and then expand into dating. We were hoping to speed things along by licensing it for dating, sign an exclusivity agreement, and then use that money for the start-up, but it looks like it won't be as expensive to build as we thought. It would be interesting to see if simply first-mover advantage can long withstand competiting with simply a much, much better product.
Permalink to Comment8. Fernando Ardenghi![[TypeKey Profile Page]](http://dating.corante.com/nav-commenters.gif)
on March 19, 2006 2:28 AM writes...
Dear Dr. Steel:
I had read/studied the paper "From the Work One Knows the Worker: A Systematic Review of the Challenges, Solutions, and Steps to Creating Synthetic Validity"
In other words:
1- FOR JOBS
The validity of a job selection procedure may be established with respect to different domains (components) of work, then 'synthesized' (combined) for use based on the domains (or components) of work relevant for a given job or job family.
Suppose: There are B available jobs / open positions
There are A applicants / persons applying to those B jobs
Number of Comparisons == A * B
E.G.: 10 available jobs / open positions,
8,000 applicants / persons applying to all those 10 jobs
Number of Comparisons == 8,000 * 10 == 80,000
Each comparison calculated to predict job performance of (applicant k) with (job l), k integer from 1 to A; l integer from 1 to B.
Job Performance of applicant predicted by a combination of ability&aptitudes from a battery of cognitive tests, physical/psychomotor/perceptual_abilities from a collection of attributes relevant to the physical requirements of a job, personality, summarized by the Big Five model and biodata: facts regarding life history in a multiple regression equation.
For personality traits, Big-5/Big-7 can be used.
- With 10 degrees per independent variable at the results of the test (1,2,3,4,5,6,7,8,9,10):
If only 5 variables are used, like a 5 Traits Inventory, Big-5 dimensions of personality: Neuroticism, Extraversion, Openness, Agreeableness and Conscientiousness; the Ensemble (the whole set of different possibilities) is 10*10*10*10*10 == 1 E5 == 100,000 different personality types.
If only 7 variables are used, Big-7 dimensions of personality: Neuroticism, Extraversion, Openness, Agreeableness, Conscientiousness, Self-Esteem and Self-Doubt; the Ensemble is 10*10*10*10*10*10*10 == 1 E7 == 10,000,000 different possibilities.
An example of Job Performance Prediction can be seen at this link (uses only 12 personality items and probably was calculated with a multiple regression equation):
http://www.computerpsychologist.com/tcp2000/careersampr.htm
Also, what is surprising to most people is that psychological testing is a better predictor of job performance than any other single measure.
http://www.computerpsychologist.com/tcp2000/news2.htm
From: Hunter, J.E., and Hunter, R. F. 1984. Validity and utility of alternative predictors of job performance. Psychological Bulletin, 96, 72-98. also cited at your paper.
------------------------------------------------------------
You had previously said:
"Synthetic Validity, while being touted as the next big thing in Human Resources, may also be a good fit for the online dating industry.
..................................................................................
By changing the metric from job performance to relationship satisfaction, you can use it to select romantic partners as well"
I do not think so!
2- FOR DATING: The main challenge.
An Online Dating Site's Database contains B profiles (persons) with M men and W women (suppose all straight men and women)
B == M + W (number of profiles)
Number of Comparisons needed to assign Compatibility between prospective mates == M * W
E.G.: Suppose an Online Dating Site's Database that contains 10,000,000 men and 12,000,000 women
B == 10,000,000 + 12,000,000 == 22,000,000 (number of profiles)
Number of Comparisons == 10,000,000 * 12,000,000 == (1.0 E7) x (1.2 E7) == (1.2 E14) == 120,000,000,000,000
Each comparison to predict relationship satisfaction of (man k) with (woman l), k integer from 1 to M; l integer from 1 to W.
Suppose relationship satisfaction between prospective mates (dyadic) predicted by a combination of personality traits, physical appearance (race, hair, eyes, BodyMassIndex, etc) and like&dislikes (food, smoking, drinking, etc)
Big-5/Big-7 as a main core of compatibility matching is not enough for online dating, because Dating Sites have very big databases (like 20,000,000 of different profiles!!!) and if Big-5/Big-7 is used, the whole precision is less than you could achieve searching by your own!!!
"For detailed feedback or predictive purposes, one should assess the more specific primary factors. Research has shown that more specific factors like the primary scales of the 16PF Questionnaire predict actual behavior better than the Big 5 Global Factors.
For example, one extravert (a bold, fearless, high-energy type) may differ considerably from another (a sweet, warm, sensitive type), depending on the extraversion-related primary scale score patterns, so deeper analysis is typically warranted" source IPAT's 16PF5 Test Results Manual.
Using the complete 16 Personality Factors model, 16PF test, the Ensemble (the whole set of different possibilities) is 10*10*10*10*10*10*10*10*10*10*10*10*10*10*10*10 == 1 * E16 == 10,000,000,000,000,000 of different possibilities (personality types), thus each person is really unique. This is only the whole set of different personality possibilities, the way to compare one to others is a another matter.
16 Personality Factors: (A)Warmth, (B)Reasoning, (C)Emotional_Stability, (E)Dominance,
(F)Liveliness, (G)Rule_Consciousness, (H)Social_Boldness, (I)Sensitivity, (L)Vigilance, (M)Abstractedness, (N)Privateness, (O)Apprehension, (Q1)Openness_to_Change, (Q2)Self_Reliance, (Q3)Perfectionism and (Q4)Tension
The World Population (WP) is nearly 6,400 millions persons == 6.4 * E9
16PF's Ensemble == 1 * E16
WP / Ensemble == (6.4 * E9) / (1 * E16) == 6.4 * E-7 == 64 * E-6
i.e. All World Population is 64 micro part of the Ensemble!!!
Although personality traits: NEO, 16FP5 can be measured with high precision (and for mature persons are quite stable), physical appearance and like&dislikes CAN NOT be measured with high precision unless you meet_in_person each person (many persons trend to overestimate/overvalue their physical appearance and unfortunately these values could change along the years. A woman with brown eyes who uses blue_contact_lenses will say: "I've got blue eyes").
As far as I could analyze, the paper "From the Work One Knows the Worker: A Systematic Review of the Challenges, Solutions, and Steps to Creating Synthetic Validity" DOES NOT EXACTLY SPECIFIES THE EQUATIONS NEEDED to calculate/predict relationship satisfaction between prospective mates.
If only Big-5/Big-7 personality traits were involved in a multiple_regression relationship_satisfaction_equation, then the whole precision is less than you could achieve searching by your own!!!. This lack_of_precision / lack_of_reliability problem is being suffered by actual "Compatibility Matching" dating sites.
But if only 16PF personality traits were involved in the relationship_satisfaction_equation, it could not be a multiple regression equation, because All World Population is 64 micro part of the whole set of different personality possibilities!!!!!
If you conclude that more precise&powerful equations than simple/multiple linear regression analysis will be needed, then it is LPM (LifeprojectMethod), the online dating method I had invented and I am trying to launch!
Kindest Regards,
Fernando Ardenghi.
Buenos Aires.
Argentina.
ardenghifer@gmail.com
LifeprojectMethod: More precise&powerful equations than the ones suggested at
- "DIFFERENCE SCORE CORRELATIONS IN RELATIONSHIP RESEARCH: A CONCEPTUAL PRIMER." Griffin, Murray, and Gonzalez.
http://lilt.ilstu.edu/personalrelationships/contentsofissues/pdf_articles/Volume%206,%20Issue%204/Griffin.pdf
- "THE CORRELATIONAL ANALYSIS OF DYAD_LEVEL DATA IN THE DISTINGUISHABLE CASE." Gonzalez and Griffin.
http://lilt.ilstu.edu/personalrelationships/contentsofissues/pdf_articles/Volume%206,%20Issue%204/Gonzalez.pdf
- "PARTNER EFFECTS IN RELATIONSHIP RESEARCH: CONCEPTUAL ISSUES, ANALYTIC DIFFICULTIES, AND ILLUSTRATIONS." Kenny and Cook .
http://lilt.ilstu.edu/personalrelationships/contentsofissues/pdf_articles/Volume%206,%20Issue%204/Kenny.pdf
Permalink to Comment9. Fernando Ardenghi![[TypeKey Profile Page]](http://dating.corante.com/nav-commenters.gif)
on March 20, 2006 7:09 PM writes...
Suppose an Online Dating Site's Big Database with 10,000,000 men and 12,000,000 women profiles.
Number of Comparisons == 10,000,000 * 12,000,000 == 120,000,000,000,000 /// each one to predict relationship satisfaction of (man k) with (woman l)
If the computer (or high_speed server arrangement) calculates the value of relationship_satisfaction_between_prospective_mates as fast as 10,000,000 per second.
Permalink to CommentIt will require 12,000,000 of seconds, nearly 139 days!!!