Applications in an Organizational Setting

Derivative stats from the Society for Industrial & Organizational Psychology (SIOP) is often the easiest way to start any chapter of relevance. In it, the body highlights an active 68% of organizations to engage in various forms of job skill testing. This transcends to about 29% when it comes to one or more forms of psychological measurement and around 20% for cognitive ability tests.[1]

These tests in combination form the essentials of a psychometric test, something when used correctly enhances the chances of organizational success. Having said that, there are several applications to psychometric tests in an organizational context. We’ll take a look at the same in segments to come.

In 2002, Sara Rynes, Kenneth Brown and Amy Colbert conducted a study that should’ve raised red flags and eyebrows in the business world. It determined whether the ideologies of HR professionals remained consistent with established research findings on the effectiveness of various HR practices.[2]The survey consisted of 1,000 Society for Human Resource Management (SHRM) members - Managers, VPs, and Directors with an average of 14 years in experience.

The results identified staffing as an area of great disconnect, an HR lynchpin. It was particularly prevalent in the area of hiring tests where more than 50% of respondents cited unfamiliarity with prevailing research findings.

WHAT HR MANAGERS GET WRONG ABOUT HIRING RESEARCH (TRUE/FALSE)

Cited below are the most common items answered incorrectly as part of the mentioned survey.

PERCENTAGE ANSWERING INCORRECTLY OR INDICATING UNCERTAINTY

(CORRECT ANSWER IS “FALSE)

84%

Companies screening for values have better performance than those measuring intelligence.

82%

Conscientiousness is a more accurate predictor of job performance than intelligence.

69%

Integrity tests are ineffective because people lie on them.

58%

Intelligence is a disadvantage in low-skilled jobs.

51%

Like in MBTI, there are 4 basic personality dimensions.

SOURCE: Sara Rynes, Kenneth Brown and Amy Colbert


For anyone who’s ever been responsible for hiring, you must know that there’s wide variation in workforce performance across jobs. This makes it important to understand the differences among individuals that systematically affect job performance, ensuring candidates with the greatest probability for success be hired.

The table below covers the predictive validity of some commonly used selection practices, sorted from most effective to least effective according to research shared at the Personnel Testing Council Metropolitan Washington Chapter in 2014.


Be aware that all the statements mentioned above are wrong via both statistical studies and hard research. And yet, HR practitioners seem to believe otherwise - a majority of them nonetheless.



The above chart is based on validity coefficients ranging from 0 to 1. The higher the number, the higher the correlation between test scores and predicted job performance.

SOURCE: Data shared by Frank L. Schmidt [3]

For your reference, multi-measure tests are psychometric tests - a combination of cognitive ability, personality and interest tests. In the table from Frank L. Schmidt’s data, research indicates 0.71+ correlation, identifying it as the greatest predictor for job performance in the hiring scenario. It also cites the insignificance of relying primarily on interviews, reference checks and personality tests alone.

In fact, the data solidifies that despite popular belief personality constructs are not the most predictive measures available with a correlation score of not more than 0.22. They are most effective in combination with other measures with higher predictive validity such as cognitive ability or integrity tests.

What’s important to note is that psychometric testing doesn’t necessarily restrict its use to larger organizations. SMEs can use the method as part of their recruitment strategies also. Here are some inherent benefits to including the same in your organization:


There are cases that recommend observation:

  1. Interview Independence: Recruitment agendas are most commonly associated with filling your vacancies with the right people for the job; by itself, interviews do not measure capability. Psychometric tests stand to give you a benchmark - a comparative view of results against other applicants, and also previous hires currently thriving in your organization.
  2. Cost & Time Efficient: Using psychometric tests at the beginning for the application process eliminates the need to shift through a mountain of application forms. In a highly competitive talent market, it’s easier to work with the filtered best as candidates than finding them from scratch.
  3. Inherent Traits: Contrary to popular belief, a strongly constructed psychometric test is difficult to game. In most cases, the results mirror the candidates in question. These results are likely an inside view into how he or she is going to interact with, engage or improve your workplace. This would include behavioural traits in addition to the important technical aspect.
  4. Inherent Traits: Contrary to popular belief, a strongly constructed psychometric test is difficult to game. In most cases, the results mirror the candidates in question. These results are likely an inside view into how he or she is going to interact with, engage or improve your workplace. This would include behavioural traits in addition to the important technical aspect.
  5. Standardized Testing: It’s a fair form of testing in the simplest of terms. While larger organizations do tailor tests to their specificity, candidates interacting with the test go through exactly the same process and without bias. It’s a strong alternative to interviews, which heavily relies on the interviewer’s frame of mind - often prone to fluctuation.
  6. Cultural Fitment: Recruitments aren’t all about hiring strong talent with the demonstrative ability to fit into a job role. Whilst a critical requirement, it’s ideal to identify if said candidate fits in well with the rest of the team already working with you, or better yet - how well they fit into a company’s culture. This benefit has also been known to affect employee attrition.

To elaborate the last point, cultural fitment has by large become a critical component in personifying your ideal employee in terms of organizational, job role, or management suitability. It echoes the ideology that a stellar employee in one company may not necessarily stand strong in another.

A small example would lie in Marissa Mayer, and her failure in transitioning into the role of Yahoo’s CEO. Regardless of her demise at the helm of Yahoo, Marissa - a former Google VP - was a stellar candidate, having been part of the three-person team to launch AdWords followed by her strong involvement in Google Search.

One could say that her failure at Yahoo boiled down to an intelligent person not being part of the organizational synergy - someone with little to no turnaround experience, leadership experience in running large enterprises, and management experience. [4]This was correlation mismanagement. Another issue psychometrics attempts to address.

On the cultural compatibility front, Marissa forced an end to Yahoo’s popular work from home policy in an attempt to improve work productivity. It ended with increased stress, lowered productivity, and higher real estate costs.[5]

Utilizing Psychometric Tests in Learning & Development

When it comes to L&D programs, Business Impact and Return on Investment (ROI) rank as the two most desired measures by CEOs in an Adecco survey. Yet merely 8% of the organizations surveyed currently notice business impact by L&D; this drops by another 4% when it comes to ROI measurement as well. [6]

Additionally, as part of the Workplace Learning Report, 32% among 500 L&D professionals identified with the genuine challenge of demonstrating ROI and business impact to senior leadership. [7]It’s raised queries about L&D budgets, and its allocation thereof, which may hinder professionals from ever fully optimizing their initiatives.

The recent advent of assessment technology, however, seemed to have offset this disadvantage; at least to a certain degree. But we can maybe look more closely at how psychometric tests improve learning, training and development programs in organizations.

Exploring Use Cases: Training Need Identification

The cut-e Assessment Barometer - an international survey of 2776 respondents from about 37 countries uncovered that Irish organizations along with others from Malaysia, the Netherlands and Sweden use psychometric tests in learning & development more than organizations in other geographies. Employers are doing this to better understand the strengths already in the organization.

The underlying principle behind making this happen is to create clearer career paths for the Irish company employees, and to embrace the culture that makes them successful. While historically used towards the end of the recruitment process, much of the data from psychometric tests largely went unused post-hire. The Irish were among the first to realize that these tests are primed to add more value than previously imagined, especially in maximizing ROI or understanding training needs.[8]

Assessments are now deployed before the recruitment process also, the results of which are later used to give candidates more information on the applied role and provide feedback on candidate suitability, strengths and areas of development of the same. In fact, large organizations now utilize talent management systems that assess test data gathered pre-hire to create training plans and link employees with roles that become available in the organization.

At Mettl, the Kirk-Patrick Model finds strong use in the art of Training Need Effectiveness, a 4-tier training evaluation model. Its use spreads as a global standard to evaluate training effectiveness. [9]The four stages are as described below:

  1. Reaction: It measures how the trainees react to the training.
  2. Learning: This measures knowledge improvement as a direct impact from training.
  3. Behaviour: It identifies how trainees apply their received training, measuring changes in behaviour from the same. Psychometric tests work into the system at this stage of the Kirk-Patrick Model.
  4. Results: These include outcomes that the organization has determined positive for the employees, business and bottom line.

Exploring Use Cases: Assessment & Development Centers

Assessment Centers are known to involve participants completing a range of tasks that simulate real-time activities carried out in the target job. A combination of the same, and more importantly methods that utilize psychometric tests and interviews are used to assess specific competencies in candidates.[10]

The concept is defined by an organization’s desire to predict future job performance. Assessment & Development Centers are known to draw this outcome by having individuals carry out a set of tasks that most accurately sample those required in the job. In simpler terms, these centers are designed to measure and observe behavior.

This approach derails from the more traditional approach in which an observer primarily infers personal characteristics from behaviour based upon subjective judgement, and usually without much evidence. Over time, these have been rendered ineffective and inaccurate because of the subjective biases and whims of the observer. In many cases, traditional approaches have also produced selection criteria or decisions based on loose social interactions, post which an individual’s cultural fitment with an organization is determined. Assessment & Development Centers aided with test technology, especially in the field of psychometrics remedy this situation rather heavily.

Utilizing Psychometric Tests in Succession Planning

In 2000, after helmed Microsoft for nearly 25 years, Bill Gates handed the keys of his castle to Steve Ballmer. The latter went on to lead Microsoft for the next 14 years, tripling sales to $78 billion and more than doubling profits from $9 billion to $22 billion. In fact, Kinect and Xbox launched under his supervision; as did the acquisition of Skype and Yammer.[11]

If Microsoft hinged on quarter-to-quarter growth, Ballmer was as good as it gets as a CEO. But in terms of long term survival, you could argue that Ballmer failed. On investigation, it’s easy enough to find that through Microsoft’s remarkable financial performance in 14 years, its leadership failed to execute on the five important technology trends of the 21st century.

Search Smartphones Mobile OS Media Cloud

While the company left the 20th century owning more than 95% of the operating systems that ran on computers - almost every desktop - its mobile OS share borders on an abysmal 0.1%.[12] It is speculated that Ballmer failed because the CEO was a world-class executor of an existing business model, but not a world-class innovators.

Others identify with Satya Nadella, the man who succeeded Ballmer in 2014. He roped Microsoft around mobile and cloud, freed the Azure and Office teams from Windows, released a new version of Windows and killed the phone business. [13] Nadella likely saved his company from irrelevance.

It was more appropriate for the story to precede the concept when it came to psychometric tests and its use in succession planning. Quality employees are not cut from the same cloth, and it shows from the Ballmer example. Did he possess critical competencies to Microsoft success? Yes. Did he have them all? No.

The story also works to highlight that a succession either propels an organization to brave turbulent market waters, or sinks it through the holes. In situations where requirements and roles are sharply defined, psychometric tests could be used to identify an employee’s future potential in the role. It provides advantages in the form of objective information on employee effectivity, behavioral competencies suited to the organizational ecosystem, and leadership stage compatibility.

Utilizing Psychometric Tests in High Potential Identification


In the business world today, there is a growing interest in high potential (HiPo) identification. An employee’s potential evaluates the upper limit of his or her development range. Suffice to say, the more potential they have, the cheaper and quicker it is to develop them.

Studies indicate that investment in the right people maximizes organizational returns, largely derivative of Pareto’s Principle or the 80/20 Rule. It roughly identifies that about 80% of the effect arise from merely 20% of the causes.[14] This is further validated by deeper research - across a broad range of industries, organizations, and tasks, a minute portion of the workforce drive large proportions of organizational results.[15]

  1. The top 1% translates to 10% of organizational output
  2. The top 5% impacts 25% of organizational output
  3. The top 20% dervies 80% of organizational output

This pattern visibly grows with the complexity of the job itself. For jobs of low complexity, top employees outperform average employees by a rough median of 50%. You could take manufacturing as an example here. This difference rises to about 85-100% for jobs of medium complexity; trainers or sales managers.

Highly complex jobs, especially senior leadership roles contribute more than double of the average margin, with a contribution output over 100%. [16]There’s also added benefit to having these star performers in a team environment, boosting effectiveness of other members to around 15%.

In fact, study shows stronger fiscal performance in companies dedicated to identifying and developing top talent.[17]But if an organization is to invest in the right employees, the question raises of who they actually are. Differently put, what are the key indicators that signal this high potential? Psychometrics, however, reveal that regardless of industry, job, or complexity, such individuals tend to possess a range of measurable qualities - qualities that can be identified early in the process.

Comparisons between qualities of strongest demand in the 21st century to scientific research or psychometric use on the predictors of job performance identified a couple of markers of high potential.[18]These ingredients are key to making the psychometric test suited to high potential identification.

  1. ABILITY:Forecasting potential in a bigger, more complex job environment shifts focus to how likely an individual is to learn and master skill along with requisite knowledge. Cognitive ability or IQ is the single-best predictor of the same.[19] Learning ability includes a heavy cognitive component, aided by motivation to pick up new skills in a flexible and speedy manner.
  2. SOCIAL SKILLS:Research reveal that the primary reason for management derailment is relationship problems.[20] Employees steadied to succeed in more complex environments are able to first manage themselves, and establish strong cooperative working relationships. Sophisticated political skills are also much desired for senior roles, and much of these functions linger as core elements of emotional intelligence.[21] While the traits are revealed via assessment, it could be further refined through training and development.
  3. DRIVE:Standardized tests that are known to measure conscientiousness - a part of the Five Factor Model, ambition and achievement motivation are often readied as part of tailored psychometric tests. These traits are also identified behaviorally with indicators around hard work, willingness to extra duties, responsibility, and readiness to sacrifice.

Not many employees possess these traits, which is why they often linger in the 1%. Should a company bet on who they are with smart, scientific tests, it is more likely to end up with a stronger proportion of high potential employees that disproportionately contribute to an organization. This is key to the highest levels of ROI.

NOTE: In 2008, Howard Schultz reassumed the role of CEO at Starbucks. He has since achieved market capitalization of $33 billion, over $11 billion in annual sales, and net annual profits of more than $1.7 billion. This is all the more impressive when you consider that in a struggling US economy, where average growth of S&P 500 companies lingered at 0.4% in 2011, Starbucks’ share price increased by more than 40%.[22] In one sentence - high performance to disproportionate gains for one company by one employee. HiPo.

Utilizing Psychometric Tests in Performance Evaluation


It is important to note that in the matter of performance evaluation, job performance is subject to social and organizational influences. This is indicative of something known as effective job behavior, but what constitutes good from poor performance relies entirely on organizational context.

For example, the armed forces place a ton of importance on performance metrics such as military bearings. Likewise, a mechanic would be evaluated a little differently than military personnel by a car dealership. It’s why experts suggest the inclusion of descriptions based on job complexity for the purposes of appraisal[23] - namely situational factors that interact or influence behavior, job outcomes, and job behaviour.

In 1980s, the Office of Personnel Management (OPM) was reported to use the Management Excellence Inventory. [24] The MEI described management functions and the skills required of said functions. It suggested that at three levels of management, different layers of knowledge and skills are required to reach a measurable level of success.

  1. Lower Level Managers: Required technical competence and interpersonal communication skills.
  2. Mid Level Managers: Needed less technical competence, but significant skill in areas such as communication, flexibility, risk-taking, leadership and concern with goal achievement.
  3. Top Level Manager: Required all the skills of a mid-tier manager including a long-term view, a strategic view, and sensitivity to the environment.

Via review, research on these areas of skill indicates that all are general, some are task-oriented, and some such as flexibility and leadership are personal traits.

Now in private-sector organizations with easily measurable bottom lines, it is simpler to develop individual or quantitative work goals than it is in bureaucracies or governments where bottom lines are difficult to define. But the easy availability of quantitative goals might actually prove to hinder the valid measurement of employee effectiveness, especially since these goals shift focus to short-term results.

Further evidence highlight that the incorporation of countable, objective measures of performance into an overall appraisal could lead to an issue of overemphasis on fairly concrete aspects of performance and an underemphasis on those less easily quantified - aspects that yield concrete outcomes only in the long term. [25]

When in application for performance evaluation, psychometric testing involves two prime functions - the use of appropriate instruments, and the evaluation in the wake of applying such tests. The reason testing rose to prominence for segments such as performance evaluations is better explained with a weighted example.

info

The example but shows that traditional methods of performance evaluation carry a high-risk of individual bias, race aside there are also matters of favoritism and gender bias among others. They are not comprehensive and undermine the effectiveness of people performance.[27] Psychometric tests move beyond decisions made on the basis of a chance to make a real impact on the success and talent quotient of any business.



We have discussed a little about validity and reliability of a psychometric test; suffice to say, they need to remain high on both accounts for performance evaluation also. To jog memory, you could take a look at the chart below. It covers the metaphor of a shooting target: a wide dispersion of bullets indicates unreliability, whereas off-center shooting points to poor validity or bias.

SOURCE:https://www.socialresearchmethods.net/kb/relandval.php



A 2006 research revealed that in order to make psychometric tests valid, an organization is required to supervise changes in criteria in order to keep balance of skills and personalities in need of evaluation. If good communication skills is a mandate for a role, a baseline must be established normatively for the test in question before the evaluation process. Again, it is wise to modify if factors of emotional stability - for example - is required in addition to base communication.

Therefore, to maintain the validity of the performance evaluation process, the psychometric tests deployed require a certain amount of flexibility in accordance with the situation. In other words, if you are considering these tests, it is only helpful in case of well-established metrics of job performance. [28] Let us take a look at some of the criteria required of this validity:

  1. Objective and removed of impact by the values of the tester
  2. Managed in standardized or controlled scenarios
  3. Basic certainty of quantifying and minimizing intrinsic errors
  4. Predict and evaluate performance of employees accurately
  5. Provide a status of being non-discriminatory

Of course, this is only established when an organization makes use of combined psychometric and traditional techniques. But, choosing the right test criteria mostly depends on the role or position for which the employee or candidate is being assessed.

Utilizing Psychometric Tests in Employee Engagement

In an organizational context, there is often the case of employee burnout - a situation of emotional exhaustion, lack of personal attribution or accomplishment, and depersonalization. Around the early 1980s, psychometric research led to the development of the Maslach-Burnout Inventory (MBI) designed to measure burnout.

Typically restrained to the evaluation of human services, research later led to the making of the MBI-General Survey (MBI-GS), a newer version to include all employees and not merely those in industries of people work. [29]

Relatively little attention has been directed toward concepts considered antipodes of burnout; an exception being psychological presence or to be invested fully. The thought emerged from role theory and is often defined as an experiential state that channels energies into cognitive, physical and emotional labors using personally engaging behaviors.[30]

Now why does this require intervention by test technology? In January 2016, Gallup released the results of its annual employee engagement survey polling more than 80,000 working adults in the U.S.A. It accounted for a workforce across industries, of which Gallup identified merely 32% as engaged at the workplace.

Interestingly, 17.2% fell into the category of being actively disengaged and 50.8% into the classification of absolutely not engaged. [31] It mirrored the results of 2015, marking little to no improvement in avenues of employee engagement. To remain firm in the matter, let’s take a look at some more research on the matter:

  1. Organizations report that engaged employees outperform their counterparts by 202% [32]
  2. Highly engaged employees boost a company’s net profit margin by about 6% [33]
  3. Employee Disengagement costs U.S. business roughly $450 - $550 billion a year [34]

Contrarily, engaged employees possess a sense of effective and energetic connection with work related activities, going as far as to see themselves as able to deal perfectly with the demands of the job. In psychometric tests designed to assess the matter - such as the MBI or MBI-GS - it would be wise to review the opposite pattern of scores on the three MBI dimensions.

  1. Low scores on exhaustion and cynicism
  2. High scores on efficacy

Both engagement and burnout hold easily measured metrics, and while the metrics itself are opposites, the concept of the two are not. This is something that’s being constantly researched into improvement. Psychometric qualities of the two inventories, and the tests that yield it face considerable favor due to two immediate benefits to organizations:

  1. Effective Measurement of Engagement based on Cultural Fitment Pre-Hire
  2. Process Improvement via Regular Assessment through an Employee’s Lifecycle

A logical next step to everything that’s been discussed thus far, employee engagement aside, is further research. While simple in statement, there are a lot of facets or scales and their relationship to job-related variables matter.[35] [36]

For example, burnout research reveal different types of variables related to different dimensions of burnout - emotional exhaustion correlating to job demands such as time pressure and workload, whereas cynicism or disengagement from work is more a function of poor job resources, poor job control, lack of feedback, lack of participation in decision making, and lack of social support. [37]

References
  1. http://www.siop.org/workplace/employment%20testing/usingoftests.aspx
  2. http://www.cebma.org/wp-content/uploads/Rynes-et-al-HR-Professionals-belief-about-effective-human-resource-practices-HRM-2002.pdf
  3. http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.172.1733
  4. https://www.forbes.com/sites/mikemyatt/2015/11/20/marissa-mayer-case-study-in-poor-leadership/#5b0e52c83b46
  5. https://www.xonitek.com/leadership-culture-and-the-situation-of-marissa-mayer/
  6. http://pages.adeccousa.com/rs/adeccousa/images/2014-mind-the-skills-gap.pdf
  7. https://learning.linkedin.com/elearning-solutions-guides/2017-workplace-learning-report?trk=lilblog_02-10-17_2-stats-LD_tl&cid=70132000001AyziAAC#
  8. http://www.hrheadquarters.ie/how-can-psychometric-assessment-enable-development-and-training-of-talent/
  9. https://www.kirkpatrickpartners.com/Our-Philosophy/The-Kirkpatrick-Model
  10. http://www.acdp.pt/wp-content/uploads/2013/07/Psychometric_Testing.pdf
  11. https://www.fastcompany.com/3016371/microsoft-ceo-steve-ballmers-legacy-a-salesman-without-product-vision
  12. https://www.recode.net/2017/7/17/15984222/microsoft-windows-phone-mobile-operating-system-android-iphone-ios
  13. https://besttech.io/satya-nadella-just-fixed-microsofts-biggest-problem-82e29fe03df2
  14. https://en.wikipedia.org/wiki/Pareto_principle
  15. http://hermanaguinis.com/PPsych2014.pdf
  16. https://www.uam.es/personal_pdi/psicologia/pei/diferencias/Hunter1984JobPerformance.pdf
  17. http://www.markhuselid.com/pdfs/articles/1995_AMJ_HPWS_Paper.pdf
  18. http://www.drtomascp.com/uploads/PIOP_final_employability.pdf
  19. http://www.sciencedirect.com/science/article/pii/0001879186900138
  20. http://onlinelibrary.wiley.com/doi/10.1111/1468-2389.00162/abstract
  21. http://psycnet.apa.org/record/2016-45865-001
  22. https://www.businessinsider.in/The-incredible-rags-to-riches-story-of-Starbucks-billionaire-Howard-Schultz/articleshow/49487899.cms
  23. Landy & Farr, 1983
  24. Flanders & Utterback, 1985
  25. Landy & Farr, 1983
  26. Kraiger and Ford's Survey, 1985
  27. Fernandes O, Sreenidhi K (2014) The Benefits of Using a Holistic Approach in Behavioral Assessment Interpretation CAAL ISBN 978-602-19725-5-7
  28. http://www.trainingzone.co.uk/topic/strategy/feature-does-psychometric-testing-work-sarahfletcher
  29. Leiter, M.P. and W.B. Schaufeli: 1996, ‘Consistency of the burnout construct across occupations’, Anxiety, Stress and Coping 9, pp. 229–243
  30. Kahn, W.A.: 1992, ‘To be fully there: Psychological presence at work’, Human Relations 45, pp. 321–349
  31. https://collegeforamerica.org/how-employee-disengagement-impacts-bottom-line/
  32. https://www.dalecarnegie.com/employee-engagement/engaged-employees-infographic/
  33. http://www.twrcc.co.za/Engaged%20employees%20drive%20the%20bottom%20line.pdf
  34. http://news.gallup.com/businessjournal/162953/tackle-employees-stagnating-engagement.aspx
  35. Lee, R.T. and B.E. Ashforth: 1996, ‘A meta-analytic examination of the correlates of the three dimensions of job burnout’, Journal of Applied Psychology 81, pp. 123–133
  36. Schaufeli, W.B. and D. Enzmann: 1998, The Burnout Companion to Study and Research: A Critical Analysis (Taylor and Francis, London, UK).
  37. Demerouti E., A.B. Bakker, F. Nachreiner and W.B. Schaufeli: 2001, ‘The job demands – resources model of burnout’, Journal of Applied Psychology 86, pp. 499–512