12 QUESTION ASSESSMENT EVALUATION GUIDE
1. Are the assessments locally supported?
To ensure that our assessments are locally supported wherever we operate, Hogan works with a global network of authorised distributors and partner organisations. From our earliest partnerships in Europe and Australia to more recent alliances in Asia and Africa, Hogan works to ensure that our clients receive all the support they need from trusted local agencies. Hogan’s global network includes nearly 30 distributors operating in over 40 countries across 6 continents, and this network continues to grow each year.
2. Have the assessments been reviewed by professional representative bodies around the world?
Hogan’s assessments have been examined by agencies across several continents. The Hogan Personality Inventory (HPI), Hogan Development Survey (HDS), and Motives, Values, Preferences Inventory (MVPI) received favourable reviews in the U.S. and U.K. from the Buros Institute of Mental Measurements and the British Psychological Society (BPS), respectively. Hogan’s assessments continue to receive similar reviews in Australia, Brazil, South Africa, and Sweden among other locations.
3. Have the assessments been translated for use in other cultures?
Hogan’s assessments have been translated for use in over 40 different languages worldwide. In these efforts, we partner with qualified professionals and use a combination of forward and back-translation to ensure congruence between the original and translated forms of our assessments.
4. Does the assessment provider adapt their assessment content to ensure cultural relevance?
Hogan’s primary goal in translation is to maintain the integrity and content of the original assessment while ensuring cultural sensitivity and relevance to the local audience. Other assessment providers implement literal translations, ignoring the impact of language and culture in the comprehension and relevance of their assessment content. Hogan, however, adapts our assessment content by focusing on congruence with the original assessment, but allowing local language and cultural issues to inform adaptations to ensure relevance to the local audience.
5. Can the assessment provider ensure that translated assessments are equivalent to the original forms?
Research indicates that nuances in languages, cultural differences, and other factors make perfect measurement equivalence impossible. Some assessment providers tout perfect measurement equivalence, but these results distort real construct differences across cultures as well as the cultural relevance of adapted forms. Nevertheless, global test publishers are responsible for ensuring the comparability of assessments across cultures and languages. Hogan takes this responsibility seriously, using a combination of techniques to ensure functional equivalence of items, scales, and factors while maintaining cultural sensitivity and relevance in adapted forms. Once sufficient data are available for adapted assessments, we examine item- and scale-level statistics to identify any content that may require revisions. We also review the overall factor structures of the adapted assessments to ensure that they are congruent with those from the original assessments. These analyses ensure that adapted Hogan assessments are equivalent to the original assessments across all levels of analysis.
6. How are assessment scores interpreted within a local culture?
Assessment scores mean little without norms to guide interpretation. However, even norms hold little meaning unless a person is compared to an appropriate comparison group. For example, organisations interested in selecting job applicants inside the Czech Republic would be interested in how applicant scores compare to other Czechs instead of Americans or other groups. As such, within cultures, we develop local norms by collecting assessment data on adapted forms of our assessments. Once sufficient data are available, we use these data to calculate itinerant norms, which summarise the local population. However, because these norms are based on the first available data, these norms may not accurately reflect proportions of occupational or demographic groups in the local population. Once additional data are available, Hogan calculates a stratified local norm to replace the itinerant norm, ensuring that the local norm reflects the demographic and workforce characteristics of the target culture.
7. How are assessment scores interpreted across cultures?
Interpreting assessment scores across cultures can be a delicate issue, as norms based on scores from one culture may not accurately reflect individuals from a different culture. For example, a multi-national organisation would need a common comparison group to interpret assessment scores of job applicants from multiple locations. To provide an apples-to-apples metric for these comparisons, Hogan uses a multi-language norm comprised of data representing many languages and cultures. These norms are useful for comparing individuals in applications where the scores of participants from diverse locations should be compared using a common metric.
8. Have the assessments been proven to consistently and accurately predict performance across cultures?
Hogan has conducted in-depth validation studies to illustrate the validity of the Hogan Personality Inventory (HPI), Hogan Development Survey (HDS), and Motives, Values, Preferences Inventory (MVPI) in predicting job performance across occupations, job levels, and industry sectors around the world. These studies cross six continents and numerous countries, client organisations, and occupations. Supporting these studies, Hogan provides Return on Investment (ROI) results to clearly illustrate the impact of using Hogan assessments for making applied personnel decisions.
9. Can the assessment provider supply technical documentation to support the use of translated
assessments?
Once translation, equivalence, and local norming efforts are complete, Hogan provides comprehensive technical documentation to support the use of the translated assessments. These materials describe the results of these processes in detail and present psychometric properties and available local validity evidence to support the use of the translated assessments. In addition, every Hogan research study concludes with the delivery of technical reports documenting validation results and providing support for the use of translated Hogan assessments for specific applications.
10. Does the assessment provider offer any global off-the-shelf solutions?
Hogan offers a number of off-the-shelf solutions for our global clients interested in using personality-based assessment solutions without going through the rigour of local validation. These solutions include selection recommendations of job candidates based on job family profiles, selection recommendations of candidates into entry-level jobs, identification of high-potential employees, and evaluation of employee safety. These products provide our global client base with efficient solutions for selecting applicants into a variety of jobs across the labour force and evaluating current employees against metrics that can facilitate future organisational performance.
11. Does the assessment provider maintain a global research archive that can be accessed to confirm the
results of individual validity studies?
The Hogan archive contains information from hundreds of research studies including Criterion-Related (CR) validation studies, Validity Generalisation (VG) studies, content validation studies, job analysis research, and competency mapping studies. Our ability to deliver selection and development solutions using data from similar jobs and industries previously studied hinges on the breadth and depth of validity evidence currently available in the Hogan archive. As this global research archive increases, so does our ability to accurately predict job performance across a range of jobs.
12. Do the assessments comply with local laws and regulations?
Hogan works closely with our international network of partners and distributors to understand any applicable laws or regulations that apply to the use of psychological assessments in organisations. Hogan routinely provides analyses, documentation, and other support services such as adverse impact analyses, participant access to results, and data privacy and archiving rules to address these needs. In addition, we work with our partners to comply with the requirements for becoming approved assessment providers in accordance with local testing commissions and other governing bodies.