Danfoss: Ranking made by 4 Superiors correlates with PEP Management 0.74, N = 26 HGC: Ranking made by 5 Superiors correlates with PEP Management 0.87, N = 21 Jumbo: Ranking made by 2 Superiors correlates with PEP Management 0.78, N = 10 Jumbo: Ranking made by 1 Superior correlates with PEP Management 0.60, N = 11 Jumbo: Ranking made by 2 Superiors correlates with PEP Management 0.58, N = 8 Reso: Ranking made by 4 Superiors correlates with PEP Management 0.78, N = 28 Ema Bank: Ranking made by 4 Superiors correlates with PEP Sales 0.88, N = 10 Ema Bank: Tunover correlates with PEP Sales 0.82, N = 10 Ema Bank: Ranking made by 3 Superiors correlates with PEP Management 1, N = 6 HGC: Ranking made by 3 Superiors correlates with PEP Sales 0.92, N = 14 HGC: PEP Management and PEP Sales correlates 0.73, N = 10      Prof. Dr. Hans Peter Kempkes Certificate for Potential Evaluation Program (PEP) by © InSyst Master Data Establishment (IMDE)   1.  Purpose of appraisal 2.  Motivation for participation in PEP 3.  Written sources and observations 4.  Evaluation of PEP 5.  Areas of application 6.  Concluding appraisal   1.  Purpose of appraisal The purpose of this paper is to first subject PEP to a critical appraisal and then to demonstrate the consequent areas of application for the instrument as part of personnel development and strategic company management. This appraisal aims to summarize previous scientific examinations and practical experiences and present them to the interested public. Those responsible for human resources in a company are given criteria to aid them in selecting candidates using the potential analysis instruments and in using the instruments more efficiently. 2. Motivation for participation in PEP The author has been involved with the subject of potential analysis for some time and in two respects: First of all, he addresses the subject area scientifically in his function as a professor for strategic business management and human resources management. Secondly, he is also familiar with the pragmatics of business and consulting, in which he has been involved for years, and therefore is aware of the needs and requirements business management and human resource managers have. It follows, therefore, that different tests and methods (for example, AC) were applied and controlled in a scientific manner. With the increasing experience, however, so, too, has the lack of satisfaction with the tests available on the market increased. Almost all of the tests displayed limited results or results that were of little use because: • they are too general in their findings • the findings are difficult to comprehend • they categorize too strongly due to the model or personal image they use (i.e. typologies) • the scope is not specific to the requirements • it is not possible to clearly define the potential for development • many are too positive in their findings and therefore not completely honest. Furthermore, many of the test providers promised a high level of validation, reliability and practicality, yet were usually not able to convincingly deliver on these claims. For the customers in the companies, this is made even more confusing because the individual tests are quite difficult to assess due to their inherent structure, contextual effect, the underlying personal image used, etc. The claim of high quality then often results in the price, the presentation of the consultant or the pleasantness of the advertising materials being the decisive factor. Often the results of the analysis do not expose the weaknesses of many tests to those responsible for human resources, as the persons tested are often content with insufficient results, particularly when these results show a euphemized picture of the reality. In other words: Tests that do not really show requirement-specific strengths and weaknesses are more readily accepted by the persons being tested - and in some cases due to the human resources manager's misinterpretation of the compatibility needs. The author sees these circumstances as utterly unsatisfactory. The additional search and large number of computer-based tests showed that PEP uses a completely different approach as compared to most other programs. PEP starts with the requirements that separate effective managers, project managers, sales consultants and administrators from those that are less effective and less successful. Because the overall objective is to put the right person in the right job or to initiate the appropriate human resources development measures, the author feels that balancing the concrete requirements of a position with the competencies of the applicant or person holding the position is the sole correct method. Instruments that diagnose characteristics are not wholly suited to do this. PEP has a somewhat unpopular, yet more success-relevant approach. Strengths and weaknesses can be recognized directly - as will be demonstrated - and supported with a high level of consistency. 3. Written sources and observations This report is based on the following written sources: • Abati, V. Dahinden, P., Thönen, D., report for review of PEP, unpublished manuscript, Bern 2001; • Simonet, S., Report on Revision of PEP for Evaluation of Management Personnel, Sales Personnel and Administrators, unpublished manuscript, Bern 2003; • O.V., Criteria-Oriented Validity Study, Self-Image/Third-Party Image, Bern 2003; • Steinhauser, Th., Human Resources Development on the Basis of Computer-Based Potential Analysis Instruments, Case Study of Banking Association, unpublished master's thesis, July 2003. The following sources find mention in addition:: • Implementation of more than one hundred potential analyses performed by the author on test persons. This allowed diverse impressions and experiences to be collected regarding the correctness of the instrument and its potential areas of application. • Observed participation during application of the instrument by instructed trainers. • Participation in IMDE training seminar for qualified PEP users, August 2003, in Bern, Switzerland. Both written sources and all other impressions of the author refer to the great seriousness and extreme care with which PEP was developed and enhanced: on the basis of rational and understandable considerations alone. The authors place particular importance on the constructive inclusion of the users for continual improvement of the product, which since the beginning of the 1990s has led to PEP becoming "best of class" for requirement-oriented potential analysis instruments. Put more accurately, the quality and multifunctionality of the product has led in a new generation of tests. To ensure that the users attain the high quality standard the product has to offer, IMDE offers product support that addresses the technical area, as well as "distance coaching" for clarification of open questions or particular problems the user might be having. This is in addition to the different training sessions for PEP (including selection and promotion discussions with test persons) offered by IMDE. This training and support, worldwide, up-to-date and accessible at any time, allows the consultant network to receive help when they need it. Thanks to the constructive cooperation during the training, the IMDE consultants create a clearly win-win attitude among one another. Other (sales) partners are not seen as competitors, but as possible complements to one's own consulting portfolio. Contacts among consultants persist after the seminars as well. The excellent results that can be attained using PEP are based on these psychological influences in combination with expert quality management. 4. Evaluation of PEP: The arguments and explanations listed above surely suggest the effectiveness of PEP. This effectiveness has solely been reaffirmed by the day-to-day work of the author with individuals and companies. However, this does not replace proof of a coherent theoretical concept or statistical substantiation. The authors of PEP have documented all important characteristics of their survey procedure in the report above, allowing for an objective evaluation based on statistical information. The authors first present their theoretical framework in the report, which is no personality theory, but rather a practice-oriented approach that is aligned with requirements and based on benchmarks. However a theory addressing needs and their satisfaction and dealing with frustration is used for assigning critical events to item groups. From a large number of sources, the authors have compiled success-relevant events for fundamental areas of management, sales, processing (these, however, were divided into sub-areas as well). These events were formulated into statements (approximately 1500 items all together) and issued to test persons in a four-level scale (++ agree completely, + tend to agree, - tend to disagree, do not agree at all). A control scale is also included in the question catalog. The degree to which the answers are realistic can be determined from this control scale. After the items have been answered, PEP calculates the results, which are placed in relation to individuals of the respective function group who have been proven to be particularly effective (for example, middle and lower management). The results are also presented in profiles. By weighting and defining the desired characteristics, the PEP profile can be adjusted to comply with requirements or the specific needs of a company. In this way it is possible on one hand to adjust PEP to the respective requirements of different functions, such as existing position profiles. On the other hand, thanks to the high standardization in the procedure (standard texts guide the test person through the program), it is possible to guarantee great objectivity during implementation, as variance with respect to the person performing the test is avoided completely. A survey of 1000 test persons substantiates the acceptance of the procedure among the test persons, clearly showing that the questions asked were understood correctly throughout. Furthermore we can assume great impartiality with regard to the objectivity in the evaluation, as the data processing is performed electronically, thereby reducing the error variance to an absolute minimum. To correctly evaluate and interpret the results, the users are offered the courses mentioned before, in which they can practice the interpretation of different results while the trainer - when required - can offer help and corrections. Thanks to this intensive training and the support mentioned before, the best possible requirements for objectivity of interpretation are met. Additionally, PEP receives numerous suggestions from discussions that can be used for such documents as interview guidelines. The authors based the revision of PEP on a thorough item analysis that allows items to be examined on the basis of different indicators. These analyses were used to reduce the procedure to dependable items with regard to the statistics taken from the test. PEP was examined as well from the perspective of analytical factors in that the analysis was performed using item groupings. The resulting factor structure contained from three to four and up to eight factors depending on the criteria used. Due to this relatively high variance in the individual perspectives and the requirement-specific considerations of the content, the factor solution chosen appears appropriate. Due to the practice-oriented, profession-based and requirement-specific approach, considerable intercorrelations sometimes arise that result in an overlap of the individual perspectives in the factor analysis. This can be justified with respect to the content by considering the need for statements referring to a wide spectrum of trainable characteristics. This fact is appropriately taken into consideration in the interpretation of the results. It must also be said that the user him/herself can perform a weighting of perspectives in a way that conforms to requirements (see explanations above). After the distribution characteristics of the answers to the individual items were tested, the results were normalized to allow for an evaluation of the information that was better adjusted to reality. PEP was analyzed according to the following quality standards as part of the thesis advised and cited by the author: • Validity, understood as degree of exactness with which the test records what is to be measured or forecasted • Reality, which states to what extent the measured values attained agree for the same person under the same conditions. • Objectivity, understood as degree with which the results of a test are independent of the person performing the test. Here a distinction is made according to the objectivity of the implementation, evaluation and interpretation. • Social validity, which includes different aspects that makes the situation for diagnosing aptitude an acceptable social situation for the test person, allowing the potential analysis to be used successfully. • Usability, understood as user friendliness • Economic viability, where here it is necessary to discuss from a theoretical standpoint whether the benefit aspects of the usability should be included as part of personnel decisions. It is the author's view that this should take place in order to make comprehensive controlling accessible to the instrument. The resulting price/performance relationship is an important factor for success on the market. With respect to the validity, there are many investigations that prove the validity of PEP. These demonstrate high correlation between the PEP total value and the characteristics of professional success. These incredibly high values (PEP management with ranking by supervisors 0.75, N = 104; PEP sales with production rate 0.69, N = 308) were not only c onfirmed by the thesis but also exceeded (PEP Management ranking = 1; N = 6; PEP sales = 0.879; N = 10). Additional comparisons made in the thesis also showed exceptional values in agreement. The test results from PEP were synchronized with the previous supervisor evaluations, with the paid salaries, with the financial results attained by employees in sales (0.818, N = 10) etc. Without going into more detail in this report, it can be said that the high quality of the instrument was clear according to this measurement criterion. Additional theses addressing this topic, for example comparison with assessment centers assigned by the author, also anticipate interesting results. The reliability of the measurement tested by the PEP author was ensured using the halving consistency coefficient (0.92, N = 1210) and the retest method (0.958 same items, N = 33 repetition after an average of 18 months). The results derived from these tests also demonstrated the reliability of the measurement method. Objectivity has already been addressed above. Experiments have confirmed the statements regarding objectivity. The high social validity is a particularly positive characteristic of PEP. Because the test person at the computer can choose the test situation and the time of the test him/herself, the instrument has considerable advantages over many other tests, assessment center in particular. Because the test person can choose a time at which he/she fells physically and psychologically healthy, erroneous or inexact results from a diurnal condition can be largely avoided. By performing a test-person survey in parallel with all test levels, it was shown that existing reservations regarding the validity of the results or the reliability of the test dissipated in the course of the process. Due to the user friendliness and the clarity of the process itself, a high degree of usability can be assumed. The question catalogs are available in numerous languages on the Internet. At first glance, the market price for the instrument appears relatively high in comparison to the other tests. However, as soon as the instrument is used in an overall personnel context, the cost/benefit relationship is clearly put into perspective and turns out to be inexpensive, as the costliness of making wrong decisions is reduced and the money for different types of training can be used more effectively. This appraisal will address additional potential areas of use for PEP from the author's consultancy below under the aspects of economic viability. 5. Potential Areas of Use: PEP has proven very helpful as part of a career consultancy for individuals. A new professional direction or promotion can be planned in a sensible manner on an objective basis that often gives the test person new insights into his or her own capabilities and into the requirements. This also applies for the development of internal development strategies for promotion-worthy employees. The formation of teams, for example in the area of sales, or the installation of a second management level, the founding of a pool for high potential, or the personnel selection for several employees are all very much possible on the basis of a sound potential analysis. Economic viability is also put into perspective in this respect if one considers the costs incurred by a company for filling a single important position with the wrong person. In this respect, PEP can be seen as an excellent safeguard against bad personnel decisions. The benefit of PEP is especially evident when it is part of the strategic personnel management of a company. In this way, a personnel portfolio for important areas of the company can be created on the basis of existing data. This portfolio can then form the basis for strategic realignments in company areas. Here PEP can be used by a company as an important component of a strength/weakness analysis that, as part of the SWOT analysis gives important incentive for a strategic realignment of the company. This was also an important consideration in the cited thesis. These considerations, which can only be touched on here, must be probed more deeply in the future and require additional scientific investigation, which is already in the works. 6. Concluding Appraisal: The sources cited document PEP in a manner that is performed thoroughly and appropriately, allowing for a clear substantiation of the testing theory. This provides the basis for critical involvement with the instrument that satisfies the criterion of transparency in the diagnostics of success-relevant characteristics in a professional context. The available validity data shows that PEP is founded on a solid basis with regard to testing statistics. The data given by the author has been impressively confirmed through examinations and measurements. PEP exhibits above-average performance in all other measurement criteria as well. From this it can be said that PEP is a suitable instrument for the evaluation of one's self and the evaluation of a person by others. This applies for members of management, sales professionals and processors. To better compare PEP with other potential analysis instruments, a scientific examination must be worked toward involving a comparison of different instruments available on the market. To the knowledge of the author of this report, however, only NEOFFI and one other test were compared with PEP. All in all this suggests that PEP is an extremely helpful instrument for practical application. It is based on a clear, scientifically supportable, statistically measured basis. This makes it a means for dealing with and evaluating personnel and strategic issues of selection, development and company leave with respect to requirement-specific and success-relevant characteristics. These kinds of instruments will be indispensable for the future of personnel management and will replace those instruments that do not have the same quality and comparable practical benefit. Freiburg, Germany - September 29, 2003 Author : Prof. Dr. Hans Peter Kempkes            (Prof. Dr. Hans Peter Kempkes)