|Danfoss: Ranking made by 4 Superiors correlates with PEP Management 0.74, N = 26|
|HGC: Ranking made by 5 Superiors correlates with PEP Management 0.87, N = 21|
|Jumbo: Ranking made by 2 Superiors correlates with PEP Management 0.78, N = 10|
|Jumbo: Ranking made by 1 Superior correlates with PEP Management 0.60, N = 11|
|Jumbo: Ranking made by 2 Superiors correlates with PEP Management 0.58, N = 8|
|Reso: Ranking made by 4 Superiors correlates with PEP Management 0.78, N = 28|
|Ema Bank: Ranking made by 4 Superiors correlates with PEP Sales 0.88, N = 10|
|Ema Bank: Tunover correlates with PEP Sales 0.82, N = 10|
|Ema Bank: Ranking made by 3 Superiors correlates with PEP Management 1, N = 6|
|HGC: Ranking made by 3 Superiors correlates with PEP Sales 0.92, N = 14|
|HGC: PEP Management and PEP Sales correlates 0.73, N = 10|
|Prof. Dr. Hans Peter Kempkes|
Certificate for Potential Evaluation Program (PEP) by
© InSyst Master Data Establishment (IMDE)
|1. Purpose of appraisal|
|2. Motivation for participation in PEP|
|3. Written sources and observations|
|4. Evaluation of PEP|
|5. Areas of application|
|6. Concluding appraisal|
|1. ||Purpose of appraisal|
The purpose of this paper is to first subject PEP to a critical appraisal and then to demonstrate the
consequent areas of application for the instrument as part of personnel development and strategic company management.
This appraisal aims to summarize previous scientific examinations and practical experiences and present them to the interested
public. Those responsible for human resources in a company are given criteria to aid them in selecting candidates using the
potential analysis instruments and in using the instruments more efficiently.
|2.||Motivation for participation in PEP|
The author has been involved with the subject of potential analysis for some time and in two respects:
First of all, he addresses the subject area scientifically in his function as a professor for strategic business management and human
resources management. Secondly, he is also familiar with the pragmatics of business and consulting, in which he has been involved for
years, and therefore is aware of the needs and requirements business management and human resource managers have. It follows, therefore,
that different tests and methods (for example, AC) were applied and controlled in a scientific manner.
With the increasing experience, however, so, too, has the lack of satisfaction with the tests available on the market increased.
Almost all of the tests displayed limited results or results that were of little use because:
|||they are too general in their findings|
|||the findings are difficult to comprehend|
|||they categorize too strongly due to the model or personal image they use (i.e. typologies)|
|||the scope is not specific to the requirements|
|||it is not possible to clearly define the potential for development|
|||many are too positive in their findings and therefore not completely honest.|
Furthermore, many of the test providers promised a high level of validation, reliability and practicality, yet were usually not
able to convincingly deliver on these claims.
For the customers in the companies, this is made even more confusing because the individual tests are quite difficult to assess
due to their inherent structure, contextual effect, the underlying personal image used, etc. The claim of high quality then often
results in the price, the presentation of the consultant or the pleasantness of the advertising materials being the decisive
factor. Often the results of the analysis do not expose the weaknesses of many tests to those responsible for human resources, as
the persons tested are often content with insufficient results, particularly when these results show a euphemized picture of the
reality. In other words: Tests that do not really show requirement-specific strengths and weaknesses are more readily accepted by the
persons being tested - and in some cases due to the human resources manager's misinterpretation of the compatibility needs.
The author sees these circumstances as utterly unsatisfactory. The additional search and large number of computer-based tests
showed that PEP uses a completely different approach as compared to most other programs. PEP starts with the requirements that
separate effective managers, project managers, sales consultants and administrators from those that are less effective and less
successful. Because the overall objective is to put the right person in the right job or to initiate the appropriate human
resources development measures, the author feels that balancing the concrete requirements of a position with the competencies
of the applicant or person holding the position is the sole correct method. Instruments that diagnose characteristics are not
wholly suited to do this.
PEP has a somewhat unpopular, yet more success-relevant approach. Strengths and weaknesses can be recognized directly - as will
be demonstrated - and supported with a high level of consistency.
|3.||Written sources and observations|
This report is based on the following written sources:
||Abati, V. Dahinden, P., Thönen, D., report for review of PEP, unpublished manuscript, Bern 2001;|
||Simonet, S., Report on Revision of PEP for Evaluation of Management Personnel, Sales Personnel and Administrators, unpublished
manuscript, Bern 2003;|
||O.V., Criteria-Oriented Validity Study, Self-Image/Third-Party Image, Bern 2003;|
||Steinhauser, Th., Human Resources Development on the Basis of Computer-Based Potential Analysis Instruments, Case Study of
Banking Association, unpublished master's thesis, July 2003.|
The following sources find mention in addition::
||Implementation of more than one hundred potential analyses performed by the author on test persons. This allowed diverse
impressions and experiences to be collected regarding the correctness of the instrument and its potential areas of application.|
||Observed participation during application of the instrument by instructed trainers.|
||Participation in IMDE training seminar for qualified PEP users, August 2003, in Bern, Switzerland.|
Both written sources and all other impressions of the author refer to the great seriousness and extreme
care with which PEP was developed and enhanced: on the basis of rational and understandable considerations alone. The authors
place particular importance on the constructive inclusion of the users for continual improvement of the product, which since the
beginning of the 1990s has led to PEP becoming "best of class" for requirement-oriented potential analysis instruments. Put more
accurately, the quality and multifunctionality of the product has led in a new generation of tests.
To ensure that the users attain the high quality standard the product has to offer, IMDE offers product support that addresses
the technical area, as well as "distance coaching" for clarification of open questions or particular problems the user might be
having. This is in addition to the different training sessions for PEP (including selection and promotion discussions with test
persons) offered by IMDE. This training and support, worldwide, up-to-date and accessible at any time, allows the consultant
network to receive help when they need it.
Thanks to the constructive cooperation during the training, the IMDE consultants create a clearly win-win attitude among one
another. Other (sales) partners are not seen as competitors, but as possible complements to one's own consulting portfolio.
Contacts among consultants persist after the seminars as well.
The excellent results that can be attained using PEP are based on these psychological influences in combination with expert
|4.||Evaluation of PEP:|
The arguments and explanations listed above surely suggest the effectiveness of PEP. This effectiveness
has solely been reaffirmed by the day-to-day work of the author with individuals and companies. However, this does not replace
proof of a coherent theoretical concept or statistical substantiation.
The authors of PEP have documented all important characteristics of their survey procedure in the report above, allowing for an
objective evaluation based on statistical information.
The authors first present their theoretical framework in the report, which is no personality theory, but rather a practice-oriented
approach that is aligned with requirements and based on benchmarks. However a theory addressing needs and their satisfaction and
dealing with frustration is used for assigning critical events to item groups. From a large number of sources, the authors have
compiled success-relevant events for fundamental areas of management, sales, processing (these, however, were divided into
sub-areas as well). These events were formulated into statements (approximately 1500 items all together) and issued to test
persons in a four-level scale (++ agree completely, + tend to agree, - tend to disagree, do not agree at all).
A control scale is also included in the question catalog. The degree to which the answers are realistic can be determined from
this control scale. After the items have been answered, PEP calculates the results, which are placed in relation to individuals of
the respective function group who have been proven to be particularly effective (for example, middle and lower management).
The results are also presented in profiles.
By weighting and defining the desired characteristics, the PEP profile can be adjusted to comply with requirements or the specific
needs of a company.
In this way it is possible on one hand to adjust PEP to the respective requirements of different functions, such as existing
position profiles. On the other hand, thanks to the high standardization in the procedure (standard texts guide the test person
through the program), it is possible to guarantee great objectivity during implementation, as variance with respect to the person
performing the test is avoided completely. A survey of 1000 test persons substantiates the acceptance of the procedure among the
test persons, clearly showing that the questions asked were understood correctly throughout.
Furthermore we can assume great impartiality with regard to the objectivity in the evaluation, as the data processing is performed
electronically, thereby reducing the error variance to an absolute minimum.
To correctly evaluate and interpret the results, the users are offered the courses mentioned before, in which they can practice
the interpretation of different results while the trainer - when required - can offer help and corrections. Thanks to this
intensive training and the support mentioned before, the best possible requirements for objectivity of interpretation are met.
Additionally, PEP receives numerous suggestions from discussions that can be used for such documents as interview guidelines.
The authors based the revision of PEP on a thorough item analysis that allows items to be examined on the basis of different indicators.
These analyses were used to reduce the procedure to dependable items with regard to the statistics taken from the test.
PEP was examined as well from the perspective of analytical factors in that the analysis was performed using item groupings. The
resulting factor structure contained from three to four and up to eight factors depending on the criteria used. Due to this
relatively high variance in the individual perspectives and the requirement-specific considerations of the content, the factor
solution chosen appears appropriate.
Due to the practice-oriented, profession-based and requirement-specific approach, considerable intercorrelations sometimes arise
that result in an overlap of the individual perspectives in the factor analysis. This can be justified with respect to the content
by considering the need for statements referring to a wide spectrum of trainable characteristics. This fact is appropriately taken
into consideration in the interpretation of the results. It must also be said that the user him/herself can perform a weighting
of perspectives in a way that conforms to requirements (see explanations above).
After the distribution characteristics of the answers to the individual items were tested, the results were normalized to allow
for an evaluation of the information that was better adjusted to reality.
PEP was analyzed according to the following quality standards as part of the thesis advised and cited by the author:
||Validity, understood as degree of exactness with which the test records what is to be measured or forecasted|
||Reality, which states to what extent the measured values attained agree for the same person under the same conditions.|
||Objectivity, understood as degree with which the results of a test are independent of the person performing the test. Here a
distinction is made according to the objectivity of the implementation, evaluation and interpretation.|
||Social validity, which includes different aspects that makes the situation for diagnosing aptitude an acceptable social
situation for the test person, allowing the potential analysis to be used successfully.|
||Usability, understood as user friendliness|
||Economic viability, where here it is necessary to discuss from a theoretical standpoint whether the benefit aspects of the
usability should be included as part of personnel decisions. It is the author's view that this should take place in order to
make comprehensive controlling accessible to the instrument. The resulting price/performance relationship is an important
factor for success on the market.|
With respect to the validity, there are many investigations that prove the validity of PEP. These
demonstrate high correlation between the PEP total value and the characteristics of professional success. These incredibly high
values (PEP management with ranking by supervisors 0.75, N = 104; PEP sales with production rate 0.69, N = 308) were not only c
onfirmed by the thesis but also exceeded (PEP Management ranking = 1; N = 6; PEP sales = 0.879; N = 10). Additional comparisons
made in the thesis also showed exceptional values in agreement. The test results from PEP were synchronized with the previous
supervisor evaluations, with the paid salaries, with the financial results attained by employees in sales (0.818, N = 10) etc.
Without going into more detail in this report, it can be said that the high quality of the instrument was clear according to this
measurement criterion. Additional theses addressing this topic, for example comparison with assessment centers assigned by the
author, also anticipate interesting results.
The reliability of the measurement tested by the PEP author was ensured using the halving consistency coefficient (0.92, N = 1210)
and the retest method (0.958 same items, N = 33 repetition after an average of 18 months). The results derived from these tests
also demonstrated the reliability of the measurement method.
Objectivity has already been addressed above. Experiments have confirmed the statements regarding objectivity.
The high social validity is a particularly positive characteristic of PEP. Because the test person at the computer can choose the
test situation and the time of the test him/herself, the instrument has considerable advantages over many other tests, assessment
center in particular. Because the test person can choose a time at which he/she fells physically and psychologically healthy,
erroneous or inexact results from a diurnal condition can be largely avoided. By performing a test-person survey in parallel with
all test levels, it was shown that existing reservations regarding the validity of the results or the reliability of the test
dissipated in the course of the process.
Due to the user friendliness and the clarity of the process itself, a high degree of usability can be assumed. The question
catalogs are available in numerous languages on the Internet.
At first glance, the market price for the instrument appears relatively high in comparison to the other tests. However, as soon
as the instrument is used in an overall personnel context, the cost/benefit relationship is clearly put into perspective and turns
out to be inexpensive, as the costliness of making wrong decisions is reduced and the money for different types of training can be
used more effectively. This appraisal will address additional potential areas of use for PEP from the author's consultancy below
under the aspects of economic viability.
|5.||Potential Areas of Use:|
PEP has proven very helpful as part of a career consultancy for individuals. A new professional
direction or promotion can be planned in a sensible manner on an objective basis that often gives the test person new insights
into his or her own capabilities and into the requirements.
This also applies for the development of internal development strategies for promotion-worthy employees. The formation of teams,
for example in the area of sales, or the installation of a second management level, the founding of a pool for high potential, or
the personnel selection for several employees are all very much possible on the basis of a sound potential analysis. Economic
viability is also put into perspective in this respect if one considers the costs incurred by a company for filling a single
important position with the wrong person. In this respect, PEP can be seen as an excellent safeguard against bad personnel
The benefit of PEP is especially evident when it is part of the strategic personnel management of a company. In this way, a
personnel portfolio for important areas of the company can be created on the basis of existing data. This portfolio can then form
the basis for strategic realignments in company areas.
Here PEP can be used by a company as an important component of a strength/weakness analysis that, as part of the SWOT analysis gives
important incentive for a strategic realignment of the company. This was also an important consideration in the cited thesis.
These considerations, which can only be touched on here, must be probed more deeply in the future and require additional scientific
investigation, which is already in the works.
The sources cited document PEP in a manner that is performed thoroughly and appropriately, allowing for
a clear substantiation of the testing theory. This provides the basis for critical involvement with the instrument that satisfies
the criterion of transparency in the diagnostics of success-relevant characteristics in a professional context.
The available validity data shows that PEP is founded on a solid basis with regard to testing statistics. The data given by the
author has been impressively confirmed through examinations and measurements.
PEP exhibits above-average performance in all other measurement criteria as well. From this it can be said that PEP is a suitable
instrument for the evaluation of one's self and the evaluation of a person by others. This applies for members of management, sales
professionals and processors. To better compare PEP with other potential analysis instruments, a scientific examination must be
worked toward involving a comparison of different instruments available on the market. To the knowledge of the author of this
report, however, only NEOFFI and one other test were compared with PEP.
All in all this suggests that PEP is an extremely helpful instrument for practical application. It is based on a clear,
scientifically supportable, statistically measured basis. This makes it a means for dealing with and evaluating personnel and
strategic issues of selection, development and company leave with respect to requirement-specific and success-relevant
characteristics. These kinds of instruments will be indispensable for the future of personnel management and will replace those
instruments that do not have the same quality and comparable practical benefit.
|Freiburg, Germany - September 29, 2003|
|Author : Prof. Dr. Hans Peter Kempkes|
| (Prof. Dr. Hans Peter Kempkes)|