GSB# 2005-3933
UNION# 2005-0546-0039
IN THE MATTER OF AN ARBITRATION
Under
THE CROWN EMPLOYEES COLLECTIVE BARGAINING ACT
Before
THE GRIEVANCE SETTLEMENT BOARD
BETWEEN
Ontario Public Service Employees Union (Myles)
Union
- and -
The Crown in Right of Ontario (Ministry of Finance)
Employer
BEFORE
Joseph D. Carrier
Vice-Chair
FOR THE UNION
Mark Barclay Grievance Officer Ontario Public Service Employees Union
FOR THE EMPLOYER
Jennifer Richards Counsel Ministry of Government Services
HEARING
June 5, 2007.
Decision
In the case before me, Ms. Jennifer Myles complained by way of grievance that she was improperly denied a position as Tax Auditor 4 (TA4) in the Retail Sales Tax (RST) area of the Ministry of Finance, Province of Ontario. The initial grievance alleged violations of Article 3.1, 3.2 and 6.3 of the Collective Agreement; however, the allegations concerning Articles 3.1 and 3.2 were not pursued leaving only a challenge by the Grievor respecting Article 6.3 which provides amongst other things that in cases of job posting, the senior applicant would be successful if he or she met a test of relative equality in comparison to other applicants.
At the time of the posting in July 2005, the Grievor, whose seniority dated from May 15, 2000, was employed by the Ministry in Corporation Tax (CT) as a TA3. In the years leading up to the posting, the Grievor had gained some exposure and experience in the Retail Sales Tax area. Be that as it may, in this competition where there were 29 TA4 positions to be filled divided between 7 locations within the Province, she was unsuccessful in her bid. This was of particular concern to the Grievor Ms. Myles since she was one of less than a handful of candidates of the many who applied who had previous experience within RST. The overwhelming majority of applicants had experience only within the Corporation Tax sector and none in RST. Indeed, the competitions were restricted to personnel who were then employed within the Corporation Tax Sectors.
In the circumstances, aside from background qualifications such as education and the like, it was not anticipated that the candidates would have had any particular experience in RST. Accordingly, a test was devised to assess the aptitude or functional qualifications of candidates who were expected to have had no prior experience in retail sales tax. Having set the test, the examining committee determined that a threshold mark of 55% would have to be achieved in order for a candidate to be considered further for one of the positions.
There was no challenge by the Union concerning the nature or substance of the test with respect to its relationship to RST. Rather, the Union challenged the arbitrary setting of a threshold of 55% as well as the finding that the Grievor had failed to reach that threshold. Accordingly, the Grievor ought not to have been disqualified from further consideration, and, her candidacy and entitlement to the TA4 position should be remitted to the Parties for further consideration and determination.
BACKGROUND
In the presentation of this case, the Parties through their representatives chose to proceed by way of an Agreed Statement of Facts. As requested by them, I will attempt to keep this Award as brief as possible without a reiteration of all those facts while referring to those which are most salient to my determination here.
The test itself was comprised of three elements: an oral presentation worth 20%, an oral interview worth 30% and finally a written test worth 50%. As indicated earlier, there was no challenge to the substance of the test so that it must be taken to have been an appropriate measure of factors which were relevant to the TA4 position. The initial written portion of the test was marked by an outside consultant group, H.R. Associates. The Grievor had achieved 55.25% when marked by that group. Subsequently, the written portion of those results where the applicant had achieved a total score of 49% or better was remarked by two members of the Transition Project Office (TPO) team “to ensure consistency”. That team had apparently been put together to oversee, amongst other things, the large transition of employees from the corporate tax offices to the retail sales tax sector. Of the 58 written tests which were remarked by this team, the results of 17 candidates including the Grievor went down, and, of those 17, 4 including the Grievor now fell below the 55% threshold.
The Grievor’s total score which was official following the remark by the TPO team was 49.25%. Ms. Myles discovered later, after filing her grievance, that her original total score when the written aspect had been marked by the outside consultant firm had been 55.25%. The Grievor had lost approximately 6 points between the H.R. Associates marking and that by the TPO team. This was the case notwithstanding that both teams apparently made use of a scoring key in reviewing the written test. Upon inquiry, counsel advised this Arbitrator that the written test was an essay type test for which a scoring key had been devised; it was not, however, a multiple choice test where answers are either correct or incorrect.
Following the filing of the grievance in this matter along with others of a similar nature, the Ministry undertook to remark all relevant tests and determined that amongst others, this Grievor’s mark did not reach or exceed the minimum threshold of 55%.
THE DECISION
Having considered the facts submitted and the submissions of counsel, it is my view that the Grievor was inappropriately disqualified from further consideration based upon her test results alone. Although there were three stages to the test, including an oral presentation, an oral interview and the written test, there was no evidence that any of those involved a review of the Grievor’s past record and work history with the Ministry. Accordingly, it was unclear as to whether or not the testing involved any consideration whatsoever to the Grievor’s past record and experience, qualifications and ability in particular in the Retail Sales Tax area. Although tests have been endorsed by this Board as a legitimate part of the selection process there were few if any cases available where the results of such tests were considered sufficient in and of themselves to disqualify a candidate for a promotion. Indeed, where the collective agreement sets out a standard of relative equality between candidates, it would seem counter-intuitive that the application of a threshold test could disqualify a senior employee scoring 54% in favour of a junior employee scoring 55%. A difference of 1% seems relatively insignificant.
Although there was no allegation that the tests were unreasonable or that there was any bad faith involved in the setting of the test or the marking of the tests with respect to the Grievor or any other candidate, it is my view that the test results should comprise but one piece of information, albeit significant, available to the Employer in the selection process.
Arbitrator Owen Gray had occasion to deal with the appropriate use of tests in a case before this Board between OPSEU (Sauve) and The Crown in Right of Ontario (Ministry of Transportation) in a decision released May 3, 1993 pursuant to GSB File No. 1695/91. At pages 19 and 20 of his Award, Arbitrator Gray reviewed some of the Board’s jurisprudence with respect to the use of tests in the selection process and the establishment of minimum qualifications or thresholds in the use of such tests:
“Properly prepared and conducted tests may be used as part of the selection process, subject to the limitation that applies to interviews: the results of a test should not be relied upon to the exclusion of other evidence of the qualifications and abilities which the test is designed to measure. Moreover, the selection committee must be alert to the possibility that a test which has not been validated, particularly a test which is designed by persons who have no expertise in test design, might not reliably measure what it is intended or expected to measure. When the test purports to measure a qualification or ability which evaluations of past performance will also have measured, the evaluations must be given appropriate weight even (and, perhaps, especially) when the test results and evaluations conflict: see Hall/Powers, 716/89 (Gorsky) at page 18.
Most job specifications list several distinct job activities. Some of those activities may form a greater part of the job than others, and each activity may require a distinct qualification or ability. As part of its selection process, the employer may establish a reasonable minimum for each required qualification or ability, and may refuse to consider further those applicants whose applications do not claim that they have, and are not known to the employer to have, (or are found not to have) one of these essential minimum qualifications or abilities: Moses, 715/89 (Barrett); Sequeiro and Lueck, 768/87 (Gandz); Bent, 0031/88 (Knopf). Once consideration has narrowed to comparison of applicants who all have the minimum required qualifications and abilities, the weight which the employer gives to the applicants’ strengths in each relevant kind of qualification or ability should correspond at least roughly with the degree to which the activities of the job require that qualification or ability; Lall, 1726/87 (Fisher); Loconte, 2573/87 (Fraser).”
In the context of the matter before him, Arbitrator Gray was clearly uncomfortable with the Employer’s reliance upon tests to disqualify the Grievor from further consideration especially when to do so resulted in a disregard of earlier evaluations and/or job history. At page 23 Arbitrator Gray expressed himself in the following terms:
“Whatever the committee members may have thought, however, we are satisfied that the results of the letter test did not justify eliminating the grievor from consideration for the position in question. The committee had not consulted the grievor’s supervisor and had no reason (other than the test results) to suppose that the positive job evaluations which accompanied his application were exaggerated or unreliable. The test results were not a sufficient basis for ignoring either those evaluations or the grievor’s job history. Considered in the context of that job history and those job evaluations, and in light of the considerations we have already identified, the test results did not support a conclusion that the grievor lacked some essential minimum qualifications or ability required in the subject job. Equally, in that context the test results could not have supported a conclusion that the grievor’s qualifications and abilities were so eclipsed by those of other candidates that he was clearly out of the running. Accordingly, we reject the argument that the employer acted properly when it at first eliminated the grievor from consideration without granting him an interview.”
With respect to the facts and issues before me, the Grievor had successful achieved the threshold mark set by the Employer when the written portion was initially graded by H.R. Associates, the third party which had been provided with the scoring key in order to properly assess the results. In the circumstances, absent any explanatory evidence, the subsequent review by the transition team members which resulted in an overall failing grade signified some weakness or inconsistency in the test itself or perhaps in the scoring key. In any event, the fact that two different results were possible, where good faith is not in question, indicates that there was sufficient subjective element either in the test or the scoring keys to undermine the validity and consistency of the test results. The subsequent review of the Grievor’s test results following the filing of the grievance does little to satisfy this concern. Still there was no explanation for the inconsistency nor was there evidence that the subsequent review confirmed the identical mark to that achieved by the Grievor in the first re-mark. At best, that information is self-serving and fails to explain the earlier inconsistent results.
In summary, it is my view that:
The inconsistency in the test results, without explanation, undermines their reliability;
To have relied on those inconsistent test results as a basis for finding that the Grievor had failed to meet the minimum requirements for the TA4 position was inappropriate;
While a threshold test to determine minimum qualifications may be appropriate in certain circumstances, that does not necessarily relieve the Employer from considering comparative results of different candidates where the differences in their marks are marginal such that they might be considered relatively equal in that regard;
Any test which is ostensibly set as a threshold test to determine minimum requirements should be based insofar as is possible on objective standards which are consistent and reliable or, if a subjective element is involved, that element must prove reliable when subjected to scrutiny;
In any event, as quoted by Arbitrator Gray in the Sauve matter “the results of a test should not be relied upon to the exclusion of other evidence of the qualifications and abilities which the test is designed to measure”.
In the circumstances, I remit the matter to the Parties to consider an appropriate solution insofar as this Grievor is concerned. I shall remain seized in the event the Parties have any difficulty in reaching a satisfactory resolution or remedy for Ms. Myles.
Dated at Toronto this 6th day of July 2007.