The Indian Child Goes To School

APPENDIX D

Levels of Achievement

Differential Diagnosis on Achievement Profiles

LEVELS OF ACHIEVEMENT

A reference was made in Chapter VII as to how a distribution of raw scores for one grade in an area may be stratified for an interpretation of meaningful levels of achievement. These levels are determined by the three raw score points in a grade distribution nearest to the mean and plus and minus one standard deviation from the mean. The standard deviation may be described as the average distance of all scores from the mean of a distribution of scores. Thus the raw scores within the range of plus and minus one standard from the mean may be considered average scores. They will include about 68% of the cases. The 34% of the scores between the mean and +1 S. D. may be called high average, while the 34% between the mean and -1 S. D. may be called low average. The achievement of the 16% of individuals scoring above +1 S. D. may be called high, while the achievement of the 16% scoring below -l S. D. may be characterized as low.

Table D- 1 presents the raw score limits for each level of achievement on each of the seven tests for each grade in the Anadarko area (and combined with the Muskogee area on the advanced level). Similar tables have been prepared for each of the other areas.

DIFFERENTIAL DIAGNOSIS ON ACHIEVEMENT PROFILES

When a battery of tests is administered to a group of individuals, three major problems of the interpretation of scores arise: the comparability of scores of one test with those of another; the significance of the difference of scores on the same test; and the significance of the difference o~ scores on different tests.

The problem of comparability asks the question: What scores on all the other tests are equivalent to a score of x on test X? In this study comparability was obtained on each level of the California Achievement Test by normalizing the distributions of scores, plotting each on the same scale of T-scores, and using the distributions of the same group in an area to obtain normalized standard scores for all seven tests.

The problem of the significance of difference of scores on the same test poses the question: Taking chance variation into account, when can we say that two scores are most probably different scores; i. e., one Is really higher or lower than another? The standard error of measurement was calculated for each test on a level, prior to which reliability coefficients were obtained. Raw scores were plotted on the profile approximately one standard error apart. A difference in raw scores exceeding two standard errors may be expected less than five times in a hundred by chance alone

The third problem, of the significance of difference of scores on different tests, asks the question: When can we say that the achievement on one test Is really higher than on another? In Chapter VII reference was made to a “rule-of-thumb” that a difference as great as or greater than 8 T-scores between two different tests could be considered significant. A short explanation as to the method of obtaining this amount is in order.


A t-test can be made to test the probability level of every possible difference between each of the tests by using the formula:

A simpler way is to establish the t value for the desired level of confidence (.05 or .01) and to solve for the z score difference on each pair of tests, using their respective reliability coefficients. These differences were calculated for the five per cent and one per cent levels of confidence for each pair of tests on each level in each area. The average T-score difference (ten times the z score difference) at the five per cent level in all areas was found to be about 8. Thus the “rule-of-thumb” was found to be fairly consistent through all the areas.

The calculated values of difference in T-scores In the Anadarko area for both the five per cent and one per cent levels are shown in the accompanying tables. On the advanced level the Anadarko and Muskogee areas were combined, it will be recalled.

The tables may be interpreted as in the following illustration. In Table D-l, we see that a difference of nine T-scores or more between reading vocabulary and spelling may be expected less than five times in a hundred by chance alone. A difference of twelve T-scores or more on these two tests may be expected less than one time in a hundred by chance alone. These differences are considered significant on their respective levels of confidence. For speed in interpreting profiles the “rule-of-thumb” may be considered adequate.

APPENDIX E

Letter of February 9, 1951, from E. Gordon Collister to Willard W. Beatty, Proposing the Content of the Predictive Test Battery

Letter of February 12, 1951, from L. Madison Coombs to the Area Directors, Advising Them of Preliminary Planning for Predictive Testing

Letter of February 27, 1951, from L. Madison Coombs to the Area Directors, Advising Them of Specific Procedures for Predictive Testing

Application Form for the Predictive Test Battery

Table E-1 Intercorrelations of Scores on the Test Battery for Haskell Commercial A~pplicants, 1951-54

Table E-2 Test Battery for Haskell Commercial-Standard Errors of Measurement and Reliability Coefficients

Table E-3 Expectancy Tables for Pass and Fail Groups Haskell Commercial Program, 1951-54

February 9, 1951

Dr. Willard W. Beatty
Director of Indian Education
Bureau of Indian Affairs
New Interior Building
Washington 25, D. C.

Dear Dr. Beatty:

After several discussions with Mr. Coombs and Mr. Kelley, the following battery is suggested for use with Indian students applying for educational loans or admission to Haskell.

For students requesting admission to the commercial sequence at Haskell, we suggest substituting the Hundred-Problem Arithmetic Test (Schorlung-Clark-Potter) for the Cooperative Mathematics Test. Provided replacement costs for test materials do not soar, the estimated cost for this battery is $.75 per Individual. This includes the cost of the test materials, answer sheets, profiles, scoring, use of electrographic pencils, and postage for mailing tests and test reports. It is our understanding that the individual applicant will pay a fee of $50 and for the remainder of this fiscal year the balance of the cost will be drawn from our contract funds.

With the exception of the Hundred-Problem Arithmetic Test none of the above tests Involve a very short time limit for a subtest. This reduces the importance of small timing errors.

The Otis test is suggested since we already have Information from previous testing programs that may lead to meaningful comparisons. Since the content of the Cooperative Mathematics test is not particularly suited to the areas Important In the Haskell commercial department, we suggest the use of a test dealing specifically with fundamental arithmetical operations for applicants to this program.

In our discussions there was a difference of opinion regarding the Cooperative English Test. There was agreement on the use of part C₁ - Reading Comprehension. The use of both A - Mechanics of Expression and B₁ - Effectiveness of Expression was questioned. The content of subtest A deals with material usually associated with drill work in many English courses. To this extent the content will probably be familiar to students. The content of subtest B₁ is not as susceptible to practice and apparently is closely associated with the usual “Freshman Composition.” To have a complete picture of the English background of an individual student it is felt that the three subtests are important.

There was considerable discussion regarding the use of the Guilford-Zimmerman Temperament Survey. It was suggested that decisions concerning the granting of an educational loan would not be based on the results of this Instrument. In addition, the findings reported in the Educational Records Bureau study indicated that personality ratings did not add to the prediction of college success. It should be noted that the latter conclusion is based on the use of personality ratings. We cannot make the same statement concerning the use of scores on a personality inventory although it is not expected that any correlation with college success would be particularly high. The most important use of personality data appears to be for research purposes. If, at some later date, a follow-up study of applicants is contemplated, the time to obtain personality estimates is at the same time other test data are collected. For this reason the use of the Guilford-Zimmerman is suggested.

There are several general considerations which are important.

1. The tests suggested are not “easy.” The battery may be criticized as being too difficult for the background of some of the Indian students. However, the people taking this battery will be a highly select group from the total group of Indian students. Since the question at issue is not one of general level of achievement but one of ability and preparation for advanced study, test instruments suitable for this purpose should be used.
2. The impact of a battery of this type on the individual student is of considerable importance. Mr. Coombs has already raised this question and is considering means of ameliorating the problem.
3. It may appear that the length of the battery is excessive. Considering the importance of the questions to be answered, it is not felt that this objection is valid.
4. Tests in the areas of natural science and social studies have not been included in the battery. Perhaps they should be considered for use at some future time when the fields of specialization for college students are known.

In case you do not have copies of the suggested test, Mr. Kelley has a set with him for your use. We will await a reply from you before proceeding with further arrangements. Tests will be ready for distribution two weeks from the receipt of your reply.

Yours truly,

E. Gordon Collister
Director

February 12, 1951

Dear Mr. Harper:

This letter is for the purpose of bringing you up to date on future plans of the Service-Wide Testing Program with the thought that you will wish to pass this information on to reservation and school principals in your area at the earliest opportunity.

During the period December 11 to 14 inclusive, a conference was held at Haskell Institute for the purpose of reviewing the Indian Education Evaluation Program and charting its course for me future. The meeting was attended by the following representatives of the Indian Service: Mr. Willard W. Beatty, Chief of the Education Branch; Dr. George Dale, Education Specialist, Research; Mr. Earl Intolubbe, Education Specialist, Supervisor of Guidance; and Madison Coombs, Education Specialist and Director of the Service-Wide Testing Program. Dr. Kenneth E. Anderson, Director of the Bureau of Research and Service; Mr. Gordon Collister, Director of the Guidance Bureau; and Mr. Carl Ladd, graduate assistant, represented the University of Kansas.

The following decisions of immediate importance to you and your personnel were made:
(1) There will be no achievement testing for survey purposes on a nation-wide basis this year. Since the University of Kansas has recently assumed the responsibility of serving as consultant to the program, it was felt that the University men should have an opportunity to make a thorough study of what has been done in the past, to survey the instruments which have been used, and to have a hand in planning any future cycle of testing. Testing on a service-wide basis will be resumed during the 1951-52 school year. (2) The administration of tests to 12th graders who wish to be considered for educational loans or who make application for admission to the Haskell Commercial Department will be continued this spring, if present plans materialize. The selection of tests to be used for this guidance function is in process. It is not presumed that all 12th graders will be tested, but only those, both in our Indian Service schools and mission and public schools, for whom we need predictive data relative to their pursuing an education at the post high school level. Detailed information concerning the procedure to be followed will be sent out from this office soon. (3) A statistical analysis of the result of last spring’s achievement testing and interpretation of the data will be completed by the University of Kansas as soon as possible. (4) An effort will be made to validate instruments used in the past for the purpose of predicting success in college by making comparative studies between scores on the tests and the student’s success at the college level. These tests would include the Gates Basic Reading, Pressey English, and the Factorial Abilities Series, among others.

Inasmuch as we had tentatively planned at one time to do survey testing early in the current semester, I felt that the above information should be passed on to you so that you and your school personnel would know that time would not need to be allotted for this purpose this year.

Sincerely yours,

L. Madison Coornbs
Education Specialist
Indian Service Testing Program

Approved:

Solon G. Ayers
Superintendent

February 27, 1951

Dear Mr. Head:

As promised in our letter of February 12, paragraph 3, item 2, this letter will acquaint you with plans and procedures for the testing this spring of 12th grade students who wish to be considered for educational loans or for admission to the Haskell Institute commercial training course. Inasmuch as the new plan represents a considerable departure from procedures used in recent years, the plan will be described in some detail below.

1. The test battery has been adopted upon the recommendation of the Guidance Bureau of the University of Kansas and has been approved by Mr. Willard W. Beatty, Chief of Education Branch, and Mr. Earl C. Intolubbe, Education Specialist, Guidance. The battery will consist of the following tests:

These tests cover most of the basic skills which are important to academic success and therefore should have validity in predicting academic success or failure at the post high school level. We anticipate a universal reaction that the tests are relatively difficult. While this is true, we wish to point out that the purpose of the tests is to aid in predicting success or failure In college, nurse’s training, or the commercial course, and not to measure achievement in high school. They should prove more discriminating as between individual students than “easier” tests would be. It cannot be emphasized too strongly that no student will be expected to answer all of the Items correctly. An important advantage of most of the tests selected is that percentile norms for entering college freshmen are available for them. It is suggested that you exercise considerable discretion about revealing, in advance of testing, the names of the tests which comprise the battery.

2. The batteries will be available for distribution about March 15 by the Guidance Bureau of the University of Kansas. You are being supplied, under separate cover, with forms on which the applicant will place his order with the Guidance Bureau. The applicant is required to include a fee of 5Ocents with his order to partly defray the cost of tile battery and the processing thereof. Also we are requiring that the applicants high school or reservation principal or area educationist approve the order so that we may be assured that the battery is being mailed to a proper person for administration. It is felt that the 5O cent fee will not preclude any persons taking the tests or work a hardship but will serve to emphasize the importance of the tests and cause the applicant to feel that he has made a small investment in his future.

3. The battery, together with explicit instructions for administering, will be mailed to the person who is to give the test. Some of the tests will have answer sheets and special pencils for machine scoring. All of the tests and answer sheets will be returned to the Guidance Bureau for scoring. It is recommended that whenever possible the battery be administered during parts of three different days, over the span of a week, to lessen the impact of the battery. In cases where this is not feasible or where a hardship would be occasioned thereby, the tests may all be given in one day but a full day should be Set aside for the purpose with provision for adequate rest periods.

4. It is recognized that the persons to whom this is addressed have a difficult problem to solve in setting up an organization for the administration of these tests. Principals of our own Indian Service high schools will undoubtedly want to test all of the applicants from their school at one time. Consequently they will probably prefer to send all of their application forms and fees to the Guidance Bureau together. Testing of mission and public school applicants will probably present more of a problem. You may wish to designate boarding schools in your area as testing centers. The main concern of this office is that the persons giving the tests prepare themselves carefully by studying the directions and that they adhere to them closely.

5. One copy of the test results for educational loan applicants will be sent to Mr. Intolubbe, Supervisor of Guidance, and one to the educationist of the area in which the applicant took the test. Copies of test results for commercial course applicants will be Sent to Mr. Intolubbe, Haskell Institute, and the area educationist.

If you have questions which are not answered in this letter do not hesitate to write us at any time. We wish to be of help in any way possible.

Sincerely yours,

L. Madison Coombs
Education Specialist
Indian Service Testing Program

Approved:

Solon G. Ayers
Superintendent