EVALUATING AMERICAN INDIAN EDUCATION

By

Thomas (Tom) R. Hopkins, Ed.D.

© 2003

Research and Management Specialists, Inc
4106 Tara Ct. NE
Albuquerque, NM 87111

January 2003

Introduction

Evaluating American Indian education requires the best of professional educators, especially as related to the principles of fairness and accuracy. The contemporary situation in American education is dominated by high stakes testing and state level accountability programs in which American Indian education receives, mostly, unfair treatment based on culturally biased, inaccurate test data. However, it is possible within high stakes testing and state accountability programs to provide fairness and accuracy to American Indian communities. We hope in this article to suggest to evaluators how this can be accomplished.

The 2000 Decennial Census (2002) reports 4.1 million people who self-reported being American Indian or part American Indian. The 4.1 million is 1.5 percent of the total U.S. population. 2.5 million of these were American Indian or Alaskan Native only, which is .9 percent of the U.S. population. An additional 1.6 million reported being part American Indian. The BIA provides services to “American Indians” who are 25 percent or more Indian by blood quantum. In this respect, “American Indian” is a legal term which applies to the BIA (and the U.S. Public Health Service Division of Indian Health) and its programs including over 200 tribes on reservations which encompass over 50 million acres of land. The BIA Office of Indian Education Programs (OIEP) web site describes its Indian education base as: “OIEP has responsibility for 185 elementary and secondary schools and dormitories as well as 27 colleges. Our colleges, schools and dormitories are located on 63 reservations in 23 states across the United States serving approximately 60,000 students representing 238 different tribes.”

Nationally, the American Indian population is complex, often confusing to non-Indians and offers education evaluators a great challenge to identify with reasonable accuracy who would be included in an evaluation. Nonetheless, there have been several national evaluations of American Indian education.

American Indian Education Evaluation, An Historical Approach ¹

For purposes of this paper, evaluation is defined as providing valid education information to decision makers. Evaluators don’t make administrative decisions. Evaluation differs from research primarily in the area of control of the education environment. Research usually involves a very limited number of subjects within a tightly controlled environment investigating strictly defined interventions. Even though evaluations are large in scope with limited control of the environment, they produce valid information on which decision makers can make decisions. In several instances, education research reports will be included in the literature.

In developing an historical approach to American Indian education evaluation it is appropriate to start with the first quarter of the twentieth century. We start at this time as it was during the first quarter of the twentieth century that American educators began to apply scientific methodology to education. Jerome V. D’Agostino (2000) tells us that something similar to standardized tests were first introduced in Boston in 1845. During the nineteenth century scientifically developed education instruments were not applied to American Indian education. It was during the first quarter of the twentieth century that scientific instruments were applied to American Indian education. D’ Agostino also points out the importance that Ralph Tyler and “Progressive educational reformers,” played important roles in early twentieth century testing. We will learn that Tyler also influenced American Indian education evaluation. Ironically, Robert Salvin (2002) in a recent article espouses the belief that, at last, scientific methodology in education will drive transformation of educational practice and research in the twenty first century. Salvin claims that science has yet to dominate U.S. education processes.

The application of scientific methodology, borrowed from the natural or hard sciences such as physics and chemistry and the employment of advanced mathematics, is not a bad development. The application of science to education has been a creative and intellectually respectable contribution to education. Scientifically developed education instruments, like all science, requires specificity and intellectual rigor. It focuses thought and education activities in a very specific manner. Scientific education instruments are usually based on a specific target population such as the total U.S., college level students, etc. If applied to that population they can be helpful to professional educators working in schools specific to the targeted population. On the other hand, achievement tests have historically been developed for the U.S. as a whole, but have reflected only the Eurocentric² based culture and related population. Without developing special norms for sub-cultures, such as Navajo, Eskimo, Sioux, etc. achievement tests produce inaccurate scores.

Preliminary, Background Education Research

Robert Havighurst (1957), the noted education researcher at the University of Chicago, contributed enormously to the education research on American Indians. His article in the 1957 issue of The Annals is a good place to start. In this article Havighurst refers to a 1928 research project on intelligence. The following paragraph makes the point of cultural fairness and standardized instruments.

Klineberg administered the Pintner-Paterson series of six tests to Indian and white children on the Yakima Indian reservation and found (1) that Indian children took longer with form boards but made fewer errors, (2) that comparison of Indian and white groups in terms of total number of points obtained on the Pintner-Paterson Point Scale showed no differences between the two because the Indians made up in accuracy for their inferior speed, and (3) that correspondence of score with degree of white blood was lacking. Whereas preceding investigations pointed to the superiority of the whites over the Indians on tests of intelligence, Klineberg’s study is among the first to offer contradictory evidence and to suggest that test performance may be affected by cultural factors.

Klineberg published his article in Archives of Psychology, 1928.

Havighurst also includes intelligence test results as related to American Indians. Said tests developed for Eurocentric, English speaking populations, have not produced reliable American Indian IQ’s. However, the several non-verbal tests, including the “Goodenough Draw-A-Man Test,” used since 1926, compare Indian children with non-Indian children in a more equal and fair manner. In the early 1940’s Havighurst did intelligence research on Navajo children. He had this to say about his work.

The existence in the Indian's experience of parallel but different cultures is illustrated by a study made in the early 1940's of Navaho Indian children having different degrees of contact with white culture in two different communities? The Shiprock community is located on a well-traveled highway and has substantial economic relations with neighboring white communities. This was probably the most acculturated of Navaho communities in the early 1940's. In contrast, the Navaho Mountain community was probably the least acculturated. Situated far away from any highway, in an area almost never visited by white men. Many of the Navaho children had never seen a white person except the school teacher: and many of the children never attended school. A few of the older children attended a boarding school in Tuba City, which was on a secondary highway, but still relatively apart from the world of white Americans.

The Shiprock children were closer to the average of white American children on a number of tests of attitudes and abilities than were the Navaho Mountain children, but still the Shiprock children showed many significant differences from the white children, indicating the persistence of the Indian culture in their lives (pp. 107-108).

Over 60 years later, things haven’t changed at all regarding the use of intelligence tests that Havighurst reported. Havighurst benefited from the BIA’s national Education Program office Chicago location. In 1941 the “Indian Education Research Project,” was initiated at the University of Chicago by the BIA (Zintz, 1969, pp. 131-132.).

Perhaps one of the earliest and most in-depth study of one American Indian education group was conducted by two Stanford University researchers, Dewey Anderson and Walter Eells (1935). They spent two years trekking throughout Territorial Alaska applying standardized tests of achievement and intelligence to Eskimos, Indians and Aleuts. Frustrated with the dramatic difference between Alaskan Native language, cultural and social conditions and the Stanford Achievement test or the Stanford-Benet Intelligence Test, the researchers attempted on-the-spot modifications in order to develop different data. It was interesting in that a main source of information on Alaskan Natives was mostly provided to the researchers by local non-Natives, many of whom operated isolated trading posts. It is important that standardized evaluation instruments provided only minimally useful results. The most important finding of this massive work was that Alaskan Natives were (and are) very different culturally from the norming groups of the instruments, consequently, standardized instruments and related statistics based on norming groups, should be used on American Indians with great caution.

Up until about 1946, the BIA headquarters office for its national education program was located in Chicago. It is not surprising that the University of Chicago played an important role in evaluating American Indian education. With the advent of the famed Meriam (1928) report and the change in administration to Franklin Roosevelt, important changes in American Indian education were implemented. Roosevelt appointed John Collier as Indian Commissioner (Philp, 1981), who appointed Progressive Educator Edward Everett Dale as his Education Director. Dale began a long influence on BIA Education by the Progressive Education movement (Goodenow 1980). When Dale left the BIA another Progressive Educator, Willard Beatty (Squire 1972, p. 214), was appointed as Education Director. Collier, who espoused “Social Engineering,” (Boyce, 1974) and his Progressive Education Directors of Education were partial to evaluation data to prove effectiveness of Federal schools compared to public schools.

Four National Evaluations, 1944 - 1970

In the contemporary standards based education reform for most states, standardized tests are used in ways the test developers say they should not be used. Standardized tests cannot measure a total education program which includes affective and cognitive domains. Further, to produce accurate evaluation data on a reform program the curriculum content, teacher education and assessment (tests) must be aligned (Schoenfeld, 2002). This is a simple, age-old education principle which says that the content taught children determines the content of the tests and that teachers are educated to teach said curriculum content.

There is an extensive literature on evaluating American Indian education most of which has been produced by serious and competent professionals. There are three components of each evaluation that lead us to our conclusion about fairness and accuracy in American Indian education evaluation. First, each publication reflects a purpose which we find is basic to the quality of the evaluation. Second, though there has been less attention to who was included in the evaluation, there was at least some statements defining the American Indian population being evaluated. Third, methodology while not receiving the discussion it deserved, was reasonably reflected in the instruments and analyses.

There were four national evaluations of American Indian education. All four were paid for by the BIA and had a major purpose of comparing the relative effectiveness of the BIA education program schools compared to public and mission schools and Indian and White students. The first was contracted to the University of Chicago, the next two to the University of Kansas and the fourth conducted without university assistance, was contracted to the Southwest Regional Laboratory, a Great Society creation. There was continuity between the four evaluation that was provided by L. Madison Coombs, an employee of the BIA’s Education office. Coombs was a major role player in BIA education evaluation and research from about 1950 to 1970. The general design of the four evaluations was set by the first evaluation. Because the first evaluation more or less established the basic design for all four, it will be discussed in more detail and its design used to describe the following three.

The Peterson Evaluation, Number One

A first national evaluation of the education of American Indians (American Indians) was contracted by the BIA to the Education Department of the University of Chicago. This evaluation started with a “Pilot study” in 1944 and gathered data in 1945 and 1946. Shalier Peterson (1948) in the Preface mentions that “Mr. Willard W. Beatty, Director of Education for the Office of Indian Affairs, consulted with Mr. Ralph W. Tyler, Chariman of the Department of Education of the university of Chicago and asked the university to provide assistance in planning this project, in guiding its execution and in analyzing the data gathered.” Tyler also contributed a section (pp. 113-117) to the book. Among others, Hilda Taba was a Research Associate and Assistant Professor at the University. It should come as no surprise that the BIA Education Program had well defined objectives. Peterson quotes from the Civil Service examination:

“The primary objectives of Indian schools are: To give students an understanding and appreciation of their own tribal lore, art, music, and community organization; to teach students through their own participation in school and community government to become constructive citizens of their communities; to aid students in analyzing the economic resources of their reservation and in planning more effective ways of utilizing these resources for the improvement of standards of living; to teach, through actual demonstration, intelligent conservation of natural resources; to give students firsthand experience in livestock management, use of native materials in housing and clothing, in subsistence gardening, cooperative marketing, farm mechanics, and whatever other vocational skills are needed to earn a livelihood in the region; to develop better health habits, improve sanitation, and higher standards of diet with a view to prevention of trachoma, tuberculosis, and infant diseases; to give students an understanding of the social and economic world immediately about them and to aid them in achieving some mastery over their environment; and to serve as a community center in meeting the social and economic needs of the community.” (p. 9)

Peterson states the purposes of the evaluation as:

The Indian Service-wide Testing Project, reported in this monograph, had two main purposes: 1) to examine the progress and achievement that the Indian students had mode in various types of educational situations; 2) to examine those factors which were thought to be related to the student’s educational development and to uncover any other factors which might prove to be related.

As pointed out above, the BIA was the major education institution providing schooling to Indians on reservations. The purpose of the evaluation was generally, in line with the famed eight year study of progressive vs traditional education. The “New Deal” BIA had instituted a new bicultural program, which was referenced above in the purpose (Szasz, 1977). The evaluation question was, “Did Indian students exposed primarily to the new program do better than those who were not exposed to the program?” As Tyler in his section reports:

The tests results show that children in the fourth grade who have had their total schooling under the new program have made relatively greater progress than pupils in the 8th or 12th grades who have had but a part of their schooling under the new program. This is true in each of the major fields, reading, writing, arithmetic, natural resources and health and safety. More specifically, the children coming through the new program gain not only knowledge of matters related to life in the Indian home and community but at the same time acquire greater competence in the tools of learning. By dealing in school with matters of interest to Indian children that they can actually use in their own lives, not only do they learn about these things but they acquire greater skill in reading, writing and arithmetic as these abilities are used in study of matters that are interesting and helpful to them. (p. 116)

Purpose. The purpose of this evaluation was to evaluate the BIA Education program and make comparisons between BIA and other schools in order to determine if the BIA progressive education program was as good as other programs. This was a large scale evaluation intended for use by administrators and policy level officials.

Indian Students. The Indian students included in the evaluation attended BIA, public and mission schools. Students in grades 4, 8 and 12 were evaluated. Indian students met the BIA legal requirement of being “Indian” with 25 percent or more by blood quantum. The report is very detailed in the instruments used for each grade level. There were 5,324 students attending six different types of schools. This was a “National” evaluation in that it evaluated the BIA Education program nationally. It did not include all American Indian students attending schools throughout the U.S. While the evaluation was longitudinal, 1944 – 1946. Students were not tracked individually. The report states, “In interpreting the tables and charts, it is necessary to keep in mind that these data describe three separate groups of students instead of the same students at three periods.” (p. 29)

Figure 1 Peterson Data

Curriculum. At that time, 1946, the U.S. Indian Service (USIS) had specific curriculum guides. Its Education purpose, as described above, was clearly stated. Evaluators tried to align tests with some aspects of the curriculum.

Instrumentation. Fourteen different evaluation instruments were used. Three of these were standardized tests two pertained to background information and five special instruments which were based on the USIS curriculum. Scientific methodology was used throughout. Chapter Two contains details on instrumentation. The USIS instruments as well as the “Free Writing” test samples are printed at the back of the report.

Background Data. Test data was linked to the “Background Questionnaire.” Background data was very important to interpretation of test results. Chapter Three, “Differences in Student Backgrounds,” provides a detailed discussion of the links to test data, including cultural backgrounds of students. It is interesting that the evaluators of 1944 thought test scores should be related to background characteristics.

Indian Blood quantum. As mentioned above, blood quantum is a legal term related to attendance at BIA funded schools. It was assumed in the Peterson evaluation that the greater the degree of Indian blood, the closer the individual was to the Indian culture. The report states, “The data do not justify a conclusion that school success is limited by degree of Indian blood.” (p. 29)

Statistics. Raw scores from standardized tests were used to illustrate relationships between scores and background characteristics. Median scores were illustrated graphically as was the range of raw scores. In fact, medians were used most often in reporting raw test scores. Number of students on which the graphs were based were also included. Percentages were used in reporting background characteristics. Range of data were also presented with a three part division between the bottom third, the middle third and the top third for each instrument. Figure 1, above, is from the Peterson report.

Test Norms. Test norms, or “National Norm Equivalents” were reported. This is nowadays referred to as “Grade Equivalents.” It is interesting to note the comment about the use of test norms with Indian students.

The "National Norm Equivalents" have not been provided with the idea of suggesting any levels of attainment that are expected. National norms are commonly based upon either urban school children or, in some cases, a combination of urban and near-urban education. In view of this, it is obvious that national norms would not be the proper yardstick to use as a means of measuring or comparing achievement in this Indian education study. (emphasis added) It should be repeated here that the students tested in the Indian evaluation study all come from a rural background and were taught in schools having a rural environment, and therefore these students should not be compared with the urban students, whose environment and curriculum are recognized to be quite different. The national norms do help to provide some concept of the range of abilities, and they also give one an understanding of the increments between grade levels in the urban school population. The national norms show the achievement of the rural students to be, in some instances, much higher than one might expect. (p. 59)

Interestingly enough, the general rule expressed above about the 1944 – 46 evaluation is just as valid today as then. The graphic presentations juxtaposed, side by side, raw score data and test norm data. The comparisons are interesting.

The Anderson Evaluation, Number Two

The second national evaluation was conducted by Anderson, Collister and Ladd (1953) at the University of Kansas. By the time this evaluation started Madison Coombs, a former Kansas county school superintendent, had joined the BIA Education program and was stationed at Lawrence, Kansas on the campus of the then Haskell Institute, now the Haskell Indian University. Coombs was instrumental in selecting Anderson to lead the evaluation and was also instrumental in setting the design for the evaluation. Coombs is acknowledged in the report frontispiece. It borrowed a great deal from the University of Chicago evaluation.

• The University of Chicago and Ralph Tyler were referenced as providing the background to the evaluation.
• The same purpose and philosophy as stated by Peterson were used by Anderson.
• The same instruments were used as much as possible.
• Background characteristics were important to interpretation of data.
• It was assumed that the same education program as described in the earlier study was still in effect.
• In 1950 Coombs established a “Service wide Testing program,” which provided data to Anderson. Data from the 1950 administration, only, were used.
• Students were not tracked
• Comparisons were made to the Chicago data.

Statistics. Since computers were used with data analysis different statistical analysis were also used. The Standard Deviation of the Mean was the main statistic used by Anderson. Statistical significance was also used as were “F” values and “t” ratios and the “Behrens-Fisher “d” test. In fact, Chapter III is titled, “Statistical Treatment of the Test Data.” Further, Chapter IV is titled, “Results of the Statistical Treatment of The Test Data.” “Raw or Untreated Scores” were used for statistical procedures. The first paragraph in Chapter IV explains the treatment of data.

 

Figure 2 Anderson Data

The results of the statistical treatment of the test data* obtained by the administration of twelve tests to the eighth grade and twelve tests to the twelfth grade will be discussed under three headings: (1) comparisons of achievement by means of line graphs for the nine geographic areas and for the six types of schools for grades eight and twelve, (2) comparisons of achievement by means of the percentage of overlap between grades eight and twelve where the tests were common to both grade levels, and (3) comparisons of achievement by means of critical ratios for the types of schools for grades eight and twelve. These three types of comparisons were made in order to provide realistic answers to the basic questions stated in Chapter II. (p. 34)

* Raw or untreated scores.

Figure 2 illustrates Anderson’s graphic illustrations of test data. Anderson’s more sophisticated statistics limited dramatically communication of findings to program and policy officials

By the time the Anderson report was published, 1953, Willard Beatty had left the BIA and been replaced by Hildegrad Thompson. Thomspon did not have the sterling Progressive Education credentials as Beatty, but she appeared to follow the philosophy of progressive education.

The Coombs Evaluation, Number Three

The Coombs evaluation reflects a clear departure from the previous two evaluations. The previous two evaluations cite the 1928 Merian report as a benchmark for assessing the education of American Indians. Coombs also cites the Merian report. However, the subtitle to Coombs evaluation is, “A Study of Interracial Differences.” Basically, Coombs changed the focus of the evaluation away from assessing the effectiveness of the BIA education program as compared to others to assessing , “Learning,” and achievement of Indians and whites as measured by achievement tests. In this study the California Achievement Test was used, which is another departure from the previous two studies which tried to maintain the integrity of test instruments between evaluations. In this respect, the Coombs evaluation becomes almost a new approach to evaluating American Indian education.

Basic policy regarding the education of American Indians is set by the Congress. Lyman Tyler (1973), a history professor at the University of Utah, wrote a book on Indian policy and the Government published it. During the 1950’s, the period in which the Coombs evaluation took place, the policy of the Federal Government, as determined by the Congress, was to “Terminate” Federal services to Indians. Regarding education, Termination meant turning over BIA schools to the public schools. It should be remembered that the previous two studies took place under the influence of the “New Deal” and the progressive education movement. With Coombs the foundation of policy shifts to something titled, “Inter racial.”

Another very important feature of the Coombs evaluation was that the overall design and work was done “In House.” This meant that Coombs, as a Civil Servant, an employee of the BIA, administered and controlled the evaluation, including the writing of the report. The professors at the University of Kansas, who shared authorship, provided technical assistance in the form of data entry and statistical analysis. Otherwise, this was an “In House” evaluation. The two previous evaluation were administered by higher education institutions and professors with the BIA providing support. The Coombs evaluation reversed the roles of the Government and higher education, and it is reflected throughout the report. Willard Beatty left the BIA and Hildegard Thompson took his place with the responsibility to implement Termination.

Major Characteristics of the Coombs Evaluation

Data Gathering. The two previous evaluation were very clear and explicit about who was included in the evaluations and when the data was gathered. Coombs does not reflect such a clear statement about data gathering. But, when one digs deeper it comes as no surprise that data gathering was controlled by the BIA field administrators. While Coombs reports a total of 23,608 and the breakdown by racial groups and schools, he was not clear on the time data was gathered. Background characteristics were included in the data structure.

Time and Data Organization. Coombs had to allow the field administrators, Agency School Administrators, to control who was to be included in the evaluation as well as when data was to be gathered. The BIA, starting with the Termination period, was divided into “Area Offices,” and it was these offices that determined that no total BIA administration of tests would take place during the 1951 – 1954. Without knowing the organizational structure of the BIA, one could think this was a longitudinal study, which it was not. Table 3-a (p. 18) provides the administrative areas and the number of “pupils” tested in each. There were six administrative areas at the time of the evaluation and all were tested in the fall of a year. While the evaluation program was initiated in the fall of 1951 and continued through 1954, each administrative area was tested only once in the fall of a year.

Grades Tested. Grades four through twelve were included in the evaluation whereas previous evaluation tested grades four, eight and twelve. The rationale for this was one of the purposes of the evaluation which was to improve learning and instruction at the local levels. Grades one through three were excluded because there did not seem to be a good achievement test available for the primary grades.

Evaluation Results. Coombs, like the other evaluations, is balanced in the discussions regarding test results, race and Indian cultures. He reports that, according to test scores, whites make higher scores than Indians. He, like the others, includes important discussions of age-grade results. Indians, because of language spoken in the home, were more often than whites, older for their grade placement. Pupils were old for their age and made lower test scores. Coombs established an “Order of achievement of Race-School Groups,” which was:

1. White children in public schools
2. Indian children in public schools
3. Indian children in Federal schools
4. Indian children in mission schools (p. 40)

Chapter VI is an excellent discussion of cultural and environmental factors on achievement. Blood quantum is again discussed but found not to be a basic factor regarding achievement. Coombs also made note of test scores and progress through the grades wherein Indians fall further behind whites as the progress through school.

Predictive Use of Tests. Coombs devoted considerable space to using tests to predict educational outcomes of Indians. Since there were no longitudinal data on which to base conclusions, his discussion was speculative. Haskell vocational program which enrolled high school graduates wanted to use test scores to predict probable success.

Figure 3 Coombs Data

Statistical Procedures. Because of the importance of administrative areas in the evaluation, statistical procedures accommodated the politics of them. Coombs developed a “Composite Norm.” for the different administrative areas. It was based on means.

The composite norms were established by calculating the raw score mean and standard deviation for the composite group on each skill, and total score, for each grade level. Raw score values were then fixed at the mean and plus-and-minus one standard deviation from the mean for each grade. Assuming that the distributions are normal, approximately 68 percent of the cases fall between the raw scores at plus-and-minus one standard deviation from the mean; about 16 percent fall above plus 1 S.D. and the remaining 16 percent below minus 1 S.D. These three strata of achievement are shown in the scale at the top of each figure and are designated as “average,” “below average,” and “above average.” This scale serves as a standard for comparison.

The graphic presentation of data is shown in Figure 3. T scores are also figured on raw scores. Almost as an after thought, Appendix D, presents additional statistical analyses on T scores level of significance. Obviously, statistical procedures other than means would not go over well with the area administrators.

The University of Kansas assisted Coombs with database management and statistical procedures..

Bass Evaluation, Number Four

Willard Bass (1971) , working at the Southwestern Cooperative Educational Laboratory, a creation of Lyndon Johnson’s Great Society, conducted the fourth and final evaluation in the series that started at the University of Chicago in 1944. It is remarkable in American education that we find some continuity that was initiated with the Meriam report in 1928 and was concluded with the Bass evaluation. It is suggested that this little known continuity which drew heavily from progressive education’s concern for effectiveness of school types, is a remarkable U.S. education phenomenon.

As much as any one individual, Madison Coombs was responsible for this remarkable continuity. He received no credit whatsoever for making the continuity possible. Coombs picked up on the Chicago evaluation when he joined the BIA staff at Haskell and worked with Anderson on his evaluation. Anderson makes clear, unambiguous reference to the Chicago evaluation. Then, Coombs and his evaluation, an internal evaluation conducted by BIA from 1951 – 1954, continued evaluating school types. Bass makes clear the role of the Coombs evaluation as a benchmark for his evaluation. Regarding the Anderson evaluation and the Bass evaluation, within the BIA, Coombs was the “Contract Officer’s Representative,” which means that he was the main Education officer overseeing the evaluations from within the BIA. The Bass evaluation therefore reflects the last study of an evaluative concern for the effectiveness of schools types that was initiated in 1928 and continued through to 1970, a period of 42 years.

Structure and Content of the Bass Evaluation

Research Design and Approach. In contemporary understanding of education research, the Bass evaluation was designed and conducted with disciplined strictness. It employed sophisticated statistics on a sample of students who attended a sample of schools. It used several scientific instruments including the California Achievement Test and the California Test of Mental Maturity as well as several others.

Individual Differences. “Background Characteristics” were included in all the evaluations and Bass refers to them as “Individual Differences.” Achievement between schools was controlled and achievement according to individual differences was also controlled. While there were no statistically significant differences between types of schools, there were also no differences in school effectiveness when the same types of students were compared. A low achieving students who had common background characteristics in a public schools made no higher achievement than a low achieving student in a BIA on-reservation school.

Using a Sample of Schools. Bass used only a selected, not a random, sample of the different types of schools. This placed the entire evaluation within the world of probability. The previous evaluations evaluated hundreds of Indian children and youth whereas Bass selected only a sample that would reflect the administrative areas of the BIA, which was in keeping with the Coombs structure.

Using a Sample of Students. A stratified sample of 3,375 students in grades nine through twelve were included in the evaluation. The students were tracked from year to year. In the spring of 1970, 837 students were tested. It is a safe assumption that the American Indian students met the BIA blood quantum requirement inasmuch as Madison Coombs selected the schools for Bass so they would reflect BIA administrative areas.

Longitudinal Evaluation. The Bass evaluation started in the fall of 1966 and continued through to the spring of 1970. A “Pre” and “Post” test was administered each fall and spring for four school years, 1966-69 through to 1979-70.

Secondary Education. Bass evaluated only high schools, grades nine through twelve. Perhaps as much as any part of the evaluation, no discussion was made of the history secondary education of American Indians. Historically, the BIA high school curriculum and instructional day were divided equally between vocational and academic instruction. Enrollment in a vocation was required of each student. Academic instruction was linked to the vocational curriculum that dominated high school education from the nineteenth century through about 1957. In 1957 the BIA started changing its high school curriculum by dropping the vocational requirement and implementing a stronger academic curriculum.

No mention was made of the lack of secondary school facilities available especially to Navajos and Alaskan Natives. For example, as late as 1964 the only high school available to Alaskan Natives was Mt. Edgecumbe, a boarding school located at Sitka. Further, only a scattering of high schools were available to Navajos. The Fort Wingate High School for Navajos, a boarding school, was not completed until 1967, which was in the middle of the Bass study. Otherwise, Navajos attended public high schools located on the Navajo Nation lands only to the extent that they were being constructed. Regarding Navajos, the lack of elementary schools on the reservation severely impacted negatively secondary education numbers. The effect of this condition was that there were literally thousands of American Indians denied a high school education simply because there were no elementary or high schools for them to attend. Obviously, had attention been given to the high school age students not in any school, a modified finding on achievement would have been reported. There is a pattern in the history of evaluating American Indians to include only those children and youth in school. If for some reason a student is not attending a school they are excluded from the evaluation. A focus of the evaluation on school types and achievement says nothing about the children and youth who leave school during the six though nine grades in large numbers.

Instruments. In addition to a questionnaire, which was included in all four evaluations, Bass used the following instruments:

• California Achievement Test raw scores to measure achievement
• California Test of Mental Maturity to used as a scholastic aptitude control
• Monde Problem Check List
• School Interest Inventory
• Semantic Differential on ten concepts, seven point scale
• California Psychological Inventory
• Vocational Aspiration Scale
• Self Concept

Statistics. Means were developed from raw scores and analysis of covariance were run. Tests of significant differences were run. Bass succinctly states the conclusion of his evaluation:

In the light of the above findings it can confidently be concluded that when initial individual differences in scholastic aptitude and academic ability were controlled there was no evidence that academic achievement of American Indian students was greater in one type of school than in another. (p. 81)

Bass did not include graphic illustrations of data but did provide numerous tables. A summary type table follows.

It can be said that the four evaluations, all of which were funded by the Federal Government’s Bureau of Indian Affairs, reflect seven decades of common school education of American Indians, 1900 – 1970. Some general observations from the evaluations can be made.

• The application of science to education through standardized tests and other instruments was basic to all the evaluations. None of the evaluators thought the direct application of test norms was fair to Indian students which resulted in the use of test raw scores in all four evaluations. Test norm scores were reported in all, but only as a minor observation. When test norms were used they reflected near grade level at grade four with a gradual falling below norms until in grade twelve where Indians were two years behind. This general test norm pattern continues to this day.
• At first, a comparison of progressive education as applied to the education of American Indians was the purpose of the evaluations. Schools were compared to schools. While the progressive program was dropped starting with the Coombs evaluation, comparing schools was continued.
• The statistical procedures used in all four evaluations seemed to reflect the professional evolution of education evaluation. The 1944 evaluation started with means and the Bass, 1970 evaluation used more sophisticated statistics. Using means of raw scores was basic to all statistical procedures.
• Students involved in all the evaluations varied from thousands to hundreds. All students were considered eligible for education services from the BIA and as such, were legally “Indian.” This means that all were one quarter or more Indian by blood quantum. Blood quantum was used as a variable in three of the evaluations. Bass did not use blood quantum as an important variable.
• The education levels ranged from grades 4, 8 and 12 to all grades to only high school. In this respect, there was inconsistency in grades included in the evaluations.
• In 1944 the BIA had a well defined curriculum which was tested with special instruments. Anderson, the second evaluation, also used USIS curriculum based tests. Starting with Coombs, which was at a time during which major BIA high school curriculum changes were taking place, the BIA curriculum was not tested. Alignment between standardized tests and curriculums was not in evidence in any of the evaluations.
• These were all large scale evaluations and were not considered useful at the classroom or local levels. Coombs did mention that he wanted his evaluation to be useful at the classroom level, but no follow-up on usefulness was included in the report. Hopkins (2002, Op. Cit.) has reported that the evaluations were not useful at the classroom level. There is no evidence that the evaluations impacted basic policy set by the U.S. Congress. There is no evidence that Indian tribes used the evaluation results. It is suggested that the evaluations were most interesting and useful to the academic community, only.
• The basic evaluation design, which was established by Ralph Tyler, is as useful today as when he implemented it in 1944. The overall evaluation projects reflect very professional and useful large scale evaluation design, instruments, students, statistics and reporting for any population not to mention American Indians.
• The age-grade placement of Indian children remained at least a year out of step with normal grade placement. This was discussed as emanating for the speaking of tribal languages in the home. The first year of school was devoted to teaching children English as a second language and by large numbers, Indian children didn’t enter first grade until age seven. This had started to change by 1970 but was still present for most of children schooled by the BIA. Age-grade placement was also pointed to as an explanation for lower achievement as compared to whites.

1970 – 1979 Evaluations

The National Study of Indian Education (NSIE). There were no large scale evaluations which took place from the Bass study culmination in 1970 through the next 30 years. Perhaps the four studies and the 30 years of Indian education covered therein established predictable standardized test results. Late in the 1960’s, which was during the time public schools assumed major responsibility for educating American Indians, there emerged new concern for the well being of American Indians, including their education. This concern was motivated by Senator Robert Kennedy’s study of the education of Indians (U.S. Senate, 1969). The Kennedy study generally described the education of Indians as being in a sad condition. It is important that the major contribution of the Kennedy study was creation of and establishment of the Office of Indian Education in the U.S. Department of Education. This office approaches Indian education on a scale similar to that of the Decennial Census definition of Indian. Within the Government, it was decided that a very large research project should be conducted. This study became the NSIE and was funded by the then U.S. Office of Education to the University of Chicago (Fuchs and Havighurst, 1972). The research was carried out by eight universities from 1967 through 1971. It is not surprising that Robert Havighurst was the “Principal Investigator.” He was assisted by a co-Principal Investigator, Estelle Fuchs, a cultural anthropologist, from Bank Street College. The committee which developed the content of the research made a special recommendation that no new testing be undertaken and none was undertaken. An important departure of the research pertained to American Indians included in the study. They did use the BIA blood quantum definition and therefore included self-reported Indians.

Figure 4 Fuchs and Havighurst Test Data

Fuchs and Havighurst did discuss achievement as reported in extant evaluations (Ibid. Chap. V). The Coombs evaluation was discussed as the most extensive ever undertaken. The Bass evaluation report was not available for the NSIE, but the Coleman (1966) study was. The NSIE compared several testing reports and concluded that the data, “. . . place Indian pupils just below the national average during the first four school years, and then they drop substantially,” (p. 123) There was an exception to this pattern which was reported by John Bryde (1970).

Bryde reported that Sioux children made at or above grade level on standardized tests up and until grade six. After grade six they fell below gradually until they were at least two years behind in grade twelve. Bryde, and others, (Coombs) called this phenomenon the, “Cross-over Effect.” Because the cross-over effect was very different from previous evaluation data, Fuchs and Havighurst went to the Bryde original data for analysis. Bryde had test scores on Sioux children in grades 7, 6, 5 and 4. He had 147 eighth grade children but only 65 of them were in the database in grade four. Fuchs and Havighurst thought the data reduced confidence in the quality the findings and the cross-over effect phenomenon. They thought Bryde would have been better off to base his study on the 65 children who were in his database from grade four through grade eight. The cross-over effect was, therefore, seriously questioned. Coombs also reported the cross-over effect happening at grade four. It should be noted that the cross-over effect is based on test norms “Grade Equivalent” scores, not raw scores.

Fuchs and Havighurst illustrated the major testing studies as presented in Figure 4. Of the five evaluations included in Figure 4, only the Rock Point and Rough Rock data is definitely based on an English as a second language Navajo school population.

Fuchs, a cultural anthropologist from Bank Street College, obviously, included a discussion of “Indian culture,” that she suggests has over time evolved and has the following characteristics: “. . . close family solidarity and cooperation with mutual support among the kin-folk, cooperation rather than competition among members of a given age group, belief in the values of a tribal tradition, belief in tribal religion, and a tribal language.” (p. 129) Educators undertaking U.S. national evaluations of American Indian education could profitably employ this definition for an “Indian” culture. All other research recognizes that culture is tribal based and different one from the other.

Cross-Over Effect. The cross-over effect wherein standardized test scores based on American Indian students score higher and in some studies, above grade norms, and then begin a gradual reduction in normed test scores until at grade twelve they are two levels below norms, has been much discussed. Regarding achievement tests, Indian children do produce higher test scores below grade four, but not always at test score grade level. Hopkins (2002, Op.cit.) has performed correlation’s on two Navajo schools which includes test scores and teacher grades. He found no correlation’s between test scores and teacher grades below grade four but strong correlation’s starting at grade five and continuing through high school. He suggests that primary teachers (he has been a primary level classroom teacher) are more interested in child development than academic achievement. Perhaps, also, the lack of focus on academic achievement at the primary level is a good approach to academic achievement.

BIA, 1970 – 1979

BIA Evaluations. From 1964 through 1979 the author was intimately involved with the BIA education evaluation function at the Washington level of the Education Program. First I was an “Education Specialist, Secondary,” and served as a team member at boarding school evaluation. Starting in 1968 I was Chief of the Division of Evaluation and Development and starting in 1971 Chief of the Division of Evaluation, Research and Development. Starting in 1958 as an administrator and continuing through to the time until he retired, and afterwards, I worked for and with Madison Coombs. I was thoroughly familiar with the use of standardized test to evaluate the BIA’s Education Program and saw no need to use them during the 1970’s. My approach to evaluation was to use a program review format, which could or could not include standardized test scores. Evaluations conducted were published as they were completed. Most of all, evaluations were to be professionally conducted with fairness and openness to all. Cultural fairness seemed most important evaluation characteristics of all.

BIA Publications. The BIA created two important publications that made available evaluations of all types. One was the “Research and Evaluation Report Series,” and the other was the “BIA Education Research Bulletin.” Both of these publications made available on a regular basis a large number of evaluation and research activities. It is important to note that Deyhle and Swisher (1979) did not include any of these publications in their review of educational research on American Indians. Dr. William J. Benham (Creek) was head of the BIA Indian Education Resource Center (IERC) that created the publications and Dr. Eugene Leitka (Choctaw) was editor of the Research Bulletin. Several of the contributors were American Indians.

The Research Bulletin was published quarterly from 1972 through 1979. A sample of the articles reflects concern for the evaluation of effectiveness of most BIA Education programs. For example, Bruce Pray (1975) addressed the use of standardized tests with Indian high school students; Annabelle Scoon-Rosenbluth (1975) researched the feasibility of test translation from English to Navajo and Ethel Capps reported on her masters thesis on reading levels for Navajo high school graduates (1977). Capps, an experienced reading teacher, expressed the belief that timed tests were a determent to test scores and if Navajos had more time they would make higher scores. More importantly, Navajos didn’t need high school grade level test scores in order to do well in school. The Bass evaluation was reported, too. James C. Martin, an American Indian with a Ph.D., contributed important articles on the Indian self identity and was co-principal investigator of the Oklahoma Indian Education Needs Assessment, which is the only quality statewide needs assessment ever conducted (1977; 1979 and USBIA No. 43). In summary, the total of 89 articles printed in the BIA Education Research Bulletin provide an excellent summary of what the agency thought important regarding evaluation and research of American Indian education during the 1970’s.

The “Research and Evaluation Report Series,” also reflected what the BIA thought important in evaluating American Indian education. For example, Report No. 03 is titled, “Rough Rock School Evaluation.” (USBIA, 1979, No. 01)³ Rough Rock was the first Indian controlled school and the IERC was requested by the school principal to conduct the evaluation. Then, there was the “Oklahoma Indian Education Needs Assessment, “Results of the Test of Proficiency in English as a Second Language in Grades 4, 5, and 6 of BIA Schools, School year 1971-72, “(USBIA, 1979, No. 13) and “Evaluation Report of the Indian Education Administrator Training Program At the Universities of Harvard, Penn State and Minnesota, Descriptive Analysis, 1975.” Almost every BIA Education program initiative is covered in the Series. There were nine individual schools evaluated.

The two publications also reflect an increasing BIA education evaluation and research sophistication. Based on contemporary discussions in the American Education Research Association’s (AERA) Educational Researcher (Anderson 2002; Haywood Metz and Page, 2002) discussions of “Practitioner Research,” most of the BIA publications would fall within this definition. This is not to say that BIA professional educators anticipated the 2002 AERA discussions for such was clearly not in the works in the 1970’s. What it means is that most all the doctoral level BIA employees earned their degrees while working closely and in some capacity, within and outside of BIA, in American Indian education. The BIA Education Research Bulletin provided immediate and welcoming publication of the research. ⁴

Figure 5 Standardized Test Scores and Teacher Grades

While there were no national BIA or other evaluations of American Indian education in the 1970’s in the style of the four previously discussed, there were national evaluations of the BIA Higher Education (USBIA, 1979, No. 20) program and an extensive survey of 17 off-reservation boarding schools (Sahmaunt, 1979). This evaluation was initiated by the Deputy Assistant Secretary for Indian Affairs who wanted to close the schools. School Administrators thought differently and believed that describing the characteristics of the students would justify not only keeping the schools open but increasing their budgets to serve the extraordinary needs of the students. A data form was developed, professional educators were sent to the schools, and data was taken from the students “Permanent Folders.” The evaluation had over 7,000 students in it. The data was massive in scope and included standardized test data and an estimated “Academic Achievement” based on teacher grades. The test data was not of good quality but achievement based on teacher grades was and reflected something close to a normal distribution. Figure 5 illustrates the historical standardized test scores and teacher grades. It is interesting that both standardized test scores based on test norms and teacher grades will reflect this pattern Nationally, regardless of the tribal culture of the students. Given the bias of normed tests, it is suggested that teacher grades might be used in juxtaposition to test norms.

Policy, Indian Self-Determination Act.

The Indian Self-Determination policy was initiated by President Johnson and formalized by President Nixon. Basically, the policy and statute provide authority for an Indian tribal government receiving services from the BIA to assume control of a program and administer it to tribal members. The policy clearly provides legal authority for tribes to assume control of and administer a BIA program. Pertaining to education, it is reflected in tribal control of BIA funded schools, the first of which was Rough Rock Demonstration School located on the Navajo Nation (Johnson, 1968; Op.cit. USBIA, 1979, No. 01). Currently, the BIA reports that 120, 65 percent, of its total of 185 funded schools, are under Indian control (USBIA, 2002).

Bilingual Education

Bilingual education which conducted instruction in two languages, Indian and English, was initiated in BIA funded schools. The first was Rough Rock, a Navajo schools. At about the same time, 1970, two language schooling was also initiated by the BIA in Alaska with the Ypik dialect of Eskimo. (Op. Cit. Hopkins, 2002). Evaluating American Indian bilingual education presented interesting evaluation challenges. At the time of implementing them, there was an anti-bilingual position which expressed the belief that teaching in English, only, was the most effective, and patriotic approach to educating any one, including American Indians. Unfortunately, this approach is still with us in the 21st Century and has many adherents among Indian populations.

In almost all programs, evaluations of American Indian bilingual education used standardized achievement tests in some manner. Most often, standardized tests were used to prove that bilingual education was as effective as English, only (Rosier and Holm, 1980; Rosier and Ferella, 1976). The Navajo Rock Point School also used the Test of Proficiency in English as a Second Language (TOPESL) which was developed by Eugene Briere at the University of Southern California under contract from the BIA ( Briere, 1969, 1970; Op.cit. USBIA 1979, No. 13). The Window Rock (Navajo) public school district in Arizona, developed and made available “The Window Rock Oral Language Test (WROL, 1981) which was for the primary grades, but it has been used at upper levels, too. The WROL has been used extensively with Navajo bilingual education programs to assess Navajo language education at the primary level.

The 1980’s

The 1980’s were very active regarding the evaluation of Indian education. However, the reports will not published for public or government consumption. The House Appropriations Committee was concerned about the quality of academic education provided by “Contract School,” which were referred to as ‘Indian Controlled Schools.” These schools had been Federal under the BIA and with the passage of P.L. 93-638 the BIA was provided authority to contract its programs or parts of programs to Indian tribes so they could administer them. Also, the House Education and Labor Committee had created a sub-committee which was very active in developing legislation regarding Indian education. Reports pertaining to the various Congressional committees can be found in memos and committee reports.

Navajo Indian Controlled Schools Position Paper (1987). This position paper was never published yet it contained excellent documentation of the pressures the House Appropriations Committee was putting on the Indian controlled schools. This Committee was using achievement tests to measure the education programs of the schools. This same design is being used today, 2002, in the three states (Arizona, New Mexico and Utah) which have education reform achievement test assessment reform programs. An interesting document attached to the Navajo Position Paper narrative were memos which established an official “Achievement Test Policy.” A memo by Dr. Kenneth Ross clearly references congressional committees as the motivation for the BIA Achievement Test policy. The culmination of this activity was a January 23, 1987 memo from the “Deputy to the Assistant Secretary/Director – Indian Affairs (Indian Education Programs.) The first paragraph of the memo stated:

All Bureau of Indian Affairs funded schools, including contract schools, shall administer either the CTBS-U/V or the CAT-E/F. The Agency Superintendents for Education/Area Education Programs Administrators will schedule and direct this for the schools under their jurisdiction.

This is a long and detailed memo which assigns the Office of Indian Education Programs (OIEP) responsibility for “collection, aggregation, and dissemination of Bureau-wide test information.” After researching to locate documentation related to this task it is highly probable that it was never performed by the OIEP.

At that time, 1987, the BIA Navajo Eastern Agency implemented a program to respond to the House Appropriations Committee’s pressure. Curiously enough, the Eastern Navajo Agency assessment program, which cast achievement tests as its foundation, was presented as a total BIA Education assessment program, which it was not. The Eastern Navajo Agency used achievement tests (CTBS) to retain students; evaluate teacher quality and assess their academic program. In response to this program, Navajo contract schools hired Jim C. Fortune (1987) to evaluate the Eastern Navajo Agency assessment program. Fortune established a control group from a North Carolina public schools that had 12 percent Indians in its membership. Fortune used a research design and was very careful about data quality and statistical procedures. Fortune used “t” tests of significant differences, means, standard deviations and a two way ANOVA. He used the normal curve equivalent (NCE) rather than the test grade equivalent because the NCS is more stable. His “Purpose of Study” could have been instructive to the Congress and the BIA. It described how most achievement tests are constructed:

Several federal programs such as Chapter I require the use of school achievement data in the evaluation of local projects. Such data are also used in monitoring school progress and in the comparison of schools governed by differing auspices. Due to the nature of the achievement test forming process, the sole use of test scores for student placement decisions may result in the appearance of gains in school progress that are merely artifacts of test interpretation. Because most developed through the sampling of global knowledge domains so as to accommodate a wide range of school curriculum and are leveled for multiple grades, norm statistics rather than content differ for most sequential grades. For instance, if a fifth grade student makes a scale score for the total battery of 696 in a spring administration of the CTBS, the student receives a normal curve equivalent score (NCE) of 50 and a grade equivalent score (GES) of 5.9. If the student is in the fourth grade, however, with the same scale score of 696 on the same test; the students NCE is 61 and his GES is 6.0. Notice that without any change in the amount of knowledge displayed there is a large change in NCE and a minor change in GES. This discrepancy of NCE scores represents a caution that must be taken in school evaluations. (p.2)

Fortune found that about all of Eastern Navajos claims, i.e. that retaining students was a motivator for increased achievement, were fallacious. The Navajo Position paper also discovered that Eastern Navajo’s outstanding teacher handed out to students actual pages of the tests and taught the children correct answers just prior to administration of the CTBS. It is interesting that neither the Congress or the BIA bothered to investigate their rich historical heritage of the use of standardized tests and their basic cultural unfairness. Ignorance ruled the day, as it does today.

Evaluating Indian Controlled Schools. It was mentioned above that the House Appropriations Committee was concerned about the quality of Indian controlled schools’ education. The Committee financed an education evaluation contract to Abt Associated, Inc. (1985) to assess the quality of the education program in Indian controlled schools. The evaluation design was elaborate and considerable effort was made to be fair to the schools. For purposes of comparisons, BIA and public schools were included. The evaluation included grades eight and twelve. The total database included School Characteristics, Student Background Characteristics, Achievement Tests, Attendance, Costs and Department of Education Title IV Enrichment Projects. At the eighth grade level the evaluation included 20 Indian controlled schools, two BIA schools and 10 public schools for a total of 581 students. At grade twelve they included 26 Indian controlled schools, 3 BIA Schools and seven public schools and 455 students. Several statistical procedures were applied to the data. The report summary stated:

In this report, we have provided comparative data between Indian controlled schools and other schools in the same regions serving Indian students with backgrounds that were as similar as possible, in order to provide a fair comparison. Objectively, then, we can say that Indian controlled schools are higher in cost than other schools, on average; that eighth grade students in Indian controlled schools are not achieving as well as students in the comparison schools, on average; that twelfth grade students in Indian controlled schools are achieving better than twelfth graders in BIA schools but not as well as twelfth graders in Public schools, on average; the attendance levels are slightly lower for students in Indian controlled schools than for students in other schools, on average; and that Indian controlled schools exhibit higher rates of withdrawal from twelfth grade than other schools, on average. Whether these results indicate serious problems in the Indian controlled schools is open to interpretation. (p. 114)

The elaborate design notwithstanding, the evaluation did not prove the inferiority of Indian controlled schools.

In all respects, it was a fair attempt to respond to the needs of the house Appropriations Committee. Here, again, we find a concern for the education quality differences between school types. The design matched student types, similar to the design of the Bass evaluation, and concluded with only some differences. When students are matched according to background characteristics, few differences can be expected between school types.

The same as with the National Study of American Indian Education conducted by Havighurst and Fuchs, this evaluation was also important because the contract was managed by the U.S. Department of Education, Office of Indian Education, not the BIA. It should be noted from above that one of the stipulations was to provide evaluative information pertaining to Title IV, Enrichment Projects, which were funded by the Office of Indian Education. This is a somewhat unusual situation because a Department other than Interior was funded to evaluate an Interior Department program. This is not often done in U.S. Government Education activities.

Gifted Attitude Inventory for Navajos. Up to 1980 there had been very few instruments that tried to provide an education dimension which was based on an Indian culture. The Din? Bi?Olta? Research Institute, a Navajo organization headed by Dillon Platero, obtained a grant from the U.S. Department of Education, Office of Indian Education to develop the “Gifted Attitudes inventory for Navajos (GAIN) (Abbott, about 1982). GAIN was piloted on 101 Navajo high school students and statistics were developed to reflect this exercise.. GAIN was designed for “adolescents in grades nine through twelve, or between the ages of approximately thirteen through nineteen.” The inventory consisted of 66 items for which there were no right or wrong answers. Each school was encouraged to develop their own statistical profile. GAIN was used at the Navajo Academy in 1986 (Platero, 1986) to help identify gifted and talented students. The report is critical of GAIN as it seemed to over emphasize moral turpitude. The statistical output did approach a normal distribution starting at the 21st percentile and going upward. GAIN has eight sub-categories:

1. Planning

2. Self-Reliance
3. Self Determination
4. Respectfulness
5. Ethics
6. Presentment
7. Attentiveness
8. Motivation

The higher the GAIN raw score, the higher the student reflects Navajo intelligence and culture. GAIN could probably be used today as one of the background characteristics to assist in matching Navajo students and in weighting standardized test scores. It would be interesting to run correlation’s and regressions which included GAIN scores.

While working at the Navajo Academy, Hopkins (1986) administered GAIN to 170 Navajo Students in the spring of 1986. He created a comprehensive database which included all the GAIN data as well as CTBS scores in Reading, Language Arts, Math and the Ottis-Lennon School Abilities I.Q. test. He ran correlation’s on the CTBS, Lennon and GAIN data and found a weak relationship between the standardized instruments and GAIN, His correlation matrix follows.

Correlation Matrix

READ = CTBS Reading sub-test, administered in fall of 1985

ENG = CTBS Language Arts sub-test, administered in fall of 1985

MATH = CTBS Math sub-test, administered in fall of 1985

OTIS = Otis-Lennon Scholastic Ability IQ test

GAINAVG = GAIN mean of percentage scores on each section

PERCTLE = GAIN percentile score for an individual

GAIN was administered April 11, 1986 to 186 students at Navajo Academy. 170 were determined useful for analysis and reporting. The database contains all the GAIN scores, including raw scores.

Figure 6 McGraw Hill 1985 Report

BIA Standardized Testing Program, 1980-1987. Though not published, the 1980’s were busy. The BIA contracted with McGraw Hill to test some of its students. McGraw Hill provided BIA a report each year. The 1985 report seemed to be available more than other years (USBIA, 1985; Hopkins, 1987). Only an unspecified sample of students were tested in the spring of the school year. 15,097 took the CTBS and 5,539 took the CAT for a total of 20,636, which would be less than half the BIA total number of students. It can only be assumed that students were not tracked and cohorts were used for statistical purposes. When test norms were used, the same pattern established by the University of Chicago is repeated i.e. growth is good in lower grades and falls behind until grade 12 Indian students are two test grade levels behind. The report was prepared by CTBS McGraw Hill. They issued a word of caution about the CTBS being harder than the California Achievement Test (CAT). It seems that the scores were going up on the CTBS so they made it harder. This means that the bell curve dominated testing rather than student learning. The CAT, so they said, was easier than CTBS as it had not been changed. This report clearly reflects that the priority of the education psychologists at McGraw Hill placed a higher priority on statistical output than on student learning. When combined with an earlier McGraw Hill specialist who recommended that under no circumstances should the time to take a sub-test be changed it is clear that standardized tests and their use of the years with Indian students are fundamentally flawed.

The 1990’s

Deyhle and Swisher (Op.cit. 1997) made the Self-Determination a major theme in their review of research literature. Their understanding of the education situation was that schools taught assimilation to Indian children. There is no doubt that individual non-Indian teachers have, contrary to program policy, have taught in such a manner to reflect assimilation. There is also little doubt that most textbooks used in Indian classrooms were and are Eurocentric based. Also, with the vast majority of American Indians now being educated in public schools, Indian Self-Determination does not apply. Public schools, alone, may represent a tacit program of assimilation through education. Interestingly enough, the 120 BIA funded contract and grant schools may represent the only non-assimilation schools in the U.S. and they are but a very small proportion of the total. Regarding assimilation as an education policy, officially, it has not been a BIA policy since the 1930’s. Nor has it been the policy of the U.S. Department of Education’s Office of Indian Education. Nonetheless, with few Indian teachers, it may be a major purpose of teachers.

Indian Professional Education Researchers. There are now hundreds, perhaps thousands of professional educators who are Indian. Compared to 1940 when the entire Peterson evaluation took place without a single Indian in a decision making position, there has been a sea change in the education of Indians. We cited above an evaluation report reflecting the BIA’s paying for the higher education of educational administrators, which was in 1975. Many of the individuals who were students in these programs eventually occupied leadership positions in government, public education and higher education. It is important that all administrative positions in the BIA’s Office of Indian Education Programs are Indian.

Indian Teachers. Starting in 1992, the Navajo Nation initiated a Teacher Education Program (Hopkins, 1998). It started as a part of the Ford Foundation Minority Teacher Education Program but is now entirely supported by the Navajo Nation. Since 1992, the program has produced over 600 elementary, Navajo, certified teachers. Gorman (2000) reported that over half the elementary teachers in Navajo Nation schools are now Navajo. Also, there are now a few schools, BIA funded, only, with a total Navajo staff, including administrators, teachers and ancillary staff. The Teacher Education program is still in effect and is gradually producing new certified teachers.

Contemporary BIA Education Evaluation. The BIA has implemented an education evaluation program for its 185 funded schools. Because of the complexity of the local situations, no single standardized test is used. Indeed, no test is mandated. Schools are required to set targets in the curriculum areas of math and language arts. In all areas included in the evaluation design, assessments are divided into three proficiency categories: (1) Advanced, (2) Partially Proficient. (USBIA, 2002) All reports are school generated with no outside evaluation assistance. There are individual school “Report Cards,” which are posted on the BIA Office of Indian Education Programs web site. It is important to observe the non-test items included in the BIA assessment program:

EXECUTIVE SUMMARY

The Office of Indian Education Programs (OIEP) functions as a State Education Agency (SEA) and thus required to issue a report for each of its schools summarizing records of performance with the implementation of a comprehensive school reform program; satisfying our customers; and, more importantly, improving student achievement. The following information is based on the data reported in the individual school report cards for SY 2000-01.

 

During the 2000-2001 school year, the schools reported that:

 

The Bureau-funded school consisted of 185 schools/dormitories; .with 65 schools or 35% operated by the Bureau and 120 schools or 65% operated by the Tribes through a PL 93-638 contract and/or PL 100-297 Grant, in 23 states.

• The Student enrollments were 48,693, a decrease of 383 students from SY 99-00.
• Ninety-six percent of the schools were accredited through the state and/or regional accreditation association.
• The average daily attendance rate was 90%.
• The yearly enrollment retention rate was 94%.
• The yearly dropout rate was 11%, for an increase of 1% from SY 99-00.
• The student math achievement levels were at 50% Partially Proficient, 44% Proficient and 6% Advanced, for a consistent result in student math achievement at the proficiency and advanced levels.
• The student language arts achievement levels were at 50% Partially Proficient, 45% Proficient and 5% Advanced, for a 2% increase in student language arts achievement levels at the Proficient and Advanced levels.
• The proficiency levels of staff in their use of technology was 26% Partially Proficient, 49% Proficient and 25% Advanced, for a 7% increase in the Proficient and Advanced levels.
• The staff proficiency levels in use of new assessments were 31% Partially Proficient, 46% Proficient and 23% Advanced, an increase of 1% in the Proficient and Advanced levels.
• There were 3,496 substance abuse incidents for a decrease of 1,035 of substance abuse incidents.
• There were 8,471 violence incidents for a decrease of 2,235 incidents in violence.

The current BIA education assessment program reflects a 30 year policy of down playing the use of standardized tests and their norms for evaluation purposes. This may be changing with the national state public schools accountability programs and with the national No Child Left Behind Act (NCLBA). Interestingly enough, the Department of Education’s OIE does not have an evaluation program.

Navajo Nation Rural Systemic Initiative. This program is funded by the National Science Foundation (NSF) to the Navajo Nation. It started in October of 1998 and will conclude in March of 2004. NSF requires an outside evaluation. Patrick Galvin of the University of Utah is the Principal Investigator of the evaluation. Galvin, et. al (2001) using Arizona, Utah and New Mexico test data, provides analyses that are related to control characteristics such as socioeconomic status (SES) and Navajo culture. The focus is on math and science test scores. NSF will accept qualitative evaluations but prefers quantified evaluations. State data is very important to Navajo education as public schools provide approximately 75 percent of schooling for Navajos. All three states have high stakes testing assessment-accountability programs. None of the states identify American Indian students by their culture name, which complicates separating Navajo data from the others. Galvin provides two basic questions which the evaluation attempts to answer:

1. What is the math/science test score achievement gap between Native American and Anglo students?
2. What evidence is there that the test scores of students attending schools supported by NN-RSI programs increase at a faster rate overtime than a comparable group of students not supported by these initiatives?

In the Executive Summary to the report, Galvin states:

The content of this evaluation report describes in detail the math/science achievement gap of public school Native American children. The average math or science scores for Native American students is about 20 points below the averages for Anglo students. However, after controlling for English proficiency and socio-economic status the achievement gap is not large (5 or 6 points). These differences are still statistically significant and need to be explained.

The evidence of achievement growth in math and science overtime is not easily summarized. Generally, over the last three or four years (depending on the data set), the growth of math achievement for Anglo students has outpaced that for Native American students. However, in schools and school districts supporting the initiatives promoted by the NN-RSI the achievement growth of Native American students has outstripped that of their Anglo counterparts. One way to summarize these points is to say that the evidence supports claims that Native American students can do well on standardized achievement test that curricular and instructional support these students are making consist and significant achievement gains overtime.

This evaluation uses extant data and does not gather test data for the stated project purposes. Galvin uses background characteristics (language and socio-economic) in analyses. This evaluation practice, we have learned, always produces data different from that which does not control background characteristics, i.e. the BIA testing program of the 1980’s. There is always more confidence in the evaluation when background characteristics are included in test data analysis. Overall, NN-RSI has generated over 10 evaluation reports, but Galvin’s addresses the achievement gap in such a manner as to relate to contemporary Navajo (American Indian) education.

Navajo Culture and Attitude Instrument. A second Navajo culture instrument was developed by Kalvin White (1998) for his dissertation at the University of Utah. This instrument has well developed statistics and White is a Navajo. Perhaps White’s instrument and GAIN will in time help provide an important Navajo culture dimension to Navajo education evaluation.

No Child Left Behind Act. The NCLBA has the potential for reversing policy regarding Indian Self-Determination in education. The BIA is considered the 51st state and has developed a state plan for implementing NCLBA (USBIA-NCLBA, 2002). The state plan discusses the complexity of following 23 different state assessment plans as well as others developed by tribal accrediting offices. It states:

Before the Bureau’s new assessment system is fully developed and implemented in 2004-2005, Bureau-funded schools will be required to continue to utilize the norm-referenced tests of their states until their states’ new criterion-referenced assessment systems are fully approved by the Department of Education. At that time, BIA-funded schools will utilize those approved state systems in their entirety to determine proficiency levels of their students. (p. 7)

The BIA Mission statement is worth noting, too.

The mission of the Bureau of Indian Affairs, Office of Indian Education Programs is to provide quality education opportunities from early childhood through life in accordance with the tribes' needs for cultural and economic well-being and in keeping with the wide diversity of Indian tribes and Alaska Native villages as distinct cultural and governmental entities. The Bureau shall manifest consideration of the whole person, taking into account the spiritual, mental, physical and cultural aspects of the person within family and tribal or Alaska Native village contexts.

The Mission Statement does not mention Indian Self Determination. Tribal governments are mentioned as “Entities,” which implies they are something other than legal governments. Without some clear defense of Self Determination, which is supported by