HOW WELL ARE INDIAN
CHILDREN EDUCATED?

APPENDIX C

ADMINISTRATION OF THE
SERVICE-WIDE TESTING PROGRAM

During the winter of 1944, the trial tests were administered in the seventh and ninth grades of the following schools:

1. Albuquerque Indian School-Southwest Area

2. Carson Boarding School-Pacific Area

3. Fort Sill Boarding School-Oklahoma Area

4. Little Wound Day School-Dakota Area

5. Oraibi Day School-Southwest Area

6. Phoenix Indian School-Southwest Area

7. Oglala Community School-Dakota Area

8. Porcupine Day School-Dakota Area

9. Riverside Boarding School-Oklahoma Area

In addition to the battery of examinations administered, background and personal information was .obtained for these groups.

For the reasons stated in Chapter II of this monograph, the eighth grade was selected as the level at which the tests would be administered in 1945. The examinations were given in all of the Indian schools in which there were eighth grade students. Rural public schools and mission schools cooperated in the program in order that comparisons between the Indian students and other students with rural backgrounds might be made.

For the most part, the tests were administered in the schools. If necessary, the individual giving the test visited the school for two or three days to administer the battery. In a few cases the students from very small or isoated schools were brought to centrally located boarding schools for examination. In 1945, the tests were administered by personnel who had had previous experience in test and measurement work. In no instance was the testing in a school done by one of the teachers or administrators of that school. Those people who had worked directly or indirectly with the preparation of the instruments acted as instructors in the sense that they helped to familiarize many in the field who were selected to administer the tests, with directions and objectives. The tests were given in the spring of 1945.

The Rural Practices tests prepared specifically for this program by the Indian Service staff and University of Chicago consultants, were printed at Haskell Institute, Lawrence, Kansas. The standardized examinations were shipped to Haskell where all the examinations were packaged and distributed to the areas. After administration, all of the examinations and data sheets were returned to the Chicago office for correction, tabulation and statistical analysis.

In 1946, when it was decided to extend the program to include all of the fourth and twelfth grades as well as all of the eighth grades, the problem of administration of the tests was increased to such an extent that it became necessary to use more persons as test administrators. In order that all of those participating in the program should be as familiar as possible, not only with the precise instructions for administration but with the purposes of the program and the attitude of the students which would yield the best results, the instructions were carefully detailed. The following is a copy of the "Test Manual" furnished to each person participating in the administration. This was assembled as three separate manuals in order that each examiner would have in front of him directions for each test in the order in which they were to be given.

MANUAL OF INSTRUCTIONS
FOR
TEST ADMINISTRATION

The test administrator must be familiar with the purpose of this testing program as well as with all of the tests and the way in which they must be administered. In some instances the classroom teacher herself may be the administrator, but whether or not this is the case, the classroom teacher also should understand the purposes of this testing program and the manner in which the tests must be administered.

Fourth grade students have been included in the study this year for the reason that this is the grade where we encounter the largest group of Indian children. The twelfth grade is also included in order that the study may reflect growth during the two four-year periods. We are testing eighth graders this year only to fill out the portions of the picture obtained last year which are still a little vague. This is not an annual all-pupil test such as some state departments have inaugurated, but an attempt to find out through a testing program some of the things which appear to be happening in Indian schools, so that needed changes in our approach may be made or so that we can continue in the knowledge that we are getting the kind of results we want.

We wish to emphasize that the items included in the various tests do not constitute a list of facts or skills that should be mastered by all students in Indian schools. They do not in any sense constitute an approved course of study. The range of the tests included in this study is wide in order that they may be used at the various grade levels and in different types of schools. Pupil scores will, in no case, be considered in the rating of a teacher's efficiency.

The March 1, 1946, issue of INDIAN EDUCATION carried a brief account of the 1945 Eighth Grade Testing Program and a study of this article should help to explain the purpose of this year's program in which the testing has been extended to include all fourth and twelfth grade students.

THIS MANUAL MUST BE CAREFULLY STUDIED PRIOR TO THE ADMINISTRATION PERIODS AND IT SHOULD BE KEPT AT HAND FOR EASY REFERENCE.

GENERAL INSTRUCTIONS

Both the classroom teachers and test administrators should feel responsible for securing the utmost cooperation from the pupils. Both the administrator and the teacher should employ the most friendly methods in order to cause the pupils to feel at ease and make them want to do as well as possible on all the tests. At the same time, pupils should be assured that test scores will not be used to determine promotions or school marks. It should be pointed, out that no pupil is expected to make a perfect score on any test since they are prepared for use in any of the grades.

The teacher has a special responsibility to see that the

1) Record of Attendance

2) Background Information are filled out accurately. The importance of this material can be seen by studying the 1945 Eighth Grade Study report.

RECORD OF ATTENDANCE

The teacher herself must take the responsibility for filling out these data. Obviously, the students' recollection of facts would not be accurate enough and the teacher will have to consult the school attendance record.

BACKGROUND INFORMATION

The teacher of fourth graders will probably want to fill in this information herself. In order for this information to be most accurate, it is impossible to give this questionnaire to 4th grade students in the same way that one would give them an arithmetic test. The teacher may want to work with each student independently or she may wish to confer with two or three students at the same time and gather the required information from all of them. In the 8th and 12th grades, the teacher may be able to let the students actually fill in most of the information themselves but it must be very carefully supervised and of course the teacher must personally check each response to be sure it is correct.

The background information sheets may be filled out by the teacher and her students either before or after the examination period, in order that the time of the examiner shall not be consumed. These do not necessarily have to be completed on the day. for which the tests are scheduled.

THE TESTS

These tests include some that have already been standardized on a national basis and others that have been prepared specifically for this testing program. It is very important that the test administrator be familiar with each of the tests that he is to give. In order to become sufficiently acquainted with the test directions and the contents of the test itself, it is important that each test administrator go over each test and its directions very carefully. It is suggested that the test administrator actually administer the tests to himself or to some other person in order to become fully aware of the problems and questions that may arise. The printed directions for administering each test should be followed precisely. However, the administrator should, by oral direction, encourage pupils to try to answer all of the items they can but he should warn against mere random guessing.

The test administrator should NOT let the pupils move around the room. He should insist that they all remain in their seats and as he moves quietly around to see that everything is going well, he can collect the papers from those pupils who have finished. The test administrator should NOT give the impression by his movement around the classroom that he is attempting to hurry them. Instead, his supervision should be an indication that he is interested in their work and anxious that they understand what is required of them. He should give them all the help they need in understanding the directions but must NOT answer any questions on the meaning of test items.

The test administrator may find that some pupils finish the test so much sooner than others that this factor may create a problem unless it is anticipated. In the case of the shorter tests, the pupils should be instructed to sit quietly in their seats until all papers have been collected. When longer tests are being given, the test administrator should confer with the classroom teacher as to the best device for bridging the interval.

The test administrator should pass out all of the tests himself or have the classroom teacher assist him. If the teacher herself is the administrator, she will probably want another teacher in the room to assist her. Before a single test is handed out, up-side down, the pupils should be told that they are not to open the tests or turn them over until they are so directed. When the tests have all been handed out, the test administrator should tell the pupils the name of the. test they are to take in order to be sure that no one has received the wrong test. The administrator should then follow the directions for administering the tests as they are given in the SPECIFIC INSTRUCTIONS. These instructions must be followed very carefully and attention should be given to the administration time as indicated. When time limits are specified, the administrator should be sure to time the test from the moment that the student starts work on the first item.

When the pupils are instructed to fill in the data sheet of each test, it is important the test administrator move quietly around the room to see that everyone is writing his name and following directions accurately. Again when the administrator collects the tests, he should look at each paper to be sure that it has the pupil's name written on it and that it is filled in correctly.

It is impossible to overemphasize the importance of following all test directions to the letter. Unless these tests are given in the same way every place, the results will mean very little. Only if every test administrator makes himself familiar with the tests he is to give, with these GENERAL INSTRUCTIONS and also with the SPECIFIC TEST INSTRUCTIONS, will the results of this test program be reliable and significant.

SUGGESTED ADMINISTRATIVE SCHEDULE
FOR
GRADE IV

NOTE: The working time does not include the time required for filling out the name, etc., giving directions or explanation of samples.

SPECIFIC INSTRUCTIONS
BACKGROUND INFORMATION SHEET

(This sheet has already been mentioned in the manual on page 2.)

1) While the pupil himself may actually fill this sheet in, yet it must be done with the closest supervision of the teacher. It is so important that this information be filled in correctly, that it would be impossible to expect the student to supply this information without a great deal of explanation of each item. Some of the data may have to be supplied or verified by the teacher by consulting school or agency records. This background sheet contains no test item so it is entirely possible for the teacher herself to fill in all of the required information. When the forms are completed, the classroom teacher should sign each one at the bottom of the second page and turn them over to the test administrator to be mailed to the Chicago Office.

2) All Background Information Sheets, as well as all tests and other data papers should be stamped in the upper right-hand corner of the front page with the Agency or School stamp, to facilitate handling, identification and grouping.

3) Every pupil taking any of these tests should fill out a Background Information sheet.

4) A White pupil will not answer No. 8, but will answer No. 9 by circling the "O." All other questions should be answered by all pupils.

5) In No. 3(b) the month, date and year of the child's birth should be given.

6) In No. 10(a) the names of the father and mother should be given, even though one or both may not be living. In case one is dead, this should be noted on line 10(b). If the pupil is not living with either parent, he should fill in names of father and mother, and also fill in the last column indicating name and relationship of the person with whom he makes his home.

7) All 17 spaces under "Mother" and "Father" should be filled in, even though one or both may not be living. If the exact information is not available, give approximate information, writing "app." underneath. Even under (k) a deceased father's last kind of work should be given, such as "farmer," "section man," "laborer." If the mother is not employed the term "housekeeper" may be used to indicate she takes care of her own home. If the pupil does not live with his parent, then, in addition to full information under "Mother" and "Father," all 12 spaces in the last column should be filled in. The last four lines on this page refer to the pupil's own grandparents. Be sure that he understands that "Mother" and Father" for these four lines mean the mother and father of the pupil's parents, i.e. his grandparents.

8) Be sure that all questions are answered completely. Note that some lines have several blanks to be filled in.

9) In items 16, 17 and 18, try to get answers showing as nearly correct pictures of attendance as possible.

READ OR RECITE THE FOLLOWING TO THE STUDENTS

" Your teacher and I are to give you a number of examinations today. The purpose of these tests is to help us understand what things you students have learned during the last years. Not all of these examinations will test what you have learned in the schoolroom for some of these things you will have learned at home, at work, or even at play.”

" We gave some of these examinations last year to school children all over the country. We found that many of these students know and understand things that we, as their teachers, had not even expected them to know. We also found that there were some things that students didn't understand although we thought we had taught it. These examinations are very helpful because they tell your teachers how well they have taught you.”

" You are not expected to know the answers to all of the questions on any of these tests. We would not even expect all of the twelfth grade students to know all of the answers. But some of them you do know. Always try all of the problems or questions and answer as many as you can.”

" It is always necessary to read the directions on every test carefully so as to understand exactly what we are to do, because some of the tests will be slightly different from others."

SPECIFIC INSTRUCTIONS
GATES ADVANCED PRIMARY READING TEST

Type 1. Word Recognition

1. See that each child has a pencil.

2. Distribute papers.

3. Have children fill in blanks at the top of the page (with your help).

4. SAY: (Examiner may read this directly from the cover page of the test.)

" I want you to look at the first picture, this one up here (holding up your copy and pointing to the picture of the dog). Next to it there are some words. One of the words goes with the picture. You are to draw a ring around that one word that tells about the picture. Put your finger on the word that belongs with the Picture. What is it? (Let one child answer.) That's right, 'dog.' The four words are 'did,' 'egg,' 'dog,' and 'two' (pointing to the words on your own copy and making sure children look up at your copy.) We are going to draw a ring around the word 'dog' because that's the one that tells the most about the picture. Everyone find the word 'dog' on your paper and draw a ring around it. (Check to make sure children have marked the correct word.) Now look at the box right underneath that one. Find the word that goes with the picture. What is it? (Let a child answer.) That's right, 'bed.' The four words are 'be,' 'bed,' 'bag,' 'she.' We are going to draw a ring around the word 'bed' because that's the one that tells us the most about the picture. Everyone find the word 'bed' and draw a ring around it. (Check to make sure that each child has marked the correct word. Continue in the same way for the third and fourth boxes. When you are illustrating with your copy ask the children to look up if need be.)

SAY:

" DO NOT OPEN YOUR BOOKS UNTIL I TELL YOU TO.”

Now I am going to show what we are to do next. Inside the book are some more pictures and words. (Examiner holds up copy of the test showing the inner pages.) You are to do the first one, then the next one below it, etc. (Examiner points down first column, then second, etc., and also demonstrates order on all three pages.) As soon as you have drawn a ring around the one word for one picture, go right ahead and do the next one. Now remember, first you are to look at the picture and make a ring around that word. Make a ring around one word only for each picture. Do you understand? All right. Open your books and BEGIN. Go ahead."

5. Inspect the work of each child; see that each works from top to bottom of columns and that each follows the pages in order. Urge children individually to try the examples in order but do not tell them the answers. Discourage dawdling over difficult problems; tell them to try the next. Watch for children who make rings indiscriminately and tell them to make only one ring for each picture.

6. The signal "STOP" is given at the end of 15 MINUTES. Collect papers immediately.

SPECIFIC INSTRUCTIONS
GATES ADVANCED PRIMARY READING TEST

Type 2. Paragraph Reading

1. See that each child has a pencil.

2. Distribute papers.

3. Have children fill in blanks at the top of the page.

4. SAY: (The examiner may read this directly from the cover page of the test.)

" We are going to see how well you can read. Do you see the stories and pictures on the front page of your booklet? Everyone look at the first story-up here (illustrating with your own copy.) What does it say to do? (Have child answer.) That's right, put an X on the ball. Everyone find the ball and put a cross on it. Be sure you put it right on the ball. (Check to see that they have all marked it correctly.) Now look at the box right under that one. What does this story tell you to do? (Have child answer.) That's right, draw a line around the milk bottle. Everyone find the milk bottle on your paper and draw a line around it. Be sure to put it all around the bottle exactly as the story asks you to. (Check to make sure it is done correctly.) Now look at the first box on the next side-up here (illustrating with your own paper.) What does that story say to do? (Have pupil answer.) That's right, draw a line under the little book. Be sure you find the little book, and be sure you draw the line under it exactly as the story asks you to. (Check to make sure papers are marked correctly.) Now look at the box under that one. What does this story ask you to do? (Have pupil answer.) That's right, draw a line from the pig to the tree. Do it on your paper. Be sure it goes from the pig to the tree exactly as the story asks you to. (Check to make sure it is done correctly.)

SAY:

" DO NOT OPEN YOUR BOOKS UNTIL I TELL YOU TO.”

Now I am going to show you what we are to do next. On the inside of the book are some more pictures and stories. (Examiner holds up a copy of the test showing the inner pages.) You are to do No. 1. (Examiner points to it on his own copy), then go and do No. 2, then do the next one, and the next one, etc. (Examiner points down first column, then second, etc., and also demonstrates order on all three pages.) As soon as you have finished one story, you must go right ahead and do the next one right below it. Now remember, first you are to read the story below the picture; then you are to take your pencil and do EXACTLY WHAT THE STORY TELLS YOU TO DO. Do you understand? All right. Open your books and BEGIN. Go ahead."

5. Inspect the work of each child; see that each works from top to bottom of columns and that each follows the pages in order. Urge the children individually to try the examples in order, but do not tell them the answers. Discourage dawdling over difficult problems; tell them to try the next.

6. The signal "STOP" is given at the end of 25 MINUTES. Collect papers immediately.

SPECIFIC INSTRUCTIONS
ARITHMETIC COMPUTATION TEST

1) Each pupil should have at least two sharpened pencils. Only pencil is to be used. The teacher should have an extra supply of sharpened pencils on hand in case of need.

2) SAY:

" We are going to have a test in arithmetic. I will give you each a copy of the test. Do not look in it or write anything on it until I tell you to." (Give each pupil a copy of the test, as well as two sheets of blank paper, and tell the pupils to fill in the blanks at the top of the front page of the test (name, age, grade, etc.) but not to read any part of the test.)

3) When this is done, SAY:

" You may not be able to work correctly every example in this test. You will not be marked on a per cent basis, but you will be compared with other pupils in your grade, so get the right answer to as many examples as you can. Look carefully at each example and find out what you are to do. All answers must be in lowest terms. If the answer is in money, don't forget the dollar sign and decimal point. If the answer is in bushels, pecks, quarts, gallons, or pints, don't forget to include this in your answer. You may do your work either in the space with the example or on the blank sheets of paper which I have just given you, but be sure to write your answers on the test paper in the space with the example. Work fast, but make sure that everything you do is right. When I give the signal to begin, start on the front page, just underneath where you have written your name, etc. When you have finished this page, turn to page two and continue the test on this page and then on the following pages. Do not stop until you have finished the whole test or until I give the signal to stop _______Ready, BEGIN!"

4) Make a note of the time as you give the signal, "Begin." ALLOW 40 MINUTES. However, if the teacher or examiner notices that all but the slowest two or three pupils have finished the test or have worked up to the limit of their abilities, she may stop the test before the expiration of the time limit specified. In most cases (especially in the lower grades) less than 40 minutes will be required to give the test.

5) When the time is up, give the signal "STOP" and collect all papers immediately.

SPECIFIC INSTRUCTIONS
PRESSEY VOCABULARY TEST

The examiner should be sure, first of all, that the children have pencils, not pens; he should then pass out the blanks. No pupil should be allowed to open any blank or turn it over before the test begins.

When all the pupils are ready, have them write name, age, grade, and so on, on the lines provided (city and state may be omitted). After this information has been filled out, the directions should be read to the class verbatim, slowly and clearly. Procedure throughout should be easy and informal, but brisk and businesslike.

SAY:

" Now look at the first example just below where you have been writing. It reads, ‘What is a horse? a book, a plant, an animal, a fruit.' What is the right answer? (Children answer.) Yes, 'an animal.' A mark has been put under 'an animal' to show that it is the right answer.”

" Now look at the next question. It reads, 'When is today? gone, coming, tomorrow, now.' What is the right answer? _______Yes, 'now.' All of you take your pencils and put a line under 'now' to show that 'now' is the right answer. (Be sure that the pupils do this correctly.)”

" Next look at question three. It reads, 'What does 'beautiful' mean? best, light, pretty, fast.' What is the right answer? _______Yes, 'pretty.' All draw a line under 'pretty' to show that this is the right answer.”

" Now look at question 4: 'For what do you use an oven? cooking, hunting, painting, sewing.' What is the right answer? _______Yes, 'cooking.' All make a line under 'cooking' to show that it is the right answer.”

" And now follow while I read to you the directions just below this question:

“ On this page and the next three pages there are some more questions like these. You are to draw a line under the right answer to each question. Work rapidly, but answer as many questions as you can. If you come to a question you cannot answer, skip it and go on. When you finish one page go on to the next. BEGIN NOW. Make a line under each right answer."

Be sure that all the children now start work, with question 1…… not with question 11 on the top of the next page. Watch also to be sure that the children do not stop at the end of the first, second, or third page. WHEN ALL BUT THE SLOWEST TWO OR THREE PUPILS HAVE FINISHED (AND IN ANY CASE AT THE END OF 25 MINUTES) stop work and collect the papers.

SPECIFIC INSTRUCTIONS
FREE WRITING TEST

READ OR RECITE:

" We have just handed out the Free Writing Test. Write your name (write name) and the name of this school (write name of school) on the front page.”

" Open the booklet and look at the picture on the left hand page. You will notice that it shows a man riding away on a horse while a girl stands at the door of the house. You probably notice many other things in the picture. You are to write a story about this picture. Let us read the directions on the page opposite the picture.”

" 'DIRECTIONS: Write one or two paragraphs telling the story which the picture on the opposite page suggests to you. In this story, be sure that you include something about

1. where the man is going,

2. what he is going to do,

3. how soon you think he will return,

4. what he will bring back with him,

5. what the girl is doing,

6. any other things that you wish to include.

WRITE YOUR STORY IN THE SPACE BELOW.’ ”

" We are anxious to have you write a good story, being sure to explain in the best way just what you imagine about the rider, where he is going, about the girl, and many other things.”

" Lines have been drawn and space given so that you can write nearly a whole page. We hope all of you will use that much space in writing your story. Those who have even more to write may turn the page and write on the back side also.”

" Write clearly. Make your sentences tell the story you want to tell. In your writing show that you know when to use capital letters, when to use periods, commas, and question marks. As long as you are telling a story, you will probably be able to use some quotation marks and in that way tell exactly what the little girl is calling out to the man.”

" We are anxious to have you tell an interesting story; that you write along enough one to tell everything you imagine about the picture; and that you show by your writing how to use many kinds of punctuation marks such as commas, quotation marks, and so on. Show us how many of them you can make use of in your story writing."

DIRECTIONS FOR SCORING THE TESTS
GRADE IV

The following are to be returned to the Chicago Office immediately after tests have been administered:

1) The Attendance Sheet,

2) All Unused Tests,

3) Background Information Sheets for all pupils tested,

4) Free Writing Tests.

Though the Free Writing tests are not to be scored in the field because this scoring is somewhat complicated, we would like to have the student's teacher, if she cares to, comment on the student's paper. We would welcome a comment on the back of each paper, particularly a statement as to whether or not she believes this paper is representative of the work which the student has been doing, and whether or not it is a good paper.

The following tests are to be scored, as arranged by your Area Superintendent, and then returned to the Chicago Office. On all tests, draw a line through the number of each correct response, and write the score on the cover of the test.

1) Gates Reading Test-Advanced Primary (Type 1. Word Recognition)

a) Mark each correct response by drawing a line through the number of the problem.

b) Mark each wrong response by placing a check beside the item.

c) Total the correct responses.

d) Total the incorrect responses and divide this number by 3.

e) Subtract this number (1/3 of the incorrect responses) from the total number of correct responses. Write this remainder in the upper right corner of the cover of the test.

2) Gates Reading Test-Advanced Primary (Type 2. Word Recognition)

a) Draw a line through the number of each correct response.

b) Write the score, or the number of the correct responses, on the upper right corner of the cover of the test.

3) Arithmetic Computation Test

a) Mark each correct response by drawing a line through the number of the problem. Each incorrect answer counts zero; each correct answer counts one point.

b) Consult the scoring key in the examiner's kit for correct responses and further instructions.

c) Write the number of the correct responses on the upper right hand corner of the cover.

4) Pressey Vocabulary Test

a) Mark each correct response by drawing a line through the number of the problem.

b) Consult the scoring key in the Examiner's Kit.

c) Write the number of correct responses on the front cover in the upper right hand corner.

SUGGESTED ADMINISTRATIVE SCHEDULE
FOR
GRADE VIII

In those schools that have elected to give optional tests, the teachers (and not the test administrators) will administer these tests on the following day.

SPECIFIC INSTRUCTIONS
SAMPLE TEST SHEET

The purpose of these few test questions is to familiarize the students with the way they are supposed to mark their tests. This test should be administered in a very informal manner and the students should be given all possible help in understanding the questions themselves, selecting the proper answer, and finally marking the answer. There is no time limit on this test for the test administrator will find that he needs more time to explain this to some groups than to others.

READ OR RECITE THE FOLLOWING TO THE STUDENTS:

" Your teacher and I are to give you a number of examinations today. The purpose of these tests is to help us understand what things you students have learned during the last years. Not all of these examinations will test what you have learned in the schoolroom for some of these things you will have learned at home, at work, or even at play.”

" We gave some of these examinations last year to school children all over the country. We found that many of these students knew and understood things that we as their teachers had not even expected them to know. We also found that there were some things the students didn't understand although we thought we had taught it. These examinations are very helpful because they tell your teachers how well they have taught you.”

" We want you to look at the questions on the sheet we are now going to give you. This test will give you a good idea of how many of the tests will look and how you should mark them.”

(Pass out tests.)

"This test will not be counted for it is only a sample. Let us read together the directions near the top of the page.”

READ TEST DIRECTIONS IN UNISON.

" DIRECTIONS: This test is made up of a number of items. Each item has four possible answers. Choose the best answer in each item and place a check (V) beside it. Check only ONE answer in each item. If you mark more than one answer, the question will be marked wrong.”

" It is always necessary to read carefully so as to understand the directions on every test you take because some of the tests will be slightly different than others. Always read and understand the directions before you begin a test. First there is a statement which is not complete. It is followed by four possible endings marked A, B, C, and D. One of these endings completes the introductory statement better than the other three. You are to choose the one which makes the best ending."

Read each of these problems, and discuss the reasons for the best answer to each. BE SURE EACH STUDENT UNDERSTANDS THE SAMPLES AND THE WAY HE IS TO MARK THE ANSWER.

SPECIFIC INSTRUCTIONS
GENERAL RESOURCES TEST

Before distributing the General Resources tests, the test administrator should give each pupil a copy of the Sample Questions Sheet. (Read Specific Instructions-Sample Questions Test.) After these have been collected, the General Resources tests may be dis- tributed, face down.

Then SAY:

" This is one of the tests in Rural Practices, which follows the same pattern as the sample which you have just completed."

Show the students your copy of the test, pointing to the top half. Have them turn their papers and fill all the blanks in the heading except "score." Tell them not to look at the bottom half until all are ready. Read aloud and explain the items in the heading. Move about the room and help pupils to fill all blanks correctly. Write on the blackboard the names of the Examiner and their teacher. After all have completed the heading, ask the pupils to follow on their papers while you read aloud slowly the printing under "Use of Resources."

" Timber, farm lands, coal and wild life are called natural resources. Nature has given us these things to help us make a living.”

" DIRECTIONS: This test is made up of a number of items about some of these resources. Each item has four possible answers. Choose the best answer in each item and place a check beside it. Check only ONE answer in each item."

Pause briefly to answer any questions and see that all are ready. .Then tell them to begin with Question No. 1, Part A, and go right through to the end of the test-through Part C, Question 66. They should not spend too long on any one question, but leave it and go on to the next question. After they have gone through the whole test, they may have time to go back and give further study to the questions they have passed over.

This is not a timed test. However, it should not take over one hour. Take up all of the papers after 90 per cent of the pupils have finished.

SPECIFIC INSTRUCTIONS
RURAL PRACTICES VOCABULARY TEST

Be sure that all pupils have pencils. Put the pupils at ease. Tell them that the test they are now about to take is to see how many of the words commonly used in talking or reading about rural life are understood by the pupils.

Then distribute the test papers, face down, telling them not to turn them over until they are told to do so. Hold up your copy of the test and point to the blanks where they are to fill in the heading, including the date the test is taken.

Then tell them to turn over their papers and fill out the heading. Move around quietly to see that all blanks are filled in correctly. Tell them to look at the Directions, and to follow as you slowly read the Directions aloud.

(These Directions may be read directly from the test itself.)

" DIRECTIONS: On the following pages, you are to select the best way of completing each of the sentences. Be sure to notice the word in black print in each sentence for the answer which you select will depend upon that word. Select only ONE answer. Draw a line under the answer which you select.'"

Read the first sample question. Ask them to underline the best answer. (Move around quietly to observe that all pupils underline the right answer.) After a short pause, ask one pupil which answer he underlined. If it is "fast" ask how many underlined that answer. Tell them they are right, that is the correct answer. If the first pupil you ask did not underline "fast," ask other pupils till you get one who underlined the correct answer. Avoid embarrassing any pupil who did not underline the correct answers, but encourage questions and discussion, keeping them within reasonable bounds. Tell them the Sample Questions will not be scored. Then continue the same procedure with Sample Questions 2 and 3. Be sure they understand clearly the method to be used in marking the 72 questions on the following pages.

Pause briefly to see that all are ready. Then tell them to turn the first page and begin with Question No. 1 and go right on through to the end of the test-through Question No. 72.

This is not a timed test. However, it should not require more than about 30 minutes. Take up all the papers as soon as about 90 per cent of all the pupils have finished.

DIRECTIONS FOR SCORING THE TESTS
GRADE VIII

The following are to be returned to the Chicago Office immediately after the tests are administered:

1) The Attendance Sheet,

2) The Background Information Sheet for every pupil tested,

3) All Unused Tests.

The following tests are to be scored, as arranged by your Area Superintendent and then returned to the Chicago Office:

1) General Resources

a) Consult the scoring key enclosed in the Examiner's Kit.

b) Draw a line through the number of each question to which the correct response has been given.

c) Write the total number of correct responses on the front cover of the test in the. space provided.

2) Rural Practices Vocabulary Test

a) Consult the scoring key enclosed in the Examiner's Kit.

b) Draw a line through the number of each question to which the correct response has been given.

c) Write the total number of correct responses on the front cover of the test in the space provided.

3) Optional Tests

Keys are provided in the "Optional Test Kits" so that teachers may score tests which they have requested. You may record the results for your own use, but it has been decided that all of these tests should be returned to the Chicago Office, after scoring, in order that we may furnish you with interpretations based not only on national norms, but on norms obtained in the 1945 Eighth Grade Service-wide Testing Program.

SPECIFIC INSTRUCTIONS-GRADE XII
HEALTH AND SAFETY TEST

Be sure that all pupils have pencils. Put the pupils at ease. Tell them they are now about to take a test to see what they know about doing things that keep them well and strong. It is called the Health and Safety test.

Then distribute the test papers, face up. Have them fill in the blanks on the cover page, except "Score." Do not allow them to open their papers until all have finished the cover page. Move quietly about the room to see that all pupils are filling all blanks correctly. Help them, wherever needed. Write the names of the Examiner and the Teacher on the blackboard.

When all have finished the cover page, tell them to open their tests and look at the Directions on page two. Read the directions aloud slowly, while pupils follow on their own copies.

" DIRECTIONS: This test is made up of fifty items about health and safety. Each item has four possible answers. Choose the best answer in each item and place a check beside it. Check only ONE answer in each item."

Pause briefly to answer any questions and see that all are ready. Then tell them to begin with No. 1 and go right on through to the end of the test. They should not dwell too long on any one question. If they cannot decide on the best answer, skip it and go on with the next question. After they have gone through the whole test to No. 50, they may have time to go back and give further study to the questions they have passed over.

THIS IS NOT A TIMED TEST. However, it should not take longer than about 40 minutes. Take up all papers as soon as about 90 per cent of the pupils have finished.

HOME ECONOMICS TEST (FOR GIRLS ONLY)

Be sure that all pupils have pencils. Put the pupils at ease. Tell them the test they are now about to take is to see what they know about the things that go on in a rural home, and the knowledge and skills that should help to make a happy home. This is called the Home Economics Test.

Then distribute the tests, face down, telling them not to turn the booklet until they are told to do so. Hold your copy of the test up and point to the blanks where they are to write their names, etc. Then have them fill in the blanks. When all have completed the cover, ask them to follow as you read the Directions.

" DIRECTIONS: This test is made up of a number of items about home economics. Each item has four possible answers. Choose the best answer in each item and place a check beside it. Check only ONE answer in each item.”

Pause briefly; answer any questions; then tell them to begin with No. 1 and go right through to the end of the test, through question No. 41. If they cannot decide on the best answer, skip it, and after they have gone through the whole test, they may have time to go back and give further study to the questions skipped.

THIS IS NOT A TIMED TEST. However, it should not take longer than about 40 minutes. Take up all papers as soon as about 90 per cent of the pupils have finished.

SPECIFIC INSTRUCTIONS
ARITHMETIC REASONING

1) Each pupil should have at least two sharpened pencils. Only pencil is to be used. The examiner should have an extra supply of sharpened pencils on hand in case of need.

2) SAY:

" We are going to have a test in arithmetic. I will give you each a copy of the test. Do not look at it or write anything on it until I tell you to." (Give each pupil a copy of the test, as well as two sheets of blank paper, and tell the pupils to fill in the blanks at the top of the front page of the test (name, age, grade, etc.) but not to read any part of the test.)

3) SAY:

" You may not be able to solve correctly every problem in this test. You will not be marked on a per cent basis, but you will be compared with other pupils in your own grade, so get the right answer to as many examples as you can. After you have solved the problem, write the answer in the ruled space for that purpose. Look at the example on the front page of your test booklet. It reads, ‘If you have two apples and I gave you one more apple, how many apples would you have?’ This problem is solved by adding 2 and 1, which gives the answer '3.' You will note that the '3' has been placed in the ruled space marked 'Answer.' You may do as much of your work as you wish 'in your head,' but do all your written work on the blank sheets of paper which I have just given you. If the answer is in money, don't forget the dollar sign and decimal point, or the cent sign. If the answer is in dozens, per cent, or the like, don't forget to include this in your answer. Work fast but make sure that everything you do is right. When I give the signal to begin, open your booklet to page two and work the example under one. When you have worked this example and have placed the answer in the ruled space for that purpose, work example number two. Continue with the other examples on that page and then on the following pages. Do not stop until you have finished the whole test or until I give the signal to stop ________Ready, BEGIN!"

4) Make a note of the time as you give the signal, "Begin." ALLOW 40 MINUTES. However, if the examiner notices that all but the slowest two or three pupils have finished the test or have worked up to the limit of their abilities, she may stop the test before the expiration of the time limit specified. In most cases (especially in the lower grades), no longer than 30 minutes should be required to give the test.

5) When the time is up, give the signal "STOP" and collect all papers immediately.

SPECIFIC ISTRUCTIONS
GATES BASIC READING TESTS

1) See that each child has a pencil.

2) Distribute the papers.

3) Have the children fill in the blanks on the cover page.

4) Read the directions aloud.

(These may be read directly from the cover page of each test.)

TYPE A

SAY:

" This is to be a reading test. You are to read a number of paragraphs. Below each paragraph is a sentence which tells you what to do. It will tell you to draw a line under one of five words which best shows that you have understood the paragraph. Be sure to draw a line under only one word. Now let us try a sample before we begin the real test. Read the following paragraph and do what the sentence under the paragraph tells you to do."

(Children read) Explain the directions with great care and make clear the reason why one answer is best. Repeat the directions and explanations if necessary.

SAY:

" On the following pages are more paragraphs similar to this one. When the signal 'Begin' is given, you should turn the page, read the first paragraph, and do what the directions tell you to do. (Show the children the order of the paragraphs on all three pages by holding up a copy of the test. Ask them to read in 'this order.') When you finish the first, go on with the second and so on until the signal 'Stop' is given. The purpose of this test is to see how many paragraphs you can read and mark correctly in a short time. Don't waste any time. Don't look at anyone's paper. DO NOT TURN THE PAGE UNTIL YOU ARE TOLD TO BEGIN."

SAY:

" BEGIN." At the end of EXACTLY SIX MINUTES, say "STOP."

TYPE B

SAY:

" This is to be a reading test. You are to read a number of paragraphs. Below each paragraph are four sentences. Each sentence tells what is most likely to follow after the happenings that are described in the paragraph. You should draw a line under oneand only one-of these sentences to show that you can tell what will probably happen next. Now, let us try a sample before we begin the real test. Read this paragraph and then draw a line under the one sentence that you think tells what will happen next."

(Children read.) After the pupils have tried the test paragraph on this page, tell them how you would do it. Give reasons why the line you marked in the sample is most likely to happen next. Repeat directions, if necessary.

SAY:

" On the following pages are more paragraphs just like this one. When the signal 'Begin' is given, you should turn the page, read the first paragraph, and underline the sentence that best tells what is coming next, just as you did above. When you finish the first, go on with the second and so on until the signal STOP is given. The purpose of this test is to see how many paragraphs you can read and mark correctly in a short time. Don't waste any time. Don't look at anyone's paper. DO NOT TURN THE PAGE UNTIL YOU ARE TOLD TO BEGIN."

The front page should be up. Give the signal "BEGIN." At the end of EXACTLY EIGHT MINUTES, say "STOP."

TYPE C

SAY:

" This is to be a reading test. You are to read a number of paragraphs. Each paragraph tells you to make some sort of mark with your pencil to show that you have understood the paragraph. Do exactly what the paragraph tells you to do. Make the marks quickly-do not waste any time trying to make pretty drawings. The purpose of the test is to see how many of the paragraphs you can read in a short time. Don't waste any time. Don't look at anyone else's paper. Remember, you must do exactly what the paragraph tells you to do. Don't make any marks other than those the paragraph tells you to make. Wait until you are told to begin, and then turn the page and work as quickly and as accurately as you can until you are told to stop. DO NOT TURN THE PAGE UNTIL YOU ARE TOLD TO BEGIN."

Demonstrate the order in all three paragraphs. Tell the children they are to read, in order, as many paragraphs as they can in the time allowed.

The front page should be up. Give the signal "BEGIN." At the end of EXACTLY EIGHT MINUTES, say "STOP."

TYPE D

SAY:

" This is to be a reading test. You are to read a number of paragraphs. Below each paragraph are three questions which you must answer by drawing a line under the one word or phrase that gives the best answer. Let us try a sample before we begin the real test. First read the paragraph. Then underline one-and only one-of the four answers to each question to show that you understand what the paragraphs said."

(Children try the test paragraph.) Ask them which word they marked. Explain carefully why one answer is correct and the others wrong. Give special attention to any who marked the wrong answer. See that all understand exactly what they are to do.

SAY:

" On the following pages are more paragraphs just like this one. When the signal begin is given, you should turn the page, read the first paragraph, and underline the best of the four answers to each question, just as you did above. When you finish the first, go on with the second and so on until the signal stop is given. The purpose of the test is to see how many paragraphs you can read and mark correctly in a short time. Don't waste any time. Don't look at anyone's paper. DO NOT TURN THE PAGE UNTIL YOU ARE TOLD TO BEGIN."

Show the order in which the paragraphs are to be read on all three pages of the test by holding up a copy of the test. Tell them to read in "this order."

The front page should be up. Say, "BEGIN." At the end of EXACTLY EIGHT MINUTES say "STOP."

Collect all papers immediately after the signal "STOP."

SPECIFIC INSTRUCTIONS
PRESSEY ENGLISH TESTS

1) Each pupil should have at least two sharpened pencils. Only pencil is to be used. The teacher should have an extra supply of sharpened pencils on hand in case of need.

2) Allow exactly the time specified for each test. The examiner is to lose no time between the tests, but is to proceed directly to the next test when one has been finished.

3) Use every reasonable effort to avoid disturbance during the examination.

4) As soon as the tests are passed to the pupils, SAY:

" Fill in the blanks at the top of the page. On the first line you are to put your name. On the second line put your grade, and your age in years and months. On the third line, put the name of your school. On the fourth line, put today's date."

5) After this has been done, SAY:

" I am going to give you four tests in English. You will be given a separate time limit for each test. If you finish a test before I tell you to stop, go over your work to make sure that you have done it correctly.”

" Now you are to do only the Capitalization Test on the left half of the first page. I will read the brief directions at the beginning of the test, while you follow silently:

'In the sentence below, you are to draw a line under each letter that should be a capital.’ ”

When all are ready, say "BEGIN!" In THREE MINUTES, say "STOP!"

6) SAY:

" Now go to the Good Usage Test and work all thirty exercises on the right half of the first page and on all of the second page. I will read the directions at the beginning of the test, while you follow silently:

'In each group below, one of the three sentences, and only one, is incorrect-contains something which is not good English. Find the wrong sentence in each group and put a cross in the square before it.” When all are ready, say, "BEGIN!" In TWELVE MINUTES, Say: "STOP!"

7) Now say:

" Now go to the Punctuation Test on the left half of the third page. I will read the directions at the beginning of the test, while you follow silently:

'The sentences below do not have punctuation marks-except the period at the end of each sentence. You are to write in all further punctuation, changing periods to exclamation points or interrogation marks where necessary, and inserting quotation marks. Do not punctuate so as to form new sentences.' "

When all are ready, Say: "BEGIN!" In FIVE MINUTES SAY: "STOP!"

8) SAY:

" Go to the Sentence Structure Test and work all twenty exercises on the right half of the third page and on all of the fourth page. I will read the directions at the beginning of the test, while you follow silently:

'In each group below, one of the three statements is not well written-is poor in sentence structure. You are to find the poor statement in each group and put a cross in the square before it.' "

9) When all are ready, say "BEGIN!" In TWELVE MINUTES, say "STOP!"

SPECIFIC INSTRUCTIONS
PRESSEY VOCABULARY TEST

The examiner should be sure, first of all, that the children have pencils, not pens; he should then pass out the blanks. No pupil should be allowed to open any blank or turn it over before the test begins.

When all the pupils are ready, have them write name, age, grade, and so on, on the lines provided (city and state may be omitted). After this information has been filled out, the directions should be read to the class verbatim, slowly and clearly. Procedure throughout should be easy and informal, but brisk and businesslike.

SAY:

" Now look at the first example just below where you have been writing. It reads, 'What is a horse? a book, a plant, an animal, a fruit?' What is the right answer? (Children answer.) Yes, 'an animal.' A mark has been put under 'an animal' to show that it is the right answer.”

" Now look at the next question. It reads, 'When is today?

gone, coming, tomorrow, now.' What is the right answer?_______Yes, 'now.' All of you take your pencils and put a line under 'now' to show that 'now' is the right answer. (Be sure that the pupils do this correctly.)”

" Next look at question three. It reads, 'What does 'beautiful' mean? best, light, pretty, fast.' What is the right answer?_______Yes, 'pretty.' All draw a line under 'pretty' to show that this is the right answer.”

" Now look at question 4: 'For what do you use an oven? cooking, hunting, painting, sewing.' What is the right answer?_______Yes, 'cooking.' All make a line under 'cooking' to show that it is the right answer.”

" And now follow while I read to you the directions just below this question:

'On this page and the next three pages there are some more questions like these. You are to draw a line under the right answer to each question. Work rapidly, but answer as many questions as you can. If you come to a question you cannot answer, skip it and go on. When you finish one page go on to the next. BEGIN NOW. Make a line under each right answer."

Be sure that all the children now start work, with question 1-not with question 11 on the top of the next page. Watch also to be sure that the children do not stop at the end of the first, second, or third page. WHEN ALL BUT THE SLOWEST TWO OR THREE PUPILS HAVE FINISHED (AND IN ANY CASE AT THE END OF 25 MINUTES) stop work and collect the papers.

SPECIFIC INSTRUCTIONS
GENERAL SCIENCE TEST

1) Be sure that all pupils have pencils (soft lead preferred).

2) SAY:

" We are living today in a world where the study of science and mechanics has changed our ways of life. Of course, many years of study and experience are necessary to understand all these scientific developments, and there is much to be learned yet-even by our greatest scientists. But we are all using in our daily life some of the products of scientific study, and everybody should learn something about the laws and facts of science. The test you are now going to take is to see how much you know about the common things in the field of science. The name of this test is "General Science.' "

3) Distribute tests, face up, telling them not to open them until they are told to. Tell them to look at the Directions at the top of the cover page, and to follow while you read:

" 'DIRECTIONS: You have two hours and 30 minutes for this examination. (ALSO ADD-MOST OF YOU WILL PROBABLY NOT NEED THIS MUCH TIME. THIS IS NOT A SPEED TEST. THE IMPORTANT THING IS THAT YOU SHOULD DO YOUR WORK CAREFULLY.) As you answer the questions, you should omit any that seem unusually difficult until you finish the others.

" Your answers to the exercises in this examination are to be recorded on the separate Answer Sheet which is loosely inserted in this examination booklet. Remove this answer sheet now; write your name and the other information called for in the blanks at the top of the answer sheet.'" (See that the pupils fill out all blanks correctly, and do not write on the test booklets.) _______After the number on the answer sheet corresponding to that of each exercise, mark the one lettered space which designates the answer you have selected as correct' " (Omit the last line of this paragraph and the first sample.) "'Your answer sheet contains rows of paired dotted lines. You are to indicate each answer with a heavy black mark between the paired dotted lines under the letter selected as your answer."

4) Explain the sample, and make sure that all understand how to use the answer sheet and that no marks are to be made on the test booklet itself. Tell them not to fold their answer sheets, and to erase any marks completely if they change their answer. Stress the fact that THERE IS ONLY ONE CORRECT ANSWER FOR EACH PROBLEM.

5) SAY:

" Remember not to spend too long on a problem if it seems difficult, but go ahead to the next one for it may be easier for you. You are not expected to know all the answers, but if you try every problem, you will find a good many that you can answer." Stress the fact that easy items are scattered among the hard ones, but the problems should be read in numerical order.

6) Now have the pupils open the test to page two, arrange the answer sheets for convenient marking, and tell them to begin with Item No. 1 and go right on through to the end of the test.

At the END OF TWO AND ONE-HALF HOURS take up all tests and answer sheets. It is believed that few students will require this much time to complete ail of the items that they are able to do. It is not necessary that they work for the entire period, but be sure that they have made an honest attempt to do all they can before they stop. A rest period (deduct time allowed from test time) is desirable, but care must be taken to so plan this rest period that there is no opportunity for students to discuss questions on the test.

DIRECTIONS FOR SCORING THE TESTS
GRADE XII

The following tests and forms are to be returned to the Chicago Office for scoring and tabulating immediately after the tests are administered:

1) The Attendance Sheet,

2) The Background Information Sheet for every pupil tested,

3) All Unused Tests,

4) The General Science Tests and Answer Sheets,

5) The Free Writing Tests.

Though the Free Writing tests are not to be scored in the field because this scoring is somewhat complicated, we would like to have the student's teacher, if she cares to, comment on the student's paper. We would welcome a comment on the back of each paper, particularly a statement as to whether. or not she believes this paper is representative of the work which the student has been doing and her judgment as to the quality of the paper.

The following tests are to be scored, as arranged by your Area Superintendent, and then returned to the Chicago Office. Mark each correct response by drawing a line through the number of the question. For each test, write the score, or the total number of correct responses either in the space provided on the front of the test, or on the upper right hand corner of the cover page.

1) General Resources Test

a) Consult the scoring key enclosed in the Examiner's Kit.

b) Draw a line through the question number to indicate each correct response.

2) Health and Safety Test

a) Consult the sample copy of this test in the Examiner's Kit for correct responses.

b) Mark each correct response, by drawing a line through the number.

3) Home Economics Test

a) Consult the sample copy of this test in the Examiner's Kit for correct responses.

b) Mark each correct response, by drawing a line through the number.

4) Rural Practices Vocabulary Test Kit

a) Consult the scoring key enclosed in the Examiner's

b) Mark each correct response, by drawing a line through the number.

5) Arithmetic Computation Test

a) Read the instructions, and follow precisely, on the key enclosed in the Examiner's Kit.

b) Mark each correct response, by drawing a line through the number.

6) Arithmetic Reasoning

a) Consult the scoring key enclosed in the Examiner's Kit.

b) Mark each correct response, by drawing a line through the number.

7) Gates Reading Test: Types A, B, C, and D

a) Consult the sample copies of these tests in the Examiner's Kit.

b) Mark each correct response, by drawing a line through the number.

8) Pressey Vocabulary Test

a) Consult the scoring key in the Examiner's Kit.

b) Mark each correct response, by drawing a line through the number.

9) Pressey English Test (Sections A, B, C, and D, and Mean Score)

a) Consult the scoring key in the Examiner's Kit.

b) Mark each correct response, by drawing a line through the number.

c) Convert scores to per cent scores, using table at the bottom of each test.

d) In section A, all words indicated must be capitalized-each sentence counts one point. In section C, all marks, and no others, must be used. Count each correct sentence one point. Mean is Average.

APPENDIX D

SCORING AND INTERPRETING THE TESTS

1945 PROGRAM

All of the papers from the 1945 program were scored in the Chicago office. In order to facilitate the rapid handling of all the papers, two people were delegated the responsibility for planning the assignments of those who worked on the scoring and tabulating. A special effort was made to increase the accuracy and consistency of the scoring by the careful selection of conscientious, interested personnel, and by providing each grader with careful specific instructions and limiting his activities to a single examination. A sufficiently large sample of tests was rechecked by others to prove that the test scorer or scorers were following directions exactly. The rechecking indicated that errors occurred in only a fraction of one per cent of the papers. Moreover, practically all of these errors were errors in matters of judgment or interpretation such as interpreting the place that a student had intended to make a mark, whether a letter was intended as a "C" or an "O," and the like. Therefore, even in the cases in which an error had been made, it would account for only a very small part of the student's total score.

In order to facilitate the recording of the scores, separate score sheets were prepared for each examination as well as special sheets for each of the eighth grade classes. The students' names were listed alphabetically on the class sheets and each given a number. This number was then placed on all examination papers and numbers, rather than names, used as a basis for transferring scores from paper to record sheets. The scores were all transferred to master class sheets on which there was space for all of the thirtyone scores. This recording was done by two people with one reading the scores and the other making the entries. These recorded data then were completely checked for accuracy. Here again, a count was made to determine the character of the errors and their frequency. It was discovered that there was a greater frequency of errors made in transferring scores than in the original scoring. The recording errors were traced to two fundamental types, namely: 1) recording scores in spaces that should have been left blank, thereby giving wrong scores to the students; and 2) omitting scores that should have been included. Omissions were sometimes sanctioned where there was evidence that the time limit of an examination had been exceeded, thereby giving the student an abnormally high score. It was thought that it would be better to omit such scores for they would not only be meaningless but they might also be wrongly interpreted if one did not notice the necessary footnote explanation. It should be noted that though recording errors did occur, they were found and corrected by checking every score.

In the 1945 program, all distributions were computed by pencil-paper tabulations. Most of the information was studied in relation to the entire group tested, while a few of the conclusions were based on data and distributions secured from a representative random sample. The various distributions permitted checking in a number of ways which made a higher accuracy possible. The various groupings used are explained fully in Appendix E, by the copies of the explanation sheets furnished to the participating schools together with the record of student scores.

1946 PROGRAM

Most of the examinations in the 1946 program were scored in the field. In some instances, the scoring was done by the teachers of the particular class taking the examination, whereas in other cases groups of teachers were brought together at the close of school and the scoring was done by the group at one time. It is believed that the second plan resulted in more consistent and accurate scoring. It was possible to delegate the responsibility of scoring to the teachers in the field because so many of the examinations were of the purely objective type and did not involve interpretation and judgment that lead to inconsistency and variation between schools and teachers. Even the objective examinations were spot-checked in the Chicago office, and in those instances in which the computation of the scores was incorrect, all of the papers from the school were recalculated. Most of the scoring was found to be extremely accurate. The type of error discovered could be blamed upon the scorer's failing to understand such things as the correction of a score for guessing by deductions for incorrect responses, which resulted in their counting omissions as errors.

The United States Armed Forces Institute General Science Examination, given to all of the twelfth graders and to a sample of eighth graders, was returned to the Chicago Office for scoring because it was planned to analyze the individual items at the time of scoring.

The Free Writing Examination given to fourth and twelfth graders, was not scored in the field because it involved more complicated scoring directions. These tests were scored by a group of teachers, all working under the direction of one of the staff of the Chicago Office.

The Free Writing Examination had been found in the 1945 program to be one of the most time-consuming and difficult tests to score and the directions for administration were radically changed in the 1946 program.

Those who participated in the scoring of the tests reported that they profited by the experience. Several are interested in further experimentation with the tests in relation to methods of instruction. Some of the observations of members of the group seem to be worthy of reporting.

1) This type of analysis tends to focus the reader's attention upon the intended meaning of the passage. A first impression of a paper is influenced markedly by legibility, neatness, and spelling. The following paper, photographed, at first appears entirely incoherent.

In reality, though the child's handwriting is very poor and he has little knowledge of punctuation or spelling, he has at least average command of English. He appears to be lacking only the tools for written expression. If such tools were employed, this story would read thus:

" He went for a long ride to the mountains to hunt spotted tail deer. He went for along time, riding through the meadow. At last he saw spotted tail deer, grazing in the meadow. So he took his rifle from its holster and fired (at) Spotted Tail, the deer. (The) deer was too fast for the warrior and sped through the forest. The deer was not too fast, for the rider sped after him and a shell "zinged" Spotted Tail's head and he started to play out, and slowed down. In that minute, the warrior shot and Spotted Tail fell dead. The Indian dragged the deer to his horse and drug him home. And his family had a great feast."

It will be noted that in this translation no construction has been changed. The only missing words are indicated in parenthesis. The story does possess one of the essentials of any good story-it is interesting. Its merit is certainly hidden by the child's lack of mastery of tools.

2) Most students interpreted the picture in terms of their own environment and experiences. In Alaska the man was going to hunt seal; in the Southwest he was going to hunt deer; in another area he was in search of rabbits; in some instances he was in search of a new mother for his children. In many areas the oven was described as a dog-house; in others it was a playhouse for the little girl. In general, even the names assigned to the characters were names common to Indians of the area of the writer.

3) Though the picture was not selected with any thought of stimulating the expression of the individual's attitudes, so many of the papers revealed the writer's attitudes that it has been suggested that a picture-stimulus, free writing test might be a useful instrument in an attitude study. The following papers interested members of the group because of attitudes which are reflected.

OKLAHOMA-NON-RESERVATION BOARDING
GRADE 12

Attitudes toward: Rationing and Indian Administration
Reservation Living

" He is probably going to the nearest Indian agent to wait for his beef rationing. It will probably be along time before he gets the rations. He will bring just about enough to get by on since he is content to live on some reservation.”

" He is bringing the girl up to live in the some drab existence that he has lived in all of his life.”

" He looks very illiterate and he must be to be content with such a life. If this is all that we have to offer the Indian, such a miserable life as this then I think the ‘vanishing' race had better hurry up and vanish all together. They would be better off. This kind of treatment is disgusting even to the eyes of the Indian himself.”

" If there is such conditions in this country today, then I take back all of the things I said about Africa, Sicily, and Italy. We have nothing to brag about. Even the lowliest of Arabs live better than this."

(Note: Written by a returned veteran.)

OKLAHOMA-NON-RESERVATION BOARDING
GRADE 12

Attitude toward: Medicine Men as contrasted to white doctors
" The Saver"

Pokes's little brother was very sick. The medicine man did not know how to cure this disease. All the Pueblo family was afraid to call on the white doctors for their aid.

" One night when father and mother was discussing the situation, there come a sound from the small boys room. He was calling, 'White doctor, white doctor.' The father knew this was a sign to get the white doctor for his son without delay.”

" Next day, father eats his breakfast and prepares for a long journey to save his beloved son. It was sun rise when he strapped his rifle to his pinto poney's back. He rides off toward the burning sun, waving good bye to his small daughter, who was standing in the doorway calling to Shep, the dog.”

" Days had gone by and the small boy was getting worse. When all hopes were given up, suddenly the mother and daughter heard chimes of bells in the far distance. A speck was seen coming over the foothills, which glittered in the sun's bright ray. Finally two specks could be seen. It was! It was father and the white doctor galloping on their ponys' as faster as they could.”

" The doctor persuaded the family to let the small boy to go to a hospital where he could have special care.”

" Now the boy, who was fail and had T.B., is the strongest lad living in Pueblo.”

" The family has now accepted the white man's medicine in place of their own, thanks to the boys call for the white doctor during the night."

ALASKA-NON-RESERVATION
GRADE 12

Attitude: Toward Family and possible separation

" Poncho," "Poncho," please don't go, cried, little Treasa. Poncho, her brother was going to the trading post to trade some things, but treason didn't want him to go, she, was afraid he, too might never come back. She remembered very distinctly one afternoon when her father and mother and the rest of the family were all together discussing the Sunday feast. Mike had come to tell her father that the white men had a trading Post and that the men were going there.

Right a way father, started to get ready, even then Treasa, was afraid, she told her father not to go, "its alright treasa, he had said, "I'll be back early in morning with new things for you; and then he was gone. He hod never come back. Now it seem her brother, who was always so good to her and who had helped her mother so much was also going. Although these things were running through her mind she went and did her nightly chores. She also would have to make his bed tonite just incase he came back late. That nite when she went to bed, she prayed for her brother. The next morning she felt something heavy around her neck. She looked, was she seeing things? No she wasn't, it was a beautiful necklace. She looked at her brothers bed. Yes he had come back."

4) The method of analysis of free writing should be extended to include an objective measure of what is said, as well as how it is said. The present study has not included any measure of the quality of content of the story. There is certainly no clear relationship between length, or even the number of errors, and the quality of what is said. The printed instructions suggested several things which should be included in the story, and then instructed the child to include any other things which he wished. A large number merely responded to each of the suggestions and then included nothing else, or merely wrote a number of unrelated sentences.

DAKOTA DAY SCHOOL
GRADE 4

179 words
11 errors per 60 words

The man where going hunting. They try to bring a animal. The girl do not know where he going. The house is in a man. The man goes to the monetin. There are two little trees. too. The little house is round. The man ride on a hares. The big house has a sgure door. The Man is goes hunting woods. The dog run fast in the hills. There were many trees in creek. The man has a Gun. too. The hous has woods at top. The father is going hunting. I think he will return back soon. I saw a flower this morening. I came to school in a bus. I see a coyot this morning. too. I play basketball this afternoon. I sow a train at Kadto. I saw a boy fall of from the swing. I have seen a bird this morning. I saw a hares in a pasture. I saw the cows in a pasture, too. and I saw a horesback yesterday. too. I am is forth grade this year. I saw a goat in the farm."

PUEBLO DAY SCHOOL
GRADE 4

141 words
11 errors per 60 words

" Mother said, "We have nothing to eat.' Sally go tell father to hunt for rabbits.' So father got up on the horse and went off. Sally yelled, bring me a baby rabbit to play with Mother said, I'm going to make bread for supper too. Sally said, I want to help you make bread. Mother said 'are you strong enough to chop wood for me so that I can build afire?" Where is the ax inquired Sally. I think father would return now. I am as hungry as a wolf. There he come shouted. He got fox, rabbits, squirrel, and a baby rabbit for me. Is your horse tried inquired Sally. "I think so," answered her father. Where is Sniff our dog? He went back to the forest by himself. I think he smell something good that why."

5) Some study should be made of the teaching techniques which are related to the achievement or failure to achieve this ability to write an interesting composition, and in relation to the ability to employ language tools which make it possible for the reader to get the author's meaning easily. Neither good nor poor writing appeared in any single area. Within any one school within an area individual differences, such as one might find on any standardized test, appeared. A few relationships have been studied this year, but some of the questions raised by members of the group because they affect their teaching plans, remain unanswered. Does one develop the ability to write fluently as a result of a curriculum which stimulates creative thinking? Is it necessary to teach formal grammar in order to eliminate the most common errors? By what approach does a child learn to express himself as effectively as have the following?

ALASKA PUBLIC
GRADE 4

A little Indian boy

Once there was a little boy named Black Moon. Block Moon was a little Indian child. One day he was playing with a pet waive he had caught when his father came up to him and said. Do you want to go hunting with me Black Moon? Black Moon was so excited that he forgot about his pet. And left it out of its cage. Why'll Black Moon was gone. The little pet crept away why'll Blackmoon was hunting with his father. Blockmoon said he would be back in a couple of hours. Two hours post and Black Moon and his father came riding up the trail with a bear. Blackmoon saw his pet Wolve gone and began to cry. And I think I will end my story wright here.

ALASKA NON-RESERVATION
GRADE 12

" Goodbye Father," shouted Three Fingers as she waved to the rapidly receding figure of Slow Talker. Blanket roll and rifle told that Slow Talker was going on along hunt, probably for several moons.

He would meet the other braves of the village at their appointed meeting place, the fork in the road.

Over the mountain ridges they would climb, and possibly on to the plains beyond in their hunt for deer and buffalo.

Presently they would return to their waiting womenfolk and children with food enough to last for some time.

OKLAHOMA NON-RESERVATION
GRADE 12

The Indian father, who is concerned with providing his family with food, is mounted on his horse with his rifle and other equipment near at hand.

He gazes in the direction of the mountains and the forest where game is plentiful.

His daughter bids him goodbye, and as he rides away, she says, "Hurry home, father."

As he rides along, he thinks of his happy family, the children who depend upon him for food.

He notices the clear sky, the cook look of the forest ahead and silently gives his thanks to the Lord.

His thoughts turn back to the present situation, and he sees himself greeted by a chorus of cheerful cries. He has been gone three days and before sunset, he has killed four deer and a few other game.

PACIFIC NON-RESERVATION
GRADE 12

The hot brilliant sun is climbing higher and higher, in the blue, cloudless skies, pouring its bright golden light down upon a small Indian village somewhere in the Western part of the United States. This new day is of great importance to the tribe of this little village, as on the following morn a great celebration is to be held, in the honor of the pretty dark-eyed-maiden Little Red Wing, the daughter of Chief Thunder Cloud, the tribe's beloved leader. On this day; however, everyone from the young to the aged are all excited and wide awake, as preparations are to be made for the coming dawn. There are clothes to be washed, food to be cooked and most important of all food to be sought and gathered from the purple colored mountains, which lie to the north of the village.

In the home of Little Flower and her father Big Bear, everyone was astir since the break of down, like everyone else. Early that morning it was planned that Big Bear was to go to hills in search of deer and rabbits, and Little Flower was to get the house in order. As soon as her father was amount his shaggy poney ready to start on his hunting trip, Little Flower came excitedly running out of the house and paused by the doorless door, to wave good-By to her father and wishing him luck in bringing back meat, as she was thinking of how delicious it would taste on the following day, when it was cooked to a dark brown over an open fire.

SOUTH EAST DAY
GRADE 4

The man is going up the hill and work there until night. The dog is going up with him. The girl is going with him I think. The horse is carrying something but I don't know I see a little house and a big house where they live. It is a pretty horse and house. The door is like window. Inside the house I see a boy, but I didn't see a women. The dog is like something but I can't tell. The horse is not all skin, but it is not all fat. I see bones on the horse leg. The house didn't have any steps. The horse has long hair. The horse is looking at the side way and not look at where He walks. I see horse foots go pet-pat. The girl is wear a long socks. I didn't see a man shoes.

The teachers of each class had been made responsible for the completeness and accuracy of the data recorded on the Background information Sheet, which was designed to obtain data relative to the student's environment, background, training, etc. These sheets were returned promptly from the field to the Chicago Office and as they were returned the data was coded and transferred to master sheets illustrated by Figure D-1. This information was arranged in such a way that both the background data and test scores could be easily transferred to punch cards. The punching of cards and mechanical distribution of scores and data was then all done by the International Business Machines Corporation. The contract required the verification of each card, thus insuring a high degree of accuracy in the computations.