Interim Assessment Report Summary Sheet for 1999-2000

September 15, 2000

(complete one form for each course assessed)

NOTE: Please attach as an appendix relevant supporting information used to complete the Interim Assessment Report Summary Sheet, for archival purposes.

Section 1: General information

1. Course information

Department	Psychology
Assessment Coordinator	Linda Reinhardt
Course title	Psychology of Childhood and Adolescence
Course number	PSY 360
Number of credits	3

2. Department participation

Number of campuses on which the course is taught during the year * (for example, on 12 campuses in Fall plus on 6 in Spring = 18)	14
Number of campuses participating in the assessment during the year	6 (5 F + 1 S) +1 250
Total number of instructors teaching the course *	14
Number of instructors assessing the course	5 + 1
Total number of course (lecture) sections offered*	15
Number of course sections assessed	6 + 1 PSY250
Total number of students enrolled in the course from final grade report*	224
Number of students participating in the assessment	128 (including PSY 250)

*This data may be requested from your department chair

Comments:

I have not yet received the *’d information from the department chair, but will send it to the committee when it becomes available.

One instructor is teaching PSY 250, Lifespan Developmental Psychology. I have included the data from that class in this report.

Section 2: Changes made based on 1997-99 results

1. Procedural changes made after the 1997-99 cycle, either proposed in the Department's 1997-99 Final Assessment Report or made in the 1999-01 Department Plan.

for courses continuing to be assessed from 97-99, a change in…

____ breadth of knowledge outcomes or proficiencies assigned to courses

__X_ assessment methods

other changes

____ change in course(s) assessed

__X_ data collection methods

____ reporting

Please explain the changes made.

Department members were not satisfied that the methods we used for assessment in 1997-99 were very useful to us because few members participated and we were not satisfied that the methods we used really assessed students’ learning.

In an effort to improve our assessment exercise on both points we changed the objective questions we used to assess breadth of knowledge, and we developed an essay question that we thought would enable us to assess the students’ proficiencies.

Both types of assessment items were given to students on the last day of class or on the final exam.

2. Changes in the instructional process since the 1997-99 cycle, either proposed in the Final Report or included in the Department Plan.

Department members share proven and/or promising exercises and supplemental instructional materials. They have also developed and shared additional classroom exercises and examples that specifically target those content areas and proficiencies where are students appear to be having difficulty.

3. Is there followup evidence to indicate that the instructional changes in 2) above improved student learning?

This is difficult for us to evaluate since we are still working on producing reliable and valid assessment instruments. It is certainly something that we all work toward.

Section 3: Breadth of Knowledge (Content) Outcomes, results 1 to 4

1. Please list the breadth of knowledge (content) outcomes assessed:

Number	Outcome
1	Understand the quasi-experimental research designs used by developmental psychologists (e.g., cross-sectional, longitudinal, sequential).
2	Understand the major theories of development, including theories of social, cognitive, and language development.
3	Understand the nature/nurture controversy and how it applies to different types of development
4

(use the next page for outcomes 5-8, if needed)

2. How frequently were each of the following assessment methods used? Put the number of times each method was used in the boxes provided. If you do not have this information by individual outcome, use the first column only.

Not separated by outcome: Outcome number:

Method	1	2	3	4
Pre and post test (all instructors used the same test)	3	3	3
Pre and post test (instructors used individual tests)
Post test only	4	4	4
Written test/ final exam questions (other than pre/ post test)
in which all instructors used the same questions
in which instructors selected from a common pool
in which instructors created their own questions
Oral test /exam questions (excluding pre or post test)
Performance activity (speech, acting, playing instrument)
Lab reports
Homework problems
Portfolio contribution
Essay or critique
Research paper
Other (please elaborate below)

3. Results in numbers, (not in percentages): (NB: Read comments in Section 3, #4)

Outcome number:

Number of responses which	1	2	3	4	Total 1-4
Exceed expectations
Meet or exceed expectations	93	69	99
Do not meet expectations	35	59	29

Section 3: Breadth of Knowledge (Content) Outcomes, results 5 to 8

1. Please list the breadth of knowledge (content) outcomes assessed:

Number	Outcome
5
6
7
8

(create additional pages, if needed)

2. How frequently were each of the following assessment methods used? Put the number of times each method was used in the boxes provided. If you do not have this information by individual outcome, use the first column only.

Not separated by outcome: Outcome number:

Method	5	6	7	8
Pre and post test (all instructors used the same test)
Pre and post test (instructors used individual tests)
Post test only
Written test/final exam questions (other than pre/ post test)
in which all instructors used the same questions
in which instructors selected from a common pool
in which instructors created their own questions
Oral test /exam questions (excluding pre or post test)
Performance activity (speech, acting, playing instrument)
Lab reports
Homework problems
Portfolio contribution
Essay or critique
Research paper
Other (please elaborate below)

3. Results in numbers, (not in percentages):

Outcome number:

Number of responses which	5	6	7	8	Total 5-8
Exceed expectations
Meet expectations
Do not meet expectations

Section 3: Breadth of Knowledge (Content) Outcomes, comments and use

4. Comments on results (including any pre-test/post-test comparisons)

The data the Psychology department collected on breadth of knowledge doesn’t readily fit into this format of the table in Section 2. We collected the number/percent of students who answered our questions correctly as we felt that is the information most informative to us.

I can report that the mean accuracy of students on six the true-false items that assessed each of the content outcomes were:

1) Understand the quasi-experimental research designs used by developmental psychologists (e.g., cross-sectional, longitudinal, sequential): 73% (Range: 52%-91%)

2) Understand the major theories of development, including theories of social, cognitive, and language development: 54% (Range: 29%-82%)

3) Understand the nature/nurture controversy and how it applies to different types of development: 77% (Range: 52%-96%)

I can only fill in crude estimates of the number of students who exceeded, met, or did not meet expectations into the table in Section 2. Rather arbitrarily, I will use the average percent accuracy of 73%, 54%, and 77% to represent the percentage of students who met or exceeded our expectations, and the reciprocal of those percents as the percentage of students who did not meet our expectations.

In comparing pretest-posttest results, we can readily see the disadvantages of true-false questions in which a student has a 50% chance of guessing the correct answer. On some items there was little difference in the number of students who answered correctly on the pretest and posttest, but on other items there was a considerable change from the pretest to post test scores

The item analysis for the 18 items is included in the Appendix.

5. What comments did individual instructors make about how they plan to use their results to improve the instructional process for 2000-2001?

In retrospect, we can see that there are problems in the assessment measures we used in that they may not validly assess what we ALL emphasize in our classes. Students tended to do well on the particular items that instructors emphasize in their classes, but did not do as well on topics that instructors do not emphasize. We need to be much more careful in the construction of our assessment instrument and make sure our assessment instrument assesses students’ knowledge of a topic at the level of specificity that we are all comfortable with. The assessment process is much more likely to improve teaching/learning if it is a reliable and valid measure of what is actually being taught.

Once we come up with a satisfactory instrument, we will be able to gauge changes and improvements in what students learn from one year to the next.

6. When these results are shared at the Fall Department meetings, please record how the department will use them to improve the instructional process for 2000-2001, so that you can include these plans in your final version of the Interim Report. For this August 1 draft, list any suggestions you plan to make to your department at the Fall meeting.

During our Fall department meeting, we discussed the results of the assessment. As psychologists, we are very much concerned that we develop methods of assessment that are reliable and valid tests. We decided that the assessment instrument we used this past year did not produce data that we could readily interpret. We had a lengthy discussion of what we could do to improve the assessment.

A subcommittee of the department worked on developing a better assessment instrument, and the department adapted it at the Fall meeting. Although it is not completed, our new assessment instrument will be based on a set of multiple choice items that we believe will do a better job of assessing both the breadth of knowledge outcomes AND the proficiencies for this course than we have used before. The assessment will be embedded in the final exam of the course so that we can be confident that students will be motivated to perform well.

With this improvement, we believe we will be better able to determine what the students have learned about developmental psychology by the end of the course. This will help us determine how effective our instructional methods have been, and should indicate whether we need to make changes in our teaching methods to improve student learning.

We plan to correlate students’ performance on our assessment instrument with the grades they earn in the course. Students who have earned As and Bs in our course should be the ones who we judge to have “met or exceeded our expectations” on the outcomes measured by our assessment instrument.

Section 4: Proficiencies, list and results

1, 3. Enter the number of responses, (not the percentages) who:

No	for the proficiency…	Exceed expectations	Meet expectations	Do not meet expectations
I A1	Analyze, synthesize, evaluate and interpret information and ideas	13	43	10
I A2	Construct and support hypotheses and arguments
I A3	Distinguish knowledge, values, beliefs, and opinions	13	43	10
I A4	Select and apply scientific and other appropriate methodologies
I B1	Solve quantitative and mathematical problems
I B2	Interpret graphs, tables, and diagrams
I B3	Use statistics appropriately and accurately
I C	Integrate knowledge and experience to arrive at creative solutions
I D1	Evaluate situations of social responsibility	13	43	10
I D2	Make decisions based on an informed understanding of the moral and ethical issues involved
I E	Articulate accurately strengths and weaknesses of one's own work
II A1	Read and listen with comprehension and critical perception	13	43	10
II A2	Recognize fallacies and inconsistencies
II A3	Respond to the media actively and analytically
II B1	Write clearly, precisely, and in a well-organized manner
No	for the proficiency…	Exceed expectations	Meet expectations	Do not meet expectations
II B2	Develop a large and varied vocabulary	13	43	10
II B3	Recognize and use a variety of written communication forms and styles
II C1	Transmit information effectively through skillful speech delivery
II C2	Respond orally to questions and challenges
II C3	Recognize and use a variety of oral communication forms and styles
II D1	Work collaboratively as part of a team
II D2	Understand and communicate with people different from themselves
II E1	Gather information from printed sources, electronic sources, and observation
II E2	Use computer technologies for communication and problem-solving
II E3	Learn independently, stimulating and satisfying intellectual curiosity
IIIA1	Employ and expand the imagination
IIIA2	Engage in creative expression
IIIB	Respond to the natural world and creative expression with knowledge and sensitivity

Section 4: Proficiencies, methods

Please report using either the original proficiencies on this page or using the newer, fewer proficiency classes on the next page.

2. Frequency of methods used to assess proficiencies Indicate number of times used.

IA1 Analyze	3
IA2 Construct
IA3 Distinguish	3
IA4 Select
IB1 Solve
IB2 Interpret
IB3 Use stats
IC Integrate
ID1 Evaluate	3
ID2 Make
IE Articulate
IIA1 Read, listen	3
IIA2 Recognize
IIA3 Respond
IIB1 Write clearly
IIB2 Develop	3
IIB3 Recognize
IIC1 Transmit
IIC2 Respond
IIC3 Recognize
IID1 Work
IID2 Understand
IIE1 Gather
IIE2 Use
IIE3 Learn
IIIA1 Employ
IIIA2 Engage
IIIB Respond

Section 4: Proficiencies, methods

Please report using either the original proficiencies on the previous page or using the newer, fewer proficiency classes below.

2. Frequency of methods used to assess proficiencies Indicate number of times used.

IAs

IBs

ICs

IDs

IEs

IIAs

IIBs

IICs

IIDs

IIEs

IIIAs

IIIBs