First Year Experience

Results of Assessment

2007-2008 (submitted 07/31/08)

FIRST YEAR EXPERIENCE PROGRAM
2007-2008 ASSESSMENT REPORT

2007-2008 RESPONSES TO 2006-2007 ASSESSMENT FINDINGS

In the 2006-2007 FYE assessment report, First Year Experience (FYE) indicated it would accomplish the following to be reported out in the 2007-2008 assessment report: (1) For the purpose of continuous improvement of FYE programs and services, inquire into the evaluated areas in which FYE did not meet the Department’s standard of excellence. (2) Start a research-based, retention-focused, student needs assessment that may result in a revision to the content of “Introduction to the University.” Our progress on these two goals is noted below.

Eliminated Course Deficiencies

Fall 2006 “Introduction to the University” (EDUC 1105) student evaluations [See Table 1] indicated that students’ responses to the following statements almost met FYE’s standard of excellence: I have an increased ability to identify a major and a career of interest (3.9), I have an increased ability to employ effective academic skills such as note taking, textbook reading, test taking, and time management (3.9), and I feel this course was worthwhile overall (3.8). Multiple choice answer options were formatted in a 5-point Likert scale ranging from least favorable rating (1) to most favorable rating (5), and achievement of FYE’s standard of excellence was defined as a mean rating of at least a 4.0. FYE instructors were informed of these student perceptions and the topics of the seven subsequent FYE instructor workshops focused on the content of some of these lower rated areas (e.g., study skills assessment, annotation, and the PORPE study technique).

Table 1: Fall 2007 FYE Student Evaluation Comparisons

Statement

2006
Mean*

2007
Mean**

Difference

Instructor…

clearly explained the course goals and requirements in a syllabus

4.4

4.6

+0.2

utilized class time effectively

4.3

4.5

+0.2

used class activities to help fulfill the course objectives outlined in the syllabus

4.4

4.6

+0.2

assigned pertinent course work

4.2

4.4

+0.2

supplied helpful feedback on work

4.2

4.4

+0.2

was enthusiastic about the course

4.5

4.6

+0.1

demonstrated professionalism

4.5

4.6

+0.1

Encouraged participation in class

4.4

4.5

+0.1

was accessible outside of class time

4.3

4.5

+0.2

Having taken “Introduction to the University”…

I have an increased ability to use a variety of campus resources such as the library, advisement center, testing center, and tutoring services

4.2

4.0

- 0.2

I feel this course was worthwhile overall

3.8

4.3

+0.5

I have an increased ability to identify a major and a career of interest

3.9

4.3

+0.4

Increased ability to employ effective academic skills such as note taking, textbook reading, test taking, and time management

3.9

4.2

+0.3

*“n” was 198
**”n” was 222

Fall 2007 evaluations indicated significant improvements to these three means (from 3.9 to 4.3, from 3.9 to 4.2, and from 3.8 to 4.3, respectively), surpassing the Department’s standard of excellence and netting them the highest mean increases in the instrument’s findings [See Table 1]. Although other factors may have triggered the increases, we are hopeful that sharing course feedback with instructors and providing instructor workshops on these topics helped improve evaluations in these areas.

Revised FYE Module Evaluations

In response to last year’s FYE assessment conclusions, three revisions were made to the evaluation of FYE Modules. First, after consulting our liaisons in the Library and Career Services, FYE implemented some minor wording changes to the evaluation instruments that were administered this assessment year. Second, starting Fall 2008 FYE will use a pre/post evaluation model for all three FYE modules (i.e., Advising, Library and Career Services); we have successfully used this model for the Advising Module for some time. Third, in Fall 2008 FYE will administer the evaluations of all three modules itself. More details on revisions to the FYE modules based on the 2007-2008 assessment follow on pages 4 and 5.

Conducted FYE Alumni Focus Group

In response to last year’s FYE assessment conclusions, the Department conducted a focus group to solicit feedback on “Introduction to the University” from FYE alumni who were juniors and seniors. In the past, FYE conducted focus groups consisting of students who completed “Introduction to the University” the previous year. The Department chose FYE alumni who were upperclassmen for this focus group to get a more inclusive sense of how prior FYE students perceive the course has helped them in their studies. By phone, FYE invited 67 WSU students to participate in the focus group. Being that it is difficult to get sufficient attendance at our focus groups, a full lunch and drawings for bookstore gift certificates were advertised incentives. Five of the 10 who RSVP’d attended the focus group held on April 9, 2008.

The questions asked were: What stands out in your memory about your “Introduction to University” class? How did FYE help you transition from high school to college? How did FYE help you succeed in your other WSU courses? Which five class topics did you think were most helpful? Which topics were least helpful? What specific things that you learned in FYE do you still use? What could have been done in your FYE class to help you succeed in college? What did you learn in FYE that you didn’t appreciate at the time? If you were to organize an FYE focus group, what would you ask the students?

Being that only five attended the focus group and their question responses varied significantly, conclusions cannot be drawn from the findings other than that alumni who attended the focus group tended to have positive, yet mixed, feelings about their FYE experience. Due to the delayed hiring of a new FYE Program Coordinator, the needs assessment beyond the FYE alumni focus group has been postponed; future plans include soliciting program improvement input from more former FYE students, FYE instructors, FYE cluster faculty, faculty who teach other WSU courses traditionally taken by first-year students, etc.

ADDITIONAL 2007-2008 ASSESSMENT FINDINGS

In addition to the aforementioned assessment-related efforts FYE committed to in the 2006-2007 assessment report, we report the following additional assessment-related work:

INSTRUCTOR TRAINING

This past year’s FYE-offered, instructor training opportunities focused on helping instructors be best prepared to support first-year students’ success. Topics were selected to avoid redundancy with initiatives of the Teaching and Learning Forum and the Retention and Persistence to Graduation Committee’s subcommittee working on developmental course instructor training.

To promote the professional development of FYE instructors, FYE conducted Fall and Spring Kick-offs [See Table 2] and offered a workshop series [See Table 3]. To be most responsive to instructor needs, these professional development opportunities were evaluated based on quality and content of the presentations. The feedback on the contents and quality of the presentations met or surpassed the Department’s standard of excellence and attendance was deemed acceptable. Future training topics will continue to be based on FYE instructor, student, and cluster faculty evaluations of the FYE programs and its services.

Table 2: 2007-2008 FYE Instructor Training Workshops

Training

Presenter

Fall Kick-Off (8/23)
Keep Them from Falling Between the Cracks: Mastery Learning in the College Classroom
Module Presentations
Library
Academic Advising
Service Learning
Career Services


Dr. Fran Butler, Associate Professor, Teacher Education



Fran Zedney, Coordinator, Reference/Instruction Coordinator, Library
Ashley Owens, Associate Director, Academic Advisement Center
Dr. Kari Petersen, Coordinator, Community Service Programs
Dr. Richard Lambert, Career Counselor/Employment Advisor, Career Services

Spring Kick-Off (12/18)
Creating a Collaborative Learning Community for Weber State Developmental Students
Module Presentations
Library
Academic Advising
Service Learning
Career Services


Eric Kohler, Tutoring Supervisor, Developmental Education Learning Center



Fran Zedney, Coordinator, Reference/Instruction Coordinator, Library
Ashley Owens, Associate Director, Academic Advisement Center
Dr. Brenda Kowalewski, Director, Civic Engagement
Dr. Richard Lambert, Career Counselor/Employment Advisor, Career Services

Table 3 : 2007-2008 FYE Instructor Training Workshops

Date

Trainings and Meetings

Presenter

9/11

Study Skills Assessment

Dr. Louise Moulding, Assistant Professor, Teacher Education

9/25

Tips for Best Teaching Developmental Students in a First-Year Course

Dr. Kathleen Lukken, Instructor of Remedial Education

10/9

Annotation Workshop

Dr. Bruce Bowen, Associate Provost, Enrollment Services

10/23

Understanding Understanding

Dr. Peggy Saunders Assistant Professor, Teacher Education

11/13

Tips on Lecturing to First-Year College Students

Dr. Claudia Eliason, Associate Professor, Teacher Education

1/15

Creating Effective Rubrics

Dr. Louise Moulding, Assistant Professor, Teacher Education

3/18

PORPE (Predict, Organize, Rehearse, Practice and Evaluate) Study Technique

Dr. Bruce Bowen, Associate Provost, Enrollment Services

With the Teaching and Learning Forum and the Retention and Persistence to Graduation Committee’s subcommittee working on developmental course instructor training, FYE sponsored a workshop conducted by nationally-recognized brain-based research expert Dr. Rita Smilkstein. FYE instructors were strongly encouraged by the department to attend Dr. Smilkstein’s workshop, “We’re Born to Learn: Creating Eager, Motivated, and Successful Learners,” that was well-received. Lastly, FYE funded the attendance of interested EDUC 1105 instructors and peer mentors to this year’s National Academic Advising Association’s (NACADA) Region X Conference and Utah College Reading & Learning Association Conference.

STUDENT EVALUATIONS OF “INTRODUCTION TO THE UNIVERSITY”

For Fall 2007, student evaluations of “Introduction to the University” instruction indicated that FYE met or exceeded its standard of excellence in all of the 13 evaluated areas [see Table 3]. Answer choices were formatted in a 5-point Likert scale ranging from least favorable rating (1) to most favorable rating (5), and achievement of FYE’s standard of excellence was defined as a mean rating of at least a 4.0.

When asked which course topics students thought would most help them be more successful students, the most frequent responses were time management (63), note taking (37), study strategies (31), campus resources (31), test taking (24), stress management (22), and academic advisement (19). When asked what one change they recommend for future EDUC 1105 classes, the most frequent responses were to provide more hands on activities (9), assign less busywork (5) and explain assignments in more detail (4). This feedback is being considered as we approach a significant review of the course. An explanation of the Fall 2007 improvements over the Fall 2006 evaluations was noted in the first section of this report.

Advising Module

For First Year Experience’s “Introduction to the University” course, Academic Advisement Center and WSU Davis Enrollment Services advisors conducted two-day modules for each course section. This FYE module is an extended version of the Academic Advisement Center’s General Studies Session.

In Fall 2007, 69% (208) of the 303 FYE students completed both a pre-module and post-module evaluation. Taking attendance is not required of instructors, so we cannot report a true response rate that is based on the number of students in attendance at the module. Also, students’ evaluations were only considered if they attended both module dates and completed both the pre-module and post-module portions of the evaluation. FYE is pleased that this response rate was again slightly higher than the previous year’s 67%.

FYE met or surpassed the Department standard of excellence in all areas evaluated in the Advising Module [See Table 4]. The highest pre-session/post-session increases were in how to navigate the AAC website (1.89 rating increase), how to declare a major (1.68 increase), and how to access web-based information on majors (1.56 increase). These findings were consistent with the Fall 2006 findings. One of the revisions to the evaluation was a separation of the statement “I know which Math and English classes I need to complete” into two statements to get a better idea of how much students think they understand about their math and English requirements, respectively.

Table 4: Fall 2007 Advising Module Evaluations

Knowledge Area

Pre-session
Mean

Post-session
Mean

Pre-Post
Difference

General education requirements

3.20

4.35

+1.15

How to navigate AAC website

2.47

4.36

+1.89

Importance of completing English and math early

3.60

4.64

+1.04

Math classes I need to complete

3.43

4.49

+1.06

English classes I need to complete

3.37

4.56

+1.23

How to build a course schedule

3.09

4.23

+1.14

How to register for classes

3.78

4.50

+0.72

How to officially drop/WD from class

3.31

4.51

+1.20

Impact of my GPA on my academic standing

3.60

4.52

+0.92

Access web-based information on majors

2.78

4.34

+1.56

How to declare a major

2.66

4.34

+1.68


Note: Only feedback from those who completed both the pre-session and post-session parts of the tool
was included.

Library Module

For FYE’s “Introduction to the University” course, the Stewart Library conducts a one-day module that was evaluated online through a library system. A member of the Library staff accidentally combined the Fall and Spring data sets so we can only report the combination of Fall and Spring data. The Library Module surpassed the FYE Department standard of excellence in 3 of the 6 evaluated areas [See Table 5]. The most frequent student comments were that the Library Module was very helpful (45) and the library staff members were excellent (17). The areas in which the standard of excellence was not met were students’ indication that they felt more comfortable asking for help from library staff (3.98), better understanding of the WSU Information Literacy Competency requirement (3.95), and the overall usefulness of the exercise (3.17).

FYE met with the Library Module Liaison in July 2008; we reviewed these Library Module findings and agreed to institute Fall 2008 improvements to the module content, revise the evaluation instrument items, implement the instrument as a pre/post tool, and have FYE administer the evaluation. In Fall, we will try a “clicker” technology that will be fun for the students who will use a computer mouse to select their responses to the evaluation questions. They will see the summary of their FYE section’s responses projected on a screen at the front of the room. To be assured there are no glitches in the administration of the Library Module evaluation, the FYE representative assigned to administer the evaluations will have hard copies of evaluation forms on hand.

Table 5: Library Module Evaluations
Fall 2007 and Spring 2008

Statement

1

2

3

4

5

Mean

I feel more comfortable asking for help from library staff.

5

4

24

89

44

3.98

I learned more about library services available to help students.

5

2

7

78

74

4.29

I am better able to use Academic Search Premier to find full text articles.

3

2

17

77

67

4.22

I have a better understanding of the WSU Information Literacy Competency requirement.

3

7

25

91

40

3.95

Overall, this was an effective session.

2

8

14

83

59

4.14

Overall, this was a useful exercise.

6

6

15

62

43

3.17

“n” was 166 (Fall and Spring combined)

Career Services Module

In Fall 2007, FYE received Career Services evaluation responses from 71% (214) of the 303 students enrolled in FYE. Taking attendance is not required of instructors, so we cannot report a true response rate that is based on the number of students in attendance at the module. Students were asked to rate the following: I have a better understanding of the services available through the Career Services Center. I have a better understanding of my interests and how they apply to decision-making. I have a better knowledge of whether I prefer to work with data, people, things or ideas. I am aware of how to use eDiscover to access online career resources. I now know that if I desire to further explore how to select a career and a major, that there is a Career Development class to help me. Overall, the exercises in this presentation were useful in my understanding of career exploration.

Due to protocol breaches in the administration of the evaluations, the findings will not be reported. The Career Services Liaison informed FYE that no significant revisions to the Fall 2008 module will be made. In early August, FYE will notify the Career Services Module Liaison of the Fall 2008 move of all FYE modules to a pre/post module evaluation format and to FYE’s administration of all FYE module evaluations. In August, FYE will submit a revised instrument for the Career Services Liaison’s review.

2007-2008 Mentor Evaluation of FYE

“Mentor Leadership Seminar” and “Introduction to the University” Instruction

Fall mentor evaluations of FYE instruction indicated that FYE met or exceeded its standard of excellence in all 18 evaluated areas [See Table 6].

When asked which topics in “Mentor Leadership Seminar” would help them be most successful as mentors, respondents mentioned Color Code (4), ethical conduct (2), Family Educational Rights and Privacy Act (FERPA), time management, critical thinking, professional development, and leadership topics. When asked what one change they recommend for future “Mentor Leadership Seminar” classes, respondents suggested more time to talk about mentor-related issues (3) and more focus on helping new mentors. All of this feedback was considered in the development of the Fall 2008 EDUC 3170 syllabus.

When asked which of their roles in “Introduction to the University” helped them mentor most effectively, mentors listed helping students (2), getting to know students, teaching a class session, and participation in class discussions. When asked what one change they recommend in the responsibilities required of a peer mentor, they suggested being able to help their EDUC 1105 instructor with more academic matters. FYE appreciates this feedback.

Table 6: Fall 2007 Mentor Evaluations of FYE Instructors and Courses

Statement

2006
Mean*

2007
Mean**

Difference

“Mentor Leadership Seminar” Instructor…

clearly explained the course goals and requirements in a syllabus

4.7

4.9

+0.2

utilized class time effectively

4.9

4.8

-0.1

used class activities to help fulfill the course objectives outlined in the syllabus

4.5

4.8

+0.3

Assigned pertinent coursework

4.8

4.8

0.0

supplied helpful feedback on my work

4.1

4.8

+0.7

was enthusiastic about the course

4.7

4.8

+0.1

demonstrated professionalism

5.0

4.9

-0.1

Encouraged participation in class

5.0

4.8

-0.2

was accessible outside of class time

4.5

4.9

+0.4

Having taken “Mentor Leadership Seminar,”

I feel this course was worthwhile overall

4.4

4.8

+0.4

I have an increased ability to communicate through interpersonal interaction, small group discussions, and large group presentations

4.7

4.8

+0.1

I have an increased understanding of my role as a peer mentor

4.5

4.9

+0.4

Participation in the “Introduction to the University” class has

helped students learn effective academic skills such as note taking, time management, textbook reading, and test taking

4.9

4.6

-0.3

taught student to demonstrate effective communication and interpersonal skills

4.9

4.7

-0.2

instilled in students a sense of belonging to the WSU community

4.6

4.6

0.0

contributed to the students’ knowledge of WSU departments and services

5.0

4.8

-0.2

assisted students in developing an overall sense of confidence in academic skills

4.4

4.7

+0.3

honed your leadership skills

4.9

4.6

-0.3

*“n” was 10
**”n” was 12

2007-2008 INSTRUCTOR Evaluations of FYE Department

Fall 2007 FYE instructors’ evaluations of the FYE Department indicate that FYE met or exceeded its standard of excellence in all areas [See Table 7]. When asked what the most positive aspect of teaching “Introduction to the University” has been, the most frequent responses were seeing a positive change in students both socially and academically (5), getting to know students individually (2), and working with excellent peer mentors (2). When asked in what specific ways the FYE Program can provide better support for future “Introduction to the University” teachers, suggestions were to continue offering quality training opportunities (3), offer more training opportunities for new instructors, focus more on teaching strategies and classroom management issues, have an online database of training resources, and further clarify objectives of the course. FYE has responded to this feedback by including a classroom management workshop in the Fall 2008 Kick-off, scheduling two best practices/activities workshops for 2008, conducting two one-on-one FYE teaching orientation meetings with each new WSU FYE instructor in Summer 2008, scheduling a summer 2008 new instructor group meeting to make sure new instructors have what they need to effectively start teaching their first WSU FYE course, recently expanding the resource library to almost 200 holdings, publishing an online bibliography with publishers’ descriptions of the resource library holdings, and planning to conduct a curriculum review to clarify the objectives of the course once a new FYE Coordinator is hired.

Table 7: Fall 2007 Instructor Evaluations of FYE

Statement

2006
Mean*

2007
Mean**

Difference

As a result of teaching “Introduction to the University” I feel I…

helped students learn effective academic skills such as note taking, time management, textbook reading, and test taking.

4.1

4.7

+0.6

taught students to demonstrate effective communications and interpersonal skills.

4.3

4.6

+0.3

instilled in students a sense of belonging to the WSU community.

4.3

4.4

+0.1

contributed to my students’ knowledge of WSU departments and services.

4.2

4.5

+0.3

assisted students in displaying an overall sense of confidence using academic skills.

4.3

4.3

0.0

increased my ability to adapt the teaching process to the needs of first year students.

4.1

4.3

+0.2

* “n” was 10
** “n” was 12

FYE ASSESSMENT CONCLUSIONS

The 2008-2009 assessment year will be one of significant transition for First Year Experience. By the time a new FYE Coordinator is hired, the department will have been under temporary part-time management for a year. A top assessment-impacting priority will be the hiring of a new FYE Coordinator who will drive the department’s assessment that will include a full curriculum review of “Introduction to the University.” Until the Coordinator is hired, the Department will (1) continue evaluating its courses and related services for the purpose of ongoing improvement, (2) continue offering instructor training opportunities to help EDUC 1105 instructors be best prepared to support first-year students’ success, (3) implement revisions to the content and evaluation of the FYE Library Module, and (4) implement evaluation format and evaluation administration revisions to FYE Career Service Module.


Weber State UniversityOgden, Utah 84408

Privacy PolicyTerms of UseNondiscrimination Policy