Response to 2005 Recommendation One

2005 Recommendations

Weber State University received two recommendations from the Northwest Commission in February 2005:

  1. The Commission recommends that the institution regularly and systematically assess the general education curriculum and document that assessment activities lead to improvement. Based on evidence from faculty, students, administration and the provided documentation, the Evaluation Committee was unable to verify regular and systematic assessment of the general education curriculum and that assessment activities have led to improvement (Standard 2.B and policy 2.2 Educational Assessment).
     
  2. The Commission recommends that clearly articulated policies and procedures for evaluating part-time and adjunct faculty be developed, distributed, implemented, and assessed university-wide. The Commission additionally recommends that improvement be made in the mentoring of part-time faculty, as well as in expanding their professional development opportunities. The Evaluation Committee was unable to find sufficient evidence to verify that the institution systematically, regularly, and across all units fulfills its “obligation...to evaluate the performance” of part-time and adjunct faculty members; and was also unable to find evidence that the University provides for their development on a regular basis (Standard 4.A and Policy 4.1 – Faculty Evaluation).

Response to 2005 Recommendation 1

Weber State University (WSU) has responded seriously to the charge to “regularly and systematically assess general education.” The assessment has provided information that the university has used to alter program design, to improve instructional technology and to allocate resources to create greater emphasis on particular general education areas.

I. Structure of Weber State University General Education

Prior to the 2004 Comprehensive Visit, WSU established a Faculty Senate Committee for General Education Assessment and Improvement. The committee’s initial charge was “to create a clearly articulated, campus-wide collective conception of the qualities of a college educated person through a series of campus-wide conversations.” The campus surveys and conversations led to the adoption in 2006 of the General Education Mission Statement which includes eight general student learning outcomes:

General education at Weber State University provides students with a foundation in the arts and sciences that transcends and complements their academic emphases. This exposure to diverse fields of study enables students to make intellectually honest, ethical decisions that reflect a knowledge of and respect for diverse people, ideas and cultures. Such breadth of education also cultivates skills critical to student success in academic, personal, professional and community endeavors both within and beyond the university. Students completing the general education program can:

    1. Communicate, understand and interpret ideas and information using written, oral and visual media.
    2. Think critically and creatively to construct well-reasoned arguments supported by documented research.
    3. Use quantitative, mathematical relationships, operations and reasoning.
    4. Demonstrate an understanding of the history, foundational principles, economics and politics of the United States.
    5. Demonstrate proficiency in computer and information literacy.
    6. Demonstrate an understanding of how the biological and physical sciences describe and explain the natural world.
    7. Demonstrate an understanding of humans, their behavior and their interaction with and within their physical, social, local and global environments.
    8. Demonstrate an understanding of diverse forms of aesthetic and intellectual expression.

Methods of Assessment and Data Points for Gathering Evidence
WSU currently uses a combination of four nationally standardized tests, four local measures and six secondary indicators to assess the general education learning outcomes.

The nationally-normed tests that are used as direct measures or secondary indicators of student learning include:

  1. The Collegiate Learning Assessment (CLA)--a direct measure of students’ critical thinking, communication, understanding of statistical data and appreciation of diverse viewpoints (This instrument provides primary indicators for WSU Learning Outcomes 1, 2, 3, 4, 7 & 8.)--has been administered on the campus three times since 2006. With our baseline data in place, we plan to repeat the exam every three years beginning in 2010. (see Appendix I)
  2. The National Survey of Student Engagement (NSSE)--a secondary measure of students’ understanding of civic systems in the United States and diverse cultures and expressions and a direct measure of students’ participation in oral and written expression, community service and student scholarship, including research (The NSSE is used as a primary indicator for WSU Learning Outcomes 1,2,4, 7&8)--has been administered four times since 2000 and is repeated at least every three years with the next planned administration in 2012. (see Appendix II)
  3. The Noel-Levitz Student Satisfaction Survey (Noel-Levitz)--which allows a direct measure of students’ perceptions regarding their learning and the academic and administrative systems that impede or support that learning (These surveys provide secondary indicators for all WSU Learning Outcomes.)--has been given five times since 1997 and will be repeated every three years with the next administration in 2011. (see Appendix III)
  4. The Higher Education Research Institute Faculty Survey (HERI)--which directly measures the academic practices of faculty (secondary indicators for all WSU Learning Outcomes) and provides an important baseline for changes in programs or pedagogical practices--has been administered once in 2006 and will be repeated every three years with the next administration scheduled for this fall.

WSU has now gathered sufficient data from these nationally-normed assessment tools to provide direction for program changes and resource decisions and to provide a baseline against which the locally developed measures can be normed.

Locally-developed assessments include:

  1. Review of educational portfolios. Several programs, including Art, Teacher Education, the Community Involvement Center, the Bachelor of Integrated Studies and the Honor’s Program, use portfolios either by themselves or in conjunction with capstone courses to assess general education outcomes. The Faculty General Education Improvement and Assessment Committee is facilitating the broader use of electronic portfolios, following up on an earlier pilot project which suggested that such portfolios showed promise as a basis for cross-discipline general education assessment. (These provide primary and secondary indicators for all WSU General Education Learning Outcomes.) Each of the programs that use portfolios report general education outcomes as part of their annual assessment report. These general education outcomes are currently normed and reported from within the disciplines, however there are efforts led by the General Education Improvement and Assessment Committee to establish shared rubrics that would allow for cross-disciplinary norming. (Examples of portfolios are found in Appendix IV.)
  2. Capstone courses. Approximately eighty percent of all WSU students participate in a capstone class or project. (These provide primary and secondary indicators for all WSU General Education Learning Outcomes.) General education attainment is assessed in each of these summative settings and is reported as part of each college’s annual assessment report. As with the portfolio data, these general education outcomes are presently normed within the disciplines. To allow the data from the portfolios and capstone courses to be more easily compared, the Faculty General Education Improvement and Assessment Committee is working on developing a set of shared rubrics for each of the general education learning outcomes. (Examples of Capstone Assessments are found in Appendix V.)
  3. Student artifacts with grading rubrics. By using representative student artifacts, the grading rubrics for those artifacts that reflect general education outcomes and the associated grades, we are able to infer attainment of general education competencies from grades in general education and major courses. (Examples of Rubrics and Student Artifacts are found in Appendix VI.)
  4. Graduating student outcomes surveys. We have administered a general survey to all graduating seniors. In addition, many departments use graduation interviews and surveys as a way of assessing summative attainment of disciplinary and general education learning outcomes. (Examples of Graduating Student Assessments are found in Appendix VII.)
  5. Targeted locally developed surveys. For example, the Community Involvement Center (CIC) administers a survey to students enrolled in community-based learning courses asking students for feedback in five general areas: a) perception of service-learning; b) evaluation of the service-learning course; c) attitude toward community involvement; d) influence of service on major or profession; and e) personal reflections on service. Other campus areas have similar surveys that elicit information specific to their program. (Examples of targeted surveys are found in Appendix VIII.)

Other data that provides both secondary evidence of general education attainment includes:

  1. Scores on nationally standardized exams that reflect general education proficiency. Such exams are associated with most of the programs in the College of Health Professions and in many of the programs within the College of Applied Science and Technology. These include:
    • CLT/MLT (medical laboratory technician) and CLS/MT (medical technologist) and Board of Registry exams by the American Society for Clinical Pathology
    • American Dental Association Nurse Hygiene Board exam
    • National Registry EMT and Paramedic exams
    • American Registry of Radiologic Technologists Certification exam, American Registry of Diagnostic Medical Sonographers exam, Nuclear Medicine Technology Certification Board exam, ISCD - International Society of Clinical Densitometry exam, Certification Board for Radiology Practitioner Assistants exam
    • Registered Health Information Technology exam and the Registered Health Information Administrator exam
    • National Board for Respiratory Care Certified Respiratory Therapist exam and the Registry Examination for Advanced Respiratory Therapists exam.
    • National Council Licensure Examination for Registered Nurses and National Council Licensure Examination for Practical Nurses
    • Certified computer technician exams including A+, MCSA and MCSE
    • Automotive technician exams given by the National Association for Automotive Service Excellence.
  2. Each department tracks discipline-based outcomes and general education outcomes. These results are reflected in each department’s annual assessment report. These reports are analyzed and representative samples are placed on a shared assessment website. http://www.weber.edu/Assessment/reports_data.html (Additional Assessment Reports are available in the Accreditation Resource Room.)
  3. Rates of participation in, and quality of, undergraduate research, which reflect successful integration of general education skills and discipline-based knowledge. (Examples of Undergraduate Research are found in Appendix IX.)
  4. Participation rates in community service or other civic engagement, which are indicators of understanding of civics and diverse cultures.
  5. Graduation rates, which reflect successful application of general education skills in upper division major classes.

The following chart indicates the sources used to assess each general education learning outcome.

WSU General Education Learning Outcomes

Primary Indicators

Secondary Indicators

1. Communicate, understand and interpret ideas and information using written, oral and visual media.

CLA, Portfolios, Capstone Classes, National Certifying Exams, Graduating Student Exams Grades in Required Composition and Communications Classes

NSSE, Noel-Levitz, HERI, GSOS, Portfolios, Capstone Classes, National Certifying Exams, Grades, Graduation Rates, Placement Rates, Undergraduate Research, Civic Engagement Surveys

2. Think critically and creatively to construct well-reasoned arguments supported by documented research.

CLA, Portfolios, Capstone Classes, National Certifying Exams, Graduating Student Exams Grades in Required Composition, Mathematics, Social Sciences, American Institutions and Communications Classes

NSSE, Noel-Levitz, HERI, GSOS, Portfolios, Capstone Classes, National Certifying Exams, Grades, Graduation Rates, Placement Rates, Undergraduate Research, Civic Engagement Surveys

3. Use quantitative, mathematical relationships, operations and reasoning.

CLA, Portfolios, Capstone Classes, National Certifying Exams, Graduating Student Exams Grades in Required Mathematics and Computer Literacy Classes

NSSE, Noel-Levitz, HERI, GSOS, Portfolios, Capstone Classes, National Certifying Exams, Grades, Graduation Rates, Placement Rates, Undergraduate Research, Civic Engagement Surveys

4. Demonstrate an understanding of the history, foundational principles, economics and politics of the United States.

CLA, Portfolios, Capstone Classes, Graduating Student Exams, National Certifying Exams, Graduating Student Exams Grades in Required Composition, Mathematics, American Institutions, Social Science and Communications Classes

NSSE, Noel-Levitz, HERI, GSOS, Portfolios, Capstone Classes, National Certifying Exams, Grades, Graduation Rates, Placement Rates, Undergraduate Research, Civic Engagement Surveys

5. Demonstrate proficiency in computer and information literacy.

Portfolios, Capstone Classes, National Certifying Exams, Graduating Student Exams Grades in Required Mathematics and Computer and Information Literacy Classes

NSSE, Noel-Levitz, HERI, GSOS, Portfolios, Capstone Classes, National Certifying Exams, Grades, Graduation Rates, Placement Rates, Undergraduate Research, Civic Engagement, Surveys

6. Demonstrate an understanding of how the biological and physical sciences describe and explain the natural world.

Portfolios, Capstone Classes, National Certifying Exams, Graduating Student Exams Grades in Required Mathematics, Computer and Information Literacy, Physical Science and Life Science Classes

NSSE, Noel-Levitz, HERI, GSOS, Portfolios, Capstone Classes, National Certifying Exams, Grades, Graduation Rates, Placement Rates, Undergraduate Research, Civic Engagement Surveys

7. Demonstrate an understanding of humans, their behavior and their interaction with and within their physical, social, local and global environments.

CLA, Portfolios, Capstone Classes, National Certifying Exams, Graduating Student Exams Grades in Required American Institutions, Social Science, Humanities, Creative Arts and Communications Classes

NSSE, Noel-Levitz, HERI, GSOS, Portfolios, Capstone Classes, National Certifying Exams, Grades, Graduation Rates, Placement Rates, Undergraduate Research, Civic Engagement Surveys

8. Demonstrate an understanding of diverse forms of aesthetic and intellectual expression.

Portfolios, Capstone Classes, National Certifying Exams, Graduating Student Exams Grades in Required American Institutions, Social Science, Humanities, Creative Arts and Communications Classes

CLA, NSSE, Noel-Levitz, HERI, GSOS, Portfolios, Capstone Classes, National Certifying Exams, Grades, Graduation Rates, Placement Rates, Undergraduate Research, Civic Engagement Surveys

WSU also tracks pedagogy relating to general education. This is important not only as a secondary indicator of learning, but also as a base point that catalogs institutional practices that may need to be changed to improve student learning.

Pedagogy Relating to General Education

Primary Indicators

Secondary Indicators

Course content

Review of syllabi

CLA, Capstone Classes, Local Surveys, National Certifying Exams, Grades

Co-curricular experiences

NSSE, GSOS, Noel Levitz

CIC Survey, Local Surveys

II. Data Collection and Analysis

The general education data is collected and analyzed by a number of campus areas. The national tests such as the CLA and the NSSE, and locally developed graduation and placement research, are coordinated and analyzed by the Institutional Research Office. Each academic department and major student service or general service area also compiles an annual statistical report that reflects any general education assessment impacts. All data is sent at least annually to the Provost’s office where further analysis of all of the data identifies emerging trends and issues with respect to general education and other assessment areas. These trends and issues are then discussed with the General Education Improvement and Assessment Committee, the Deans’ Council, and the President’s Council. Any needed changes in resource allocations and/or program modifications are identified and determined within these groups.

III. Periodic Review of Approved General Education Courses

Although there are a number of primary and secondary indicators of the overall level of general education attainment of WSU students, faculty were concerned that little of the data can be analyzed at a course level. To allow for this additional level of review, the Faculty General Education Improvement and Assessment Committee and The Faculty Curriculum Committee jointly developed a new policy that was adopted by Faculty Senate this past year that will require all courses that are currently approved as satisfying a general education requirement to be reviewed at least every five years. The review is to further ensure that the students in the approved courses are acquiring the intended general education competence.

While the policy provides protections to ensure that students who have taken the identified general education courses will not be penalized in the determination of their satisfaction of the required general education requirements if a particular course loses its general education certification, it also places the faculty on notice that they must produce evidence that students are acquiring the intended general education outcomes before a course will be reapproved as satisfying a general education requirement.

The first courses were reviewed under this program this fall. All remaining courses will be reviewed during the next four years.

IV. Case Studies Illustrating the Use of the General Education Outcomes Data to Improve Teaching and Learning

While many changes that affect students’ learning occur within an individual classroom, others, such as the change in structure described above relating to the creation of a General Education Assessment Committee and the regular review of general education classes, are structural changes that facilitate learning. The following case study reflects the process of a structural change to illustrate the ways that the various data sources are used to influence a set of structural changes.

Case Study One: The Weber State Mission Statement emphasizes the importance of civic engagement as an element of students’ general education. “The university, in partnership with the broader community, engages in research, artistic expression, public service, economic development, and community-based learning experiences in an environment that encourages freedom of expression while valuing diversity.” The paired concepts of engagement and diversity undergird this general education expectation. The General Education Mission Statement reinforces this emphasis by stating that general education at Weber State University should include “exposure to diverse fields of study [that] enables students to make intellectually honest, ethical decisions that reflect knowledge of, and respect for, diverse people, ideas and cultures. Such breadth of education also cultivates skills critical to student success in academic, personal, professional and community endeavors both within and beyond the university.” These issues are particularly critical for WSU. WSU is a commuter institution located in the one of the most culturally diverse communities in Utah. Local civic issues often focus on race, religion or other diversity issues.

  1. A review of the 2005 NSSE data (2005) suggested that students perceived themselves as having relatively low levels of community engagement as part of their academic coursework. Although seniors respond more favorably than freshman to the statement “participated in a community-based project as part of a regular course,” the difference between the two groups was small. (see Appendix II)
  2. The Noel-Levitz data for 2006 suggests that students perceive the institution’s “concern for the individual” as marginally lower than the national average, while the perceptions about institutional “responsiveness to diverse populations” is somewhat higher than the national average. The Noel-Levitz also reflected relatively lower student-reported rates of collaborative and active learning or enriching educational experiences. (see Appendix III)
  3. Data from both the NSSE and the Noel-Levitz suggested that WSU students had lower than average exposure to individuals from diverse backgrounds.
  4. Data from repeated administrations of the CLA suggested that, although WSU senior students overall outperformed their expected results in all of the CLA tasks, the rate of improvement in those tasks that require students to “acknowledge alternative explanations/viewpoints” which includes “consider[ing] other viewpoints and acknowledg[ing] that his/her answer is not the only perspective,” were meaningfully lower than the scores on the portions of the exam that required analysis and synthesis of the data, but did not require the student to take multiple perspectives. (see Appendix I)
  5. The 2005 HERI data (2005) indicated that almost half of the faculty included civic engagement in their instruction (18.2% taught a service-learning course; 48.6% collaborated with the local community in research/teaching) or co-curricular activities (50.7% advised student groups involved in service/volunteer work). However, the HERI data also suggested incongruence for faculty. Almost 62% of faculty members believed WSU makes building community partnerships a high priority, but only 43.4% believed WSU makes “providing resources for faculty to engage in community-based teaching or research” a high priority.
  6. One of the key findings of a CIC survey administered in 2006 regarding the community-based learning experience was that, although the majority of student respondents to the survey saw the application of their course material to everyday life via the service-learning experience (78.9%), only half suggested that the service experience helped them to better understand course material (52.1%). This suggested that students may need more assistance making direct connections between their service experience and their coursework.

Taken together, the students’ perceptions of their relatively low campus-related civic engagement experiences and their inability to take a position different than their own reflected a general education outcome that was quite different than institution’s intended outcomes with respect to civic engagement and diversity. This finding impacts on at least four of the general education outcomes that WSU has adopted.

  1. The institutional response:
    1. Setting an institutional example. The WSU President, who feels strongly about modeling outcomes that the university is seeking from its students, has become deeply involved with Campus Compact and with local civic organizations. Other senior administrators and faculty hold elected political positions and/or serve on community boards.
    2. Providing shared instructional materials that encourage diverse viewpoint. Faculty and administration have collaborated to provide free copies of the New York Times to all students. Many faculty members include current issues from these papers in their lectures. These discussions are intended to help the students recognize the importance of civic engagement and to acknowledge diverse perspectives.
    3. Designing a curricular and co-curricular program of student engagement. Faculty and administration (including a member of the WSU Board of Trustees) designed and are implementing a new “Civitas” program that involves curricular and co-curricular opportunities for students to interact with individuals of diverse backgrounds and to become meaningfully involved with the larger community.
    4. Involving the Honor’s Program and the Student Council in sponsoring forums that highlight issues of diversity.
    5. Conducting a Deliberative Democracy experience that brought students, faculty and community members together to identify and assess students’ perceptions about undocumented aliens.
    6. Successfully submitting an application to be designated as a Carnegie “Community Engaged Institution.” This designation allows the university to focus greater on-campus and community attention on civic engagement and understanding of diverse cultures.
    7. Dedicating new space, additional personnel and other resources to establish a Civic Involvement Center. The center is co-chaired by a faculty member and a student life professional. Each has a half-time assignment in the center. Students, faculty and staff have been recruited to facilitate service learning, community-based research and service projects.
    8. Creating a community-based learning supplemental instructor (CBLSI) program for community-based learning classes. Courses with a CBLSI designation offers students additional instruction on community-based learning assignments and reflection exercises.
    9. Integration of a service-learning track in the Bachelor of Integrated Studies program. This program allows students to design their own degree by fulfilling minor requirements in three disciplines. Each student is required to complete one of three kinds of capstone projects, one of which is a service-learning experience. Other programs that require capstone experiences encourage students to choose a community-based project. For example, in the Manufacturing and Mechanical Engineering Technology program, a group of students last year created a crash simulator for the Utah Highway Patrol to help them better demonstrate the need for seatbelts.
    10. Instituting a service project as part of The First Year Experience (FYE) program. The 500 students enrolled in FYE each year utilize the service opportunity to explore career possibilities or to teach the time management skills they’re learning to at-risk youth in local schools and afterschool programs.
    11. Building community-based learning into core courses in specific disciplines so that all students majoring in those disciplines will have a community-based learning experience. For example, the athletic training program requires students to participate in service-learning opportunities in each year of the program through a series of required courses.
    12. Integrating service learning into general education courses. For example, the Small Group and Interpersonal Communication course requires service-learning in every section and approximately 59% of WSU students taking core general education classes in a given year satisfy their humanities breadth area requirement by taking this course.
  2. Initial Outcomes: While the majority of freshmen who have come in during the period of greater emphasis on civic engagement and diversity will not be seniors for two or more years, there are some encouraging initial results.
    1. In the last academic year, over 90,000 hours of service were tracked in the CIC.
    2. The percentage of faculty and students involved in formal service learning classes has increased.
    3. The Deliberative Democracy experience was recognized as a model for facilitating changed perceptions through active engagement in learning about diversity and will be expanded this year to include a Fall and Spring semester experience.
    4. WSU was recently recognized by the Carnegie Foundation for the Advancement of Teaching as a Community Engaged University for its Curricular Engagement and Outreach and Partnerships.
  3. Metrics to Track Changes
    1. We have just received data on the 2009 administrations of the CLA and NSSE. An initial review suggests that there has been improvement in indicators of diversity and engagement on both exams. In order to isolate the effect of service learning, the CLA will be administered this year to a sample of graduating students who have achieved a service learning designation and a sample drawn from the general population of graduating seniors. If, as we suspect, service learning has a significant impact on the student’s understanding of diversity, the General Education and Assessment Committee will consider modifying the diversity general education requirement to include a service component. Further information drawn from the HERI, CLA, NSSE and Noel-Levitz through the next two administrations of each instrument should give additional evidence of any changes in levels of students’ appreciation for diversity and in their capacity to take perspective other than their own.
    2. Data collected in conjunction with First Year Experience, Bachelors of Integrated Studies, the CIVITAS programs and the college-level assessments should provide ways to triangulate the data collected through the nationally standardized tests.

Case Study Two: To illustrate the process at an earlier stage, this case study focuses on Weber State’s continuing experimentation with ways to more accurately identify and standardize student attainment of general education objectives. One of the challenges with the institutional summative measures, such as the CLA and the NSSE, is that it is very difficult to have a sample large enough to allow for inferences at the department, or even the college level. While each of the colleges and schools compile annual assessment reports that include general education assessment data, it has been difficult to correlate those results to ensure both evidence of the cross-disciplinary impact of the general education curriculum and information that allows for modifications of general education instruction at the departmental level. In an attempt to respond to this dilemma, during the past several years Weber has conducted pilot tests of cross disciplinary electronic portfolios. While the initial results from those pilots are encouraging and that approach may yet prove to be an effective way of getting the data that we would like to have, the near-term technological hurdles of getting a thousand instructors and twenty two thousand students into that environment have proven to be daunting.

As a step in that direction, Weber last spring suggested that, where appropriate, departments include artifacts of student work, grading rubrics and grades assigned on the basis of the rubrics in their annual assessment reports. Many departments chose to include such data. The analysis of the rubrics has allowed the university to identify the common general education outcomes reflected by the rubrics. These common outcomes will be used by the Faculty General Education Improvement and Assessment Committee to review the existing institutionally identified general education outcomes and the nationally normed institutional measures to identify the correlation of institutional level measures with the departmental measures. This will help to determine whether alterations need to be made at any level to ensure consonance between departmental general education instruction and assessment.

For example, Weber is conducting research with the performance of targeted populations on national exams as a further indicator of cross-disciplinary general education attainment. The composition program involves the two classes taken by most students to satisfy the general education written communication outcomes. The instructors for those classes have agreed to use a shared rubric for grading. The artifacts, rubrics and grades collected by the instructors will be correlated with the performance of those students on the CLA to determine whether our internal assessment processes correlate strongly with the nationally-normed measures.

We will be doing similar research to determine the effect of service on students’ understanding of diversity by oversampling the students who are graduating this year with serve distinction and comparing their understanding of diversity, as reflected in their CLA scores, with that of the general student population. Other targeted research projects include assessing the relative performance of Bachelor of Integrated Studies majors against the performance of students in more traditional majors, assessing the relative performance of students involved in undergraduate research against the performance of students who are not engaged in such research, comparing the performance of students completing their Scientific Inquiry requirement by taking a Health class versus those taking more traditional science classes, and assessing the performance of students who complete a computerized math 1010 class compared to those who complete the class in a more traditional learning setting.


Weber State UniversityOgden, Utah 84408

Privacy PolicyTerms of UseNondiscrimination Policy